The Release Readiness Dashboard chose Elastic Search as the source of Bugzilla data because it was fast and provided a convenient means of retrieving historic information for plotting trends. However, the native queries used to request data from Elastic Search clusters are long, ugly, and horrible JSON objects.
To solve the issue of dealing with ugly Elastic Search queries, the administrator of Bugzilla’s Elastic Search cluster has set up a helpful Javascript library for web developers. It acts as a middle layer for requesting data from the Elastic Search cluster using Qb queries instead of native Elastic Search queries. The RRDashboard uses the Qb Javascript library.
Qb queries, similar to Elastic Search queries, are JSON formatted objects. One might now ask the question of what Qb queries are, and how it has an advantage over native Elastic Search queries if both are just about as obscure as the other. Qb queries for one, are much shorter than native Elastic Search queries. They are also a lot more read-able.
Coming from an obscure background, the problem with writing Qb queries still persists. The release management team will have significant difficulty in adopting the RRDashboard if they are unable to form Qb queries that request the same data they are currently pulling using the Bugzilla search interface. I am now ready to introduce the Qb query builder.
When the release management team builds a query using the Bugzilla search interface, a long Bugzilla URL is generated containing the query parameters. As an input to the Qb query builder, this Bugzilla URL can be directly translated into a Qb query after specifying some other required parameters like the cluster to query against. In the background, here is what actually happens.
Upon submitting the form on the Qb query builder, a big Javascript event handler is triggered. The search parameters are extracted from the submitted Bugzilla URL, then re-appended to the Bugzilla search interface’s URL. Using a CURL request, we receive an enormous HTML string containing the Bugzilla search interface’s page filled with user-specified search parameters. Scraping then occurs on each individual page element to extract the search parameters, which are then appended into a JSON object formatted as a Qb query.
The benefit of such an implementation is that this is as simple as it will ever get for an end-user to work between Bugzilla and Qb queries. Unfortunately, there are also significant limitations to this implementation.
Scraping HTML of the Bugzilla search interface implies that the Qb query tool is very dependent on how the views of Bugzilla are generated. If the HTML layout of the Bugzilla search interface changes, the Qb query builder will lose its accuracy and start missing search parameters in the output Qb queries. A possible alternative is directly parsing the Bugzilla URL without doing a CURL request to get the HTML. However, this can be a potentially complex task to handle and will require more time beyond what I current have as an intern.
Another limitation to the Qb query builder is that it can be buggy. Bugzilla’s search interface is a mature product that can handle many different combinations of queries. The Qb query builder, on the other hand, is new and may not be able to handle as many varieties of querying as Bugzilla can. As a result, until the Qb query builder matures enough to handle all the corner cases that Bugzilla may throw at it, users of the tool may occasionally have to modify the output Qb queries manually to get what they want.
More to come!