1:20 p.m.
New Hawkular blog post from noreply(a)hawkular.org (Joel Takvorian): http://ift.tt/2uPct0T
There have been improvements lately in the Hawkular Grafana datasource that are worth
mentioning.
The way to query metrics by tags has changed since plugin v1.0.8. It now takes advantage
of the Hawkular Metrics' tags query language, that was introduced server-side in
Metrics 0.24.0 and enhanced in 0.25.0. To sum it up, Metrics integrates a parser that
allows queries such as: tag1 = 'awesome!' AND tag2 NOT IN ['foo',
'bar'].
The datasource in now able to fetch bucketized metrics stats, instead of raw metrics. It
consists in aggregating datapoints in slices of time (buckets) and providing, for each
slice, some statistics like min, max, average and more. The exact content of a bucket is
described in Metrics REST API. Hawkular has always been able to provide metric stats
server-side, but being able to use them in the Grafana plugin is new, introduced in
v1.0.9.
The new query interface
Tags query language
The first change is that you don’t have to choose between query by tag and query by metric
id anymore, you can do both at the same time. Querying by tag will refine the available
list of metric names (much like a filter) and can result in multiple metrics from a single
query. By selecting a metric name, you restrict the query to only display that one. This
filtering is really nice when there’s tons of metrics available, like in the case of
hundreds of OpenShift pods being monitored with the same tenant.
The simple key/value pairs interface is now replaced with a more elaborated query builder,
following the pattern: 'tag-key' 'operator' 'tag-value(s)'
['AND/OR' etc.]
The following images show a walk-through:
Selecting the tag key
Selecting the tag operator
Selecting the tag value
The text fields include dynamic suggestions, you can use Grafana template variables within
tag values, or enter free text. Once you’ve set up a tag query expression, the relevant
metrics immediately show up on the chart and the list of available metrics in the dropdown
list in updated.
Filtered metrics
This query builder lets you build almost any tag query that the Hawkular server
understands. There are however some corner cases. For now this builder doesn’t allow you
to prioritize expressions with parentheses. For instance, you cannot build c1 =
'foo' OR (b1 != 'B' AND a1 = 'abcd'). As a workaround you can turn
off the query builder and directly type in your query expression.
Toggle editor mode
It will be sent as is to the Hawkular Metrics server. This will also be useful to fill the
gap if the language evolves server-side and this plugin isn’t updated immediately.
Stats query
The other important feature is the ability to run stats queries against Hawkular Metrics,
instead of raw queries. There are several reasons to do this:
it reduces the network load, and client-side processing load, especially when raw data
would contain tons of datapoints
it enables some aggregation methods
it also allows higher-level analysis with stats such as percentiles
To enable it, just clear the raw checkbox.
Toggle stats mode
When you clear the raw checkbox, you can configure Multiple series aggregation to None,
Sum or Average and can configure Stat type as avg, min, max, median, sum and different
percentiles. You can display several different Stat types within the same query.
Stats without aggregation: each two metrics show avg, min and max
Same query with series aggregation: the two metrics are averaged into one, which shows
avg, min and max
If the query returns multiple series, use Multiple series aggregation to define if and how
to aggregate them. None will show them individually on the chart. But consider for
instance the case of an OpenShift pod with many replicas, and you’re tracking their memory
usage. It may be more relevant here to aggregate all of them, both sum and average are
meaningful here.
The Stat type option refers to an aggregation at a different level: not between multiple
metrics, but within a single metric, all raw datapoints are aggregated within time
buckets.
Conclusion
These two improvements aim a common goal, that is facilitating querying over large amounts
of data. This is becoming crucial especially in the context of microservices and
applications running on container platforms, as the number of metrics explodes. Proper
metrics tagging is the corner stone to make sense of this data.
from Hawkular Blog