On 4/17/2017 6:10 PM, Benjamin Heiskell wrote:

Hey all,

I recently deployed Hawkular because I found its scaling model appealing. It has been working well for our basic monitoring usage when combined with Grafana.

One of my consumers is trying to graph KPIs that involve cross-metric mathematics like sums, division, and moving averages (i.e., post-ingress transformations). I looked into how to accomplish this with Hawkular, but I didn't see any obvious way to do it. The stats endpoint seems to only support aggregations on a single metric/tag at a time. I'm finding that we can't visualize these metrics the same way you can with other TSDBs like Graphite/Prometheus.

It's maybe not what you're looking for but the model for aggregating across metrics is to use a tag query to define the set of applicable metrics. The tag query expressions are fairly robust [1], of course the tags need to be there to make it useful. But given the tags it is possible to do sums, percentiles, etc... As an aside, there is small expression language for using those aggregates for alerting. If you're interested check out http://www.hawkular.org/hawkular-metrics/docs/user-guide/#_query_based_alerting.

[1] The tag query language has just recently been released and I'm not sure if the documentation has been updated (Snegrea?). But you can look here https://github.com/hawkular/hawkular-metrics/pull/725.

I did a little research, and found Heiko Rupp's blog post on HawkFX where he seems to already be exploring this idea [1]. Perhaps the query language item on the 2017 road map refers to incorporating this?

I would like to assist in developing this feature. Given that there is already a tags DSL, it seems like it would make sense to extend that to support metric transformations.

Grafana is also considering adding this to Grafana itself, but it doesn't look like the ticket has any traction [2]. Additionally, shipping potentially megabytes of data to Grafana (or the user's browser) to aggregate doesn't seem ideal.

I documented a few TSDB DSLs styles here [3].

What do you all think?

Thanks,

Ben

[1] http://pilhuhn.blogspot.com/2016/09/computed-metrics-for-hawkfx.html

[2] https://github.com/grafana/grafana/issues/3677

[3] https://gist.github.com/bheiskell/25af2543ee8588f70895b0311ef291d3
_______________________________________________
hawkular-dev mailing list
hawkular-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev