[Hawkular-dev] Calculated metrics: sharing ideas

[Joel Takvorian]

Hi,

I just want to share some ideas about the eventuality of having a language
to perform some arithmetic / aggregations on metrics, before it goes out of
my head...

Here's an example of what I would personally love to see in Hawkular:

----------------------------------------------
*Example:*
*sum(stats(rate((id(my_metric), tags(a=foo AND b=bar),
regexp(something_.+)), 5m), 10m))*
|
|=> "id", "tags" and "regexp" all return a set of raw metrics (0-1 for id,
0-n for tags and regexp)
|==> "(a,b,c)" takes n parameters, all sets of metrics, and flatten them in
a single set
|===> rate(set_of_raw_metrics, rate_period) computes the rate for each of
them and return a set of metrics (map n=>n)
|====> stats(set_of_raw_metrics, bucket_size) bucketize the raw metrics,
returning the same number of bucketized metrics (map n=>n)
|=====> sum(set_of_stats_metrics) sums every buckets, returning a single
bucketized metric (fold n=>1)


*Other:*
Functions like "sum" that take stats_metrics could have overloaded shortcut
"sum(set_of_raw_metrics, bucket_size)" to perform the bucketing.
In other words above example could be rewritten:
*sum(rate((id(my_metric), tags(a=foo AND b=bar), regexp(something_.+)),
5m), 10m)*

Note: we can do aggregations like "sum" on raw data if necessary, it just
means we have to interpolate.

*Scalar operations:*
*sum((id(a_metric_in_milliseconds), 1000*id(a_metric_in_seconds)), 10m)*
----------------------------------------------

Of course many other functions could come growing the library.

Now I suppose the big question, if we want to do such thing, is "are we
going to invent our own language?" I don't know if there are standards for
this, and if they are good.

The Prometheus query language cannot be transposed because a label is not a
tag and it makes no sense for us to write something like
"my_metric{tag=foo}", it's either "my_metric" or "tag=foo".

The same language could be used both for read-time on-the-fly aggregations
and write-time / cron-based rollups.

WDYT?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170719/fc0a7be5/attachment-0001.html