[Hawkular-dev] Calculated metrics: sharing ideas

[Michael Burman]

Hi,

I'd personally would want to see another type of functional language, 
such that would look like the one when you do Streams / RxJava. I guess 
it's closer to the one in MongoDB/RethinkDB/InfluxDB 2.0/Riemann, than 
the one in Prometheus/Graphite (PromQL is mostly evolution of Graphite's 
query language).

And it could allow us to provide features such as streaming metrics, 
since we could transform those queries directly to reactive queries (we 
should take advantage of the fact that we process the metrics by 
reacting to them in the backend).

    - Micke


On 07/19/2017 06:36 PM, Joel Takvorian wrote:
> Hi,
>
> I just want to share some ideas about the eventuality of having a 
> language to perform some arithmetic / aggregations on metrics, before 
> it goes out of my head...
>
> Here's an example of what I would personally love to see in Hawkular:
>
> ----------------------------------------------
> *Example:*
> /sum(stats(rate((id(my_metric), tags(a=foo AND b=bar), 
> regexp(something_.+)), 5m), 10m))/
> |
> |=> "id", "tags" and "regexp" all return a set of raw metrics (0-1 for 
> id, 0-n for tags and regexp)
> |==> "(a,b,c)" takes n parameters, all sets of metrics, and flatten 
> them in a single set
> |===> rate(set_of_raw_metrics, rate_period) computes the rate for each 
> of them and return a set of metrics (map n=>n)
> |====> stats(set_of_raw_metrics, bucket_size) bucketize the raw 
> metrics, returning the same number of bucketized metrics (map n=>n)
> |=====> sum(set_of_stats_metrics) sums every buckets, returning a 
> single bucketized metric (fold n=>1)
>
>
> *Other:*
> Functions like "sum" that take stats_metrics could have overloaded 
> shortcut "sum(set_of_raw_metrics, bucket_size)" to perform the bucketing.
> In other words above example could be rewritten:
> /sum(rate((id(my_metric), tags(a=foo AND b=bar), 
> regexp(something_.+)), 5m), 10m)/
>
> Note: we can do aggregations like "sum" on raw data if necessary, it 
> just means we have to interpolate.
>
> *Scalar operations:*
> /sum((id(a_metric_in_milliseconds), *1000**id(a_metric_in_seconds)), 10m)/
> ----------------------------------------------
>
> Of course many other functions could come growing the library.
>
> Now I suppose the big question, if we want to do such thing, is "are 
> we going to invent our own language?" I don't know if there are 
> standards for this, and if they are good.
>
> The Prometheus query language cannot be transposed because a label is 
> not a tag and it makes no sense for us to write something like 
> "my_metric{tag=foo}", it's either "my_metric" or "tag=foo".
>
> The same language could be used both for read-time on-the-fly 
> aggregations and write-time / cron-based rollups.
>
> WDYT?
>
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev