[Hawkular-dev] agent using custom metric IDs

[Lucas Ponce]

Hi John,

Thanks for bringing this as it might have potential side effects.

I haven't reviewed your proposal yet but you can take an idea about how MiQ links Hawkular metrics inside something that can be managed internally of MiQ.

https://github.com/ManageIQ/manageiq/blob/master/product/live_metrics/middleware_server.yaml
https://github.com/ManageIQ/manageiq/blob/master/product/live_metrics/middleware_datasource.yaml

I guess that if the id of the type is still valid, then perhaps nothing affects but just to be sure I chime in here just to validate it.

I am afraid about potential changes at this stage.

----- Mensaje original -----
> De: "John Mazzitelli" <mazz at redhat.com>
> Para: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
> Enviados: Jueves, 7 de Julio 2016 3:13:37
> Asunto: [Hawkular-dev] agent using custom metric IDs
> 
> <tl;dr> ======
> 
> Agent is introducing two changes:
> 
> 1. Metric Type definitions created by the agent will have the same ID as
> before, but its Name is changing (probably does not affect anyone).
> 
> 2. Clients (like UI, HawkFX, etc) should no longer assume the agent's
> h-inventory metric definition IDs match h-metric metric IDs - instead, they
> must look at the "metric-id" property on the h-inventory metric definition
> to know how to look up the actual metric data in h-metrics. Will affect all
> clients, but metric IDs will default to what they are today - so nothing
> changes and thus nothing will break today if you run with out of box
> configuration.
> 
> </tl;dr> ======
> 
> There is a use-case where the agent needs to support custom metric IDs (that
> is, rather than accepting the out-of-box metric IDs created by the agent,
> allow the user to define what the metric IDs should look like). See
> https://issues.jboss.org/browse/HWKAGENT-78
> 
> As a refresher, remember that when you create resources in inventory, those
> resources can be associated with one or more "metric" definitions. Those
> resource metrics are themselves associated with a "metric type" definition.
> Today, when the agent stores metric data into Hawkular Metrics, it stores
> the data under the ID of the "metric" that is associated with the resource
> (so the h-inventory metric ID is the same as the h-metric metric ID by
> definition - at least for the data the hawkular wildfly agent inserts).
> 
> I am proposing two changes in the PR:
> https://github.com/hawkular/hawkular-agent/pull/226
> 
> First, today, the "metric type" definition that the agent creates has an ID
> and a Name that are identical. I am changing this so the ID stays the same
> (which is the metric set name, followed by "~", followed by the name of the
> metric -- e.g. if there was a <metric-set-dmr name="this"> that contains a
> <metric-dmr name="that">, the metric type ID would be "this~that"), but the
> Name is only the name without the set name (e.g. the name would be "that" in
> the previous example).
> 
> The above is a minor change, and I doubt anyone is affected by it. But I
> point it out just in case.
> 
> Second, it should no longer be assumed that the inventory's resource metric
> ID is identical to the h-metric's metric ID.
> 
> This second change will potentially affect everyone (I know it affects
> Heiko's HawkFX :)
> 
> Now, that said, nothing really changes now, because the defaults will remain
> as they are (that is, the h-inventory's metric ID will still be exactly the
> same as the h-metrics ID - the agent keeps them identical). The change
> happens when the user actually configures the agent with a custom metric ID
> template (e.g. <remote-jmx metric-id-template="${pod.id}-%MetricTypeName"
> ...>). This means h-metric IDs will be DIFFERENT than h-inventory metric
> IDs.
> 
> How then does a client know what h-metric IDs to look for if they only have
> h-inventory metric definitions? Well, recall that inventory allows for
> properties to be associated with any entity. I use this feature here. Rather
> than rely on an implicit rule ("h-inventory metric ID is the same as
> h-metric metric ID") I explicitly define this linkage in a property called
> "metric-id" on the h-inventory metric definition. Out of box, that
> property's value will be identical to the h-inventory metric ID (and hence
> why nothing really changes - since the explicit rule in this case provides
> the same behavior as if following the old implicit rule). In fact, I'm
> considering if I should set that property at all if its the same as the
> h-inventory ID - I think it might be better to only set a "metric-id"
> property if it is different. But this would require clients to know about
> the implicit rule if there is no metric-id property set ("is there a
> metric-id property set? No? Then use the h-inven!
>  tory metric ID for the h-metric metric ID").
> 
> For example, see here (this is a live example I copied from the "raw"
> inventory JSON that HawkFX gave me for a metric) - this is the h-inventory
> entity definition for the metric "Heap Used" on my WildFly Server resource -
> notice the "properties" map has a "metric-id" value that is DIFFERENT than
> the "id" - that "metric-id" is something I customized in my agent config in
> standalone.xml (well, I used the swarm agent, so I put it in the swarm
> config, but its basically the same thing):
> 
> {
>   "path":
>   "/t;hawkular/f;mazz/m;MI~R~%5Bmazz%2FWildFly~~%5D~MT~WildFly%20Memory%20Metrics~Heap%20Used",
>   "properties": {
>     "__identityHash": "70e59a5d427632223da36c225ba6ef8572985",
>     "metric-id": "feed=mazz__msn=WildFly__typeName=Heap Used__resName=WildFly
>     Server [WildFly]__resId=WildFly~~__typeId=WildFly Memory Metrics~Heap
>     Used"
>   },
>   "name": "Heap Used",
>   "identityHash": "70e59a5d427632223da36c225ba6ef8572985",
>   "type": {
>     "path": "/t;hawkular/f;mazz/mt;WildFly%20Memory%20Metrics~Heap%20Used",
>     "name": "Heap Used",
>     "identityHash": "3be5b5fdabed925ac46fdc6d8295e34bbd3147a",
>     "unit": "BYTES",
>     "type": "GAUGE",
>     "collectionInterval": 30,
>     "id": "WildFly Memory Metrics~Heap Used"
>   },
>   "id": "MI~R~[mazz/WildFly~~]~MT~WildFly Memory Metrics~Heap Used"
> }
> 
> Notice this:
> 
>    feed=mazz__msn=WildFly__typeName=Heap Used__resName=WildFly Server
>    [WildFly]__resId=WildFly~~__typeId=WildFly Memory Metrics~Heap Used
> 
> is different from this:
> 
>    MI~R~[mazz/WildFly~~]~MT~WildFly Memory Metrics~Heap Used
> 
> And that's the issue. Clients have to know to look for the "metric-id"
> property and use it when looking up metric data in h-metrics (so if you want
> to graph the data, you have to ask h-metrics for the data associated with
> the value found in the "metric-id" property).
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>