[Hawkular-dev] agent using custom metric IDs

[Lukas Krejci]

On středa 6. července 2016 21:13:37 CEST John Mazzitelli wrote:
> <tl;dr> ======
> 
> Agent is introducing two changes:
> 
> 1. Metric Type definitions created by the agent will have the same ID as
> before, but its Name is changing (probably does not affect anyone).
> 
> 2. Clients (like UI, HawkFX, etc) should no longer assume the agent's
> h-inventory metric definition IDs match h-metric metric IDs - instead, they
> must look at the "metric-id" property on the h-inventory metric definition
> to know how to look up the actual metric data in h-metrics. Will affect all
> clients, but metric IDs will default to what they are today - so nothing
> changes and thus nothing will break today if you run with out of box
> configuration.
> 
> </tl;dr> ======
> 
> There is a use-case where the agent needs to support custom metric IDs (that
> is, rather than accepting the out-of-box metric IDs created by the agent,
> allow the user to define what the metric IDs should look like). See
> https://issues.jboss.org/browse/HWKAGENT-78
> 
> As a refresher, remember that when you create resources in inventory, those
> resources can be associated with one or more "metric" definitions. Those
> resource metrics are themselves associated with a "metric type" definition.
> Today, when the agent stores metric data into Hawkular Metrics, it stores
> the data under the ID of the "metric" that is associated with the resource
> (so the h-inventory metric ID is the same as the h-metric metric ID by
> definition - at least for the data the hawkular wildfly agent inserts).
> 
> I am proposing two changes in the PR:
> https://github.com/hawkular/hawkular-agent/pull/226
> 
> First, today, the "metric type" definition that the agent creates has an ID
> and a Name that are identical. I am changing this so the ID stays the same
> (which is the metric set name, followed by "~", followed by the name of the
> metric -- e.g. if there was a <metric-set-dmr name="this"> that contains a
> <metric-dmr name="that">, the metric type ID would be "this~that"), but the
> Name is only the name without the set name (e.g. the name would be "that"
> in the previous example).
> 
> The above is a minor change, and I doubt anyone is affected by it. But I
> point it out just in case.
> 
> Second, it should no longer be assumed that the inventory's resource metric
> ID is identical to the h-metric's metric ID.
> 
> This second change will potentially affect everyone (I know it affects
> Heiko's HawkFX :)
> 
> Now, that said, nothing really changes now, because the defaults will remain
> as they are (that is, the h-inventory's metric ID will still be exactly the
> same as the h-metrics ID - the agent keeps them identical). The change
> happens when the user actually configures the agent with a custom metric ID
> template (e.g. <remote-jmx metric-id-template="${pod.id}-%MetricTypeName"
> ...>). This means h-metric IDs will be DIFFERENT than h-inventory metric
> IDs.
> 
> How then does a client know what h-metric IDs to look for if they only have
> h-inventory metric definitions? Well, recall that inventory allows for
> properties to be associated with any entity. I use this feature here.
> Rather than rely on an implicit rule ("h-inventory metric ID is the same as
> h-metric metric ID") I explicitly define this linkage in a property called
> "metric-id" on the h-inventory metric definition. Out of box, that
> property's value will be identical to the h-inventory metric ID (and hence
> why nothing really changes - since the explicit rule in this case provides
> the same behavior as if following the old implicit rule). In fact, I'm
> considering if I should set that property at all if its the same as the
> h-inventory ID - I think it might be better to only set a "metric-id"
> property if it is different. But this would require clients to know about
> the implicit rule if there is no metric-id property set ("is there a
> metric-id property set? No? Then use the h-inven! tory metric ID for the
> h-metric metric ID").
> 
> For example, see here (this is a live example I copied from the "raw"
> inventory JSON that HawkFX gave me for a metric) - this is the h-inventory
> entity definition for the metric "Heap Used" on my WildFly Server resource
> - notice the "properties" map has a "metric-id" value that is DIFFERENT
> than the "id" - that "metric-id" is something I customized in my agent
> config in standalone.xml (well, I used the swarm agent, so I put it in the
> swarm config, but its basically the same thing):
> 
> {
>   "path":
> "/t;hawkular/f;mazz/m;MI~R~%5Bmazz%2FWildFly~~%5D~MT~WildFly%20Memory%20Met
> rics~Heap%20Used", "properties": {
>     "__identityHash": "70e59a5d427632223da36c225ba6ef8572985",
>     "metric-id": "feed=mazz__msn=WildFly__typeName=Heap
> Used__resName=WildFly Server [WildFly]__resId=WildFly~~__typeId=WildFly
> Memory Metrics~Heap Used" },
>   "name": "Heap Used",
>   "identityHash": "70e59a5d427632223da36c225ba6ef8572985",
>   "type": {
>     "path": "/t;hawkular/f;mazz/mt;WildFly%20Memory%20Metrics~Heap%20Used",
>     "name": "Heap Used",
>     "identityHash": "3be5b5fdabed925ac46fdc6d8295e34bbd3147a",
>     "unit": "BYTES",
>     "type": "GAUGE",
>     "collectionInterval": 30,
>     "id": "WildFly Memory Metrics~Heap Used"
>   },
>   "id": "MI~R~[mazz/WildFly~~]~MT~WildFly Memory Metrics~Heap Used"
> }
> 
> Notice this:
> 
>    feed=mazz__msn=WildFly__typeName=Heap Used__resName=WildFly Server
> [WildFly]__resId=WildFly~~__typeId=WildFly Memory Metrics~Heap Used
> 
> is different from this:
> 
>    MI~R~[mazz/WildFly~~]~MT~WildFly Memory Metrics~Heap Used
> 
> And that's the issue. Clients have to know to look for the "metric-id"
> property and use it when looking up metric data in h-metrics (so if you
> want to graph the data, you have to ask h-metrics for the data associated
> with the value found in the "metric-id" property).
> 

What if you used the inventory's canonical path as an the metric ID for h-
metrics (e.g. used "path" from the json above)? After all, if inventory should 
be the storage of all things Hawkular knows about, the canonical path is the 
one and only thing that can identify any one thing in inventory. So it lends 
itself nicely to be used as an identifier in other components.

> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev


-- 
Lukas Krejci