[Hawkular-dev] Inventory storage on metrics

[Joel Takvorian]

Quick reminder for those who may not have followed the discussions: this is
about removing the inventory project as it currently stands, and replace it
with Metrics-based (so C*-based) storage, using String metrics and json
blobs.

So, I've been analyzing the usages of inventory in the wildfly agent and in
manageiq / ruby client.

As posted on another topic, I started with the following pattern for
putting inventories in string metrics:

- metrics are named:
*inventory.[feedid].[r|rt|mt].[id]*

which means that we would store JSON blobs by resource type, by metric type
and by root resource (all per feed id). Which also means we don't allow
anything outside of a feed.

This naming pattern seems to be quite easily transposable to the wildfly
agent feeding.

Things become more complicated when we look at the ruby client / manageIQ.
To help in querying, we can add those tags on every metric:

module: inventory

feed: [feedid]

type: [r|rt|mt]

With this, we can easily query, for instance, all resource types of a given
feed, or all feeds, or child resources at a given path, etc.

Now, after analysis of the ruby code (and I hope I didn't miss important
stuff), I have two remarks:

- There is a lot a methods in the ruby client that are not used in manage
IQ (listed below [1]). Of course those methods may have an interest for a
general purpose client, but since we're removing a lot of functionalities
in inventory, I would suggest not to focus on them, and just remove them
for the time being. This is possible, of course, if no other product than
manageIQ consumes then... is it the case?

- I've found just 1 method in the ruby client that would be badly impacted
by the usage of string metrics as currently defined. And unfortunately it's
also the one that is most called (in number of occurrence in code) by
manage IQ, it's "list_resources_for_type"; With the implementation I
described above, to list all resources of a given type we would have to get
all resource blobs of a feed and search in all of them to check all root &
child resource types. That would probably be terrible for performances.

A suggestion to this problem, is to tag each root resource with one tag per
resource type, something like: "
*rt.MyResourceType=resource1,resource2,resource3*".
Then, we would still have to load the whole blobs and search resources in
them, but we would exclude from the search all root resources that doesn't
have those types, and we wouldn't have to search extensively in the blobs
as we would know the paths.

*Question to metrics devs* about that: can we use any character for
tagging, or is there some limitation?

Or if you can suggest a better way, I'm all ears  :)

[1] list of methods that seem to be unused from the ruby client:
- list_resources_for_feed
- list_relationships
- list_relationships_for_feed
- get_entity
- list_metrics_for_metric_type
- list_metrics_for_resource_type
- create_feed
- delete_feed
- create_resource_type
- create_resource
- create_resource_under_resource
- create_metric_type
- list_operation_definitions
- list_operation_definitions_for_resource
- create_metric_for_resource
- no_more_events

Joel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170306/25419944/attachment.html