Quick reminder for those who may not have followed the discussions: this is about removing the inventory project as it currently stands, and replace it with Metrics-based (so C*-based) storage, using String metrics and json blobs. So, I've been analyzing the usages of inventory in the wildfly agent and in manageiq / ruby client. As posted on another topic, I started with the following pattern for putting inventories in string metrics: - metrics are named: inventory.[feedid].[r|rt|mt].[id] which means that we would store JSON blobs by resource type, by metric type and by root resource (all per feed id). Which also means we don't allow anything outside of a feed. This naming pattern seems to be quite easily transposable to the wildfly agent feeding. Things become more complicated when we look at the ruby client / manageIQ. To help in querying, we can add those tags on every metric: module: inventory feed: [feedid] type: [r|rt|mt] With this, we can easily query, for instance, all resource types of a given feed, or all feeds, or child resources at a given path, etc. Now, after analysis of the ruby code (and I hope I didn't miss important stuff), I have two remarks: - There is a lot a methods in the ruby client that are not used in manage IQ (listed below [1]). Of course those methods may have an interest for a general purpose client, but since we're removing a lot of functionalities in inventory, I would suggest not to focus on them, and just remove them for the time being. This is possible, of course, if no other product than manageIQ consumes then... is it the case? - I've found just 1 method in the ruby client that would be badly impacted by the usage of string metrics as currently defined. And unfortunately it's also the one that is most called (in number of occurrence in code) by manage IQ, it's "list_resources_for_type"; With the implementation I described above, to list all resources of a given type we would have to get all resource blobs of a feed and search in all of them to check all root & child resource types. That would probably be terrible for performances. A suggestion to this problem, is to tag each root resource with one tag per resource type, something like: " rt.MyResourceType=resource1,resource2,resource3 ". Then, we would still have to load the whole blobs and search resources in them, but we would exclude from the search all root resources that doesn't have those types, and we wouldn't have to search extensively in the blobs as we would know the paths. Question to metrics devs about that: can we use any character for tagging, or is there some limitation? Or if you can suggest a better way, I'm all ears :) [1] list of methods that seem to be unused from the ruby client: - list_resources_for_feed - list_relationships - list_relationships_for_feed - get_entity - list_metrics_for_metric_type - list_metrics_for_resource_type - create_feed - delete_feed - create_resource_type - create_resource - create_resource_under_resource - create_metric_type - list_operation_definitions - list_operation_definitions_for_resource - create_metric_for_resource - no_more_events Joel _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

I haven't checked that part of the puzzle yet, will do it tomorrow. However I hope it is possible to keep the ResourcePath as they are currently used. Since we can retrieve the root resource from a canonical path of a child resource, it's should be ok to find the desired associated string metric. On Mon, Mar 6, 2017 at 7:28 PM, John Mazzitelli <mazz(a)redhat.com&gt; wrote: > This also affects the command-gateway stuff since virtually all of the > messages require an Inventory ResourcePath (that is how the MiQ UI targets > where it wants the message to go, and how the server knows what agent to > pass the message to, and how the agent knows what specific resource to > operate on). > > These ResourcePaths are Hawkular-Inventory resource paths - if resource > path formats change, we need to change the way the command-gateway stuff > works. > > I can't speak for the MiQ UI side, but from server/agent sides, it isn't > trivial but also not that hard to do these changes. It is just work we need > to remember we have to do - work in the MiQ UI, server, and agent. > > ----- Original Message ----- > > Quick reminder for those who may not have followed the discussions: > this is > > about removing the inventory project as it currently stands, and > replace it > > with Metrics-based (so C*-based) storage, using String metrics and json > > blobs. > > > > So, I've been analyzing the usages of inventory in the wildfly agent > and in > > manageiq / ruby client. > > > > As posted on another topic, I started with the following pattern for > putting > > inventories in string metrics: > > > > - metrics are named: > > inventory.[feedid].[r|rt|mt].[id] > > > > which means that we would store JSON blobs by resource type, by metric > type > > and by root resource (all per feed id). Which also means we don't allow > > anything outside of a feed. > > > > This naming pattern seems to be quite easily transposable to the wildfly > > agent feeding. > > > > Things become more complicated when we look at the ruby client / > manageIQ. To > > help in querying, we can add those tags on every metric: > > > > > > > > module: inventory > > > > feed: [feedid] > > > > type: [r|rt|mt] > > > > With this, we can easily query, for instance, all resource types of a > given > > feed, or all feeds, or child resources at a given path, etc. > > > > Now, after analysis of the ruby code (and I hope I didn't miss important > > stuff), I have two remarks: > > > > - There is a lot a methods in the ruby client that are not used in > manage IQ > > (listed below [1]). Of course those methods may have an interest for a > > general purpose client, but since we're removing a lot of > functionalities in > > inventory, I would suggest not to focus on them, and just remove them > for > > the time being. This is possible, of course, if no other product than > > manageIQ consumes then... is it the case? > > > > - I've found just 1 method in the ruby client that would be badly > impacted by > > the usage of string metrics as currently defined. And unfortunately it's > > also the one that is most called (in number of occurrence in code) by > manage > > IQ, it's "list_resources_for_type"; With the implementation I described > > above, to list all resources of a given type we would have to get all > > resource blobs of a feed and search in all of them to check all root & > child > > resource types. That would probably be terrible for performances. > > > > A suggestion to this problem, is to tag each root resource with one tag > per > > resource type, something like: " > > rt.MyResourceType=resource1,resource2,resource3 ". > > Then, we would still have to load the whole blobs and search resources > in > > them, but we would exclude from the search all root resources that > doesn't > > have those types, and we wouldn't have to search extensively in the > blobs as > > we would know the paths. > > > > Question to metrics devs about that: can we use any character for > tagging, or > > is there some limitation? > > > > Or if you can suggest a better way, I'm all ears :) > > > > [1] list of methods that seem to be unused from the ruby client: > > - list_resources_for_feed > > - list_relationships > > - list_relationships_for_feed > > - get_entity > > - list_metrics_for_metric_type > > - list_metrics_for_resource_type > > - create_feed > > - delete_feed > > - create_resource_type > > - create_resource > > - create_resource_under_resource > > - create_metric_type > > - list_operation_definitions > > - list_operation_definitions_for_resource > > - create_metric_for_resource > > - no_more_events > > > > Joel > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev(a)lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev >