[Hawkular-dev] Metrics and use of metrics_idx

[Michael Burman]

Hi,

This sparked my interest after the discussions in PR #523 (adding cache to avoid metrics_idx writes). Stefan commented that he still wants to write to this table to keep metrics available instantly, jsanda wants to write them asynchronously. Maybe we should instead just stop writing there?

Why? We do the same thing in tenants also at this time, we don't write there if someone writes a metric to a new tenant. We fetch the partition keys from metrics_idx table. Now, the same ideology could be applied to the metrics_idx writing, read the partition keys from data. There's a small performance penalty, but the main thing is that we don't really need that information often - in most use cases never.

If we want to search something with for example tags, we search it with tags - that metricId has been manually added to the metrics_idx table. No need to know if there's metrics which were not initialized. This should be the preferred way of doing things in any case - use tags instead of pushing metadata to the metricId.

If we need to find out if id exists, fetching that from the PK (PartitionKey) index is fast. The only place where we could slow down is if there's lots of tenants with lots of metricIds each and we want to fetch all the metricIds of a single tenant. In that case the fetching of definitions could slow down. How often do users fetch all the tenant metricIds without any filtering? And how performance critical is this sort of behavior? And what use case does list of ids serve (without any information associated to them) ?

If you need to fetch datapoints from a known metricId, there's no need for metrics_idx table writing or reading. So this index writing only applies to listing metrics.

  -  Micke