On Mar 20, 2015, at 11:08 AM, mike thompson <mithomps(a)redhat.com&gt; wrote: > > On 20 Mar 2015, at 08:00, John Sanda <jsanda(a)redhat.com&gt; wrote: > > There has been some good discussion around availability lately. I want to add one more to the mix, but hopefully this one is not as in-depth as some of the other topics. Right in metrics we have endpoints like, > > POST /metrics/numeric/data > GET /metrics/numeric/{id}/data > POST /metrics/availability/data > GET /metrics/availability/{id}/data > > I would like to change these to, > > POST /metrics/data > GET /metrics/{id}/data > POST /availability/data > GET /availability/{id}/data > > I think the “metrics” prefix is awkward and unnecessary. I think that it is intuitive enough that metrics on its own refers to numeric data. Thoughts? I think the ‘data’ postfix is redundant as well.

There has been some good discussion around availability lately. I want to add one more to the mix, but hopefully this one is not as in-depth as some of the other topics. Right in metrics we have endpoints like, POST /metrics/numeric/data GET /metrics/numeric/{id}/data POST /metrics/availability/data GET /metrics/availability/{id}/data I would like to change these to, POST /metrics/data GET /metrics/{id}/data POST /availability/data GET /availability/{id}/data I think the “metrics” prefix is awkward and unnecessary. I think that it is intuitive enough that metrics on its own refers to numeric data. Thoughts? - John _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

On 20 Mar 2015, at 08:19, Lukas Krejci <lkrejci(a)redhat.com&gt; wrote: I think the "metrics" should actually be dropped instead of "numeric”.

Hawk metrics will never deal with anything else than some kind of metric, so way have it in the path? On the other hand having the type of the metric in the path is IMHO more useful. It is the context root of the metrics rest web app that should be called "metrics", IMHO. So the paths in metrics would look like: /{tenantId}/numeric/data /{tenantId}/availability/data and the REST API user would reach those on the URLs like: http://asdf.com/hawkular/metrics/tenant1/numeric/{id}/data conversely, inventory could be reached on the same host like: http://asdf.com/hawkular/inventory/tenant1/production/resources/{id} On Friday, March 20, 2015 11:00:22 John Sanda wrote: > There has been some good discussion around availability lately. I want to > add one more to the mix, but hopefully this one is not as in-depth as some > of the other topics. Right in metrics we have endpoints like, > > POST /metrics/numeric/data > GET /metrics/numeric/{id}/data > POST /metrics/availability/data > GET /metrics/availability/{id}/data > > I would like to change these to, > > POST /metrics/data > GET /metrics/{id}/data > POST /availability/data > GET /availability/{id}/data > > I think the “metrics” prefix is awkward and unnecessary. I think that it is > intuitive enough that metrics on its own refers to numeric data. Thoughts? > > > - John > _______________________________________________ > hawkular-dev mailing list > hawkular-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

> On 20 Mar 2015, at 08:19, Lukas Krejci <lkrejci(a)redhat.com&gt; wrote: > > I think the "metrics" should actually be dropped instead of "numeric”. The only thing about ‘numeric’ is that it is not very specific. Numeric just means number — every metric is a number so its really not adding anything. Even availability data is numeric.

> Hawk metrics will never deal with anything else than some kind of metric, > so way have it in the path? On the other hand having the type of the > metric in the path is IMHO more useful. > > It is the context root of the metrics rest web app that should be called > "metrics", IMHO. > > So the paths in metrics would look like: > > /{tenantId}/numeric/data > /{tenantId}/availability/data > > and the REST API user would reach those on the URLs like: > > http://asdf.com/hawkular/metrics/tenant1/numeric/{id}/data > > conversely, inventory could be reached on the same host like: > > http://asdf.com/hawkular/inventory/tenant1/production/resources/{id} > > On Friday, March 20, 2015 11:00:22 John Sanda wrote: >> There has been some good discussion around availability lately. I want to >> add one more to the mix, but hopefully this one is not as in-depth as >> some >> of the other topics. Right in metrics we have endpoints like, >> >> POST /metrics/numeric/data >> GET /metrics/numeric/{id}/data >> POST /metrics/availability/data >> GET /metrics/availability/{id}/data >> >> I would like to change these to, >> >> POST /metrics/data >> GET /metrics/{id}/data >> POST /availability/data >> GET /availability/{id}/data >> >> I think the “metrics” prefix is awkward and unnecessary. I think that it >> is >> intuitive enough that metrics on its own refers to numeric data. >> Thoughts? >> >> >> - John >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev

On Mar 20, 2015, at 12:47 PM, John Sanda <jsanda(a)redhat.com&gt; wrote: > > On Mar 20, 2015, at 12:23 PM, Lukas Krejci <lkrejci(a)redhat.com <mailto:lkrejci@redhat.com>> wrote: > > On Friday, March 20, 2015 09:03:19 mike thompson wrote: >>> On 20 Mar 2015, at 08:19, Lukas Krejci <lkrejci(a)redhat.com <mailto:lkrejci@redhat.com>> wrote: >>> >>> I think the "metrics" should actually be dropped instead of "numeric”. >> >> The only thing about ‘numeric’ is that it is not very specific. Numeric just >> means number — every metric is a number so its really not adding anything. >> Even availability data is numeric. > > Isn't metrics going to store "events" eventually, too? Those would be textual, > wouldn't they? > > Also, IMHO, even config could be stored in metrics. Even though that is a bit > of a stretch, configuration is after all a time series of structured data... > > You're right that at the moment numeric doesn't add much information. So maybe > instead of that, the type would specify what should be done with the metric? > > Something along the lines of dropwizard-metrics' distinction between a meter, > counter, gauge, histogram, etc? > > So like: > > http://asdf.com/hawkular/metrics/tenant1/histogram/{id} <http://asdf.com/hawkular/metrics/tenant1/histogram/{id}> > http://asdf.com/hawkular/metrics/tenant1/counter/{id} > My aggregate metrics design doc[1] talks about formally introducing gauges and counters. The question I have about endpoints like this is what about pushing a batch of metrics? Do we need to make a separate request for each type? That seems cumbersome and inefficient. [1] http://bit.ly/1BAfF8d <http://bit.ly/1BAfF8d>

But first: do we know that we need counters and gauges right now?

On 23 Mar 2015, at 11:04, Thomas Segismont wrote: > Le 23/03/2015 09:04, Heiko W.Rupp a écrit : >> But first: do we know that we need counters and gauges right now? > > I can't see why need special semantics for storing values coming from > counters and gauges (and availability). If they need no special semantics, why would one want to add counters and gauges? And for availability, do we really want to have feeds deliver random numbers as their availability and then in the UI show the random numbers as we can not determine what availability state they mean? If all you have is a double, then everything will look like 2.999999975687 pentium processors :)

On Mar 23, 2015, at 9:52 AM, Heiko W.Rupp <hrupp(a)redhat.com&gt; wrote: > What I meant is that we shouldn't care about how the data is stored, if > it's a counter, a gauge, a string. It's a "value point" in time. While I think it is good to separate the storage from the semantics, I do not think we should not care how it is stored. For counters, which are integral values, we should probably not store them as double. Similar, a log line should not be stored as double either :-) > > What matters is how we query this data. For example, if you're working > with a counter you'll probably be interested in applying a derivative > function. Yes

On Mar 31, 2015, at 12:52 PM, Thomas Segismont <tsegismo(a)redhat.com&gt; wrote: Hi, Last week I had a long phone conversation with John about the impact of the coming changes to metrics on the Influx API support. I'd like to share the conclusion I've come to. So in the future we'll have counters and gauges series. The goal is to qualify the data inserted upfront so that things like statistics/aggregates are automatically computed.

If we're going this way it's going to be difficult (and less relevant) to maintain the Influx API support, because we'll lose the 1 to 1 mapping between an Influx series and an Hawkular Metrics series. When a user will post/read data, should it be stored/read to/from a counter or a gauge? As Micke is working on a Go lang sink for cAdvisor/Heapster, the Influx support API is less important regarding our immediate needs. But we'll lose the ability to integrate with all the tools which support Influx already (Grafana and JMeter among others). It doesn't mean we can't write our own integration code for these tools. But that's not for free. Regards, Thomas Le 23/03/2015 15:18, John Sanda a écrit : > >> On Mar 23, 2015, at 9:52 AM, Heiko W.Rupp <hrupp(a)redhat.com&gt; wrote: >> >>> What I meant is that we shouldn't care about how the data is stored, if >>> it's a counter, a gauge, a string. It's a "value point" in time. >> >> While I think it is good to separate the storage from the semantics, >> I do not think we should not care how it is stored. >> >> For counters, which are integral values, we should probably not >> store them as double. >> >> Similar, a log line should not be stored as double either :-) >> >>> >>> What matters is how we query this data. For example, if you're working >>> with a counter you'll probably be interested in applying a derivative >>> function. >> >> Yes > > The semantics of metric type are precisely about querying. The type information tells us which aggregation functions to apply to the data points. And these functions should be utilized in both ad-hoc queries as well as the generation of pre-computed aggregates/rollups; however, I am not suggesting that this should preclude the use of other functions. _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

Similar, a log line should not be stored as double either:-)

I think that there was some general consensus or at least convergence towards the notion that availability is a function that reports resource state change. It could be a function of metric data or

I think we need to expand on what was done in RHQ. I think that availability functions should be explicit and stored in inventory so that users can easily see how availability is determined and also change how availability is computed should the need arise. The

monitoring agent, not the server, should compute availability. I am using the term agent loosely here because it applies to agents that

There needs to be more done in terms of correlation. When there is a state change, we should be reporting an accompanying event, e.g., server shutdown, server restarted, etc. If we compute availability based on some metric, then maybe the event reported is something like, http status code is X or the response time is Y.

On 20 Mar 2015, at 17:23, Lukas Krejci wrote: > Isn't metrics going to store "events" eventually, too? Those would be > textual, > wouldn't they? > > Also, IMHO, even config could be stored in metrics. Even though that > is a bit > of a stretch, configuration is after all a time series of structured > data... If so, would we need to rename this to Hawkular-storage (?) :-) Seriously, I think we should consider better decoupling the actual storage from api and semantics. Sounds probably pretty strange, so let me explain: If we consider everything a "metric" we are sucking in more and more dependencies into metrics or vice versa. But in fact (and we are seeing it with the availability discussion), items like availability / computed resource state, inventory data, ... have a semantic that differs a lot from metrics. We may still want to store all that in Cassandra, but not necessarily in the same place inside it. And even less have one api "POST /stuff" where people throw everything at, but rather specialized ones. _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

Metrics might not just be numeric. Right now we have simple numeric and availability. We will probably have events at in the future. We might also have multi-value numeric metrics, e.g., call time in RHQ. And I could also see multi-value metrics consisting of numeric and text data.

I think the "metrics" should actually be dropped instead of "numeric". Hawk metrics will never deal with anything else than some kind of metric, so way have it in the path? On the other hand having the type of the metric in the path is IMHO more useful. It is the context root of the metrics rest web app that should be called "metrics", IMHO. So the paths in metrics would look like: /{tenantId}/numeric/data /{tenantId}/availability/data and the REST API user would reach those on the URLs like: http://asdf.com/hawkular/metrics/tenant1/numeric/{id}/data conversely, inventory could be reached on the same host like: http://asdf.com/hawkular/inventory/tenant1/production/resources/{id}