Hi Stefan, I haven't looked at the PR, the changes only apply to GET requests, correct? In other words, will POST requests to '*/data' (e.g. from feeds) still work? Thanks, Thomas 2016-04-02 0:24 GMT+02:00 Stefan Negrea <snegrea(a)redhat.com&gt;: > Hello Everybody, > > I submitted today a PR for a long standing JIRA: > https://issues.jboss.org/browse/HWKMETRICS-24 ; here is a related one: > https://issues.jboss.org/browse/HWKMETRICS-57 , and the PR: > https://github.com/hawkular/hawkular-metrics/pull/473 > > The JIRAs recommend splitting '*/data' endpoint in two separate > endpoints, '/raw' and '/stats'. There are two issues with the current > '*/data'. First, the code was really hard to maintain because it was > serving two purposes. Depending on the input parameters, the endpoint would > serve raw ingested data in some cases and bucketed results in some other > cases. The PR still has the old code (since it got just deprecated in this > release) and it's not pretty. The second problem was the actual API > interface. There was no simple way to know exactly what you get back from > the endpoint because sometimes it would return simple data points and > sometimes bucketed data. This was based on the query parameters specified > in the request. And to make things worse, some parameters could not be > mixed, for example, the user could not limit or order bucketed results; but > the documentation had to include all under the same endpoint. > > The plan is to deprecate existing '*/data' endpoints in the upcoming > release (0.15.0) and remove them in the release after that (0.16.0). That > gives enough time (roughly 2 months) for all existing projects to migrate > to the newer API. Does anybody see any problem with this timeline? > > I expect the transition to be simpler because most of the consumers were > using '*/data' with the intent to get bucketed results, which is now > '*/stats'. So it is just a simple change in most cases. > > Are there any projects negatively affected by this change in the > long-term? Does the change make sense? Is the API interface for retrieving > data easier to understand after the change? > > > Thank you, > Stefan Negrea > > Software Engineer > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -- Thomas Segismont JBoss ON Engineering Team _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

From: "Heiko W.Rupp" <hrupp(a)redhat.com&gt; To: "Discussions around Hawkular development" <hawkular-dev(a)lists.jboss.org&gt; Sent: Monday, April 4, 2016 4:56:15 AM Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint On 2 Apr 2016, at 0:24, Stefan Negrea wrote: > The plan is to deprecate existing '*/data' endpoints in the upcoming > release (0.15.0) and remove them in the release after that (0.16.0). That > gives enough time (roughly 2 months) for all existing projects to migrate > to the newer API. Does anybody see any problem with this timeline? Yes. There are users out there that use the old api in production. They may not even be aware that there is a new one. If they deploy the new hawkular-metrics (or someone does for them), their apps will break and they may not even know why.

REST allows for content negotiation. So we should keep the old endpoint, deprecate them (and perhaps even spit regular warnings in the logs) and on top add the new endpoints that you suggested but with a new content type like "application/json+hawkular-v2".

In fact if we keen old and new, that new content type is not exactly needed if the data format exchanged does not change. Btw: I doubt that GET ../data and POST ../data is confusing anyone and in Jax-Rs you can have those on two different methods anyway. _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

Making sure we don’t break clients is the most important aspect of this; however, there are some other questions I want to throw out. Today we only support clients adding raw data points. What if in the future though we add support for complex types like histograms and allow clients to directly store them. Would clients post that data to /raw, /stats, or something else? Today we compute aggregated metrics at query time. We are going to add support for pre-computed aggregate metrics that are persisted. Do we have different endpoints for each? I haven’t thought about it too much yet, but I don’t separate endpoints is the way to go.

With HWKMETRICS-373, we are introducing support for tag based bucketing as opposed to the date range bucketing already have. There are some minor differences in the data structures returned in the response, namely a map is returned instead of an array. Should we have a different endpoints for tag based buckets vs date range buckets?

We currently have one increasingly complex method to handle queries for raw data, various permutations of date range buckets, and eventually tag based buckets once my changes for HWKMETRICS-373 are merged. These changes definitely will simplify things on the server side, but do they simply or otherwise improve things for clients? I am a little concerned that we might be letting our implementation bleed into the API a little too much. For example, we segment our data in our schema design by metric type. Do we necessarily need to do that everywhere in the API?

On Apr 4, 2016, at 10:21 AM, Matt Wringe <mwringe(a)redhat.com&gt; wrote: ----- Original Message ----- From: "Heiko W.Rupp" <hrupp(a)redhat.com&gt; To: "Discussions around Hawkular development" < hawkular-dev(a)lists.jboss.org&gt; Sent: Monday, April 4, 2016 4:56:15 AM Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint On 2 Apr 2016, at 0:24, Stefan Negrea wrote: The plan is to deprecate existing '*/data' endpoints in the upcoming release (0.15.0) and remove them in the release after that (0.16.0). That gives enough time (roughly 2 months) for all existing projects to migrate to the newer API. Does anybody see any problem with this timeline? Yes. There are users out there that use the old api in production. They may not even be aware that there is a new one. If they deploy the new hawkular-metrics (or someone does for them), their apps will break and they may not even know why. +1 I think we need to make sure that we don't break the existing API for current users who are already using it. There will most likely be a bunch of complaints if we update this in OpenShift and then everyone's setup stops working. REST allows for content negotiation. So we should keep the old endpoint, deprecate them (and perhaps even spit regular warnings in the logs) and on top add the new endpoints that you suggested but with a new content type like "application/json+hawkular-v2". If we are going to introduce a new version for the REST API, it might make sense to release Hawkular Metrics with a new major version as well. But major versions can be tricky depending on how the project handles them. Some projects will have the same major versions for years, others will update the major version almost monthly (eg kernel version versus browser version) In fact if we keen old and new, that new content type is not exactly needed if the data format exchanged does not change. Btw: I doubt that GET ../data and POST ../data is confusing anyone and in Jax-Rs you can have those on two different methods anyway. _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

On 5 Apr 2016, at 0:18, Stefan Negrea wrote: > On Mon, Apr 4, 2016 at 4:03 PM, John Sanda <jsanda(a)redhat.com&gt; wrote: > >> Making sure we don’t break clients is the most important aspect of >> this; >> however, there are some other questions I want to throw out. Today we >> only >> support clients adding raw data points. What if in the future though >> we add >> support for complex types like histograms and allow clients to >> directly >> store them. Would clients post that data to /raw, /stats, or >> something else? >> >> Today we compute aggregated metrics at query time. We are going to >> add >> support for pre-computed aggregate metrics that are persisted. Do we >> have >> different endpoints for each? I haven’t thought about it too much >> yet, but >> I don’t separate endpoints is the way to go. >> > > In the actual JIRA ticket there were 3 ideas floated around: separate > endpoint, or /raw endpoint (because that is not bucketed data), or > /stats > endpoint (because is bucketed, just server side). And I think those > are all > the possible variations. One thing that I do not like about '/data' > and > pre-computed aggregates, if we do not split that method will serve 3 > or 4 > different formats depending on query params. That is way too > complicated > for both us and the users. > > >> >> With HWKMETRICS-373, we are introducing support for tag based >> bucketing as >> opposed to the date range bucketing already have. There are some >> minor >> differences in the data structures returned in the response, namely a >> map >> is returned instead of an array. Should we have a different endpoints >> for >> tag based buckets vs date range buckets? >> > > I like the idea of just raw and stats. I associate /raw with raw data > stored by Hawkular Metrics, and that includes pre-computed aggregates > too, > and /stats with some sort of statistical computation done do the raw > data > before returned to the user. Based on this, the tag querying would > fall > under '/stats' if data is bucketed and would fall under '/raw' if you > query > for just raw data that is tagged. John wrote about "tag based bucketing as opposed to the date range bucketing" So both is bucket data and both /stats -- but with different return types. If this is true, they need to return different content-types. This can not just be returned like so without any notice to the used and the user doing the guesswork. _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

----- Original Message ----- > From: "Heiko W.Rupp" <hrupp(a)redhat.com&gt; > To: "Discussions around Hawkular development" < hawkular-dev(a)lists.jboss.org&gt; > Sent: Monday, April 4, 2016 4:56:15 AM > Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint > > On 2 Apr 2016, at 0:24, Stefan Negrea wrote: > > The plan is to deprecate existing '*/data' endpoints in the upcoming > > release (0.15.0) and remove them in the release after that (0.16.0). That > > gives enough time (roughly 2 months) for all existing projects to migrate > > to the newer API. Does anybody see any problem with this timeline? > > Yes. > There are users out there that use the old api in production. > They may not even be aware that there is a new one. > > If they deploy the new hawkular-metrics (or someone does > for them), their apps will break and they may not even know > why. +1 I think we need to make sure that we don't break the existing API for current users who are already using it. There will most likely be a bunch of complaints if we update this in OpenShift and then everyone's setup stops working. > REST allows for content negotiation. So we should keep the > old endpoint, deprecate them (and perhaps even spit regular > warnings in the logs) and on top add the new endpoints > that you suggested but with a new content type > like "application/json+hawkular-v2". If we are going to introduce a new version for the REST API, it might make sense to release Hawkular Metrics with a new major version as well. But major versions can be tricky depending on how the project handles them. Some projects will have the same major versions for years, others will update the major version almost monthly (eg kernel version versus browser version) > In fact if we keen old and new, that new content type > is not exactly needed if the data format exchanged does not > change. > > Btw: I doubt that GET ../data and POST ../data is confusing anyone > and in Jax-Rs you can have those on two different methods anyway. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

2) we create new endpoints perhaps even under a different url-root (inside /hawkular/metrics) and only add features to the new endpoints. 3) At some point in time we declare metrics-1.0 and for this one remove the old endpoints.