[Hawkular-dev] Hawkular Metrics - /data enpoint

[Stefan Negrea]

Hello,

One point not made so far, even today developers have to code twice against
GET '*/data/' because of different possible outputs. First is GET '*/data/'
with some parameters to get raw data and then another instance with
identical URL but different parameters to get bucketed results. The split
in API will formalize the differences and will no longer burden the
developer with reading the documentation to make sure the right parameters
are passed.

The typical CRUD REST API design principles cannot be applied in some cases
to the Hawkular Metrics interface because of the project's nature. There is
little CRUD in ingesting metrics once and making that data available in a
few different ways (raw, bucketed, by tags, and eventually pre-computed
aggregates). And that makes it somewhat challenging to design and find good
articles and examples. But here a few resources that I think are good in
this context:

1)
https://www.thoughtworks.com/insights/blog/rest-api-design-resource-modeling
- section 'An example from the public GitHub API'
2) http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api -
section 'What about actions that don't fit into the world of CRUD
operations?'
3) https://developer.github.com/v3/repos/statistics/  - documentation page
to get statistical data for a repo, the rest of the documentation is also
interesting
4)
https://developers.facebook.com/docs/marketing-api/reference/ad-keyword-stats
- documentation for getting ad keyword statistics

I see two common themes in the links above. First, the public APIs are
really complex and the number of end-points is really not the concern; more
important are ease of use, single purpose, and good documentation. And
second, outside of CRUD there is little material (blogs, books, articles)
about what to do.

With this in mind here is the plan of action based on the feedback in this
thread and many discussions we had in the community and amongst Hawkular
Metrics contributors. Hawkular Metrics will proceed with merging the first
PR to split and restructure '/data', the old code is deprecated but intact.
We will work with the community and current users to find the best path to
fully remove '*/data'. We will publish more details about the deprecation
and plans for removal as soon as we have more details; also except this
change to be documented in detail in the release notes until fully removed.
Just to be clear, this is not a single release transition to fully remove
'*/data/', so far we identified a minimum of 3 releases, but it could
extend beyond that based on feedback.


Thank you,
Stefan Negrea

Software Engineer


On Tue, Apr 5, 2016 at 8:04 AM, Heiko W.Rupp <hrupp at redhat.com> wrote:

> On 5 Apr 2016, at 0:18, Stefan Negrea wrote:
>
> > On Mon, Apr 4, 2016 at 4:03 PM, John Sanda <jsanda at redhat.com> wrote:
> >
> >> Making sure we don’t break clients is the most important aspect of
> >> this;
> >> however, there are some other questions I want to throw out. Today we
> >> only
> >> support clients adding raw data points. What if in the future though
> >> we add
> >> support for complex types like histograms and allow clients to
> >> directly
> >> store them. Would clients post that data to /raw, /stats, or
> >> something else?
> >>
> >> Today we compute aggregated metrics at query time. We are going to
> >> add
> >> support for pre-computed aggregate metrics that are persisted. Do we
> >> have
> >> different endpoints for each? I haven’t thought about it too much
> >> yet, but
> >> I don’t separate endpoints is the way to go.
> >>
> >
> > In the actual JIRA ticket there were 3 ideas floated around: separate
> > endpoint, or /raw endpoint (because that is not bucketed data), or
> > /stats
> > endpoint (because is bucketed, just server side). And I think those
> > are all
> > the possible variations. One thing that I do not like about '/data'
> > and
> > pre-computed aggregates, if we do not split that method will serve 3
> > or 4
> > different formats depending on query params. That is way too
> > complicated
> > for both us and the users.
> >
> >
> >>
> >> With HWKMETRICS-373, we are introducing support for tag based
> >> bucketing as
> >> opposed to the date range bucketing already have. There are some
> >> minor
> >> differences in the data structures returned in the response, namely a
> >> map
> >> is returned instead of an array. Should we have a different endpoints
> >> for
> >> tag based buckets vs date range buckets?
> >>
> >
> > I like the idea of just raw and stats. I associate /raw with raw data
> > stored by Hawkular Metrics, and that includes pre-computed aggregates
> > too,
> > and /stats with some sort of statistical computation done do the raw
> > data
> > before returned to the user. Based on this, the tag querying would
> > fall
> > under '/stats' if data is bucketed and would fall under '/raw' if you
> > query
> > for just raw data that is tagged.
>
>
> John wrote about "tag based bucketing as opposed to the date range
> bucketing"
>
> So both is bucket data and both /stats -- but with different return
> types.
> If this is true, they need to return different content-types. This can
> not just
> be returned like so without any notice to the used and the user doing
> the
> guesswork.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160406/9c5d68e8/attachment-0001.html