Hello,

One point not made so far, even today developers have to code twice against GET '*/data/' because of different possible outputs. First is GET '*/data/' with some parameters to get raw data and then another instance with identical URL but different parameters to get bucketed results. The split in API will formalize the differences and will no longer burden the developer with reading the documentation to make sure the right parameters are passed.

The typical CRUD REST API design principles cannot be applied in some cases to the Hawkular Metrics interface because of the project's nature. There is little CRUD in ingesting metrics once and making that data available in a few different ways (raw, bucketed, by tags, and eventually pre-computed aggregates). And that makes it somewhat challenging to design and find good articles and examples. But here a few resources that I think are good in this context:

1) https://www.thoughtworks.com/insights/blog/rest-api-design-resource-modeling  - section 'An example from the public GitHub API'
2) http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api - section 'What about actions that don't fit into the world of CRUD operations?'
3) https://developer.github.com/v3/repos/statistics/  - documentation page to get statistical data for a repo, the rest of the documentation is also interesting
4) https://developers.facebook.com/docs/marketing-api/reference/ad-keyword-stats - documentation for getting ad keyword statistics

I see two common themes in the links above. First, the public APIs are really complex and the number of end-points is really not the concern; more important are ease of use, single purpose, and good documentation. And second, outside of CRUD there is little material (blogs, books, articles) about what to do.

With this in mind here is the plan of action based on the feedback in this thread and many discussions we had in the community and amongst Hawkular Metrics contributors. Hawkular Metrics will proceed with merging the first PR to split and restructure '/data', the old code is deprecated but intact. We will work with the community and current users to find the best path to fully remove '*/data'. We will publish more details about the deprecation and plans for removal as soon as we have more details; also except this change to be documented in detail in the release notes until fully removed. Just to be clear, this is not a single release transition to fully remove '*/data/', so far we identified a minimum of 3 releases, but it could extend beyond that based on feedback.


Thank you,
Stefan Negrea

Software Engineer


On Tue, Apr 5, 2016 at 8:04 AM, Heiko W.Rupp <hrupp@redhat.com> wrote:
On 5 Apr 2016, at 0:18, Stefan Negrea wrote:

> On Mon, Apr 4, 2016 at 4:03 PM, John Sanda <jsanda@redhat.com> wrote:
>
>> Making sure we don’t break clients is the most important aspect of
>> this;
>> however, there are some other questions I want to throw out. Today we
>> only
>> support clients adding raw data points. What if in the future though
>> we add
>> support for complex types like histograms and allow clients to
>> directly
>> store them. Would clients post that data to /raw, /stats, or
>> something else?
>>
>> Today we compute aggregated metrics at query time. We are going to
>> add
>> support for pre-computed aggregate metrics that are persisted. Do we
>> have
>> different endpoints for each? I haven’t thought about it too much
>> yet, but
>> I don’t separate endpoints is the way to go.
>>
>
> In the actual JIRA ticket there were 3 ideas floated around: separate
> endpoint, or /raw endpoint (because that is not bucketed data), or
> /stats
> endpoint (because is bucketed, just server side). And I think those
> are all
> the possible variations. One thing that I do not like about '/data'
> and
> pre-computed aggregates, if we do not split that method will serve 3
> or 4
> different formats depending on query params. That is way too
> complicated
> for both us and the users.
>
>
>>
>> With HWKMETRICS-373, we are introducing support for tag based
>> bucketing as
>> opposed to the date range bucketing already have. There are some
>> minor
>> differences in the data structures returned in the response, namely a
>> map
>> is returned instead of an array. Should we have a different endpoints
>> for
>> tag based buckets vs date range buckets?
>>
>
> I like the idea of just raw and stats. I associate /raw with raw data
> stored by Hawkular Metrics, and that includes pre-computed aggregates
> too,
> and /stats with some sort of statistical computation done do the raw
> data
> before returned to the user. Based on this, the tag querying would
> fall
> under '/stats' if data is bucketed and would fall under '/raw' if you
> query
> for just raw data that is tagged.


John wrote about "tag based bucketing as opposed to the date range
bucketing"

So both is bucket data and both /stats -- but with different return
types.
If this is true, they need to return different content-types. This can
not just
be returned like so without any notice to the used and the user doing
the
guesswork.
_______________________________________________
hawkular-dev mailing list
hawkular-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev