[Hawkular-dev] Hawkular Metrics - Roadmap

[John Sanda]

> On Mar 16, 2016, at 6:52 AM, Thomas Segismont <tsegismo at redhat.com> wrote:
> 
> Le 15/03/2016 21:00, Stefan Negrea a écrit :
>> Hello Everybody,
>> 
>> Hawkular Metrics contributors have been working for the past few weeks on a roadmap for the upcoming year. The goal is to give clarity on the project direction, serve as a planning tool for releases, and show our strong commitment to open source.
>> 
>> The project and community are enjoying excellent growth. A maturing code base, an ever growing set of integrations, and consistent community contributions are ingredients that make this project successful and also an indication of the future.
>> 
>> For those not familiar with Hawkular Metrics, the project is a high performance and high availability storage engine for large volume metric data. The project uses Cassandra as a storage engine because of its flexible data model well suited for time-series data storage and linear scalability with no single point of failure.
>> 
>> 
>> Here is the roadmap:
>> 
>> 1) Cassandra 3.x
>>   * The 3.x release of Cassandra is maturing, making the perfect timing for the project to transition from current 2.2.x line
>>   * Expect this transition to happen rather soon since work is already in progress (driver updates, and a schema management tool)
>> 
>> 2) Pre-computed aggregates
>>   * Needed to support long term data storage and retrieval for high volume metrics
>>   * Single metrics roll-ups are also the foundation for pre-computed multi-metric aggregations, that goal is to work on this subsequent to single metric roll-ups
>> 
>> 3) Metric Enhancements
>>   * Histogram metrics are fairly common in other time series databases. The plan is to add histogram metrics as a sub-metric to existing gauge metrics, analogous to what counter-rate metrics are counter metrics. It is common to do the calculations need for the histogram on the client side, but there are a lot of advantages to push the calculations to the server.
> 
> +1
> 
>>   * Add support for metrics baselines; automatically computed server-side and stored
>>   * Implement an Apdex score, similar purpose to baselines, but based on the open standard
>> 
>> 4) Native Grafana integration
>>   * Grafana integration is important for Hawkular Metrics due to lack of a dedicated UI. Currently Grafana integration works through an InfluxDB compatibility layer that has obvious disadvantages (maintaining compatibility with InfluxDB, limited set of features based on the InfluxDB capability).
>>   * A native Grafana provider will be easier to maintain and expose the full feature set of Hawkular Metrics
> 
> +1000
> 
> It is important because Grafana is a *widely* used tool and we must 
> provide a first class experience when connecting Grafana to Hawkular 
> Metrics. Lack of dedicated UI just reinforce this need.
> 
>> 
>> 5) Developer Support
>>   * Provide a Hawkular Metrics distribution with all components needed for third-party developers to get a developer environment running with minimal effort
>>   * An easy-to-use and all-inclusive distribution will avoid having platform developers configure Wildfly server and a Casasndra cluster just to test or write integration code
>> 
> 
> +1
> 
>> 6) Import & Export Data APIs
>>   * The project already provides a growing set of APIs for querying metric data, but there are scenarios that require bulk data export into another system for further analysis. And vice-versa, import large amounts of data from another system for longer term storage and aggregation by Hawkular Metrics.
>>   * The goal is to provide APIs optimized for bulk importing or exporting data. Tools need to be both fast and easy to use, with the primary use case of moving a large amounts data well beyond the capability of current REST interface (eg. moving 100GB of data).
>> 
>> 7) ElasticSearch integration
>>   * An optional integration with Elastic Search for tasks beyond the capability of Cassandra.
>>   * Basic examples for this are whole tenant searches and aggregation of text based data, such as tags, events, and even availability.
>> 
> 
> I'm reluctant to introduce a new dependency for the use cases listed 
> above. Has the new support for text based index in C* 3.x been explored? 
> Why isn't it enough? Also, the container already comes with Infinispan 
> which also has text index support.
> 

The key part is that it is optional. I think we already have some pretty nice tag filtering, but if we want a lot more search capabilities, then I think looking at integration with ES or similar tools makes sense. This would need to be a pluggable layer with our current impl being the default.

> 
> 
>> 
>> If you have any other suggestion or would like to contribute to the project, please contact me; feedback is more than welcomed.
>> 
>> 
>> Thank you,
>> Stefan Negrea
>> 
>> Software Engineer
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160316/a8dc6784/attachment-0001.html