Hawkular Metrics - Roadmap - hawkular-dev

Tuesday, 15 March 2016

Hello Everybody,

Hawkular Metrics contributors have been working for the past few weeks on a roadmap for
the upcoming year. The goal is to give clarity on the project direction, serve as a
planning tool for releases, and show our strong commitment to open source.

The project and community are enjoying excellent growth. A maturing code base, an ever
growing set of integrations, and consistent community contributions are ingredients that
make this project successful and also an indication of the future.

For those not familiar with Hawkular Metrics, the project is a high performance and high
availability storage engine for large volume metric data. The project uses Cassandra as a
storage engine because of its flexible data model well suited for time-series data storage
and linear scalability with no single point of failure.

Here is the roadmap:

1) Cassandra 3.x
  * The 3.x release of Cassandra is maturing, making the perfect timing for the project to
transition from current 2.2.x line
  * Expect this transition to happen rather soon since work is already in progress (driver
updates, and a schema management tool)

2) Pre-computed aggregates
  * Needed to support long term data storage and retrieval for high volume metrics
  * Single metrics roll-ups are also the foundation for pre-computed multi-metric
aggregations, that goal is to work on this subsequent to single metric roll-ups

3) Metric Enhancements
  * Histogram metrics are fairly common in other time series databases. The plan is to add
histogram metrics as a sub-metric to existing gauge metrics, analogous to what
counter-rate metrics are counter metrics. It is common to do the calculations need for the
histogram on the client side, but there are a lot of advantages to push the calculations
to the server.
  * Add support for metrics baselines; automatically computed server-side and stored
  * Implement an Apdex score, similar purpose to baselines, but based on the open
standard

4) Native Grafana integration
  * Grafana integration is important for Hawkular Metrics due to lack of a dedicated UI.
Currently Grafana integration works through an InfluxDB compatibility layer that has
obvious disadvantages (maintaining compatibility with InfluxDB, limited set of features
based on the InfluxDB capability). 
  * A native Grafana provider will be easier to maintain and expose the full feature set
of Hawkular Metrics

5) Developer Support
  * Provide a Hawkular Metrics distribution with all components needed for third-party
developers to get a developer environment running with minimal effort
  * An easy-to-use and all-inclusive distribution will avoid having platform developers
configure Wildfly server and a Casasndra cluster just to test or write integration code

6) Import & Export Data APIs
  * The project already provides a growing set of APIs for querying metric data, but there
are scenarios that require bulk data export into another system for further analysis. And
vice-versa, import large amounts of data from another system for longer term storage and
aggregation by Hawkular Metrics.
  * The goal is to provide APIs optimized for bulk importing or exporting data. Tools need
to be both fast and easy to use, with the primary use case of moving a large amounts data
well beyond the capability of current REST interface (eg. moving 100GB of data).

7) ElasticSearch integration
  * An optional integration with Elastic Search for tasks beyond the capability of
Cassandra. 
  * Basic examples for this are whole tenant searches and aggregation of text based data,
such as tags, events, and even availability.

If you have any other suggestion or would like to contribute to the project, please
contact me; feedback is more than welcomed.

Thank you,
Stefan Negrea

Software Engineer

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Hawkular Metrics - Roadmap