Hawkular Metrics - Roadmap
by Stefan Negrea
Hello Everybody,
Hawkular Metrics contributors have been working for the past few weeks on a roadmap for the upcoming year. The goal is to give clarity on the project direction, serve as a planning tool for releases, and show our strong commitment to open source.
The project and community are enjoying excellent growth. A maturing code base, an ever growing set of integrations, and consistent community contributions are ingredients that make this project successful and also an indication of the future.
For those not familiar with Hawkular Metrics, the project is a high performance and high availability storage engine for large volume metric data. The project uses Cassandra as a storage engine because of its flexible data model well suited for time-series data storage and linear scalability with no single point of failure.
Here is the roadmap:
1) Cassandra 3.x
* The 3.x release of Cassandra is maturing, making the perfect timing for the project to transition from current 2.2.x line
* Expect this transition to happen rather soon since work is already in progress (driver updates, and a schema management tool)
2) Pre-computed aggregates
* Needed to support long term data storage and retrieval for high volume metrics
* Single metrics roll-ups are also the foundation for pre-computed multi-metric aggregations, that goal is to work on this subsequent to single metric roll-ups
3) Metric Enhancements
* Histogram metrics are fairly common in other time series databases. The plan is to add histogram metrics as a sub-metric to existing gauge metrics, analogous to what counter-rate metrics are counter metrics. It is common to do the calculations need for the histogram on the client side, but there are a lot of advantages to push the calculations to the server.
* Add support for metrics baselines; automatically computed server-side and stored
* Implement an Apdex score, similar purpose to baselines, but based on the open standard
4) Native Grafana integration
* Grafana integration is important for Hawkular Metrics due to lack of a dedicated UI. Currently Grafana integration works through an InfluxDB compatibility layer that has obvious disadvantages (maintaining compatibility with InfluxDB, limited set of features based on the InfluxDB capability).
* A native Grafana provider will be easier to maintain and expose the full feature set of Hawkular Metrics
5) Developer Support
* Provide a Hawkular Metrics distribution with all components needed for third-party developers to get a developer environment running with minimal effort
* An easy-to-use and all-inclusive distribution will avoid having platform developers configure Wildfly server and a Casasndra cluster just to test or write integration code
6) Import & Export Data APIs
* The project already provides a growing set of APIs for querying metric data, but there are scenarios that require bulk data export into another system for further analysis. And vice-versa, import large amounts of data from another system for longer term storage and aggregation by Hawkular Metrics.
* The goal is to provide APIs optimized for bulk importing or exporting data. Tools need to be both fast and easy to use, with the primary use case of moving a large amounts data well beyond the capability of current REST interface (eg. moving 100GB of data).
7) ElasticSearch integration
* An optional integration with Elastic Search for tasks beyond the capability of Cassandra.
* Basic examples for this are whole tenant searches and aggregation of text based data, such as tags, events, and even availability.
If you have any other suggestion or would like to contribute to the project, please contact me; feedback is more than welcomed.
Thank you,
Stefan Negrea
Software Engineer
8 years, 1 month
Ambiguity in behaviour of token generated by token page in web UI
by Anuj Garg
I was working on fetching data for different personas. At that time I am
facing that key/secret pair generated by /secret-store/v1/tokens/create
is being able to respond according to Hawkular-Persona provided in the
header.
But One generated by "Create Token" on token page is not able to work that
way. I have tried only on resources till yet though.
http://pastebin.com/Rp9idM7s
I am providing Persona of created org which should have returned empty list
but proving the resource URLs which are present in first Persona.
8 years, 1 month
Error when trying to connect to the server Hawcular
by Rodrigo Osmar Garcete
Hi everyone
I am interested in this idea: Improve Existing Hawkular-Android client. I
have experience in Java and Android. I have already hawkular-android-client
on my Android Studio, but not connect to the Web Admin.
My client version is 1.0.0 Alpha3 and sever is 1.0.0 Alpha9. I am on a
local network from my laptop and phone android. I try connect to the URL
192.168.0.106 with por 18080 from hawcular-android-client I get the
following error in my console:
03-19 15:24:17.167 30343-31518/org.hawkular.client.android
E/HttpRestProvider﹕ Error on POST of
http://192.168.0.106:18080/secret-store/v1/tokens/create
java.net.ConnectException: failed to connect to /192.168.0.106 (port
18080): connect failed: ECONNREFUSED (Connection refused)
at libcore.io.IoBridge.connect(IoBridge.java:114)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:192)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:459)
at java.net.Socket.connect(Socket.java:872)
at
libcore.net.http.HttpConnection.<init>(HttpConnection.java:76)
at
libcore.net.http.HttpConnection.<init>(HttpConnection.java:50)
at
libcore.net.http.HttpConnection$Address.connect(HttpConnection.java:340)
at
libcore.net.http.HttpConnectionPool.get(HttpConnectionPool.java:87)
at
libcore.net.http.HttpConnection.connect(HttpConnection.java:128)
at
libcore.net.http.HttpEngine.openSocketConnection(HttpEngine.java:315)
at libcore.net.http.HttpEngine.connect(HttpEngine.java:310)
at
libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:289)
at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:239)
at
libcore.net.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:80)
at
libcore.net.http.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:188)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.addBodyRequest(HttpRestProvider.java:220)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:148)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:135)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:117)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:105)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:864)
Caused by: libcore.io.ErrnoException: connect failed: ECONNREFUSED
(Connection refused)
at libcore.io.Posix.connect(Native Method)
at libcore.io.BlockGuardOs.connect(BlockGuardOs.java:85)
at libcore.io.IoBridge.connectErrno(IoBridge.java:127)
at libcore.io.IoBridge.connect(IoBridge.java:112)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:192)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:459)
at java.net.Socket.connect(Socket.java:872)
at
libcore.net.http.HttpConnection.<init>(HttpConnection.java:76)
at
libcore.net.http.HttpConnection.<init>(HttpConnection.java:50)
at
libcore.net.http.HttpConnection$Address.connect(HttpConnection.java:340)
at
libcore.net.http.HttpConnectionPool.get(HttpConnectionPool.java:87)
at
libcore.net.http.HttpConnection.connect(HttpConnection.java:128)
at
libcore.net.http.HttpEngine.openSocketConnection(HttpEngine.java:315)
at libcore.net.http.HttpEngine.connect(HttpEngine.java:310)
at
libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:289)
at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:239)
at
libcore.net.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:80)
at
libcore.net.http.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:188)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.addBodyRequest(HttpRestProvider.java:220)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:148)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:135)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:117)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:105)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:864)
03-19 15:24:17.167 30343-31518/org.hawkular.client.android
D/SecretStoreAuthzModule﹕ SecretStoreAuthzModule
03-19 15:24:17.187 30343-30343/org.hawkular.client.android
D/LoginActivity﹕ Authorization failed.
java.lang.RuntimeException: java.net.ConnectException: failed to
connect to /192.168.0.106 (port 18080): connect failed: ECONNREFUSED
(Connection refused)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:153)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:135)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:117)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:105)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:864)
Caused by: java.net.ConnectException: failed to connect to /
192.168.0.106 (port 18080): connect failed: ECONNREFUSED (Connection
refused)
at libcore.io.IoBridge.connect(IoBridge.java:114)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:192)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:459)
at java.net.Socket.connect(Socket.java:872)
at
libcore.net.http.HttpConnection.<init>(HttpConnection.java:76)
at
libcore.net.http.HttpConnection.<init>(HttpConnection.java:50)
at
libcore.net.http.HttpConnection$Address.connect(HttpConnection.java:340)
at
libcore.net.http.HttpConnectionPool.get(HttpConnectionPool.java:87)
at
libcore.net.http.HttpConnection.connect(HttpConnection.java:128)
at
libcore.net.http.HttpEngine.openSocketConnection(HttpEngine.java:315)
at libcore.net.http.HttpEngine.connect(HttpEngine.java:310)
at
libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:289)
at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:239)
at
libcore.net.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:80)
at
libcore.net.http.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:188)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.addBodyRequest(HttpRestProvider.java:220)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:148)
at
org.jboss.aerogear.android.pipe.http.HttpRestProvider.post(HttpRestProvider.java:135)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:117)
at
org.hawkular.client.android.auth.SecretStoreAuthzModule$2.doInBackground(SecretStoreAuthzModule.java:105)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:864)
Caused by: libcore.io.ErrnoException: connect failed: ECONNREFUSED
(Connection refused)
at libcore.io.Posix.connect(Native Method)
at libcore.io.BlockGuardOs.connect(BlockGuardOs.java:85)
at libcore.io.IoBridge.connectErrno(IoBridge.java:127)
at libcore.io.IoBridge.connect(IoBridge.java:112)
Someone can lend a hand with this
Rodrigo Garcete
8 years, 1 month
Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1
by Michael Foley
A single instance of the following meeting has been modified:
Subject: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1
Organizer: "Michael Foley" <mfoley(a)redhat.com>
Location: Bluejeans http://www.bluejeans.com/mfoley51
Time: Wednesday, March 16, 2016, 3:00:00 PM - 3:30:00 PM GMT -05:00 US/Canada Eastern
Required: pruan(a)redhat.com; mmahoney(a)redhat.com; vnguyen(a)redhat.com; snegrea(a)redhat.com; jsanda(a)redhat.com; mwringe(a)redhat.com
Optional: jon-qa-list(a)redhat.com; jboss-on-team(a)redhat.com; hawkular-dev(a)lists.jboss.org
*~*~*~*~*~*~*~*~*~*
Hi,
Let's have a discussion and planning session on testing Openshift & Hawkular Integration!
Let's use this etherpad to coordinate the discussion -->> http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning
5 Point Plan for Openshift 3.1 GA
* Unit tests .... owned by Hawk-Metrics developers
* Integration tests ... owned by Hawk-Metrics developers and Hawk-Metrics QE
* Performance CI on Hawk-Metrics (this one is actually new and was not discussed on Wednesday , but I now see it makes sense)
* Functional Integration tests on Hawk-Metrics latest + Openshift Origin latest
*
Funtional/UI .... Cucumber/Ruby tests ...owned by Openshift QE * Functional/Rest ... Cucumber/Ruby tests ... owned by Openshift QE
* Performance & Scalability .... owned by Openshift QE
Regards,
Michael Foley
QE Supervisor, Middleware BU
8 years, 1 month
GSoC 2016
by Rishabh Makhija
Hi,
I am Rishabh Makhija, CSE undergrad student from Bharati Vidyapeeth's
College of Engineering, India. I would like to work on Improving existing
Hawkular-Android client.
I have previous experience of working on Android as a core developer at
ownCloud-android and wordPress-android (http://www.github.com/rishabh7m).
Can someone help me with listing introductory bugs to work on?
My project mentors are Heiko Rupp and Thomas Segismont but there is no
contact information provided on the site.
8 years, 1 month
[GSOC 2016], "Hawkular-agent for vert.x"
by Anton Okolnychyi
Hi, all!
My name is Anton Okolnychyi. I am keen on participating in GSOC 2016.
That's why I am trying to gather relevant information to write a nice
proposal for a "Hawkular-agent for vert.x" project on GSoC 2016.
Thanks to Thomas Segismont for his initial advice. I have analyzed the
existing functionality of the vertx-hawkular-metrics module, its current
features, and architecture. In addition, I have investigated Hawkular's
REST APIs of Metrics and Inventory services. Currently, the
vertx-hawkular-metrics module reports only to the Metrics service. The idea
that is available in the scope of GSOC 2016 is to enhance
vertx-hawkular-metrics' capabilities. In particular, enable reporting to
the Inventory service of Hawkular.
As a result of my research, I have some questions. It would be great if
someone can answer them.
1. Since the main goal of the project is to actually enable reporting to
the Inventory service, one should come up with an appropriate graph
architecture of Inventory which should contain resources(vertices),
relationships between them(edges) and optionally assigned metrics to each
resource (I am aware of many to many relationships here). The
vertx-hawkular-metrics module currently has metrics that cover the
following: datagram/udp, event bus, tcp, http client, http server. That's
why one can define the corresponding resources for these metrics in the
Inventory (e.g. Event Bus, Net Client, HTTP Client, Datagram socket, HTTP
Server, Net Server etc.). Will that make sense? Am I going in the right
direction? Alternatively, we can define a resource that will cover several
metrics at once. If you have anything that is important to include as a
resource in the Inventory, please, share that with me.
2. The vertx-hawkular-metrics module collects only counters and gauges and
reports them via the POST request to /metrics/data (correct me if I am
wrong). Would it make sense/be interesting to also handle some availability
metrics? The Hawkular API allows to do that and the current architecture of
the vertx-hawkular-metrics is flexible enough to integrate a new metric
type quite easily. Probably, this question should be asked on the vertx-dev
list. But if someone knows the answer or has any possible
applications/suggestions in mind I would be glad to hear.
Thanks in advance! Any help is highly appreciated!
Sincerely,
Anton Okolnychyi
8 years, 1 month
Criteria for a trigger to show in alert center?
by Heiko W.Rupp
Hey,
so I've created a trigger via the rest-api, that is also
correctly firing. But I don't see it in the alert center.
What are the criteria for a trigger to show up there?
Is that currently limited to resources of type wildfly
and its children?
8 years, 1 month
Questions about Inventory in agent
by Austin Kuo
Hi, all.
This is Austin.
I've been trying to understand how the inventory in agent works.
As far as I understand, an inventory is a graph, and the agent sends the
graph to the hawkular server with some kind of format. Such that the
hawkular server can answer queries with the data that the agent sends.
I have a few of questions:
1. What are the various protocols for(dmr, jmx, platform)?
2. And there's Resource and ResourceType, each of them holds a graph,
what's the relation between them? Are they isomorphic and each node of
ResouceType annotates the corresponding Resource node?
Thanks!
8 years, 1 month
