Cancelled: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1
by Michael Foley
A single instance of the following meeting has been cancelled:
Subject: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1
Organizer: "Michael Foley" <mfoley(a)redhat.com>
Location: Bluejeans http://www.bluejeans.com/mfoley51
Time: Wednesday, January 20, 2016, 3:00:00 PM - 3:30:00 PM GMT -05:00 US/Canada Eastern
Required: pruan(a)redhat.com; mmahoney(a)redhat.com; vnguyen(a)redhat.com; snegrea(a)redhat.com; jsanda(a)redhat.com; mwringe(a)redhat.com
Optional: jon-qa-list(a)redhat.com; jboss-on-team(a)redhat.com; hawkular-dev(a)lists.jboss.org
*~*~*~*~*~*~*~*~*~*
Hi,
Let's have a discussion and planning session on testing Openshift & Hawkular Integration!
Let's use this etherpad to coordinate the discussion -->> http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning
5 Point Plan for Openshift 3.1 GA
* Unit tests .... owned by Hawk-Metrics developers
* Integration tests ... owned by Hawk-Metrics developers and Hawk-Metrics QE
* Performance CI on Hawk-Metrics (this one is actually new and was not discussed on Wednesday , but I now see it makes sense)
* Functional Integration tests on Hawk-Metrics latest + Openshift Origin latest
*
Funtional/UI .... Cucumber/Ruby tests ...owned by Openshift QE * Functional/Rest ... Cucumber/Ruby tests ... owned by Openshift QE
* Performance & Scalability .... owned by Openshift QE
Regards,
Michael Foley
QE Supervisor, Middleware BU
8 years, 3 months
agent can inventory and monitor JMS
by John Mazzitelli
I added resource types for the Artemis JMS server and its queues/topics as well as the queue/topic metrics.
So we can monitor JMS components. You won't see anything because the UI will need to be enhanced to show their graphs. But at least the data is there for the UI to consume.
8 years, 3 months
agent: define your own feed ID from command line
by John Mazzitelli
When the agent starts for the very first time, it needs to generate a feed ID for itself - if inside WF9 the feed ID is typically the hostname (its convoluted, but normally that is what it is). On WF10, its the WildFly server's UUID. About a week ago, I let the agent be told what feed ID to use via its feedId attribute in <storage-adapter> for those that don't want the UUID to be the feed ID.
But people didn't want to have to edit standalone.xml to set their feed ID. So I released a new agent (0.15.6.Final) that adds a new feature to the agent installer - you can now pass in --feed-id (or set feed-id in the installer config file) to the installer which will set the feedId attribute in the agent <storage-adapter> config.
This still doesn't get you all the way to what people want (which is being able to pass in -Dhawkular.agent.feedid=myfeedid when starting the Hawkular Server). So this PR is now submitted:
https://github.com/hawkular/hawkular/pull/791
When this is merged, that feature will be enabled - if you pass in -Dhawkular.agent.feedid=XYZ the first time the agent is started it will use that when it creates its feed id. If you don't pass it in, the agent autogenerates its feed id like it always has in the past.
I don't know if people want this merged into master for the release tomorrow or not - I'll leave that for someone else to decide. The PR is there if you want it.
--John Mazz
8 years, 3 months
Getting started with Docker
by Anton Hughes
Hi guys
Im following the quick-start guide using Docker guide on your website.
It says if I go to localhost:8080 then I should get the login screen.
However, I get the standard WF9 screen, without any login page.
Should I delete the docker image and try again? Or is the remote docker
image out of date?
thanks,
Anton
[image: Inline image 1]
8 years, 3 months
Release schedule
by Juraci Paixão Kröhling
Team,
Based on the discussed that followed the last release, here's the
adjusted scheduled based on the next release. Although those are only
*suggestions*, I'll be following that for Accounts.
- Thursday before the release: Parent, Commons and Accounts
- Friday before the release: Inventory, Metrics, BTM
- Monday before the release: Command Gateway, Agent, Alerts
- Tuesday (release day): Hawkular
Once Accounts is released, I'll be submitting a PR for the downstream
components.
- Juca.
8 years, 3 months
Hawkular Alerts 0.6.0.Final Released!
by Jay Shaughnessy
Hawkular Alerts 0.6.0.Final has been Released!
* Event Support
This major new feature incorporates Events into Hawkular Alerts. Events:
* Are more lightweight than Alerts
* Record things of interest
* Can be injected via an API call
* Can be generated via a Trigger
* Can have actions performed on them (for Trigger-generated Events)
* Can be used as Trigger conditions
For more on Events there is some general information here (more to come):
http://www.hawkular.org/docs/dev/alerts.html
And Lucas provides some great examples here:
https://github.com/lucasponce/hawkular-examples/tree/master/events
* WebHooks Action Plugin
Hawkular Alerts 0.6.0.Final introduces a new Action plugin for WebHook
notifications.
There have been some changes to the REST endpoints as we refine the model.
Of course there are fixes and minor enhancements as well. A Jira
Summary for the release is here:
* https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12315924&versi...
The next scheduled release is 0.7.0, targeted for December 9, 2015.
Hawkular Alerts Team
Jay Shaughnessy (jshaughn(a)redhat.com)
Lucas Ponce (lponce(a)redhat.com)
8 years, 3 months
Gulp version 3.9.0+ required for Hawkular UI Dev builds
by mike thompson
NOTE: this is only relevant if you are doing Hawkular UI development. This process is transparent to the normal build process.
In an effort to ‘ES6 All the things!” All of the Hawkular UI repos have been updated to use Gulp 3.9.0+ which allows gulp to use ES6 in its gulpfile (via Babel ES6 translator). This means that gulpfile.js filename has changed to gulpfile.babel.js and is written in ES6. All of the previous gulp commands work the same as they always have.
To determine what version of gulp is installed:
gulp —version
If the version returned is less than 3.9.0 then you need update the version. To update the version:
sudo npm update -g gulp
or
sudo npm install -g gulp
8 years, 3 months
First round of perf tests with more cassandra nodes - results
by Filip Brychta
Hello,
I did first quick perf testing of haw metrics STANDALONE with more cassandra nodes and it showed some interesting results.
Important note is that hawkular and cassandra cluster were running on VMs with shared storage. Which is very poor design for cassandra cluster but it still showed some patterns which will be true for every setup.
Witch proper cassandra cluster (dedicated disks, CommitLog and SSTables on different disks, ...) the results should be definitely better.
Summary of what was found (something is obvious even without testing):
- small messages (1 datapoint per request) utilize heavily cpu on hawkular host and cassandra hosts are utilized gently
- bigger messages (100 datapoints per request) are less demanding on hawkular host's cpu, cassandra hosts are utilized little bit more
- with week cpu on hawkular host, adding more cassandra nodes makes performance even worst
- for small messages (1 datapoint per request) even with sufficient cpu on hawkular host the performance improvement was only ~ 25% when number of nodes in the cluster was increased from 1 to 2
- for bigger messages (100 datapoints per request) with sufficient cpu on hawkular host the performance improvement was ~ 75% when number of nodes in the cluster was increased from 1 to 2
- for small messages (1 datapoint per request) even with sufficient cpu on hawkular host the performance does NOT scale up with more cassandra nodes (see results - performance dropped when 4th node was added)
- for bigger messages (100 datapoints per request) with sufficient cpu on hawkular host the performance scales up but not linearly (this could be caused by shared storage and with proper cassandra cluster the results will be better)
Questions:
- why is the performance getting worst when adding 3th and 4th storage nodes when sending small messages and having sufficient cpu on hawkular host?
About the test:
- load generator was hitting following endpoint http://${server.host}:${server.port}/hawkular/metrics/gauges/data
- one test run takes 4 minutes
- message with one datapoint looks like this [{"id":"gaugeID","data":[{"timestamp": "@{CurrentTimestamp}", "value": 10.12}]}]
- load generator was using 300 threads (each thread is acting like single client) and was sending messages containing 1 or 100 datapoints
- hawkular metrics is deployed on wildfly-9.0.2.Final
- metrics version: "Implementation-Version":"0.12.0-SNAPSHOT","Built-From-Git-SHA1":"c35deda5d6d03429e97f1ed4a6e4ef12cf7f3a00"
Results:
===================================================
VMs with 2 cores and shared storage, 4GB of memory.
===================================================
300 threads, 1 datapoint, hawkular_metrics | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"
++++++++++++++++++++++++++++++++
1 cassandra node ~ 3945 req/sec
2 cassandra nodes ~ 3751 req/sec
3 cassandra nodes ~ 3318 req/sec
4 cassandra nodes ~ 2726 req/sec
In this case the cpu on hawkular VM was fully used and adding more cassandra nodes actually made performance worst.
Cpu on cassandra nodes was never fully used
300 threads, 100 datapoint, hawkular_metrics | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"
++++++++++++++++++++++++++++++++
1 cassandra nodes ~ 102 req/sec
2 cassandra nodes ~ 138 req/sec
3 cassandra nodes ~ 188 req/sec
4 cassandra nodes ~ 175 req/sec
With week cpu on hawkular VM and big messages (100 datapoints in each) there is still some improvement when adding more cassandra nodes.
Cpu on cassandra nodes was never fully used
===================================================
Hawkular VM with 4 cores, cassandra VMs 2 cores and shared storage,4GB of memory.
===================================================
300 threads, 1 datapoint, hawkular_metrics | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"
++++++++++++++++++++++++++++++++
1 cassandra node ~ 5150 req/sec
2 cassandra nodes ~ 5667 req/sec
3 cassandra nodes ~ 5799 req/sec
4 cassandra nodes ~ 5476 req/sec
With stronger cpu on hawkular VM adding more cassandra nodes improves performance but there is a drop when 4th node is added.
Cpu on cassandra nodes was never fully used
300 threads, 100 datapoint, hawkular_metrics | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"
++++++++++++++++++++++++++++++++
1 cassandra nodes ~ 111 req/sec
2 cassandra nodes ~ 173 req/sec
3 cassandra nodes ~ 206 req/sec
4 cassandra nodes ~ 211 req/sec
With stronger cpu on hawkular VM adding more cassandra nodes improves performance.
Cpu on cassandra nodes was never fully used
===================================================
Hawkular VM with 8 cores, cassandra VMs 2 cores and shared storage,4GB of memory. Cpu on hawkular machine is used 30-40%
===================================================
300 threads, 1 datapoint, hawkular_metrics | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"
++++++++++++++++++++++++++++++++
1 cassandra node ~ 5424 req/sec
2 cassandra nodes ~ 6810 req/sec
3 cassandra nodes ~ 6576 req/sec
4 cassandra nodes ~ 6094 req/sec
Why there is a drop for 3th and 4th node?
300 threads, 100 datapoint, hawkular_metrics | org.apache.cassandra.locator.SimpleStrategy | {"replication_factor":"1"
++++++++++++++++++++++++++++++++
1 cassandra nodes ~ 97 req/sec
2 cassandra nodes ~ 168 req/sec
3 cassandra nodes ~ 222 req/sec
4 cassandra nodes ~ 241 req/sec
Please let me know what you would like to see in next rounds of testing.
Filip
8 years, 3 months
where should integration/glue code and tests live?
by John Sanda
I am currently working on accounts integration in metrics. The integration code lives in the hawkular-component module in the metrics repo. Now I want to write some integration tests to verify that things work as expected. I don’t really think that integration/glue code belongs in the component repo. I want to make sure that wherever it lives I can still get feed back from CI builds when changes are made. When I make a change to component code in the metrics repo, the CI build runs and checks for regressions by running tests. There is a pretty quick feedback loop which is very important. If the glue code is living elsewhere, I would like for a a CI build to get kicked off at some point after the metrics CI builds completes successfully so that I can find out if any of my changes to the component code caused a regression in the glue code. And of course if I make a change directly in the glue code, I should get the feedback from the check-in CI job. In RHQ we had a build pipeline with jenkins that handled this for us. Is this possible with travis?
For now I am putting the glue code and tests in the metrics repo so that I can have the CI feedback loop, but I see this as a short term solution. It may work fine in this instance, but what about when I want to test some other integration? I will be pulling in more hawkular dependencies and eventually hit circular dependencies. And the quick feedback that I get from the component CI build will get a lot slower in the process.
- John
8 years, 3 months
Cassandra upgrade
by John Sanda
We are currently running Cassandra 2.2.2. I believe that 3.1.1 is the latest version, and it was released on December 21. There are some pretty significant changes in 3.x. From a development standpoint, I think it absolutely makes sense to upgrade. I would like to know what others think (Heiko in particular). There are other considerations like productization build changes and OpenShift integration. There would be a data migration involved with OpenShift since we already shipped Cassandra 2.x. When upgrading across major versions, you are supposed to run the sstableupgrade utility on each table. We would need to do this for OpenShift. Whether or not we choose to upgrade now, we will need to deal with this migration step in OpenShift at some point.
- John
8 years, 3 months
