November 2015 - hawkular-dev - Jboss List Archives

Hawkular Metrics 0.6.0 - Release & Beyond

by Stefan Negrea

Hello Everybody, I am happy to announce release 0.6.0 of Hawkular Metrics. The release is anchored by enhancements for counter metrics, updates to the container infrastructure, and overall performance and stability enhancements. Here is a list of major changes in this release: 1) Querying * Metrics can now be searched with regexp filtering capabilities on tags * Metrics queries now support logical AND for tags 2) Internal * TenantId and type were moved to the MetricId instead of Metric class * Implicit tenants (not manaully created prior to sending first set of metric data) are now created on metrics insertion 3) Container * Updated to the latest Heapster version (0.17) * The communication between the Hawkular Metrics and Cassandra containers have now been secured. This means that only Hawkular Metrics can communicate with its Cassandra containers. 4) Counters * Per-minute rates can be retrieved via the /hawkular/metrics/counters/{id}/rate endpoint. 5) Task scheduling * The scheduler API and schema has undergone several changes. It is now possible to group related tasks so that order of execution can be controlled when there are interdependencies. There is still ongoing work to support asynchronous job execution and to provide fault tolerance. 6) Ruby Client * The project has now an official Ruby client, this joins the existing Go and Python clients. One of Hawkular Metrics' objectives is to be easy to integrate with; providing language specific clients is an important component for fulfilling this objective. Hawkular Metrics Clients * Ruby: https://github.com/hawkular/hawkular-client-ruby * Python: https://github.com/hawkular/hawkular-client-python * Go: https://github.com/hawkular/hawkular-client-go Github Release: https://github.com/hawkular/hawkular-metrics/releases/tag/0.6.0 JBoss Nexus Maven artifacts: http://origin-repository.jboss.org/nexus/content/repositories/public/org/... Jira release tracker: https://issues.jboss.org/browse/HWKMETRICS/fixforversion/12327689/ Hawkular Metrics 0.7.0 & Beyond 1) Gauge Aggregates - The task scheduling work is now in place to enable server side aggregates at ingestion. The expectation is to release an initial implementation for this feature in 0.7.0. 2) Improved docker and kubernetes support - this is a long term goal for the project 3) Support for counters in Python & Golang clients A big "Thank you" goes to John Sanda, Thomas Segismont, Mike Thompson, Matt Wringe, Michael Burman, Libor Zoubek, Jirka Kremser, and Heiko Rupp for their project contributions. Thank you, Stefan Negrea Software Engineer

9 years, 3 months

3
6
0 / 0

Hawkular performance reports are now available in perfrepo

by Filip Brychta

Hello, you can now see results of perf tests like: - Response time - min/max/avg - Throughput min/max/avg - Disk usage - count of datapoints for different parameters like: - # of threads used for load generator - # of datapoints in each request - metric type in perfrepo [1]. You should be able to add more charts and edit existing ones to see any combination you like. Charts are interactive and you can display certain test execution by clicking on chart series. Each test execution has number in name which match build number of jenkins job [2] where you can find artifacts like server.log, garbage collection log, iostat log for each run. (Those artifacts will be stored in perfrepo as well in a future) Email me or vprusa to get credentials. Please note that this is just our internal testing instance of perfrepo. Plan is to move to official redhat perfrepo instance as soon as the setup is tuned. Many annoying perfrepo bugs will be fixed there. Please let me know what combinations and parameters you would like to see. [1] - http://perfrepo.bc.jonqe.lab.eng.bos.redhat.com:8080/reports/metric/5 [2] - http://jenkins.jonqe.lab.eng.bos.redhat.com:9080/job/haw-perf-stability-w... Filip

9 years, 3 months

2
1
0 / 0

Hawkular metrics - generating load from more generators - results

by Filip Brychta

Hello, as discussed on last haw perf meeting I tried to generate load from more generators. Motivation for this was to find out If it's possible to max out hw resources on tested machine (PowerEdge R630, 16 cores) when generation heavy load from more load generators. Quick answer is NO. Even though the load was generated from 5 machines from 1500 threads in total, CPU usage was never higher than 50% and maximal disk write speed was 15MB/sec. Maximum throughput was ~19000 requests/sec when using ~ 900 threads. When adding more threads the throughput and cpu usage went actually down. When bypassing cassandra calls in hawkular metrics and returning 200 response directly (see https://github.com/hawkular/hawkular-metrics/pull/411), single generator from 300 thread was able to generate load 45000 requests/sec. All this means that the problem is really not on load generator side not being able to generate sufficient load. Hawkular is not able to max out resources on that machine in current configuration no matter how many threads are used for load generation. Important note about perfcake (load generator we are using) is that it's not trying to break the server. For each thread it's sending new request only if previous request was done (response returned). So this means it's reporting maximum throughput which is hawkular metrics able to serve for given number of threads. Results: 1 datapoint per request: 600 threads -> throughput ~18100 requests/sec 900 threads -> throughput ~19000 requests/sec 1200 threads -> throughput ~10300 requests/sec 1500 threads -> throughput ~ 10000 requests/sec 10 datapoint per request: 300 threads -> throughput ~5000 requests/sec 600 threads -> throughput ~3500 requests/sec 900 threads -> throughput ~3100 requests/sec 1200 threads -> throughput ~3100 requests/sec 1500 threads -> throughput ~ 3100 requests/sec For all cases garbage collection was happening in regular periods ~3s. ( WF is running with -Xms64m -Xmx512m -XX:MaxPermSize=256m) I tried a few runs with increased memory -Xms2048m -Xmx2048m -XX:MaxPermSize=256m: 1 datapoint per request: 900 threads -> throughput ~ 19800 requests/sec 1500 threads -> throughput ~ 19800 requests/sec So for bigger number of threads the increased memory helped a lot. For 900 threads it helped a little. Throughput was stabilized before fist garbage collection occurred so it seems another increase of memory doesn't make sense. See attached txt for more detailed results. Filip

9 years, 3 months

1
0
0 / 0

Datamining integration with Alerts

by Pavol Loffay

Hello, I would like to integrate Datamining module with Alerts. For alert prediction users should configure globally/on MetricType/Metric prediction interval for how long ahead they want to be notified. For example notify my if in two hours alert conditions are met (based on prediction). For start we could reuse defined conditions and evaluate them not only for incoming data from Metrics(value at time t) but also from Datamining module (predicted value at time t+prediction interval). If defined conditions are met it will use the same actions(email, sms) as are defined for real Metrics with changed description text(this is prediction). This would require minimal configuration for users (only prediction interval, when zero no predictions). What do you think? Alert module would have cloned triggers+conditions for metrics being predicted with some changed properties like dataId(match condition to metric) and context information about metrics. Datamining module would send predicted metrics(with changed id) to the same topic as Metrics module does. With this approach there are no changes required in Alert module. Any comments or suggestions are welcome. Console or other modules can still ask for predictions for any time in the future. Datamining module does predictions on demand. Regards, Pavol

9 years, 3 months

1
0
0 / 0

oshi and MacOS

by John Mazzitelli

Heiko - I think I remember you reporting this (but I can't remember where or when). See: https://issues.jboss.org/browse/HAWKULAR-838 This is because on MacOS the OSHI library uses the Swing file chooser dialog widget to obtain file system data. Since MacOS is not an officially supported platform, do we want to do what this JIRA asks? That is, change kettle's startup options? This would also mean we'd have to somehow get the agent installer to also change WF/EAP startup options (not something I think we want to do - especially since this only affects MacOS). We could add this to the agent install documentation for those on MacOS. The other alternative is to disable file system data collection in the agent, assuming the user doesn't care about those metrics.

9 years, 3 months

2
3
0 / 0

Proposal: ban srcdeps from releases

by Peter Palaga

Hi *, this is yet another proposal from the series of changes that should make the life easier for productization engineers and sustaining engineers. While srcdeps [1] are showing themselves at their best while tinkering across components, it is not 100% legal to depend on srcdeps artifacts in releases. Why: (1) Releases are deployed to public Maven repositories where they are made accessible to tools (IDEs, dependency/compatibility analyzers, ...) that do not know how to handle srcdeps. For them, srcdeps are simply unsatisfied dependencies and our artifacts depending on them appear as disobedient to the common rules holding in public maven repositories. (2) The productization tool chain can actually count to such tools too. They are programmatically very strict about what is there in the dependency hierarchy and I quite doubt that they are ready to adapt to srcdeps. The proposed solution: With the next release, the srcdeps-maven-plugin will offer the option [2] to fail with certain (configurable) profiles. The default being just the "release" profile. This would reach Hawkular components with the next release 27 of hawkular-parent. (Note that parent 26 is out, although it still has not reached all Hawkular components.) [1] http://ppalaga.github.io/presentations/150929-srcdeps-maven-plugin/150929... [2] https://github.com/l2x6/srcdeps-maven-plugin/pull/31 Best, Peter

9 years, 3 months

1
0
0 / 0

Proposal: introduce DependencyConvergence enforcer rule

by Peter Palaga

Hi *, The proposal is to introduce an enforcer plugin rule called DependencyConvergence to hawkular-parent. From the rule's docs [1]: > This rule requires that dependency version numbers converge. If a > project has two dependencies, A and B, both depending on the same > artifact, C, this rule will fail the build if A depends on a > different version of C then the version of C depended on by B. Why do we need this: because it brings more consistency and better control during the productization and sustaining of the project. Known pitfalls - see [2] Any comments on this? Thanks, Peter [1] http://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html [2] http://stackoverflow.com/a/16239482

9 years, 3 months

7
12
0 / 0

Breaking change in Accounts

by Juraci Paixão Kröhling

If you use "@Inject Persona" or "@Inject @CurrentUser HawkularUser", keep reading. If all you consume from Accounts are the Services ("@Inject PermissionChecker" or "@Inject BlaService"), you are not affected. Still, you should update to Accounts 2.0.2.Final. On Accounts code, I was mis-using a behavior of WF8/WF9, where it was possible to inject request-related beans into stateless beans. More concretely, I was able to inject the persona for the current request into a SLSB. With WF10, the bean pooling is now enabled by default, meaning that beans created for a persona are pooled and re-used on future requests. In this case, user "X" would make a request and the bean would see it as being from user "Y". This is being fixed in Accounts 2.0.2.Final, which also comes with the update to WF 10. Unfortunately, I could not find a reasonable workaround for current consumers of Accounts that are using direct injection, so, you'll need to change your code. From: @Inject Persona persona; @Inject @CurrentUser HawkularUser user; To: @Inject Instance<Persona> personaInstance; @Inject @CurrentUser Instance<HawkularUser> userInstance; If your component has no changes required to work on WF 10, let me know and I'll send a PR for this change. Otherwise, please incorporate such change when changing your code to work on WF 10. Code changes are required *only* is you use direct injection. If you get the current persona via PersonaService#getCurrent, you should just update to Accounts 2.0.2.Final. - Juca.

9 years, 3 months

2
1
0 / 0

Cancelled: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1

by Michael Foley

A single instance of the following meeting has been cancelled: Subject: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1 Organizer: "Michael Foley" <mfoley(a)redhat.com> Location: Bluejeans http://www.bluejeans.com/mfoley51 Time: Wednesday, November 25, 2015, 3:00:00 PM - 3:30:00 PM GMT -05:00 US/Canada Eastern Required: pruan(a)redhat.com; mmahoney(a)redhat.com; vnguyen(a)redhat.com; snegrea(a)redhat.com; jsanda(a)redhat.com; mwringe(a)redhat.com Optional: jon-qa-list(a)redhat.com; jboss-on-team(a)redhat.com; hawkular-dev(a)lists.jboss.org *~*~*~*~*~*~*~*~*~* Hi, Let's have a discussion and planning session on testing Openshift & Hawkular Integration! Let's use this etherpad to coordinate the discussion -->> http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning 5 Point Plan for Openshift 3.1 GA * Unit tests .... owned by Hawk-Metrics developers * Integration tests ... owned by Hawk-Metrics developers and Hawk-Metrics QE * Performance CI on Hawk-Metrics (this one is actually new and was not discussed on Wednesday , but I now see it makes sense) * Functional Integration tests on Hawk-Metrics latest + Openshift Origin latest * Funtional/UI .... Cucumber/Ruby tests ...owned by Openshift QE * Functional/Rest ... Cucumber/Ruby tests ... owned by Openshift QE * Performance & Scalability .... owned by Openshift QE Regards, Michael Foley QE Supervisor, Middleware BU

9 years, 3 months

1
0
0 / 0

Termin gestrichen: Hawkular-Team Update - Di 24. Nov. 2015 3:30PM - 4PM (hrupp@redhat.com)

by hrupp＠redhat.com

Dieser Termin wurde gestrichen und aus Ihrem Kalender entfernt. Titel: Hawkular-Team Update This is a all-Hawkular team meeting to give updates where we are and so on. This is *open to the public*. Location: on bluejeans: https://redhat.bluejeans.com/hrupp/ or alternatively teleconference Reservationless+ , passcode 204 2160 481 You can find Dial-In telephone numbers here: https://www.intercallonline.com/listNumbersByCode.action?confCode=2042160481 RedHat internal short dial numbers are 16666 and 15555 (and probably others, depends on your location) Wann: Di 24. Nov. 2015 3:30PM - 4PM Berlin Wo: pc 204 2160 481 Kalender: hrupp(a)redhat.com Wer * Heiko Rupp - Organisator * Viliam Rockai * gbrown(a)redhat.com * Thomas Segismont * amendonc(a)redhat.com * Gabriel Cardoso * John Mazzitelli * snegrea(a)redhat.com * Jiri Kremser * Simeon Pinder * hawkular-dev(a)lists.jboss.org * miburman(a)redhat.com * mwringe(a)redhat.com * John Sanda * Lucas Ponce * theute(a)redhat.com * lkrejci(a)redhat.com * jcosta(a)redhat.com * Peter Palaga * Jay Shaughnessy * Mike Thompson Einladung von Google Kalender: https://www.google.com/calendar/ Sie erhalten diese E-Mail unter hawkular-dev(a)lists.jboss.org, da Sie ein Gast bei diesem Termin sind. Lehnen Sie diesen Termin ab, um keine weiteren Informationen zu diesem Termin zu erhalten. Sie können auch unter https://www.google.com/calendar/ ein Google-Konto erstellen und Ihre Benachrichtigungseinstellungen für Ihren gesamten Kalender steuern. Wenn Sie diese Einladung weiterleiten, kann jeder Empfänger Ihre Antwort auf die Einladung ändern. Weitere Informationen finden Sie unter https://support.google.com/calendar/answer/37135#forwarding

9 years, 3 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

hawkular-dev November 2015