April 2016 - hawkular-dev - Jboss List Archives

Termin gestrichen: Hawkular-Team Update - Di 19. Apr. 2016 3:30PM - 4PM (hrupp@redhat.com)

by hrupp＠redhat.com

Dieser Termin wurde gestrichen und aus Ihrem Kalender entfernt. Titel: Hawkular-Team Update This is a all-Hawkular team meeting to give updates where we are and so on. This is *open to the public*. Location: on bluejeans: https://redhat.bluejeans.com/hrupp/ or alternatively teleconference Reservationless+ , passcode 204 2160 481 You can find Dial-In telephone numbers here: https://www.intercallonline.com/listNumbersByCode.action?confCode=2042160481 RedHat internal short dial numbers are 16666 and 15555 (and probably others, depends on your location) Wann: Di 19. Apr. 2016 3:30PM - 4PM Berlin Wo: pc 204 2160 481 Kalender: hrupp(a)redhat.com Wer * Heiko Rupp - Organisator * Jiri Kremser * theute(a)redhat.com * abonas(a)redhat.com * jcosta(a)redhat.com * Lucas Ponce * Simeon Pinder * Gabriel Cardoso * snegrea(a)redhat.com * John Sanda * John Mazzitelli * lkrejci(a)redhat.com * Jay Shaughnessy * gbrown(a)redhat.com * mwringe(a)redhat.com * miburman(a)redhat.com * Peter Palaga * Mike Thompson * amendonc(a)redhat.com * Thomas Segismont * hawkular-dev(a)lists.jboss.org Einladung von Google Kalender: https://www.google.com/calendar/ Sie erhalten diese E-Mail unter hawkular-dev(a)lists.jboss.org, da Sie ein Gast bei diesem Termin sind. Lehnen Sie diesen Termin ab, um keine weiteren Informationen zu diesem Termin zu erhalten. Sie können auch unter https://www.google.com/calendar/ ein Google-Konto erstellen und Ihre Benachrichtigungseinstellungen für Ihren gesamten Kalender steuern. Wenn Sie diese Einladung weiterleiten, kann jeder Empfänger Ihre Antwort auf die Einladung ändern. Weitere Informationen finden Sie unter https://support.google.com/calendar/answer/37135#forwarding

9 years, 11 months

1
0
0 / 0

Re: [Hawkular-dev] Do not depend on Keycloak anymore

by Juraci Paixão Kröhling

Bringing this discussion to the list. Summary so far: === If all those requirements got dropped, specially "multiple organizations" and "multi tenancy", then there's really no point in having Accounts. Without the UI/self-registration/SSO, I doubt we need to depend on Keycloak. I added SSO to this list because if there will be no user interaction on Hawkular, there's no "single sign on" required there. It's still not clear to me, though, if we need to care about authorization. Are we getting calls only from "trusted" clients? Also, should we use the same user base as MiQ, or should we have a "system account" only? Having a system account might make more sense here, and we won't need to integrate with their user database. With this system account, we would act like a database: any user with valid credentials would be allowed to read/write data. We would trust the client if the client said "store this data for this tenant". Instead of tokens, we could have just system accounts. Revoking a token would mean just removing this system account. === > On 15 Apr 2016, at 14:38, John Mazzitelli wrote: > > What about agents/feeds? They aren't talking to CFME ; they are > talking directly to the Hawkular Server (or Hawkular-Metrics if in > METRICS-ONLY mode, such as in Open Shift). Is there not going to be > any Basic Auth headers anymore? If there will still need to be > authentication, then it must be the JAAS credentials and not the > Accounts tokens, I suppose. If we have system accounts, then agents would use a system account, like they would use a token. Admins are free to create one system account per agent (key/secret), or use one shared account to all agents (jdoe/password). For all "we" (Hawkular backend) care, it's all done by JAAS. In fact, if all goes well, there will be no need to change anything for the agents. And I'm considering that the auth is done by Basic auth. If we *need* to have other authentication mechanisms, we need to deviate from JAAS. On 15.04.2016 14:43, Heiko W.Rupp wrote: > Yes, that *may* require a change. Or not if we e.g. have > - accounts-keycloak > - accounts-jaas > where the latter does the mapping as a jaas provider/plugin. > But yes we need to secure the agent/server comm. And we > need to apply SSL for the product (isn't that part of http/2 anyway). So far, I don't see the need in having Keycloak, or Accounts. Securing the agent/server communication is not relevant to the discussion, as it's independent from the authentication mechanism being used. - Juca.

9 years, 11 months

3
9
0 / 0

Invitation: Watercooler: MIQ provider show + tell @ Tue Apr 19, 2016 10am - 11am (Hawkular)

by jshaughn＠redhat.com

You have been invited to the following event. Title: Watercooler: MIQ provider show + tell This is an optional gathering for MIQ Hawkular provider devs to show what they have done (merged or in the pipeline) and to informally discuss what we have and where we should go. So, minimally I will give a brief overview and demo of the event stuff we have so far. Hopefully other will do a brief demo or presentation as well. All are welcome, no one is required. When: Tue Apr 19, 2016 10am - 11am Eastern Time Where: https://redhat.bluejeans.com/u/jshaughn/ Calendar: Hawkular Who: * jshaughn(a)redhat.com - creator * jbmc-team * hawkular-dev(a)lists.jboss.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=dTJjdm03cWJqdDk2cjl... Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account hawkular-dev(a)lists.jboss.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding

9 years, 11 months

1
0
0 / 0

agents

by John Mazzitelli

[sending to dev list] If you wrote an agent/feed (even if its something in your own little corner of the world in your github repo), let me know where it is, what it is, what it can do now, what you hoped it could do in the future. Don't assume I know anything about it, because I probably don't. I just want to get a "lay of the land" - what is out there, what people are thinking about wrt agents, what we have today that we can build on. That kind of thing. Thanks, John Mazz

9 years, 11 months

1
0
0 / 0

Ideas about CD with Maven

by Peter Palaga

Hi all interested in continuous delivery and integration, Contrary to srcdeps, this guy [1] does not try to avoid releases and maven repos, instead, he just makes the release process extremely effective. Some of his ideas can be used to improve the srcdeps plugin. [1] https://axelfontaine.com/blog/dead-burried.html Best, Peter

9 years, 11 months

3
4
0 / 0

Reducing foot print of embedded Cassandra

by John Sanda

Earlier Heiko asked me if there are any changes we can make to reduce the foot print of the embedded Cassandra instance we use for development and testing. There are several configuration changes that we can make which could be helpful. I did a quick check and it looks like the hawkular server is configured with a 512 MB heap. Some recommended values are based on that assumption. * disable hinted handoff There is no need to run hinted handoff with a single node. Set the hinted_handoff_enabled property in cassandra.yaml to false. * disable key and counter caches This may decrease read performance but should reduce Cassandra’s overall memory usage. Set the key_cache_size_in_mb and counter_cache_size_mb properties in cassandra.yaml to 0. * reduce the concurrent reads/writes This will reduce the number of threads that Cassandra uses. Set the concurrent_reads, concurrent_writes, and concurrent_counter_writes properties in cassandra.yaml to 2. * reduce file cache size This is off-heap memory used for pooling SSTable buffers. For a 512 MB heap it will be 128 MB. Change the file_cache_size_in_mb property to 64. * reduce memory used for memtables This will increase the number of flushes which should be fine for write-heavy workloads. We probably don’t want to make this too small for read-heavy scenarios. Change the memtable_heap_space_in_mb and memtable_offheap_space_in_mb properties to 64. * reduce memory for index summaries This defaults to a little over 25 MB. Change the index_summary_capcaity_in_mb property to 13. * reduce the native transport threads The native transport server will use a max of 128 threads by default. Change the native_transport_max_threads property to 32. * reduce thrift server threads Change the rpc_min_threads property to 8 and pc_max_threads to 512. * tune compaction Set concurrent_compactors to 1. * disable gossip No need to run gossip with a single node. Since Cassandra runs in its own deployment, the easiest way to do this is via JMX using the stopGossiping() method on StorageServiceMBean. I need to do some testing with this to see how it works. It looks like the call has to happen after some initialization has completed. * disable compression Compression is enabled by default and is configured per table. It reduces disk space but increases CPU utilization. ALTER TABLE my_table WITH compression = { ’sstable_compression’: ‘’ };

9 years, 11 months

1
1
0 / 0

PoC hawkular wildfly agent and prometheus working

by John Mazzitelli

I have a PoC working with the agent scraping a Prometheus endpoint and storing the metric data in Hawkular-Metrics. It came together faster than I expected :) This can be used as a standalone feature - in other words, you don't have to run the agent collecting everything (DMR + JMX + Platform + Prometheus data) though you can if you want a "uber-agent" to collect from multiple sources. But you can just have a minimally configured agent collect just Prometheus data. The config in standalone.xml would look like this (this is the entire agent subsystem config): <subsystem xmlns="urn:org.hawkular.agent:agent:1.0" enabled="true"> <diagnostics enabled="false"/> <storage-adapter password="password" serverOutboundSocketBindingRef="hawkular" type="HAWKULAR" username="jdoe"/> <managed-servers> <remote-prometheus enabled="true" name="My Prometheus Endpoint" url="http://my-prometheus-endpoint:8080/metrics" interval="10" timeUnits="seconds" />  </managed-servers> </subsystem> This is just a prototype. Still lots of questions to answer like: 1) What do we want the keys to look like in H-Metrics? How to translate Prometheus keys/labels to H-Metrics? What this PoC does right now is build a huge H-Metrics key that appends the feed ID, the Prometheus metric name and Prometheus labels so it looks like this "b6fb9814-e6b5-435a-b533-411d85867720_prometheus_rule_evaluation_failures_total_rule_type=alerting_" (rule_type=alerting is a label). 2) Prometheus supports summary and histograms - what do we do with those? I ignore them for now. (btw: even histograms isn't supported in Prometheus' Java model API that is released, only supported in local snapshot build form only) 3) Do we want to create labels in H-Metrics to distinguish the Prometheus endpoint data? How? 4) No security whatsoever in here - PoC assumes in-the-clear http access. We'd need to add https, basic auth, ssl/security realms and anything else needed to access any Prometheus endpoints that are secured. I'm sure Open Shift has some things we need to work around to scrape Prometheus endpoints that are running in an OS environment - this is something we need to figure out. 5) Notice there is no metadata in the above config. There is no integration with H-Inventory - no resource types, no resources, no metric types. Prometheus doesn't really have the notion of inventory - at best, each Prometheus endpoint could be a "resource". But this PoC does nothing like that - it is a metrics-only solution (which as I understand it is all we need for the Open Shift requirement anyway). And I'm sure plenty more questions. But, point is, we have a working example of how this can be done. Code is in this branch: https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-66-dynamic-... --John Mazz

9 years, 11 months

1
0
0 / 0

java-based prometheus metrics scraper

by John Mazzitelli

FYI: I finished a Java-based Prometheus scraper - seems to be working. It is just code that scrapes any remote Prometheus endpoint given a URL and let's you easily walk the metric data for further processing (like, if you want to insert that data into Hawkular Metrics or if you want to dump the data in XML or JSON format - the CLI does this today, see below). I'll eventually blog about this, but not until I actually merge it into master. It's in my branch right now here: https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-66-promethe... This also includes a CLI tool - build the scraper maven module and you'll get an uber-jar that let's you scrape a remote Prometheus endpoint and dump the metric data in XML, JSON, or simple form. See the README.md for more. Now I just have to figure out how to integrate this into the Hawkular WildFly Agent :-)

9 years, 12 months

3
3
0 / 0

hawkular-agent, prometheus, openshift

by John Mazzitelli

I'm trying to figure out how the Hawkular WildFly Agent needs to be enhanced to collect metrics from Prometheus (which is where a lot of Open Shift metrics are going). Here is how I originally understood the problem (which may be completely wrong): I am looking at this: https://prometheus.io/docs/querying/api/ So if Open Shift components are storing metrics in Prometheus, the Agent would need to query the data via something like: http://localhost:9090/api/v1/query?query=my_metric_name_seconds{label_one... The agent can take the data it gets and store it in Hawkular Metrics (using a different metric name and/or labels if we want). I am hoping Matt W. can clarify and if this is completely wrong, how does he see it working? --John Mazz

9 years, 12 months

2
6
0 / 0

Hawkular Alerts 1.x Roadmap

by Jay Shaughnessy

Today, Hawkular Alerts 1.0 offers a rich and comprehensive set of features for adding alerts and events to an application: * Robust REST API * Complex Alerting * Powerful Event Handling * Pluggable Action Handlers with many provided out-of-box (e-mail, SMS, etc) * Alert Lifecycle (Ack, Resolved, etc) * Flexible Deployment, either standalone or as part of Hawkular * Group trigger management * Much More... This is the proposed roadmap for upcoming Hawkular Alerts 1.x releases. The main goal of the 1.x releases will be hardening of existing features, upgrades aligned with the main Hawkular project, fixes and enhancements identified by the community. Currently planned or proposed: * Upgrade to Cassandra 3 (as part of common upgrade in all Hawkular components). o Review and optimize schema * Improvements in the Actions plugins architecture. o Add plugins properties validation. o Introduce deployed/undeployed status on actions definitions. * Extend listeners support to clustering scenarios. * Improve REST API documentation and add additional examples. There is no release date established for Hawkular Alerts 1.1, releases will be made as demanded by the Hawkular project or other community needs. Please let us know if you have questions or suggested additions. Thanks! Hawkular Alerts Team Jay Shaughnessy (jshaughn(a)redhat.com) Lucas Ponce (lponce(a)redhat.com)

9 years, 12 months

4
4
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

hawkular-dev April 2016