From jmazzite at redhat.com Mon Oct 3 10:04:02 2016 From: jmazzite at redhat.com (jmazzite at redhat.com) Date: Mon, 03 Oct 2016 14:04:02 +0000 Subject: [Hawkular-dev] Invitation: open shift agent discussion @ Mon Oct 3, 2016 11:30am - 12pm (EDT) (jmazzite@redhat.com) Message-ID: <001a11459c7c90063c053df66ac7@google.com> You have been invited to the following event. Title: open shift agent discussion Just a quick talk about where Matt is with his own code to prototype an Open Shift hawkular agent, and what I can do to further push it along to create a h-agent prototype. When: Mon Oct 3, 2016 11:30am ? 12pm Eastern Time Where: https://bluejeans.com/2105364687 Calendar: jmazzite at redhat.com Who: * jmazzite at redhat.com - organizer * hawkular-dev at lists.jboss.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=bXV0dGdxanVybzhqajQ1c2l0anZibWY3NzggaGF3a3VsYXItZGV2QGxpc3RzLmpib3NzLm9yZw&tok=MTkjam1henppdGVAcmVkaGF0LmNvbTIxN2Q2N2NlYjZiOTY5ZDk3MDI1YTE5MGQ1ZjYzMzNkY2NlYTJlYTg&ctz=America/New_York&hl=en Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account hawkular-dev at lists.jboss.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161003/080acb8a/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 1272 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161003/080acb8a/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 1302 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161003/080acb8a/attachment-0003.bin From hrupp at redhat.com Tue Oct 4 05:20:51 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 04 Oct 2016 11:20:51 +0200 Subject: [Hawkular-dev] Inventory: transient feeds - or how to tackle the pets vs cattle scenario Message-ID: <95DB8D26-E6F3-4A87-B603-16EDCC03BC00@redhat.com> Hey, Right now we identify "agents" via their feed-id. An instrumented wildfly comes online, registers its feed with the server, sends its resource discovery and later metrics with the feed id. Over its lifecycle, the server may be stopped and re-started several times. This is great in the classical use case with installations on tin or VMs. In container-land especially with systems like Kubernetes, containers are started once and after they have died for whatever reason they are not restarted again. So the id of an individual container is less and less interesting. The interesting part is the overall app, that contains of many containers linked together with several of them representing an individual service of the app. So basically we would rather need to record the app and other metadata for identifying individual parts of the app (e.g. the web servers or the data bases) and then get pointers to individual stuff. The feed would not need to survive for too long, but some of its collected data perhaps. And then e.g. the discovery of resources in a new container of the exact same type as before should be sort of a no-op, as we know this already. Could we short-circuit that by storing the docker-image-hash (or similar) and once we see this known one abort the discovery? Another aspect is certainly that we want to keep (some) historic records of the died container - e.g. some metrics and the point when it died. Suppose k8s kills a container and spins a new one up (same image) on a different node, then logically it is a continuation of the first one, but in a different place (but they have different feed ids) Now a more drastic scenario: As orchestration systems like k8s or Docker-Swarm have their own registries, that can be queried : do we need a hawkular-inventory for this at all? ( We still need it for the non-OpenShift/K8s/Docker-Swarm envs ) -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander From theute at redhat.com Tue Oct 4 05:41:01 2016 From: theute at redhat.com (Thomas Heute) Date: Tue, 4 Oct 2016 11:41:01 +0200 Subject: [Hawkular-dev] Inventory: transient feeds - or how to tackle the pets vs cattle scenario In-Reply-To: <95DB8D26-E6F3-4A87-B603-16EDCC03BC00@redhat.com> References: <95DB8D26-E6F3-4A87-B603-16EDCC03BC00@redhat.com> Message-ID: On Tue, Oct 4, 2016 at 11:20 AM, Heiko W.Rupp wrote: > Hey, > > Right now we identify "agents" via their feed-id. > An instrumented wildfly comes online, registers > its feed with the server, sends its resource discovery > and later metrics with the feed id. > Over its lifecycle, the server may be stopped and re-started > several times. > > This is great in the classical use case with installations > on tin or VMs. > > In container-land especially with systems like Kubernetes, > containers are started once and after they have died for > whatever reason they are not restarted again. > So the id of an individual container is less and less interesting. > The interesting part is the overall app, that contains of many > containers linked together with several of them representing > an individual service of the app. > > So basically we would rather need to record the app and other > metadata for identifying individual parts of the app (e.g. the web > servers or the data bases) and then get pointers to individual > stuff. > The feed would not need to survive for too long, but some of > its collected data perhaps. And then e.g. the discovery of resources > in a new container of the exact same type as before should be sort > of a no-op, as we know this already. Could we short-circuit that > by storing the docker-image-hash (or similar) and once we see this > known one abort the discovery? > > Another aspect is certainly that we want to keep (some) historic > records of the died container - e.g. some metrics and the point > when it died. Suppose k8s kills a container and spins a new one > up (same image) on a different node, then logically it is a continuation > of the first one, but in a different place (but they have different feed > ids) > > Now a more drastic scenario: As orchestration systems like k8s or > Docker-Swarm have their own registries, that can be queried : do we need > a > hawkular-inventory for this at all? > My gut feeling: no or not now... We may want/need it if we ever wanted to keep info about what's inside the containers (such as which war are in that EAP running in that container), but the need hasn't been expressed so far. Thomas > > ( We still need it for the non-OpenShift/K8s/Docker-Swarm envs ) > > -- > Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, > Werner-von-Siemens-Ring 14, D-85630 Grasbrunn > Handelsregister: Amtsgericht M?nchen HRB 153243 > Gesch?ftsf?hrer: Charles Cachera, Michael Cunningham, Michael O'Neill, > Eric Shander > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161004/eace3f30/attachment.html From yvedpath at redhat.com Tue Oct 4 07:21:17 2016 From: yvedpath at redhat.com (Yogesh Vedpathak) Date: Tue, 4 Oct 2016 07:21:17 -0400 (EDT) Subject: [Hawkular-dev] Hawkular on Windows In-Reply-To: <2116614595.792543.1475580005711.JavaMail.zimbra@redhat.com> Message-ID: <631886979.793227.1475580077699.JavaMail.zimbra@redhat.com> Hi, Let me know if I can install Hawkular on Windows. I do not find any reference for installing Hawkular on windows in Installation instructions. Regards, Yogesh From hrupp at redhat.com Wed Oct 5 05:32:39 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 05 Oct 2016 11:32:39 +0200 Subject: [Hawkular-dev] Reminder: Hawkular-services release timing Message-ID: Hey, a short reminder: Hawkular-services are (normally) done on Tuesday (mid-) morning Europe time. (Updates of) components that want to be included need to have a PR ready at this point in time. The PR for the component must not be for a src-dep nor a Snapshot. From hrupp at redhat.com Thu Oct 6 04:56:05 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 06 Oct 2016 10:56:05 +0200 Subject: [Hawkular-dev] Heads up: PRs in h-services are now also tested with the ruby-client tests Message-ID: <8837098F-C0AF-4394-BE6A-83936A46F33E@redhat.com> Hey, thanks to Josejulios work, we are now running an additional test job on hawkular-services after the normal build that puts the pr-outcome into a Docker container, starts this and runs the Ruby-client tests agains it. This may trigger false positives / negatives but should gives us an additional indication if changes in components would have a negative effect on the work with ManageIQ. From snegrea at redhat.com Thu Oct 6 14:25:02 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Thu, 6 Oct 2016 13:25:02 -0500 Subject: [Hawkular-dev] Hawkular Metrics 0.20.0 - Release Message-ID: Hello Everybody, I am happy to announce release 0.20.0 of Hawkular Metrics. This is a major release anchored by the inclusion of Hawkular Alerting in the distribution, a brand new compression algorithm, and numerous API enhancements and fixes. *Hawkular Metrics + Alerting* Hawkular Metrics now includes Hawkular Alerting out of the box; it's no longer necessary to install and manage two separate components. Hawkular Metrics delivers a single EAR file containing components from both projects. Incoming metric data is efficiently filtered and evaluated, generating alerts and events for defined triggers. The combined EAR artifact is the primary binary distribution of Hawkular Metrics going forward. For more details: HWKMETRICS-491 Hawkular Alerting Details - Version 1.2.1 is included in this release - Project details and repository: Hawkular Alerting - Github - Documentation: Hawkular Alerting REST API Documentation , Hawkular Alerting Examples , Hawkular Alerting Developer Guide Deployment Notes 1. Deployment Naming - The new EAR should be deployed as 'hawkular-metrics.ear' in WildFly. This naming is required to allow for proper dependencies to be established by the EAR components. 2. standalone.xml Configuration - The following snippet must be added to standalone.xml to support the Hawkular Alerting component. Add this additional cache-container entry after the existing cache-container entries: Usage Notes To use Hawkular Alerting with Hawkular Metrics there is a naming convention when defining trigger conditions. For a metric with name 'X', the alerting DataId to reference it will be '*X',* where the depends on the metric's type. For example, let's consider a metric with name 'HeapUsed' and type 'gauge'. This example defines a trigger condition to test if the used heap exceeded 80%: "hm_g_HeapUsed > .8". The 'hm_g' prefix indicates that HeapUsed is a gauge. Because Hawkular Metrics allows the same metric name for different types, the prefix is needed to make clear the target metric. The prefixes are: - hm_a: availability - hm_c: counter - hm_cr: counter rate - hm_g: gauge - hm_gr: gauge rate - hm_s: string *Compression* Hawkular Metrics will now compress all the data with a compression scheme based on Facebook's Gorilla paper. A job will be run every two hours that compresses data received during the 2 hour block since last compression (there's one hour time for out-of-order writes to be written to the Cassandra). These data points are then written to a new storage format that's designed after the Gorilla compression and also compressed with the LZ4 algorithm to give even better compression ratio. Benefits include a faster reading time of larger block of datapoints and very large disk space savings. For more details: HWKMETRICS-464 *Other major changes* - REST API - Query Improvements - Querying raw & rate data by tags is now also possible via POST {metric_type}/query endpoints for all types of metrics (HWKMETRICS-466 ) - Added new POST endpoints for stats gauges/stats/query and counters/stats/query (HWKMETRICS-465 ) - It is now possible to fetch metric definitions filtered by a list of ids not just tags (HWKMETRICS-461 ) - Cassandra Driver Configuration - Driver configuration options are now exposed for connection and request timeouts - Connection timeout can be configured via CASSANDRA_CONNECTION_TIMEOUT environment variable, or hawkular-metrics.cassandra.connection.timeout system property; value is specified in milliseconds and defaults to 5 seconds. - Request timeout can be configured via CASSANDRA_REQUEST_TIMEOUT environment variable, or hawkular-metrics.cassandra.request.timeout system property; value is specified in milliseconds and defaults to 12 seconds. - For more details: HWKMETRICS-490 - Job Scheduler - Duplicate instances of jobs were wrongly scheduled on server restart because the job scheduler was not checking if the job was already scheduled (HWKMETRICS-461 ) - Triggers now have a delay to properly schedule jobs in the future ( HWKMETRICS-224 ) - Resolved an issue where long running job can miss future executions because the current execution takes longer than the repeat interval ( HWKMETRICS-477 ) - Prevent execution of a job multiple times for the same execution time (HWKMETRICS-486 ) - RepeatingTrigger now allows specifying trigger start time ( HWKMETRICS-476 ) - Admin Endpoints - Admin related endpoints are going to be protected via an admin token, to be sent for admin related REST endpoints via Hawkular-Admin-Token request header - The first endpoint to be protected by this mechanism is /tenants since its functionality is cross tenant - The admin token can be set via command line arguments or environment variable at container startup; by default it is not set which means no access to protected endpoints - This is the foundation for adding more admin related functionality in upcoming releases (such as Cassandra cluster metrics, system telemetry, or admin operations) - The details: - system property key: hawkular.metrics.admin-token - environment variable: ADMIN_TOKEN - default value: null (admin endpoints are not accessible until set) - header name: Hawkular-Admin-Token - stored in system config using one-way hashing - For more details: HWKMETRICS-478 *Hawkular Metrics Clients* - Python: https://github.com/hawkular/hawkular-client-python - Go: https://github.com/hawkular/hawkular-client-go - Ruby: https://github.com/hawkular/hawkular-client-ruby - Java: https://github.com/hawkular/hawkular-client-java *Release Links* Github Release: https://github.com/hawkular/hawkular-metrics/releases/tag/0.20.0 JBoss Nexus Maven artifacts: http://origin-repository.jboss.org/nexus/content/repositories/public/org/ hawkular/metrics/ Jira release tracker: https://issues.jboss.org/browse/HWKMETRICS/fixforversion/12331360 A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, Joel Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their project contributions. Thank you, Stefan Negrea -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161006/08ee89db/attachment-0001.html From rvargasp at redhat.com Thu Oct 6 16:49:07 2016 From: rvargasp at redhat.com (Ruben Vargas Palma) Date: Thu, 6 Oct 2016 15:49:07 -0500 Subject: [Hawkular-dev] Cassandra Cluster with docker. In-Reply-To: <2aa8d059-0e6d-54e4-922f-896b87ffc333@redhat.com> References: <0a5f4097-2492-8990-f1dd-032bc0c7d124@redhat.com> <2aa8d059-0e6d-54e4-922f-896b87ffc333@redhat.com> Message-ID: Hi, I updated my repository with a basic bash script that allow you to launch more nodes in different hosts and query the status of the cluster, it is in pure bash and I used ssh for connecting to the remote hosts. (https://github.com/rubenvp8510/hwk-cassandra-docker/tree/master/bash) This is very basic but pure bash (no composer), there is a lot of things to do to improve this.. I was looking for another alternatives, Docker has his own orchestration mechanism but it is only available for the latest version (1.12), for old versions you could use docker-swarm but I think like with compose, this won't be available on this environment. Other alternative to SSH is use docker-api but it requires some configurations on the remote hosts.. On 09/29/2016 03:57 PM, Juraci Paix?o Kr?hling wrote: > On 29.09.2016 17:21, Stefan Negrea wrote: >> docker-compose is no-go for orchestration. > Why's that? I think we use it somewhere in APM as well. > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Fri Oct 7 04:00:07 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 07 Oct 2016 10:00:07 +0200 Subject: [Hawkular-dev] Hawkular-services 0.15 released Message-ID: Hello, we have released Hawkular-services 0.15 with two updates * hawkular-metrics 0.20 as described in http://www.hawkular.org/blog/2016/10/06/hawkular-metrics-0.20.0.Final-released.html * hawkular-inventory 0.19.3, which allows to specify the relational database to use (if nothing is provided, the internal one will be used). See http://www.hawkular.org/hawkular-services/docs/user-guide/inventory/index.html#configuration-properties. Please test this release as it will lead us to a Hawkular-services 1.0 release in the not too distant future. Unofficial Docker images are available at https://hub.docker.com/r/pilhuhn/hawkular-services/ From hrupp at redhat.com Fri Oct 7 04:07:16 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 07 Oct 2016 10:07:16 +0200 Subject: [Hawkular-dev] Hawkular-services 0.15 released In-Reply-To: References: Message-ID: <4F1C76E5-1E51-4DA7-BDC8-E1E60683675B@redhat.com> I forgot to mention: Thanks to the work by Josejulio we have identified two issues in the current RubyGem https://github.com/hawkular/hawkular-client-ruby/issues/155 : Tenant handling - I believe this does not matter for ManageIQ https://github.com/hawkular/hawkular-client-ruby/issues/154 : Handling of ackBy - I believe this also does not matter for ManageIQ, but may have an impact on cm-ops On 7 Oct 2016, at 10:00, Heiko W.Rupp wrote: > Hello, > > we have released Hawkular-services 0.15 with two updates > * hawkular-metrics 0.20 as described in > http://www.hawkular.org/blog/2016/10/06/hawkular-metrics-0.20.0.Final-released.html > * hawkular-inventory 0.19.3, which allows to specify the relational > database to use (if nothing is provided, the internal one will be > used). > See > http://www.hawkular.org/hawkular-services/docs/user-guide/inventory/index.html#configuration-properties. > > Please test this release as it will lead us to a Hawkular-services 1.0 > release in the not too distant future. > > Unofficial Docker images are available at > https://hub.docker.com/r/pilhuhn/hawkular-services/ > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander From hrupp at redhat.com Fri Oct 7 12:17:57 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 07 Oct 2016 18:17:57 +0200 Subject: [Hawkular-dev] RfC: no longer support Ruby 2.0 in Ruby-Gem? Message-ID: Hey, ManageIQ is using Ruby 2.2.4+ and Ruby 2.0 is pretty old now (Jruby 9.1.x is Ruby 2.3+), so I would like to no longer "support" Ruby 2.0. For the time being that only means that we do not test against 2.0 anymore in Travis. What do people think? What about 2.1 ? Heiko From jshaughn at redhat.com Fri Oct 7 13:46:57 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Fri, 7 Oct 2016 13:46:57 -0400 Subject: [Hawkular-dev] RfC: no longer support Ruby 2.0 in Ruby-Gem? In-Reply-To: References: Message-ID: <801feeee-e508-2f06-bc56-a64e4c47d929@redhat.com> no objection from me. On 10/7/2016 12:17 PM, Heiko W.Rupp wrote: > Hey, > > ManageIQ is using Ruby 2.2.4+ and Ruby 2.0 is > pretty old now (Jruby 9.1.x is Ruby 2.3+), so > I would like to no longer "support" Ruby 2.0. > For the time being that only means that > we do not test against 2.0 anymore in Travis. > > What do people think? > > What about 2.1 ? > > Heiko > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161007/ac5f9765/attachment.html From mithomps at redhat.com Fri Oct 7 14:00:08 2016 From: mithomps at redhat.com (mike thompson) Date: Fri, 7 Oct 2016 11:00:08 -0700 Subject: [Hawkular-dev] RfC: no longer support Ruby 2.0 in Ruby-Gem? In-Reply-To: <801feeee-e508-2f06-bc56-a64e4c47d929@redhat.com> References: <801feeee-e508-2f06-bc56-a64e4c47d929@redhat.com> Message-ID: <7D4EE2D9-08C9-4D69-9489-3F8C81FC60F7@redhat.com> > On Oct 7, 2016, at 10:46 AM, Jay Shaughnessy wrote: > > no objection from me. > > On 10/7/2016 12:17 PM, Heiko W.Rupp wrote: >> Hey, >> >> ManageIQ is using Ruby 2.2.4+ and Ruby 2.0 is >> pretty old now (Jruby 9.1.x is Ruby 2.3+), so >> I would like to no longer "support" Ruby 2.0. >> For the time being that only means that >> we do not test against 2.0 anymore in Travis. >> >> What do people think? Yes! No need for these older versions. >> >> What about 2.1 ? Why stop there? Why not 2.2? Since current version is 2.3 (which ships in Red Hat Software Collections). >> >> Heiko >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161007/68a69bef/attachment.html From jmartine at redhat.com Fri Oct 7 14:04:51 2016 From: jmartine at redhat.com (Josejulio Martinez Magana) Date: Fri, 7 Oct 2016 13:04:51 -0500 Subject: [Hawkular-dev] RfC: no longer support Ruby 2.0 in Ruby-Gem? In-Reply-To: <7D4EE2D9-08C9-4D69-9489-3F8C81FC60F7@redhat.com> References: <801feeee-e508-2f06-bc56-a64e4c47d929@redhat.com> <7D4EE2D9-08C9-4D69-9489-3F8C81FC60F7@redhat.com> Message-ID: On Fri, Oct 7, 2016 at 1:00 PM, mike thompson wrote: > > On Oct 7, 2016, at 10:46 AM, Jay Shaughnessy wrote: > > no objection from me. > > On 10/7/2016 12:17 PM, Heiko W.Rupp wrote: > > Hey, > > ManageIQ is using Ruby 2.2.4+ and Ruby 2.0 is > pretty old now (Jruby 9.1.x is Ruby 2.3+), so > I would like to no longer "support" Ruby 2.0. > For the time being that only means that > we do not test against 2.0 anymore in Travis. > > What do people think? > > Yes! No need for these older versions. > > What about 2.1 ? > > Why stop there? Why not 2.2? Since current version is 2.3 (which ships in > Red Hat Software Collections). > +1 > > Heiko > _______________________________________________ > hawkular-dev mailing listhawkular-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161007/e7dc7348/attachment.html From hrupp at redhat.com Fri Oct 7 14:14:37 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 07 Oct 2016 20:14:37 +0200 Subject: [Hawkular-dev] no longer support Ruby 2.0 in Ruby-Gem? In-Reply-To: References: <801feeee-e508-2f06-bc56-a64e4c47d929@redhat.com> <7D4EE2D9-08C9-4D69-9489-3F8C81FC60F7@redhat.com> Message-ID: <81E63BA7-DFF9-4D63-8F81-CD31359D8463@redhat.com> On 7 Oct 2016, at 20:04, Josejulio Martinez Magana wrote: > On Fri, Oct 7, 2016 at 1:00 PM, mike thompson wrote: >> Why stop there? Why not 2.2? Since current version is 2.3 As I wrote ManageIQ is on 2.2.4+ so it has to work with 2.2 From mithomps at redhat.com Fri Oct 7 14:38:33 2016 From: mithomps at redhat.com (mike thompson) Date: Fri, 7 Oct 2016 11:38:33 -0700 Subject: [Hawkular-dev] no longer support Ruby 2.0 in Ruby-Gem? In-Reply-To: <81E63BA7-DFF9-4D63-8F81-CD31359D8463@redhat.com> References: <801feeee-e508-2f06-bc56-a64e4c47d929@redhat.com> <7D4EE2D9-08C9-4D69-9489-3F8C81FC60F7@redhat.com> <81E63BA7-DFF9-4D63-8F81-CD31359D8463@redhat.com> Message-ID: <0B805C87-51C8-4812-93D6-61CA5823EB76@redhat.com> > On Oct 7, 2016, at 11:14 AM, Heiko W.Rupp wrote: > > On 7 Oct 2016, at 20:04, Josejulio Martinez Magana wrote: > >> On Fri, Oct 7, 2016 at 1:00 PM, mike thompson wrote: >>> Why stop there? Why not 2.2? Since current version is 2.3 > > As I wrote ManageIQ is on 2.2.4+ so it has to work with 2.2 Agreed. It probably makes most sense to stay in lockstep with MiQ for simplicity and bug reasons. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From luye19870406 at gmail.com Sat Oct 8 03:27:03 2016 From: luye19870406 at gmail.com (=?UTF-8?B?5Z+c6ZmG?=) Date: Sat, 8 Oct 2016 15:27:03 +0800 Subject: [Hawkular-dev] [hawkular-apm]How to inspect query string within a HttpRequest Message-ID: hi everyone I just want to make the query string as a property within a httprequest by setting the businessTransaction e.g. got two httprequest GET /foo/bar?text=hello GET /foo/bar?text=world and want to classify them with query text by evaluateURL then node will got a property with name text and values hello and world after seeing the apm-javax-servlet.json only the 'processHeaders' will trigger a businessTranaction process when all other 'setdetails' has not been invoked which will attach the query string to 'node details' , so the action 'evaluateUrl' will never take effect, and all other actions like 'setContent' or 'setProperty' only findvalues in Headers and Content, unfortunately they don't hold the query string which only exists in details by default. I am new to this, so am I missing something ? is there any way to achieve what I want without modify the default config? thanks best regards -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161008/13f0252e/attachment.html From gbrown at redhat.com Mon Oct 10 03:24:49 2016 From: gbrown at redhat.com (Gary Brown) Date: Mon, 10 Oct 2016 03:24:49 -0400 (EDT) Subject: [Hawkular-dev] [hawkular-apm]How to inspect query string within a HttpRequest In-Reply-To: References: Message-ID: <1709114769.1062450.1476084289423.JavaMail.zimbra@redhat.com> Hi You are correct that it is not currently supported - but would probably be a good idea to convert the query parameters into properties by default. Can you create a feature request in our jira: https://issues.jboss.org/projects/HWKAPM Regards Gary ----- Original Message ----- > hi everyone I just want to make the query string as a property within a > httprequest by setting the businessTransaction > e.g. got two httprequest > GET /foo/bar?text=hello > GET /foo/bar?text=world > and want to classify them with query text by evaluateURL > then node will got a property with name text and values hello and world > > after seeing the apm-javax-servlet.json only the 'processHeaders' will > trigger a businessTranaction process when all other 'setdetails' has not > been invoked which will attach the query string to 'node details' , so the > action 'evaluateUrl' will never take effect, and all other actions like > 'setContent' or 'setProperty' only findvalues in Headers and Content, > unfortunately they don't hold the query string which only exists in details > by default. > > I am new to this, so am I missing something ? is there any way to achieve > what I want without modify the default config? thanks > > best regards > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From lkrejci at redhat.com Mon Oct 10 07:15:15 2016 From: lkrejci at redhat.com (Lukas Krejci) Date: Mon, 10 Oct 2016 13:15:15 +0200 Subject: [Hawkular-dev] Hawkular Inventory 0.20.0.Final released Message-ID: <1584494.SDifyP20pY@localhost.localdomain> Hi all, I'm proud to announce that Hawkular Inventory 0.20.0.Final has been released. Apart from an updated version of Sqlg and slightly reduced distribution size, the *big* new feature this version brings is the support for keeping history of changes to inventory. From now on, you are able to retrieve the history of updates made to an entity, but you can also "look at" an entity, or whole inventory, as it looked at certain point in time - questions like "what DB was this application connected to last month?", "What was the list of feeds reporting on our production environment in March", etc. All that possible using a single new query parameter to the REST endpoints - "?at=". 0.20.0.Final doesn't offer the ability to compact or delete the "old" data from inventory (delete now merely marks the entity in question as deleted from the specified time onward) - that will be the focus of our very next release. We wanted to let this release out though so that people can start enjoying the nostalgic views at the good old days of their inventory ;) Special thanks go out to Joel Takvorian for reviews, fixes and the work in upstream Sqlg. Thanks, -- Lukas Krejci From lkrejci at redhat.com Mon Oct 10 09:46:03 2016 From: lkrejci at redhat.com (Lukas Krejci) Date: Mon, 10 Oct 2016 15:46:03 +0200 Subject: [Hawkular-dev] Inventory: transient feeds - or how to tackle the pets vs cattle scenario In-Reply-To: <95DB8D26-E6F3-4A87-B603-16EDCC03BC00@redhat.com> References: <95DB8D26-E6F3-4A87-B603-16EDCC03BC00@redhat.com> Message-ID: <1840636.RNN5E92E1E@localhost.localdomain> On Tuesday, October 4, 2016 11:20:51 AM CEST Heiko W.Rupp wrote: > Hey, > > Right now we identify "agents" via their feed-id. > An instrumented wildfly comes online, registers > its feed with the server, sends its resource discovery > and later metrics with the feed id. > Over its lifecycle, the server may be stopped and re-started > several times. > > This is great in the classical use case with installations > on tin or VMs. > > In container-land especially with systems like Kubernetes, > containers are started once and after they have died for > whatever reason they are not restarted again. > So the id of an individual container is less and less interesting. > The interesting part is the overall app, that contains of many > containers linked together with several of them representing > an individual service of the app. > > So basically we would rather need to record the app and other > metadata for identifying individual parts of the app (e.g. the web > servers or the data bases) and then get pointers to individual > stuff. > The feed would not need to survive for too long, but some of > its collected data perhaps. And then e.g. the discovery of resources > in a new container of the exact same type as before should be sort > of a no-op, as we know this already. Could we short-circuit that > by storing the docker-image-hash (or similar) and once we see this > known one abort the discovery? > Don't agents support user-defined feed IDs? Can we not pass in the feed ID as any other configuration (env var or something) and therefore, no matter how and where the feed "materializes" it identifies itself the same? If it identifies the same and inventory already has record of it, the discovery can stop quite soon, depending on how the tree hashes look like locally and on the inventory server. If you're talking about "local" discovery - feed "waking up" and realizing it woke up in the same container as last time and therefore it is guaranteed the inventory will look the same - I'm not sure it is that easy to assume that - I can imagine the discovery results being dependent on the contents of some attached data volumes, values of env vars the container was started with, etc. > Another aspect is certainly that we want to keep (some) historic > records of the died container - e.g. some metrics and the point > when it died. Suppose k8s kills a container and spins a new one > up (same image) on a different node, then logically it is a continuation > of the first one, but in a different place (but they have different feed > ids) > Again, feed id can be part of the pod configuration (passed in as env var) - that way feed id stays the same. > Now a more drastic scenario: As orchestration systems like k8s or > Docker-Swarm have their own registries, that can be queried : do we need > a > hawkular-inventory for this at all? Using the same logic, our agent would be replaced by heapster with k8s. I think we always meant Hawkular to go deeper than the orchestration services. But in general sense, yes, inventory is easily replacable by the registries, if the granularity of the data in the registry suffices. > > ( We still need it for the non-OpenShift/K8s/Docker-Swarm envs ) -- Lukas Krejci From hrupp at redhat.com Tue Oct 11 02:42:00 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 11 Oct 2016 08:42:00 +0200 Subject: [Hawkular-dev] Memory-usage of Hawkular-services Message-ID: Hey, tl;dr: we need to investigate heap usage - especially in the case of compression kicking in - it looks like there could be a memory leak. Compression timing seems mostly ok. I originally wanted to see how more feeds influence the metrics compression timing. So I started the server with -Xmx512m as I did in all the weeks before and pointed a few feeds at the server - to see it crash with OOME shortly after the compression started. Now I restarted the server with -Xmx1024m and also -XX:MaxMetaspaceSize=512m (up from 256m before) and am running the server now with 1 feed for a day. To be continued below ... ( Time is in GMT, which is 2h off of my local time) hawkular_1 | 15:00:44,764 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 15:00:45,452 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 687 ms hawkular_1 | 17:00:44,757 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 17:00:46,796 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 2039 ms hawkular_1 | 19:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 19:00:47,293 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 2541 ms hawkular_1 | 21:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 21:00:46,267 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1517 ms hawkular_1 | 23:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 23:00:45,472 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 722 ms hawkular_1 | 01:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 01:00:46,241 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1492 ms hawkular_1 | 03:00:44,747 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 03:00:45,780 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1033 ms hawkular_1 | 05:00:44,746 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 05:00:45,781 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1034 ms hawkular_1 | 07:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 07:00:46,386 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1636 ms hawkular_1 | 09:00:44,748 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 09:00:45,682 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 934 ms hawkular_1 | 11:00:44,750 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 11:00:46,339 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1589 ms hawkular_1 | 13:00:44,748 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 13:00:45,880 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 1132 ms Looking at the memory usage I see the memory usage there is often a peak in heap usage around and after the compression - looking at the accumulated GC-time also shows heavy GC activity. I guess the peak in heap usage (and thus committed heap) comes from promoting objects from young generation into old during the compression and then later after compression is over, they are garbage collected, so that heap usage goes down and the system is able to free some memory. Starting at around 4:10pm (2pm in the above output) I am running with 10 extra feeds (hawkfly on an external box) While it looks like there is a slowly growing non-heap, it was growing a lot when the 10 extra feeds connected. Also it looks like it is growing a bit more each time compression kicks in ("non-heap") The first compression run with 11 feeds did not take too long, the next was hawkular_1 | 17:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 17:00:50,277 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 5528 ms Which used a lot more memory than before. Non-heap was able to reclaim some memory though. hawkular_1 | 19:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 19:00:50,093 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 5341 ms hawkular_1 | 21:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 21:00:49,465 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 4714 ms hawkular_1 | 23:00:44,753 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 23:00:48,925 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 4171 ms hawkular_1 | 01:00:44,750 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 01:00:48,554 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 3803 ms hawkular_1 | 03:00:44,761 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 03:00:48,659 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 3898 ms hawkular_1 | 05:00:44,748 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 05:00:49,134 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 4385 ms hawkular_1 | 07:00:44,755 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 07:00:49,831 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 5076 ms hawkular_1 | 09:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 09:00:49,508 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 4757 ms Now at 11:20 (9:20 for above logs) I started 5 more feeds (with a 2min sleep between starts) hawkular_1 | 11:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 11:00:49,751 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 5002 ms hawkular_1 | 13:00:44,749 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 13:00:56,594 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 11845 ms hawkular_1 | 15:00:44,754 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 15:00:53,985 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 9231 ms And another 5 starting at 17:12 (15:00 in above logs timezone) hawkular_1 | 17:00:44,768 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 17:00:57,824 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 13056 ms And another 5 starting at 19:57 (17:57 in above log timezone ) hawkular_1 | 19:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution hawkular_1 | 19:01:24,401 WARN org.apache.activemq.artemis.ra AMQ152007: Thread Thread[TearDown/ActiveMQActivation,5,] could not be finished hawkular_1 | 19:01:40,918 WARN org.apache.activemq.artemis.ra AMQ152007: Thread Thread[TearDown/ActiveMQActivation,5,default-threads] could not be finished hawkular_1 | 19:02:04,619 WARN org.apache.activemq.artemis.ra AMQ152007: Thread Thread[TearDown/ActiveMQActivation,5,default-threads] could not be finished hawkular_1 | 19:02:22,423 WARN org.apache.activemq.artemis.ra AMQ152007: Thread Thread[TearDown/ActiveMQActivation,5,default-threads] could not be finished hawkular_1 | 19:02:30,247 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 105495 ms This took almost 2 minutes, where the server was non-responsive 21:00:44,753 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution 21:01:06,520 INFO org.hawkular.metrics.core.jobs.CompressData Finished compressing data in 21767 ms 23:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting execution And here it went south and the server more or less died with OOME exceptions (it is still responding to queries, and potentially even ingesting new data, but the scheduler seems not to run anymore. I can imagine that once the OOME happened, the scheduler thread died, freeing up memory that the compression job held and which then allowed the server itself to continue. But it certainly is in an unusable state. This is the final "one day" with "max" values plotted: ![](cid:7C83CDE5-A45F-42C6-853A-2956DC3E1719 at redhat.com "Bildschirmfoto 2016-10-11 um 08.21.53.png") The peaks at compression time for heap-max (green line) are clearly visible. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/2dd59838/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: Bildschirmfoto 2016-10-11 um 08.21.53.png Type: image/png Size: 144681 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/2dd59838/attachment-0001.png From miburman at redhat.com Tue Oct 11 03:52:24 2016 From: miburman at redhat.com (Michael Burman) Date: Tue, 11 Oct 2016 10:52:24 +0300 Subject: [Hawkular-dev] Get rid of Travis? Message-ID: <1efb6fe5-29fb-f5a5-e4d3-99e495a630a5@redhat.com> Hi, I'm proposing we get rid of the Travis for now (at least for metrics) and stick to something else, such as the old Jenkins. At the moment, it can take several runs for metrics PRs to finish, since Cassandra stops responding in the Travis runs every time. Yesterday I started my PR 4 times, last week one PR for 10 times. 22:28:24,629 ERROR [com.datastax.driver.core.ControlConnection] (cluster1-reconnection-1) [Control connection] Cannot connect to any host, scheduling retry in 600000 milliseconds Sometimes it doesn't even get this far and fails with running the installation script of Cassandra. These errors have made the Travis runs completely irrelevant, whenever they fail it just makes sense to restart them without reading logs (we should get automatic script to restart always when it fails) until they succeed. No errors reported by Travis are trustworthy. We need a working CI solution and this isn't it. There's no "community visibility" if the results have no meaning, using Jenkins at least provides us with "failed / not failed", Travis doesn't provide even this. - Micke From lponce at redhat.com Tue Oct 11 03:59:36 2016 From: lponce at redhat.com (Lucas Ponce) Date: Tue, 11 Oct 2016 03:59:36 -0400 (EDT) Subject: [Hawkular-dev] Get rid of Travis? In-Reply-To: <1efb6fe5-29fb-f5a5-e4d3-99e495a630a5@redhat.com> References: <1efb6fe5-29fb-f5a5-e4d3-99e495a630a5@redhat.com> Message-ID: <1084786031.197952.1476172776807.JavaMail.zimbra@redhat.com> I am not a fun of travis. I admit that sometimes it has weird behaviour, specially when you suspect your machine comes from an hibernate state and it needs a "warm up", so a first round fails but a second and next works good. But I think that get rid of Travis perhaps is too aggressive. Is there a possibility to study if we can "upgrade" and have a travis subscription/plan that gives us more resources ? Other projects as ManageIQ uses travis in big setups and perhaps we can follow/adopt some of their practices. One thing I miss from travis is that the machines are small to automate some performance/clustering tests for alerting, and those ones need to be executed locally. But for integration tests it works "reasonable" in the sense that after "warm up" (first round) executions are stable and consistency. Just my 2 cents. ----- Mensaje original ----- > De: "Michael Burman" > Para: hawkular-dev at lists.jboss.org > Enviados: Martes, 11 de Octubre 2016 9:52:24 > Asunto: [Hawkular-dev] Get rid of Travis? > > Hi, > > I'm proposing we get rid of the Travis for now (at least for metrics) > and stick to something else, such as the old Jenkins. At the moment, it > can take several runs for metrics PRs to finish, since Cassandra stops > responding in the Travis runs every time. Yesterday I started my PR 4 > times, last week one PR for 10 times. > > 22:28:24,629 ERROR [com.datastax.driver.core.ControlConnection] > (cluster1-reconnection-1) [Control connection] Cannot connect to any > host, scheduling retry in 600000 milliseconds > > Sometimes it doesn't even get this far and fails with running the > installation script of Cassandra. These errors have made the Travis runs > completely irrelevant, whenever they fail it just makes sense to restart > them without reading logs (we should get automatic script to restart > always when it fails) until they succeed. No errors reported by Travis > are trustworthy. > > We need a working CI solution and this isn't it. There's no "community > visibility" if the results have no meaning, using Jenkins at least > provides us with "failed / not failed", Travis doesn't provide even this. > > - Micke > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ploffay at redhat.com Tue Oct 11 04:01:16 2016 From: ploffay at redhat.com (Pavol Loffay) Date: Tue, 11 Oct 2016 10:01:16 +0200 Subject: [Hawkular-dev] Get rid of Travis? In-Reply-To: <1efb6fe5-29fb-f5a5-e4d3-99e495a630a5@redhat.com> References: <1efb6fe5-29fb-f5a5-e4d3-99e495a630a5@redhat.com> Message-ID: Hello, in APM we have changed travis image from 'STANDARD' to 'TRUSTY BETA' [1]. So far it works great without any failed builds, before builds were failing all the time. Even the build time has improved see [3] for master and [4] for PRs. PR changing to 'TRUSTY BETA' [2]. [1]: https://docs.travis-ci.com/user/ci-environment/ [2]: https://github.com/hawkular/hawkular-apm/pull/494 [3]: https://travis-ci.org/hawkular/hawkular-apm/builds [4]: https://travis-ci.org/hawkular/hawkular-apm/pull_requests Pavol On Tue, Oct 11, 2016 at 9:52 AM, Michael Burman wrote: > Hi, > > I'm proposing we get rid of the Travis for now (at least for metrics) > and stick to something else, such as the old Jenkins. At the moment, it > can take several runs for metrics PRs to finish, since Cassandra stops > responding in the Travis runs every time. Yesterday I started my PR 4 > times, last week one PR for 10 times. > > 22:28:24,629 ERROR [com.datastax.driver.core.ControlConnection] > (cluster1-reconnection-1) [Control connection] Cannot connect to any > host, scheduling retry in 600000 milliseconds > > Sometimes it doesn't even get this far and fails with running the > installation script of Cassandra. These errors have made the Travis runs > completely irrelevant, whenever they fail it just makes sense to restart > them without reading logs (we should get automatic script to restart > always when it fails) until they succeed. No errors reported by Travis > are trustworthy. > > We need a working CI solution and this isn't it. There's no "community > visibility" if the results have no meaning, using Jenkins at least > provides us with "failed / not failed", Travis doesn't provide even this. > > - Micke > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -- Pavol Loffay Cell: +421948286055 Red Hat Purky?ova 111 TPB-B 612 45 Brno -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/610ac36b/attachment.html From jmartine at redhat.com Tue Oct 11 04:10:34 2016 From: jmartine at redhat.com (Josejulio Martinez Magana) Date: Tue, 11 Oct 2016 03:10:34 -0500 Subject: [Hawkular-dev] Get rid of Travis? In-Reply-To: References: <1efb6fe5-29fb-f5a5-e4d3-99e495a630a5@redhat.com> Message-ID: Hello, Maybe instead of installing Cassandra [1] we could try to use a dockerized Cassandra. We have been using it [2] to test the ruby client (along with a dockerized Hawkular-services) and (so far) has been working well. [1] https://github.com/hawkular/hawkular-metrics/blob/master/.travis.install.cassandra.sh [2] https://github.com/hawkular/hawkular-client-ruby/blob/master/docker-compose.yml El martes, 11 de octubre de 2016, Pavol Loffay escribi?: > Hello, > > in APM we have changed travis image from 'STANDARD' to 'TRUSTY BETA' [1]. > So far it works great without any failed builds, before builds were > failing all the time. > Even the build time has improved see [3] for master and [4] for PRs. > PR changing to 'TRUSTY BETA' [2]. > > > [1]: https://docs.travis-ci.com/user/ci-environment/ > [2]: https://github.com/hawkular/hawkular-apm/pull/494 > [3]: https://travis-ci.org/hawkular/hawkular-apm/builds > [4]: https://travis-ci.org/hawkular/hawkular-apm/pull_requests > > Pavol > > On Tue, Oct 11, 2016 at 9:52 AM, Michael Burman > wrote: > >> Hi, >> >> I'm proposing we get rid of the Travis for now (at least for metrics) >> and stick to something else, such as the old Jenkins. At the moment, it >> can take several runs for metrics PRs to finish, since Cassandra stops >> responding in the Travis runs every time. Yesterday I started my PR 4 >> times, last week one PR for 10 times. >> >> 22:28:24,629 ERROR [com.datastax.driver.core.ControlConnection] >> (cluster1-reconnection-1) [Control connection] Cannot connect to any >> host, scheduling retry in 600000 milliseconds >> >> Sometimes it doesn't even get this far and fails with running the >> installation script of Cassandra. These errors have made the Travis runs >> completely irrelevant, whenever they fail it just makes sense to restart >> them without reading logs (we should get automatic script to restart >> always when it fails) until they succeed. No errors reported by Travis >> are trustworthy. >> >> We need a working CI solution and this isn't it. There's no "community >> visibility" if the results have no meaning, using Jenkins at least >> provides us with "failed / not failed", Travis doesn't provide even this. >> >> - Micke >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > > > -- > Pavol Loffay > Cell: +421948286055 > Red Hat > Purky?ova 111 TPB-B > 612 45 Brno > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/5396a3c0/attachment.html From miburman at redhat.com Tue Oct 11 06:57:13 2016 From: miburman at redhat.com (Michael Burman) Date: Tue, 11 Oct 2016 13:57:13 +0300 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: References: Message-ID: Hi, There's on going discussions and more data coming from Heiko, but since not everyone might be online, I'll fill in here as well.. Testing with the Hawkular-Metrics (not services), I can't repeat these. I started by testing just the compression library and after compressing some 9 billion datapoints, I was still running constant ~10MB heap usage. So I started HWKMETRICS in Wildfly 10.0.0.Final and ran with our load-test, gatling and clients=20, metrics=1000 and loops enough to keep it running forever. The instance was modified to run with 20 minutes interval of compression. Monitoring with jconsole, I can't see any difference in memory usage from first 5 minutes to the last 5 minutes (running for two hours now). It's running between 147MB of memory and 400MB. Requires more investigation then.. - Micke On 10/11/2016 09:42 AM, Heiko W.Rupp wrote: > > Hey, > > tl;dr: we need to investigate heap usage - especially in the case of > compression > kicking in - it looks like there could be a memory leak. Compression > timing seems > mostly ok. > > I originally wanted to see how more feeds influence the metrics > compression timing. > So I started the server with -Xmx512m as I did in all the weeks before > and pointed a > few feeds at the server - to see it crash with OOME shortly after the > compression started. > > Now I restarted the server with -Xmx1024m and also > -XX:MaxMetaspaceSize=512m (up from 256m before) and am running the > server now with 1 feed for a day. > > To be continued below ... > > ( Time is in GMT, which is 2h off of my local time) > hawkular_1 | 15:00:44,764 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 15:00:45,452 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 687 ms > hawkular_1 | 17:00:44,757 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 17:00:46,796 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 2039 ms > hawkular_1 | 19:00:44,751 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 19:00:47,293 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 2541 ms > hawkular_1 | 21:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 21:00:46,267 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1517 ms > hawkular_1 | 23:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 23:00:45,472 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 722 ms > hawkular_1 | 01:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 01:00:46,241 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1492 ms > hawkular_1 | 03:00:44,747 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 03:00:45,780 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1033 ms > hawkular_1 | 05:00:44,746 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 05:00:45,781 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1034 ms > hawkular_1 | 07:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 07:00:46,386 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1636 ms > hawkular_1 | 09:00:44,748 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 09:00:45,682 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 934 ms > hawkular_1 | 11:00:44,750 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 11:00:46,339 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1589 ms > hawkular_1 | 13:00:44,748 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 13:00:45,880 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 1132 ms > > Looking at the memory usage I see the memory usage there is often a > peak in heap usage around and after the compression - looking at the > accumulated GC-time also shows heavy GC activity. I guess the peak in > heap usage (and thus committed heap) comes from promoting objects from > young generation into old during the compression and then later after > compression is over, they are garbage collected, so that heap usage > goes down and the system is able to free some memory. > > Starting at around 4:10pm (2pm in the above output) I am running with > 10 extra feeds (hawkfly on an external box) > > While it looks like there is a slowly growing non-heap, it was growing > a lot when the 10 extra feeds connected. > Also it looks like it is growing a bit more each time compression > kicks in ("non-heap") > > The first compression run with 11 feeds did not take too long, the > next was > > hawkular_1 | 17:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 17:00:50,277 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 5528 ms > > Which used a lot more memory than before. Non-heap was able to reclaim > some memory though. > > hawkular_1 | 19:00:44,751 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 19:00:50,093 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 5341 ms > hawkular_1 | 21:00:44,751 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 21:00:49,465 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 4714 ms > hawkular_1 | 23:00:44,753 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 23:00:48,925 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 4171 ms > hawkular_1 | 01:00:44,750 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 01:00:48,554 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 3803 ms > hawkular_1 | 03:00:44,761 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 03:00:48,659 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 3898 ms > hawkular_1 | 05:00:44,748 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 05:00:49,134 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 4385 ms > hawkular_1 | 07:00:44,755 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 07:00:49,831 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 5076 ms > hawkular_1 | 09:00:44,751 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 09:00:49,508 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 4757 ms > > Now at 11:20 (9:20 for above logs) I started 5 more feeds (with a 2min > sleep between starts) > > hawkular_1 | 11:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 11:00:49,751 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 5002 ms > hawkular_1 | 13:00:44,749 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 13:00:56,594 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 11845 ms > hawkular_1 | 15:00:44,754 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 15:00:53,985 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 9231 ms > > And another 5 starting at 17:12 (15:00 in above logs timezone) > > hawkular_1 | 17:00:44,768 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 17:00:57,824 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 13056 ms > > And another 5 starting at 19:57 (17:57 in above log timezone ) > > hawkular_1 | 19:00:44,751 INFO > org.hawkular.metrics.core.jobs.CompressData Starting execution > hawkular_1 | 19:01:24,401 WARN org.apache.activemq.artemis.ra > AMQ152007: Thread Thread[TearDown/ActiveMQActivation,5,] could not be > finished > hawkular_1 | 19:01:40,918 WARN org.apache.activemq.artemis.ra > AMQ152007: Thread > Thread[TearDown/ActiveMQActivation,5,default-threads] could not be > finished > hawkular_1 | 19:02:04,619 WARN org.apache.activemq.artemis.ra > AMQ152007: Thread > Thread[TearDown/ActiveMQActivation,5,default-threads] could not be > finished > hawkular_1 | 19:02:22,423 WARN org.apache.activemq.artemis.ra > AMQ152007: Thread > Thread[TearDown/ActiveMQActivation,5,default-threads] could not be > finished > hawkular_1 | 19:02:30,247 INFO > org.hawkular.metrics.core.jobs.CompressData Finished compressing data > in 105495 ms > > This took almost 2 minutes, where the server was non-responsive > > 21:00:44,753 INFO org.hawkular.metrics.core.jobs.CompressData Starting > execution > 21:01:06,520 INFO org.hawkular.metrics.core.jobs.CompressData Finished > compressing data in 21767 ms > 23:00:44,751 INFO org.hawkular.metrics.core.jobs.CompressData Starting > execution > > And here it went south and the server more or less died with OOME > exceptions (it is still responding to queries, and potentially even > ingesting new data, but the scheduler seems not to run anymore. > I can imagine that once the OOME happened, the scheduler thread died, > freeing up memory that the compression job held and which then allowed > the server itself to continue. But it certainly is in an unusable state. > > This is the final "one day" with "max" values plotted: > > The peaks at compression time for heap-max (green line) are clearly > visible. > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/1f20e095/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 144681 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/1f20e095/attachment-0001.png From hrupp at redhat.com Tue Oct 11 07:17:32 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 11 Oct 2016 13:17:32 +0200 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: References: Message-ID: <4D895E6D-CB7E-43EF-B5A4-B9C9A439D66E@redhat.com> Hi, On 11 Oct 2016, at 12:57, Michael Burman wrote: > (running for two hours now). It's running between 147MB of memory and > 400MB. That is a 250MB difference. Can you try doing a full-gc right before compression start and right after end to and measure the heap usage at those points (before start and at end before gc) so that we get an idea how much memory the compression really takes? I started a 0.16 h-services at ~?10:35, let it run until 11:13 (first compression happened at 11) then added 25 feeds - one every 2 minutes. The compression at 1pm went well, but the graph certainly shows a much increased need for heap during that time. ![](cid:F4DB9C04-A766-4C72-B253-73030D4487AB at redhat.com "Bildschirmfoto 2016-10-11 um 13.13.16.png") We may perhaps try to spread out the compression over more time and thus trade time for memory to get rid of this large memory usage peak. After the next crash, I will try to run with external DB for Inventory to see how much this contributes to the issue here. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/4d5b03a7/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: Bildschirmfoto 2016-10-11 um 13.13.16.png Type: image/png Size: 97232 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/4d5b03a7/attachment-0001.png From miburman at redhat.com Tue Oct 11 07:34:56 2016 From: miburman at redhat.com (Michael Burman) Date: Tue, 11 Oct 2016 14:34:56 +0300 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: <4D895E6D-CB7E-43EF-B5A4-B9C9A439D66E@redhat.com> References: <4D895E6D-CB7E-43EF-B5A4-B9C9A439D66E@redhat.com> Message-ID: <8310caa2-7f7c-ab6a-30b7-a8776cf9852b@redhat.com> Hi, 250MB is about the amount which the GC lets the heap live. And it grows and shrinks all the time based on the load. But in the end and start, the lowest point was the same. And of course, heap requirement is increased when we do the compression, as we load the rows from the database. That includes both the metricIds to be compressed as well as the datapoints. - Micke On 10/11/2016 02:17 PM, Heiko W.Rupp wrote: > > Hi, > > On 11 Oct 2016, at 12:57, Michael Burman wrote: > > (running for two hours now). It's running between 147MB of memory > and 400MB. > > That is a 250MB difference. > Can you try doing a full-gc right before compression start > and right after end to > and measure the heap usage at those points > (before start and at end before gc) > so that we get an idea how much memory > the compression really takes? > > I started a 0.16 h-services at ~ 10:35, let it run until 11:13 > (first compression happened at 11) > then added 25 feeds - one every 2 minutes. > > The compression at 1pm went well, but the graph certainly > shows a much increased need for heap during that time. > > We may perhaps try to spread out the compression over more time and > thus trade time for memory > to get rid of this large memory usage peak. > > After the next crash, I will try to run with external DB for Inventory > to see how much this contributes > to the issue here. > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161011/04f8a04e/attachment.html From hrupp at redhat.com Wed Oct 12 02:07:40 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 12 Oct 2016 08:07:40 +0200 Subject: [Hawkular-dev] Inventory and postgres ? Message-ID: Lukas, I suspect we have an issue in inventory (or two) - when I run inventory as is with h2db, it works, but may or may not contribute (a lot) to the growth of the heap as seen in the "Hawkular-services and memory" thread. - When I try to run against postgres 9.5, the WildFly of the platform never is available inside inventory. I see tables in postgres, but the WF never shows. Also later on I got below error I am also a bit puzzled, that using postgres starts of a c3p0 connection pool instead of using what is already present in WildFly hawkular_1 | 20:11:37,309 WARN [org.hawkular.inventory.rest] (default task-26) RestEasy exception, : java.lang.IllegalArgumentException: A metric type with path '/t;hawkular/f;f38c6e77-6ee0-47da-a80a-bdac9a249457/mt;Singleton%20EJB%20Metrics~Wait%20Time' not found relative to '/t;hawkular/f;f38c6e77-6ee0-47da-a80a-bdac9a249457/r;Local~~/r;Local~%2Fdeployment%3Dhawkular-metrics.ear/r;Local~%2Fdeployment%3Dhawkular-metrics.ear%2Fsubdeployment%3Dhawkular-alerts.war/r;Local~%2Fdeployment%3Dhawkular-metrics.ear%2Fsubdeployment%3Dhawkular-alerts.war%2Fsubsystem%3Dejb3%2Fsingleton-bean%3DPartitionManagerImpl'. hawkular_1 | at org.hawkular.inventory.base.BaseMetrics$ReadWrite.wireUpNewEntity(BaseMetrics.java:79) hawkular_1 | at org.hawkular.inventory.base.BaseMetrics$ReadWrite.wireUpNewEntity(BaseMetrics.java:51) hawkular_1 | at org.hawkular.inventory.base.Mutator.doCreate(Mutator.java:168) hawkular_1 | at org.hawkular.inventory.base.Mutator.lambda$doCreate$95(Mutator.java:81) hawkular_1 | at org.hawkular.inventory.base.TransactionPayload$Committing.lambda$committing$44(TransactionPayload.java:34) hawkular_1 | at org.hawkular.inventory.base.Traversal.lambda$inCommittableTxWithNotifications$94(Traversal.java:119) hawkular_1 | at org.hawkular.inventory.base.Util.onFailureRetry(Util.java:110) hawkular_1 | at org.hawkular.inventory.base.Util.inCommittableTx(Util.java:81) hawkular_1 | at org.hawkular.inventory.base.Traversal.inCommittableTxWithNotifications(Traversal.java:118) hawkular_1 | at org.hawkular.inventory.base.Traversal.inTxWithNotifications(Traversal.java:91) hawkular_1 | at org.hawkular.inventory.base.Mutator.doCreate(Mutator.java:81) hawkular_1 | at org.hawkular.inventory.base.BaseMetrics$ReadWrite.create(BaseMetrics.java:122) hawkular_1 | at org.hawkular.inventory.base.BaseMetrics$ReadWrite.create(BaseMetrics.java:51) hawkular_1 | at org.hawkular.inventory.api.WriteInterface.create(WriteInterface.java:60) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher$1.visitMetric(SingleSyncedFetcher.java:313) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher$1.visitMetric(SingleSyncedFetcher.java:305) hawkular_1 | at org.hawkular.inventory.paths.ElementTypeVisitor.accept(ElementTypeVisitor.java:36) hawkular_1 | at org.hawkular.inventory.paths.Path$Segment.accept(Path.java:648) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:305) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:350) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:350) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.create(SingleSyncedFetcher.java:350) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.lambda$syncTrees$121(SingleSyncedFetcher.java:246) hawkular_1 | at java.lang.Iterable.forEach(Iterable.java:75) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.syncTrees(SingleSyncedFetcher.java:246) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.lambda$synchronize$119(SingleSyncedFetcher.java:123) hawkular_1 | at org.hawkular.inventory.base.TransactionPayload$Committing.lambda$committing$44(TransactionPayload.java:34) hawkular_1 | at org.hawkular.inventory.base.Traversal.lambda$inCommittableTx$93(Traversal.java:106) hawkular_1 | at org.hawkular.inventory.base.Util.onFailureRetry(Util.java:110) hawkular_1 | at org.hawkular.inventory.base.Util.inCommittableTx(Util.java:81) hawkular_1 | at org.hawkular.inventory.base.Traversal.inCommittableTx(Traversal.java:105) hawkular_1 | at org.hawkular.inventory.base.Traversal.inTx(Traversal.java:96) hawkular_1 | at org.hawkular.inventory.base.Traversal.inTx(Traversal.java:79) hawkular_1 | at org.hawkular.inventory.base.SingleSyncedFetcher.synchronize(SingleSyncedFetcher.java:93) hawkular_1 | at org.hawkular.inventory.base.BaseResources$Single.synchronize(BaseResources.java:206) hawkular_1 | at org.hawkular.inventory.rest.RestSync.sync(RestSync.java:80) hawkular_1 | at org.hawkular.inventory.rest.RestSync$Proxy$_$$_WeldClientProxy.sync(Unknown Source) hawkular_1 | at sun.reflect.GeneratedMethodAccessor108.invoke(Unknown Source) hawkular_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) hawkular_1 | at java.lang.reflect.Method.invoke(Method.java:498) hawkular_1 | at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) hawkular_1 | at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) hawkular_1 | at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) hawkular_1 | at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) hawkular_1 | at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:395) hawkular_1 | at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202) hawkular_1 | at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) hawkular_1 | at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) hawkular_1 | at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) hawkular_1 | at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) hawkular_1 | at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) hawkular_1 | at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) hawkular_1 | at io.undertow.websockets.jsr.JsrWebSocketFilter.doFilter(JsrWebSocketFilter.java:129) hawkular_1 | at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) hawkular_1 | at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) hawkular_1 | at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) hawkular_1 | at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) hawkular_1 | at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) hawkular_1 | at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) hawkular_1 | at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) hawkular_1 | at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) hawkular_1 | at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) hawkular_1 | at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) hawkular_1 | at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) hawkular_1 | at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51) hawkular_1 | at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) hawkular_1 | at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) hawkular_1 | at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56) hawkular_1 | at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) hawkular_1 | at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) hawkular_1 | at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) hawkular_1 | at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) hawkular_1 | at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) hawkular_1 | at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) hawkular_1 | at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) hawkular_1 | at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) hawkular_1 | at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284) hawkular_1 | at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263) hawkular_1 | at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) hawkular_1 | at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174) hawkular_1 | at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) hawkular_1 | at io.undertow.server.HttpServerExchun(HttpServerExchange.java:793) hawkular_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) hawkular_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) hawkular_1 | at java.lang.Thread.run(Thread.java:745) hawkular_1 | -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161012/b74e00f7/attachment-0001.html From mithomps at redhat.com Wed Oct 12 22:18:15 2016 From: mithomps at redhat.com (mike thompson) Date: Wed, 12 Oct 2016 19:18:15 -0700 Subject: [Hawkular-dev] MiQ log/middleware.log Message-ID: So currently, we aren?t really logging anything to this middleware.log (as far as I can tell). Should we be? What is our policy around using this log (versus evm.log)? This may be more important once we are in CFME and have to debug some customer issues. Should all of our logging goto this log? Should some? If so what? Sorry, just a bit confused about the purpose of this log since it shows empty for me. ? Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161012/b3c609db/attachment.html From mithomps at redhat.com Wed Oct 12 22:36:01 2016 From: mithomps at redhat.com (mike thompson) Date: Wed, 12 Oct 2016 19:36:01 -0700 Subject: [Hawkular-dev] MiQ log/middleware.log In-Reply-To: References: Message-ID: <918A3779-DFAF-4638-A71D-BA7C9629A712@redhat.com> > On Oct 12, 2016, at 7:18 PM, mike thompson wrote: > > So currently, we aren?t really logging anything to this middleware.log (as far as I can tell). Should we be? What is our policy around using this log (versus evm.log)? > This may be more important once we are in CFME and have to debug some customer issues. > > Should all of our logging goto this log? Should some? If so what? > > Sorry, just a bit confused about the purpose of this log since it shows empty for me. Seems to me we should be using it more. Since it much easier and relevant to look through this than the huge evm.log (which has also has worker thread traces every X seconds as well). > > > ? Mike > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161012/d0d3b902/attachment.html From theute at redhat.com Thu Oct 13 01:24:51 2016 From: theute at redhat.com (Thomas Heute) Date: Thu, 13 Oct 2016 07:24:51 +0200 Subject: [Hawkular-dev] MiQ log/middleware.log In-Reply-To: <918A3779-DFAF-4638-A71D-BA7C9629A712@redhat.com> References: <918A3779-DFAF-4638-A71D-BA7C9629A712@redhat.com> Message-ID: On Thu, Oct 13, 2016 at 4:36 AM, mike thompson wrote: > > On Oct 12, 2016, at 7:18 PM, mike thompson wrote: > > So currently, we aren?t really logging anything to this middleware.log > (as far as I can tell). Should we be? What is our policy around using this > log (versus evm.log)? > This may be more important once we are in CFME and have to debug some > customer issues. > > Should all of our logging goto this log? Should some? If so what? > > Sorry, just a bit confused about the purpose of this log since it shows > empty for me. > > > Seems to me we should be using it more. Since it much easier and relevant > to look through this than the huge evm.log (which has also has worker > thread traces every X seconds as well). > Same here, any idea how/when/by whom it was introduced ? Thomas > > > > ? Mike > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161013/77a034d5/attachment.html From jkremser at redhat.com Thu Oct 13 08:21:10 2016 From: jkremser at redhat.com (Jiri Kremser) Date: Thu, 13 Oct 2016 14:21:10 +0200 Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor Message-ID: Hello, today, I was on L&L about storage in OpenShift and I learn interesting thing. I always thought, that everything needs to be immutable and stateless and all the state needs to be handled by means of NFS persistent volumes. Luckily, there is a feature in Kubernetess (since 1.3) that allows the PODs to be treated as pets. It's called PetSet [1] and it assigns a unique ID (and persistent DNS record) to a POD that runs in this "mode". Common use-case for PetSet is a set of pods with a relational DBs that uses some kind of master-slave replication and slaves needs to know the master's address. But it can be used for anything. We can use the hostname as the feed id for instance. I don't know how much this will be popular because it kind of defeats the purpose of immutable infrastructure but it can save us some work with the feed identity. And of course we need to support also the "normal" POD scenario. [1]: http://kubernetes.io/docs/user-guide/petset/ jk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161013/378156aa/attachment.html From jshaughn at redhat.com Thu Oct 13 08:43:50 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 13 Oct 2016 08:43:50 -0400 Subject: [Hawkular-dev] MiQ log/middleware.log In-Reply-To: References: <918A3779-DFAF-4638-A71D-BA7C9629A712@redhat.com> Message-ID: <275b7831-35ef-378d-09b9-683e07d2bf0e@redhat.com> I added it way back in my first or second PR, I think. It's exactly what it looks like, a way for us to log stuff to a file that pertains only to our provider. I have a couple of log messages going there, like when there is a problem with the EventCatcher, I think. I do think it could be useful if used a bit more, when the information may be useful outside of, or in addition to, the main evm log. On 10/13/2016 1:24 AM, Thomas Heute wrote: > > > On Thu, Oct 13, 2016 at 4:36 AM, mike thompson > wrote: > > >> On Oct 12, 2016, at 7:18 PM, mike thompson > > wrote: >> >> So currently, we aren?t really logging anything to this >> middleware.log (as far as I can tell). Should we be? What is our >> policy around using this log (versus evm.log)? >> This may be more important once we are in CFME and have to debug >> some customer issues. >> >> Should all of our logging goto this log? Should some? If so what? >> >> Sorry, just a bit confused about the purpose of this log since it >> shows empty for me. > > Seems to me we should be using it more. Since it much easier and > relevant to look through this than the huge evm.log (which has > also has worker thread traces every X seconds as well). > > > Same here, any idea how/when/by whom it was introduced ? > > Thomas > > >> >> >> ? Mike >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161013/f67b4fa8/attachment-0001.html From gbrown at redhat.com Thu Oct 13 10:27:40 2016 From: gbrown at redhat.com (Gary Brown) Date: Thu, 13 Oct 2016 10:27:40 -0400 (EDT) Subject: [Hawkular-dev] Hawkular APM 0.11.0.Final now available In-Reply-To: <2007795173.1853369.1476368583918.JavaMail.zimbra@redhat.com> Message-ID: <258269481.1854074.1476368860614.JavaMail.zimbra@redhat.com> Hi The Hawkular APM team are pleased to announce the release of version 0.11.0.Final. The release details, including distributions, can be found here: https://github.com/hawkular/hawkular-apm/releases/tag/0.11.0.Final The release includes: * Improvements in the UI for displaying service dependency and trace instance information * Zipkin integration now includes Kafka support (with json and thrift encoded data) * Initial implementation of a Java opentracing provider * Integration with Hawkular Alerts to trigger alerts based on trace completion events Blogs and videos will follow in the next couple of days to demonstrate these capabilities. Regards Gary From hrupp at redhat.com Fri Oct 14 05:55:04 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 14 Oct 2016 11:55:04 +0200 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: References: Message-ID: <781ED7FD-EA4C-47A8-BB1C-AC2C72042350@redhat.com> On 11 Oct 2016, at 8:42, Heiko W.Rupp wrote: > Hey, > > tl;dr: we need to investigate heap usage - especially in the case of > compression > kicking in - it looks like there could be a memory leak. Compression > timing seems > mostly ok. Looks like we are making some progress - metric forwarding to alerts seems to do some memory-wise heavy work, that can be reduced (hwkmetrics-507 and -508 and hawkular-1130 iirc) - internal hsqldb uses a lot of memory - that is "only" a developer concern - using postgres does not exhibit the memory usage for caching as does hsqldb, but - the postgres driver in inventory (v...1201) seems to use finalizers too much - we are investigating to use a newer driver that may perform better I just replaced in a local copy inventory 0.20 with 0.19.3 and this seems to be smoother sailing - a lot less finalizers that also hold less memory. I did try to run with -XX:+UseStringDeduplication which requires -XX:+UseG1GC, but according to MAT it did not have an effect for me. We may want to look into string interning for the future like we did in JON. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161014/e4788ee3/attachment.html From ploffay at redhat.com Fri Oct 14 13:10:21 2016 From: ploffay at redhat.com (Pavol Loffay) Date: Fri, 14 Oct 2016 19:10:21 +0200 Subject: [Hawkular-dev] Hawkular APM Distributed Tracing of Polyglot Application using Zipkin Instrumentations Message-ID: Hello, there is a blog post about monitoring polyglot applications with Hawkular APM using Zipkin instrumentation libraries. Blog: http://www.hawkular.org/blog/2016/10/14/hawkular-apm- polyglot-zipkin.html Regards, -- Pavol Loffay -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161014/84c44bc2/attachment.html From hrupp at redhat.com Sat Oct 15 01:56:10 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Sat, 15 Oct 2016 07:56:10 +0200 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: <781ED7FD-EA4C-47A8-BB1C-AC2C72042350@redhat.com> References: <781ED7FD-EA4C-47A8-BB1C-AC2C72042350@redhat.com> Message-ID: <9AB7C1BC-C30B-4FCA-8DCB-DDABDC36D288@redhat.com> Hey, I ran this over night with 24 feeds and the patch to HAM wrt alert forwarding and it was stable. Had to stop it now, but will try to pick up where I left today. Anyway: good sign. Thanks to all On 14 Oct 2016, at 11:55, Heiko W.Rupp wrote: > On 11 Oct 2016, at 8:42, Heiko W.Rupp wrote: > >> Hey, >> >> tl;dr: we need to investigate heap usage - especially in the case of >> compression >> kicking in - it looks like there could be a memory leak. Compression >> timing seems >> mostly ok. > > Looks like we are making some progress > > - metric forwarding to alerts seems to do some memory-wise > heavy work, that can be reduced (hwkmetrics-507 and -508 and > hawkular-1130 iirc) > - internal hsqldb uses a lot of memory > - that is "only" a developer concern > - using postgres does not exhibit the memory usage for caching as does > hsqldb, but > - the postgres driver in inventory (v...1201) seems to use > finalizers too much > - we are investigating to use a newer driver that may perform > better > > I just replaced in a local copy inventory 0.20 with 0.19.3 and this > seems to be > smoother sailing - a lot less finalizers that also hold less memory. > > I did try to run with -XX:+UseStringDeduplication which requires > -XX:+UseG1GC, but > according to MAT it did not have an effect for me. We may want to look > into > string interning for the future like we did in JON. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161015/94ec0015/attachment.html From mazz at redhat.com Sun Oct 16 21:38:24 2016 From: mazz at redhat.com (John Mazzitelli) Date: Sun, 16 Oct 2016 21:38:24 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics go client missing string metric type In-Reply-To: <1467123544.1304984.1476668203454.JavaMail.zimbra@redhat.com> Message-ID: <561320724.1305082.1476668304337.JavaMail.zimbra@redhat.com> Looking here: https://github.com/hawkular/hawkular-client-go/blob/master/metrics/types.go#L80-L84 Shouldn't this have a "string" metric type, too? I assume this means the Hawkular Metrics Go client does not yet support string metrics. Unless this strange "generic" type means "string" ??? From mazz at redhat.com Mon Oct 17 01:19:58 2016 From: mazz at redhat.com (John Mazzitelli) Date: Mon, 17 Oct 2016 01:19:58 -0400 (EDT) Subject: [Hawkular-dev] gohawk - need Go code to write to H-Metrics In-Reply-To: <1810431103.1312108.1476680510449.JavaMail.zimbra@redhat.com> Message-ID: <899082861.1313227.1476681598227.JavaMail.zimbra@redhat.com> I am close to having GoHawk [1] be able to take flight :) He's still a fledgling, not quite ready to leave the nest yet.. but close. I could even demo what I got if some folks are interested in learning how GoHawk is configured (YAML!!!), seeing it react to changes to an Open Shift node environment on the fly, collecting Prometheus data, and mock-storing the metrics to h-Metrics. BUT! Right now I'm at the point where I need code that writes data to Hawkular Metrics from a Go client. Anyone have code that shows how to do this? This isn't code that QUERIES the H-Metrics for existing metric data - it is code that WRITES metrics to Hawkular Metrics. I already have a array of MetricHeader objects ([]metrics.MetricHeader) - I just need code that builds up the HTTP request and sends it (including any encryption/credential parameters/settings required?). [1] https://github.com/jmazzitelli/gohawk From miburman at redhat.com Mon Oct 17 02:49:06 2016 From: miburman at redhat.com (Michael Burman) Date: Mon, 17 Oct 2016 09:49:06 +0300 Subject: [Hawkular-dev] hawkular metrics go client missing string metric type In-Reply-To: <561320724.1305082.1476668304337.JavaMail.zimbra@redhat.com> References: <561320724.1305082.1476668304337.JavaMail.zimbra@redhat.com> Message-ID: Hi, No, there's no support for String type in the golang client. - Micke On 10/17/2016 04:38 AM, John Mazzitelli wrote: > Looking here: > > https://github.com/hawkular/hawkular-client-go/blob/master/metrics/types.go#L80-L84 > > Shouldn't this have a "string" metric type, too? > > I assume this means the Hawkular Metrics Go client does not yet support string metrics. > > Unless this strange "generic" type means "string" ??? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From miburman at redhat.com Mon Oct 17 02:56:47 2016 From: miburman at redhat.com (Michael Burman) Date: Mon, 17 Oct 2016 09:56:47 +0300 Subject: [Hawkular-dev] gohawk - need Go code to write to H-Metrics In-Reply-To: <899082861.1313227.1476681598227.JavaMail.zimbra@redhat.com> References: <899082861.1313227.1476681598227.JavaMail.zimbra@redhat.com> Message-ID: Hi, I'm not sure why I write documentation ;) " Writing datapoints" section says: err := c.Write([]MetricHeader{header}) So just use c.Write(yourMetricHeaderArray). If you initialized with correct settings (from the "Initialization " part) with NewHawkularClient(), you don't need to do anything else. SSL settings are provided by giving &tls.Config to it (standard way in Golang to set TLS settings). Unit test with TLS: https://github.com/hawkular/hawkular-client-go/blob/master/metrics/client_test.go#L267 And almost every unit test uses Write in client_test.go. Remember there's internal pooling in the Golang client, so if you need concurrency, don't be afraid to spawn goroutines and let the client take care of the concurrency limits to the server (adjustable with Parameters in initialization). - Micke On 10/17/2016 08:19 AM, John Mazzitelli wrote: > I am close to having GoHawk [1] be able to take flight :) He's still a fledgling, not quite ready to leave the nest yet.. but close. I could even demo what I got if some folks are interested in learning how GoHawk is configured (YAML!!!), seeing it react to changes to an Open Shift node environment on the fly, collecting Prometheus data, and mock-storing the metrics to h-Metrics. > > BUT! Right now I'm at the point where I need code that writes data to Hawkular Metrics from a Go client. Anyone have code that shows how to do this? This isn't code that QUERIES the H-Metrics for existing metric data - it is code that WRITES metrics to Hawkular Metrics. I already have a array of MetricHeader objects ([]metrics.MetricHeader) - I just need code that builds up the HTTP request and sends it (including any encryption/credential parameters/settings required?). > > [1] https://github.com/jmazzitelli/gohawk > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jmazzite at redhat.com Mon Oct 17 11:11:37 2016 From: jmazzite at redhat.com (jmazzite at redhat.com) Date: Mon, 17 Oct 2016 15:11:37 +0000 Subject: [Hawkular-dev] Invitation: review GoHawk PoC as it stands now @ Tue Oct 18, 2016 12pm - 12:30pm (EDT) (Hawkular) Message-ID: You have been invited to the following event. Title: review GoHawk PoC as it stands now Just a quick 30 minute review to go over what we've got so far to help monitor things (Prometheus, Jolokia) in Open Shift (and outside of Open Shift) using the Go agent. Source and README is here: https://github.com/jmazzitelli/gohawk Let me know if this time is OK. I tried putting it for a time where it's not on anyone's calendar that I could see. Thomas - it came up with a warning saying its outside of your normal working hours :) Let me know if you want to move it up, say, 30 minutes. Though Matt has a meeting then, so we'd need his input. When: Tue Oct 18, 2016 12pm ? 12:30pm Eastern Time Where: https://bluejeans.com/jmazzite Calendar: Hawkular Who: * jmazzite at redhat.com - creator * theute at redhat.com * mwringe at redhat.com * Jay Shaughnessy * hrupp at redhat.com * hawkular-dev at lists.jboss.org - optional Your attendance is optional. Event details: https://www.google.com/calendar/event?action=VIEW&eid=c2RkOG12OGFxZjVkcG5lOTdpYTduZzFqZGcgaGF3a3VsYXItZGV2QGxpc3RzLmpib3NzLm9yZw&tok=NjMjcmVkaGF0LmNvbV9mbWlnMm9zdTY5a21hNDdqcmRjMnZlbjRtb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29tOWFlNzg2ZGNhYzYzZmMxZDUzYTE1YTIwOTc1YmEyZGNjMTVjNDFiOQ&ctz=America/New_York&hl=en Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account hawkular-dev at lists.jboss.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161017/110cc26a/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2298 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161017/110cc26a/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 2346 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161017/110cc26a/attachment-0001.bin From mazz at redhat.com Tue Oct 18 13:08:35 2016 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 18 Oct 2016 13:08:35 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> Message-ID: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. --John Mazz P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? From mithomps at redhat.com Tue Oct 18 13:17:25 2016 From: mithomps at redhat.com (mike thompson) Date: Tue, 18 Oct 2016 10:17:25 -0700 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: ?kubehawk?, ?kubular? or ?hawkshift? seem obvious? ? Mike > On Oct 18, 2016, at 10:08 AM, John Mazzitelli wrote: > > OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. > > We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) > > We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). > > "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jshaughn at redhat.com Tue Oct 18 13:32:21 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Tue, 18 Oct 2016 13:32:21 -0400 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <3c6b1203-a7a1-c399-2910-97f72034cac7@redhat.com> Hawkster :) On 10/18/2016 1:17 PM, mike thompson wrote: > ?kubehawk?, ?kubular? or ?hawkshift? seem obvious? > > ? Mike > > >> On Oct 18, 2016, at 10:08 AM, John Mazzitelli wrote: >> >> OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. >> >> We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) >> >> We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? >> >> Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. >> >> I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" >> >> I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). >> >> "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? >> >> I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. >> >> --John Mazz >> >> P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161018/b63cfab3/attachment.html From mithomps at redhat.com Tue Oct 18 14:04:13 2016 From: mithomps at redhat.com (mike thompson) Date: Tue, 18 Oct 2016 11:04:13 -0700 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <3c6b1203-a7a1-c399-2910-97f72034cac7@redhat.com> References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <3c6b1203-a7a1-c399-2910-97f72034cac7@redhat.com> Message-ID: > On Oct 18, 2016, at 10:32 AM, Jay Shaughnessy wrote: > > > Hawkster :) Mazz was too afraid to send this: ?Mazzster? > > On 10/18/2016 1:17 PM, mike thompson wrote: >> ?kubehawk?, ?kubular? or ?hawkshift? seem obvious? >> >> ? Mike >> >> >>> On Oct 18, 2016, at 10:08 AM, John Mazzitelli wrote: >>> >>> OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. >>> >>> We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) >>> >>> We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? >>> >>> Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. >>> >>> I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" >>> >>> I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). >>> >>> "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? >>> >>> I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. >>> >>> --John Mazz >>> >>> P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161018/d4585986/attachment.html From snegrea at redhat.com Tue Oct 18 15:19:37 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 18 Oct 2016 14:19:37 -0500 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <3c6b1203-a7a1-c399-2910-97f72034cac7@redhat.com> Message-ID: How about naming the project Hawkular KMon? Thank you, Stefan Negrea On Tue, Oct 18, 2016 at 1:04 PM, mike thompson wrote: > > On Oct 18, 2016, at 10:32 AM, Jay Shaughnessy wrote: > > > Hawkster :) > > > Mazz was too afraid to send this: ?Mazzster? > > > > On 10/18/2016 1:17 PM, mike thompson wrote: > > ?kubehawk?, ?kubular? or ?hawkshift? seem obvious? > > ? Mike > > > > On Oct 18, 2016, at 10:08 AM, John Mazzitelli wrote: > > OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. > > We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) > > We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). > > "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing listhawkular-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing listhawkular-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161018/39d55c15/attachment.html From jdoyle at redhat.com Tue Oct 18 15:38:11 2016 From: jdoyle at redhat.com (John Doyle) Date: Tue, 18 Oct 2016 15:38:11 -0400 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <3c6b1203-a7a1-c399-2910-97f72034cac7@redhat.com> Message-ID: On Tue, Oct 18, 2016 at 2:04 PM, mike thompson wrote: > > On Oct 18, 2016, at 10:32 AM, Jay Shaughnessy wrote: > > > Hawkster :) > > > Mazz was too afraid to send this: ?Mazzster? > I'm going to guess that one does not make it past the PC police. > > > > On 10/18/2016 1:17 PM, mike thompson wrote: > > ?kubehawk?, ?kubular? or ?hawkshift? seem obvious? > > ? Mike > > > > On Oct 18, 2016, at 10:08 AM, John Mazzitelli wrote: > > OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. > > We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) > > We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). > > "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing listhawkular-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing listhawkular-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161018/49a7fa17/attachment-0001.html From jdoyle at redhat.com Tue Oct 18 15:41:28 2016 From: jdoyle at redhat.com (John Doyle) Date: Tue, 18 Oct 2016 15:41:28 -0400 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: I actually like "hawkulark". Why have only one bird when you can have two. I'm sure to repeatedly misspell it, but that's true of most things I type. ~jd On Tue, Oct 18, 2016 at 1:08 PM, John Mazzitelli wrote: > OK, folks, as much as I hate these "what should we name this thing?" > threads, I have to do it. > > We are at the point where we are going to start going full-throttle on > building out an agent that can monitor things on Open Shift (and Heiko > wants to be able to monitor things outside of Open Shift - I'll let him > chime in on what his use cases are to get a better feel for what he's > thinking) > > We need a name ASAP so we can create a repository under the Hawkular > github namespace and put the code up there so people can start working on > it. I would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == > kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented > in "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its > really targeting Open Shift and even then doesn't have to run in Open Shift > (back to the ideas Heiko has for this thing). > > "GoHawk" doesn't seem to be a winner simply because what happens if we > implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a > group - but I nominate Thomas H, Heiko R, and John D. as the committee to > give the final approval/tie-breaking authority :) It won't be me. I suck at > coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you > can enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161018/a3a734c7/attachment.html From mazz at redhat.com Tue Oct 18 16:30:15 2016 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 18 Oct 2016 16:30:15 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <858270072.373687.1476822615549.JavaMail.zimbra@redhat.com> Here's what we got in alphabetical order - and I can't believe I'm typing some of these :p GoHawk Hawkshift Hawkster Hawkulark KMon Kubehawk Kubular We need more. All of these smell. You people disappoint me :) Where's Thomas S ??? We need him. Who knows how to create one of those online surveys? From mazz at redhat.com Tue Oct 18 16:54:25 2016 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 18 Oct 2016 16:54:25 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <858270072.373687.1476822615549.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <858270072.373687.1476822615549.JavaMail.zimbra@redhat.com> Message-ID: <1168473379.375610.1476824065579.JavaMail.zimbra@redhat.com> I hate these polls websites, but whatever: http://doodle.com/poll/4xky52qwzdszrpz4 I want this done by Thursday. If you want me to add something to the list let me know. I can't find a way to tell doodle to let a user add their own option. So go vote. You won't feel dirty pulling the lever for whatever you pick, unlike our presidential choices. From theute at redhat.com Wed Oct 19 03:20:54 2016 From: theute at redhat.com (Thomas Heute) Date: Wed, 19 Oct 2016 09:20:54 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: Personally I would vote for: - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" and reduce its scope to the embedded WF scenario (+ remote for domains). Small in scope == easier to maintain, document, understand... - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift Agent" if it really depends on OpenShift (but I'm not sure why), because the discovery is a key feature and I guess (will) use Kubernetes API. I can understand this could be extended to other usecases that don't involve WildFly or Kubernetes, but I'm afraid that we just make things more complex/harder to understand while those 2 targets are key. PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark (and in theory requires legal implication) Thomas On Tue, Oct 18, 2016 at 7:08 PM, John Mazzitelli wrote: > OK, folks, as much as I hate these "what should we name this thing?" > threads, I have to do it. > > We are at the point where we are going to start going full-throttle on > building out an agent that can monitor things on Open Shift (and Heiko > wants to be able to monitor things outside of Open Shift - I'll let him > chime in on what his use cases are to get a better feel for what he's > thinking) > > We need a name ASAP so we can create a repository under the Hawkular > github namespace and put the code up there so people can start working on > it. I would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == > kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented > in "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its > really targeting Open Shift and even then doesn't have to run in Open Shift > (back to the ideas Heiko has for this thing). > > "GoHawk" doesn't seem to be a winner simply because what happens if we > implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a > group - but I nominate Thomas H, Heiko R, and John D. as the committee to > give the final approval/tie-breaking authority :) It won't be me. I suck at > coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you > can enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/27dbd3df/attachment.html From theute at redhat.com Wed Oct 19 03:39:57 2016 From: theute at redhat.com (Thomas Heute) Date: Wed, 19 Oct 2016 09:39:57 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: I would really hate to have: - Hawkular Agent (is our current name for the Wildfly agent) - Hawkulark or hawkster or hawkshift... for this Kuber/Openshift one - hawkraf or karhawk if we ever build an embedded karaf agent - hawkdroid andrhawk when someone contributes an Android monitoring agent - ... On Wed, Oct 19, 2016 at 9:20 AM, Thomas Heute wrote: > Personally I would vote for: > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" > and reduce its scope to the embedded WF scenario (+ remote for domains). > Small in scope == easier to maintain, document, understand... > - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift > Agent" if it really depends on OpenShift (but I'm not sure why), because > the discovery is a key feature and I guess (will) use Kubernetes API. > > I can understand this could be extended to other usecases that don't > involve WildFly or Kubernetes, but I'm afraid that we just make things more > complex/harder to understand while those 2 targets are key. > > PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark > (and in theory requires legal implication) > > Thomas > > On Tue, Oct 18, 2016 at 7:08 PM, John Mazzitelli wrote: > >> OK, folks, as much as I hate these "what should we name this thing?" >> threads, I have to do it. >> >> We are at the point where we are going to start going full-throttle on >> building out an agent that can monitor things on Open Shift (and Heiko >> wants to be able to monitor things outside of Open Shift - I'll let him >> chime in on what his use cases are to get a better feel for what he's >> thinking) >> >> We need a name ASAP so we can create a repository under the Hawkular >> github namespace and put the code up there so people can start working on >> it. I would like to do this sooner rather than later - say, by Thursday??? >> >> Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == >> kubernetes. >> >> I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented >> in "Go" >> >> I wasn't keen on relying on "kubernetes" as part of the name since its >> really targeting Open Shift and even then doesn't have to run in Open Shift >> (back to the ideas Heiko has for this thing). >> >> "GoHawk" doesn't seem to be a winner simply because what happens if we >> implement other hawkular feeds in Golang? >> >> I'm assuming we'll come up with a name and agree to it collectively as a >> group - but I nominate Thomas H, Heiko R, and John D. as the committee to >> give the final approval/tie-breaking authority :) It won't be me. I suck at >> coming up with names. >> >> --John Mazz >> >> P.S. Who knows how to setup one of those online polls/surveys where you >> can enter your submissions and vote for other submissions? >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/ce8d7481/attachment-0001.html From hrupp at redhat.com Wed Oct 19 03:58:49 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 19 Oct 2016 09:58:49 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <5BA401EB-85ED-4D3E-9206-1F77322151F8@redhat.com> On 19 Oct 2016, at 9:20, Thomas Heute wrote: > Personally I would vote for: > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" and > reduce its scope to the embedded WF scenario (+ remote for domains). Small > in scope == easier to maintain, document, understand... > - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift > Agent" if it really depends on OpenShift (but I'm not sure +1 > PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark > (and in theory requires legal implication) The binary certainly needs one - but something like hawkagent should be good enough here. From jtakvori at redhat.com Wed Oct 19 04:06:41 2016 From: jtakvori at redhat.com (Joel Takvorian) Date: Wed, 19 Oct 2016 10:06:41 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <5BA401EB-85ED-4D3E-9206-1F77322151F8@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <5BA401EB-85ED-4D3E-9206-1F77322151F8@redhat.com> Message-ID: Other cryptic names we won't use anyway: hawkenshift (sonority close to openshift), variant: Hawk'n Shift But +1 for the desperately serious "Hawkular Kubernetes/OpenShift Agent" and renaming Hawkular Agent :) On Wed, Oct 19, 2016 at 9:58 AM, Heiko W.Rupp wrote: > On 19 Oct 2016, at 9:20, Thomas Heute wrote: > > > Personally I would vote for: > > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" > and > > reduce its scope to the embedded WF scenario (+ remote for domains). > Small > > in scope == easier to maintain, document, understand... > > - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift > > Agent" if it really depends on OpenShift (but I'm not sure > > +1 > > > PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark > > (and in theory requires legal implication) > > The binary certainly needs one - but something like hawkagent > should be good enough here. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/fc06d73d/attachment.html From jshaughn at redhat.com Wed Oct 19 08:12:18 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 19 Oct 2016 08:12:18 -0400 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <5BA401EB-85ED-4D3E-9206-1F77322151F8@redhat.com> Message-ID: <9b3e5c8b-872e-18f0-4abe-bc7a14464ab8@redhat.com> Yes, hawkster was meant as a joke with a nod to heapster, I am +1 to a series of 'Hawkular * Agent' namings. Although, one more just for fun, hawkfeed! On 10/19/2016 4:06 AM, Joel Takvorian wrote: > Other cryptic names we won't use anyway: hawkenshift (sonority close > to openshift), variant: Hawk'n Shift > > But +1 for the desperately serious "Hawkular Kubernetes/OpenShift > Agent" and renaming Hawkular Agent :) > > On Wed, Oct 19, 2016 at 9:58 AM, Heiko W.Rupp > wrote: > > On 19 Oct 2016, at 9:20, Thomas Heute wrote: > > > Personally I would vote for: > > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly > Agent" and > > reduce its scope to the embedded WF scenario (+ remote for > domains). Small > > in scope == easier to maintain, document, understand... > > - Name this one "Hawkular Kubernetes Agent", or "Hawkular > OpenShift > > Agent" if it really depends on OpenShift (but I'm not sure > > +1 > > > PS: I don't think we need yet another cryptic name as GoHawk / > Hawkulark > > (and in theory requires legal implication) > > The binary certainly needs one - but something like hawkagent > should be good enough here. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/6f7cca7f/attachment.html From jkremser at redhat.com Wed Oct 19 08:37:24 2016 From: jkremser at redhat.com (Jiri Kremser) Date: Wed, 19 Oct 2016 14:37:24 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <5BA401EB-85ED-4D3E-9206-1F77322151F8@redhat.com> Message-ID: Hello, It's hard to come up with some name that fits well with the hawk and ocular metaphor. I like what the ELK stack has, they have units called beats (it evokes some periodicity to me and also it sounds like a bee). imho, "bee" is the best name for a lightweight agent that collect metrics, but it doesn't work for us. What about having multiple agents called "ocular" or okular. Like OpenShift Okular, WildFly Okular, ${you name it} Okular? Although, flying hawk w/ multiple oculars imply the pull architecture. Hawkular X Agent isn't bad, but it's too long and too enterprisy, worse is only Red Hat Hawkular OpenShift Agent :] jk On Wed, Oct 19, 2016 at 10:06 AM, Joel Takvorian wrote: > Other cryptic names we won't use anyway: hawkenshift (sonority close to > openshift), variant: Hawk'n Shift > > But +1 for the desperately serious "Hawkular Kubernetes/OpenShift Agent" > and renaming Hawkular Agent :) > > On Wed, Oct 19, 2016 at 9:58 AM, Heiko W.Rupp wrote: > >> On 19 Oct 2016, at 9:20, Thomas Heute wrote: >> >> > Personally I would vote for: >> > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" >> and >> > reduce its scope to the embedded WF scenario (+ remote for domains). >> Small >> > in scope == easier to maintain, document, understand... >> > - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift >> > Agent" if it really depends on OpenShift (but I'm not sure >> >> +1 >> >> > PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark >> > (and in theory requires legal implication) >> >> The binary certainly needs one - but something like hawkagent >> should be good enough here. >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/3bb5b564/attachment.html From vnguyen at redhat.com Wed Oct 19 08:52:02 2016 From: vnguyen at redhat.com (Viet Nguyen) Date: Wed, 19 Oct 2016 08:52:02 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <1701046791.2370744.1476881522270.JavaMail.zimbra@redhat.com> We can also try a name that aligns with our upstream parent projects, Origin and Atomic. I can't think of any at the moment :) Viet ----- Original Message ----- From: "John Mazzitelli" To: "Discussions around Hawkular development" Sent: Tuesday, October 18, 2016 7:08:35 PM Subject: [Hawkular-dev] open shift agent - what to call it? OK, folks, as much as I hate these "what should we name this thing?" threads, I have to do it. We are at the point where we are going to start going full-throttle on building out an agent that can monitor things on Open Shift (and Heiko wants to be able to monitor things outside of Open Shift - I'll let him chime in on what his use cases are to get a better feel for what he's thinking) We need a name ASAP so we can create a repository under the Hawkular github namespace and put the code up there so people can start working on it. I would like to do this sooner rather than later - say, by Thursday??? Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == kubernetes. I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in "Go" I wasn't keen on relying on "kubernetes" as part of the name since its really targeting Open Shift and even then doesn't have to run in Open Shift (back to the ideas Heiko has for this thing). "GoHawk" doesn't seem to be a winner simply because what happens if we implement other hawkular feeds in Golang? I'm assuming we'll come up with a name and agree to it collectively as a group - but I nominate Thomas H, Heiko R, and John D. as the committee to give the final approval/tie-breaking authority :) It won't be me. I suck at coming up with names. --John Mazz P.S. Who knows how to setup one of those online polls/surveys where you can enter your submissions and vote for other submissions? _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From mwringe at redhat.com Wed Oct 19 09:19:45 2016 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 19 Oct 2016 09:19:45 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <8661226.4979782.1476883185149.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Tuesday, 18 October, 2016 1:08:35 PM > Subject: [Hawkular-dev] open shift agent - what to call it? > > OK, folks, as much as I hate these "what should we name this thing?" threads, > I have to do it. > > We are at the point where we are going to start going full-throttle on > building out an agent that can monitor things on Open Shift (and Heiko wants > to be able to monitor things outside of Open Shift - I'll let him chime in > on what his use cases are to get a better feel for what he's thinking) > > We need a name ASAP so we can create a repository under the Hawkular github > namespace and put the code up there so people can start working on it. I > would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == > kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in > "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its really > targeting Open Shift and even then doesn't have to run in Open Shift (back > to the ideas Heiko has for this thing). I am not sure if I like the idea of one agent handling multiple scenarios, I think it makes sense to keep them as separate agents. I do think it makes sense to share a common go library though, so that we can avoid code duplication. But the end result would still be separate agents. Eg the OpenShift agent would have code to connect to monitor the OpenShift cluster, figure out where the endpoints are, add in the tags, determine the schedule etc and then would just pass this information to a common go lang agent library to actually fetch the metrics and send to Hawkular Metrics. > > "GoHawk" doesn't seem to be a winner simply because what happens if we > implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a > group - but I nominate Thomas H, Heiko R, and John D. as the committee to > give the final approval/tie-breaking authority :) It won't be me. I suck at > coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you can > enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Wed Oct 19 09:22:48 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 09:22:48 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <8661226.4979782.1476883185149.JavaMail.zimbra@redhat.com> References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <8661226.4979782.1476883185149.JavaMail.zimbra@redhat.com> Message-ID: <1668362555.509440.1476883368197.JavaMail.zimbra@redhat.com> > then would just pass this information to a common go lang > agent library to actually fetch the metrics and send to Hawkular Metrics. We have part of that already - Micke's hawkular-client-go library sends to Hawkular Metrics. We could split out the prometheus scraping code that currently exists in the Go agent for a "fetch the metrics" library. From mazz at redhat.com Wed Oct 19 09:24:52 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 09:24:52 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be named: hawkular/hawkular-kubernetes-agent Yes? ----- Original Message ----- > Personally I would vote for: > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" and > reduce its scope to the embedded WF scenario (+ remote for domains). Small > in scope == easier to maintain, document, understand... > - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift > Agent" if it really depends on OpenShift (but I'm not sure why), because > the discovery is a key feature and I guess (will) use Kubernetes API. > > I can understand this could be extended to other usecases that don't > involve WildFly or Kubernetes, but I'm afraid that we just make things more > complex/harder to understand while those 2 targets are key. > > PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark > (and in theory requires legal implication) > > Thomas > > On Tue, Oct 18, 2016 at 7:08 PM, John Mazzitelli wrote: > > > OK, folks, as much as I hate these "what should we name this thing?" > > threads, I have to do it. > > > > We are at the point where we are going to start going full-throttle on > > building out an agent that can monitor things on Open Shift (and Heiko > > wants to be able to monitor things outside of Open Shift - I'll let him > > chime in on what his use cases are to get a better feel for what he's > > thinking) > > > > We need a name ASAP so we can create a repository under the Hawkular > > github namespace and put the code up there so people can start working on > > it. I would like to do this sooner rather than later - say, by Thursday??? > > > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == > > kubernetes. > > > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented > > in "Go" > > > > I wasn't keen on relying on "kubernetes" as part of the name since its > > really targeting Open Shift and even then doesn't have to run in Open Shift > > (back to the ideas Heiko has for this thing). > > > > "GoHawk" doesn't seem to be a winner simply because what happens if we > > implement other hawkular feeds in Golang? > > > > I'm assuming we'll come up with a name and agree to it collectively as a > > group - but I nominate Thomas H, Heiko R, and John D. as the committee to > > give the final approval/tie-breaking authority :) It won't be me. I suck at > > coming up with names. > > > > --John Mazz > > > > P.S. Who knows how to setup one of those online polls/surveys where you > > can enter your submissions and vote for other submissions? > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > > From mazz at redhat.com Wed Oct 19 09:28:07 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 09:28:07 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> Message-ID: <320854777.510327.1476883687714.JavaMail.zimbra@redhat.com> > > Personally I would vote for: > > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" and > > reduce its scope to the embedded WF scenario (+ remote for domains). Small > > in scope == easier to maintain, document, understand... For the record, that has been what I've been calling that agent for a while now. "Hawkular WildFly Agent". The git repo isn't called that, but the internals are - e.g. https://github.com/hawkular/hawkular-agent/tree/master/hawkular-wildfly-agent The "hawkular-agent" repo was created and named as such with the naive thought that all hawkular feeds/agents would go under that one big umbrella repo. But that ended up being a stupid idea :) From mwringe at redhat.com Wed Oct 19 09:29:31 2016 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 19 Oct 2016 09:29:31 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> Message-ID: <1321051243.4982655.1476883771494.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Thomas Heute" > To: "John Mazzitelli" , "Discussions around Hawkular development" > Sent: Wednesday, 19 October, 2016 3:20:54 AM > Subject: Re: [Hawkular-dev] open shift agent - what to call it? > > Personally I would vote for: > - Renaming the existing "Hawkular Agent" to "Hawkular WildFly Agent" and > reduce its scope to the embedded WF scenario (+ remote for domains). Small > in scope == easier to maintain, document, understand... +1 > - Name this one "Hawkular Kubernetes Agent", or "Hawkular OpenShift Agent" +1 > if > it really depends on OpenShift (but I'm not sure why), because the discovery > is a key feature and I guess (will) use Kubernetes API. This is a bit interesting. The agent itself should only be using the Kubernetes API and shouldn't have anything specific to OpenShift. So the agent here shouldn't care or know if its really connecting to Kubernetes or OpenShift. The problem is more that the Hawkular Metrics integration we have with OpenShift wont work in plain Kubernetes and we don't test it there. It would be possible to make it work on Kubernetes, but it wouldn't be a trivial change and would take some effort and testing to get it working there. We don't have the spare cycles or manpower to support the Kubernetes use case right now and this isn't even on our radar. It might be cool to have something usable with the upstream Kubernetes community though. > > I can understand this could be extended to other usecases that don't involve > WildFly or Kubernetes, but I'm afraid that we just make things more > complex/harder to understand while those 2 targets are key. > > PS: I don't think we need yet another cryptic name as GoHawk / Hawkulark (and > in theory requires legal implication) Yeah, they are cool for codenames, but I don't think we want to go through the whole legal stuff for this. And as already mentioned, when we start to have more agents, its going to be confusing over each one of them having unique names. > > Thomas > > On Tue, Oct 18, 2016 at 7:08 PM, John Mazzitelli < mazz at redhat.com > wrote: > > > OK, folks, as much as I hate these "what should we name this thing?" threads, > I have to do it. > > We are at the point where we are going to start going full-throttle on > building out an agent that can monitor things on Open Shift (and Heiko wants > to be able to monitor things outside of Open Shift - I'll let him chime in > on what his use cases are to get a better feel for what he's thinking) > > We need a name ASAP so we can create a repository under the Hawkular github > namespace and put the code up there so people can start working on it. I > would like to do this sooner rather than later - say, by Thursday??? > > Matt was thinking "hawkulark" (Hawk-U-Lark, Hawkular-K) because "k" == > kubernetes. > > I was thinking "GoHawk" (rhymes with "mohawk") because it is implemented in > "Go" > > I wasn't keen on relying on "kubernetes" as part of the name since its really > targeting Open Shift and even then doesn't have to run in Open Shift (back > to the ideas Heiko has for this thing). > > "GoHawk" doesn't seem to be a winner simply because what happens if we > implement other hawkular feeds in Golang? > > I'm assuming we'll come up with a name and agree to it collectively as a > group - but I nominate Thomas H, Heiko R, and John D. as the committee to > give the final approval/tie-breaking authority :) It won't be me. I suck at > coming up with names. > > --John Mazz > > P.S. Who knows how to setup one of those online polls/surveys where you can > enter your submissions and vote for other submissions? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From hrupp at redhat.com Wed Oct 19 09:32:01 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 19 Oct 2016 15:32:01 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <8661226.4979782.1476883185149.JavaMail.zimbra@redhat.com> References: <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <8661226.4979782.1476883185149.JavaMail.zimbra@redhat.com> Message-ID: On 19 Oct 2016, at 15:19, Matt Wringe wrote: >> I wasn't keen on relying on "kubernetes" as part of the name since >> its really >> targeting Open Shift and even then doesn't have to run in Open Shift >> (back >> to the ideas Heiko has for this thing). > > I am not sure if I like the idea of one agent handling multiple > scenarios, I think it makes sense to keep them as separate agents. My thinking went along the lines, that it should be possible to test it and/or the interaction with the monitored resource without the need for a full (mini)kube/shift. As you say, we can also have a "library" for interaction with hawkular and the monitored resource and then build multiple apps on top that talk to k8s or OS or just run on a 'normal' command line From hrupp at redhat.com Wed Oct 19 09:32:31 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 19 Oct 2016 15:32:31 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> Message-ID: On 19 Oct 2016, at 15:24, John Mazzitelli wrote: > OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be named: > > hawkular/hawkular-kubernetes-agent Sounds good to me From mazz at redhat.com Wed Oct 19 09:39:46 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 09:39:46 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> Message-ID: <987401426.513214.1476884386129.JavaMail.zimbra@redhat.com> > > OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be named: > > > > hawkular/hawkular-kubernetes-agent > > Sounds good to me And the executable name? The name we call it in code and in logging messages? Based on what people are saying, I'm just going to call the executable "hawkular-kubernetes-agent" and have for example "hawkular-kubernetes-agent.go" and when logging needs to name it call it "Hawkular Kubernetes Agent" This is what I was getting at - I wasn't asking for the "product name" (we can call it anything we want and it doesn't affect me at this point in time :) but what are we to call it in the code, for logging messages, executable names, github repo, etc. I'm fine with "hawkular-kubernetes-agent" and everything that entails if that is what people want. From jshaughn at redhat.com Wed Oct 19 09:40:45 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 19 Oct 2016 09:40:45 -0400 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <987401426.513214.1476884386129.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> <987401426.513214.1476884386129.JavaMail.zimbra@redhat.com> Message-ID: <80fb97a0-df31-7564-8e20-b49ff12f8de1@redhat.com> +1 to the verbose naming, it's totally clear if we just do it that way. On 10/19/2016 9:39 AM, John Mazzitelli wrote: >>> OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be named: >>> >>> hawkular/hawkular-kubernetes-agent >> Sounds good to me > And the executable name? The name we call it in code and in logging messages? > > Based on what people are saying, I'm just going to call the executable "hawkular-kubernetes-agent" and have for example "hawkular-kubernetes-agent.go" and when logging needs to name it call it "Hawkular Kubernetes Agent" > > This is what I was getting at - I wasn't asking for the "product name" (we can call it anything we want and it doesn't affect me at this point in time :) but what are we to call it in the code, for logging messages, executable names, github repo, etc. I'm fine with "hawkular-kubernetes-agent" and everything that entails if that is what people want. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/f269b103/attachment-0001.html From mwringe at redhat.com Wed Oct 19 09:49:56 2016 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 19 Oct 2016 09:49:56 -0400 (EDT) Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor In-Reply-To: References: Message-ID: <2057740863.5014134.1476884996253.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Jiri Kremser" > To: "Discussions around Hawkular development" > Sent: Thursday, 13 October, 2016 8:21:10 AM > Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > Hello, > today, I was on L&L about storage in OpenShift and I learn interesting thing. > I always thought, that everything needs to be immutable and stateless and > all the state needs to be handled by means of NFS persistent volumes. > Luckily, there is a feature in Kubernetess (since 1.3) that allows the PODs > to be treated as pets. It's called PetSet [1] and it assigns a unique ID > (and persistent DNS record) to a POD that runs in this "mode". For OpenShift, we would have moved to using PetSets for our Cassandra pod, but its not a fully supported feature yet. In the next version we will be moving over to using it. It will make changing the cluster size for Cassandra nodes a lot easier once we can use this. > > Common use-case for PetSet is a set of pods with a relational DBs that uses > some kind of master-slave replication and slaves needs to know the master's > address. But it can be used for anything. We can use the hostname as the > feed id for instance. > > I don't know how much this will be popular because it kind of defeats the > purpose of immutable infrastructure but it can save us some work with the > feed identity. And of course we need to support also the "normal" POD > scenario. PetSets are meant for clustering of pods which have persistent storage. If this is not your use case, what exactly are you trying to do? There may be better ways to handle it. > > [1]: http://kubernetes.io/docs/user-guide/petset/ > > jk > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From theute at redhat.com Wed Oct 19 09:51:10 2016 From: theute at redhat.com (Thomas Heute) Date: Wed, 19 Oct 2016 15:51:10 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> Message-ID: +1 on the format and hawkular/hawkular-wildfly-agent (renaming sucks but we're still young enough to allow this) But we still need to decide betwen Openshift vs Kubernetes. On Wed, Oct 19, 2016 at 3:32 PM, Heiko W.Rupp wrote: > On 19 Oct 2016, at 15:24, John Mazzitelli wrote: > > > OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be > named: > > > > hawkular/hawkular-kubernetes-agent > > Sounds good to me > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/bf7be37e/attachment.html From theute at redhat.com Wed Oct 19 09:55:32 2016 From: theute at redhat.com (Thomas Heute) Date: Wed, 19 Oct 2016 15:55:32 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> Message-ID: So assume Hawkular Metrics is running "somewhere", is the agent itself dependent on OpenShift or Kubernetes ? On Wed, Oct 19, 2016 at 3:51 PM, Thomas Heute wrote: > +1 on the format and hawkular/hawkular-wildfly-agent (renaming sucks but > we're still young enough to allow this) > > But we still need to decide betwen Openshift vs Kubernetes. > > > > On Wed, Oct 19, 2016 at 3:32 PM, Heiko W.Rupp wrote: > >> On 19 Oct 2016, at 15:24, John Mazzitelli wrote: >> >> > OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be >> named: >> > >> > hawkular/hawkular-kubernetes-agent >> >> Sounds good to me >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/ce66ed35/attachment.html From mazz at redhat.com Wed Oct 19 10:19:44 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 10:19:44 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> Message-ID: <1247626380.536307.1476886784247.JavaMail.zimbra@redhat.com> As far as I can tell this agent (as of today, think of the demo I showed yesterday) does not use anything Open Shift specific. Its all k8s APIs we are calling. But I've only ever run it against an Open Shift installation. ----- Original Message ----- > So assume Hawkular Metrics is running "somewhere", is the agent itself > dependent on OpenShift or Kubernetes ? > > On Wed, Oct 19, 2016 at 3:51 PM, Thomas Heute wrote: > > > +1 on the format and hawkular/hawkular-wildfly-agent (renaming sucks but > > we're still young enough to allow this) > > > > But we still need to decide betwen Openshift vs Kubernetes. > > > > > > > > On Wed, Oct 19, 2016 at 3:32 PM, Heiko W.Rupp wrote: > > > >> On 19 Oct 2016, at 15:24, John Mazzitelli wrote: > >> > >> > OK, so if we use Hawkular Kubernetes Agent then the gitrepo would be > >> named: > >> > > >> > hawkular/hawkular-kubernetes-agent > >> > >> Sounds good to me > >> _______________________________________________ > >> hawkular-dev mailing list > >> hawkular-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > >> > >> > >> > > > From mazz at redhat.com Wed Oct 19 10:21:06 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 10:21:06 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <1247626380.536307.1476886784247.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> <1247626380.536307.1476886784247.JavaMail.zimbra@redhat.com> Message-ID: <580287388.541724.1476886866050.JavaMail.zimbra@redhat.com> > As far as I can tell this agent (as of today, think of the demo I showed > yesterday) does not use anything Open Shift specific. Its all k8s APIs we > are calling. But I've only ever run it against an Open Shift installation. Here's the Go imports - notice all k8s - so nothing OS specific: core "k8s.io/client-go/1.4/kubernetes/typed/core/v1" "k8s.io/client-go/1.4/pkg/api" "k8s.io/client-go/1.4/pkg/api/v1" "k8s.io/client-go/1.4/pkg/fields" "k8s.io/client-go/1.4/pkg/watch" From theute at redhat.com Wed Oct 19 10:28:22 2016 From: theute at redhat.com (Thomas Heute) Date: Wed, 19 Oct 2016 16:28:22 +0200 Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <580287388.541724.1476886866050.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <484234469.341629.1476810515562.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> <1247626380.536307.1476886784247.JavaMail.zimbra@redhat.com> <580287388.541724.1476886866050.JavaMail.zimbra@redhat.com> Message-ID: How about the forseeable future ? I know it's a big question :) Thomas On Wed, Oct 19, 2016 at 4:21 PM, John Mazzitelli wrote: > > As far as I can tell this agent (as of today, think of the demo I showed > > yesterday) does not use anything Open Shift specific. Its all k8s APIs we > > are calling. But I've only ever run it against an Open Shift > installation. > > Here's the Go imports - notice all k8s - so nothing OS specific: > > core "k8s.io/client-go/1.4/kubernetes/typed/core/v1" > "k8s.io/client-go/1.4/pkg/api" > "k8s.io/client-go/1.4/pkg/api/v1" > "k8s.io/client-go/1.4/pkg/fields" > "k8s.io/client-go/1.4/pkg/watch" > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/31434153/attachment.html From mazz at redhat.com Wed Oct 19 10:31:07 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 19 Oct 2016 10:31:07 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <1412637517.509667.1476883492065.JavaMail.zimbra@redhat.com> <1247626380.536307.1476886784247.JavaMail.zimbra@redhat.com> <580287388.541724.1476886866050.JavaMail.zimbra@redhat.com> Message-ID: <116679804.569567.1476887467369.JavaMail.zimbra@redhat.com> I defer to Matt :) He seems to have his finger on the pulse of Open Shift and could say if he thinks there are some specific OS features (current or future features) we will want to take advantage of. ----- Original Message ----- > How about the forseeable future ? I know it's a big question :) > > Thomas > > On Wed, Oct 19, 2016 at 4:21 PM, John Mazzitelli wrote: > > > > As far as I can tell this agent (as of today, think of the demo I showed > > > yesterday) does not use anything Open Shift specific. Its all k8s APIs we > > > are calling. But I've only ever run it against an Open Shift > > installation. > > > > Here's the Go imports - notice all k8s - so nothing OS specific: > > > > core "k8s.io/client-go/1.4/kubernetes/typed/core/v1" > > "k8s.io/client-go/1.4/pkg/api" > > "k8s.io/client-go/1.4/pkg/api/v1" > > "k8s.io/client-go/1.4/pkg/fields" > > "k8s.io/client-go/1.4/pkg/watch" > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > > From jkremser at redhat.com Wed Oct 19 11:49:26 2016 From: jkremser at redhat.com (Jiri Kremser) Date: Wed, 19 Oct 2016 17:49:26 +0200 Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor In-Reply-To: <2057740863.5014134.1476884996253.JavaMail.zimbra@redhat.com> References: <2057740863.5014134.1476884996253.JavaMail.zimbra@redhat.com> Message-ID: Hi Matt, thanks for response. PetSets are meant for clustering of pods which have persistent storage. If this is not your use case, what exactly are you trying to do? There may be better ways to handle it. I am trying to figure out how to monitor wildfly in the openshift. If I am not mistaken all the metric ids contain the feed id, feed id (at least for wildfly is autogenerated if it's not provided in the xml config). If container/pod is killed and re-created its history is lost with the feed id. That's why I thought the Pet Sets with persistent ids can help. jk On Wed, Oct 19, 2016 at 3:49 PM, Matt Wringe wrote: > ----- Original Message ----- > > From: "Jiri Kremser" > > To: "Discussions around Hawkular development" < > hawkular-dev at lists.jboss.org> > > Sent: Thursday, 13 October, 2016 8:21:10 AM > > Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > > > Hello, > > today, I was on L&L about storage in OpenShift and I learn interesting > thing. > > I always thought, that everything needs to be immutable and stateless and > > all the state needs to be handled by means of NFS persistent volumes. > > Luckily, there is a feature in Kubernetess (since 1.3) that allows the > PODs > > to be treated as pets. It's called PetSet [1] and it assigns a unique ID > > (and persistent DNS record) to a POD that runs in this "mode". > > For OpenShift, we would have moved to using PetSets for our Cassandra pod, > but its not a fully supported feature yet. In the next version we will be > moving over to using it. > > It will make changing the cluster size for Cassandra nodes a lot easier > once we can use this. > > > > > Common use-case for PetSet is a set of pods with a relational DBs that > uses > > some kind of master-slave replication and slaves needs to know the > master's > > address. But it can be used for anything. We can use the hostname as the > > feed id for instance. > > > > I don't know how much this will be popular because it kind of defeats the > > purpose of immutable infrastructure but it can save us some work with the > > feed identity. And of course we need to support also the "normal" POD > > scenario. > > PetSets are meant for clustering of pods which have persistent storage. If > this is not your use case, what exactly are you trying to do? There may be > better ways to handle it. > > > > > [1]: http://kubernetes.io/docs/user-guide/petset/ > > > > jk > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/eeeeea05/attachment.html From mwringe at redhat.com Wed Oct 19 15:34:06 2016 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 19 Oct 2016 15:34:06 -0400 (EDT) Subject: [Hawkular-dev] open shift agent - what to call it? In-Reply-To: <116679804.569567.1476887467369.JavaMail.zimbra@redhat.com> References: <981522595.338003.1476809764092.JavaMail.zimbra@redhat.com> <1247626380.536307.1476886784247.JavaMail.zimbra@redhat.com> <580287388.541724.1476886866050.JavaMail.zimbra@redhat.com> <116679804.569567.1476887467369.JavaMail.zimbra@redhat.com> Message-ID: <1184999159.5157652.1476905646894.JavaMail.zimbra@redhat.com> >From my comments earlier today: "The agent itself should only be using the Kubernetes API and shouldn't have anything specific to OpenShift. So the agent here shouldn't care or know if its really connecting to Kubernetes or OpenShift. The problem is more that the Hawkular Metrics integration we have with OpenShift wont work in plain Kubernetes and we don't test it there. It would be possible to make it work on Kubernetes, but it wouldn't be a trivial change and would take some effort and testing to get it working there. We don't have the spare cycles or manpower to support the Kubernetes use case right now and this isn't even on our radar. It might be cool to have something usable with the upstream Kubernetes community though." Since we don't have a Hawkular Metrics version that is being used and tested in Kubernetes, I would be against using a name with 'Kubernetes' in it at this moment. Its going to be confusing to anyone who wants to try this in the Kubernetes directly, as the agent may work, but what it writes to will not. We may also want to add in some OpenShift specific features in there as well. I would vote to call it the "Hawkular OpenShift Agent" rather than the "Hawkular Kubernetes Agent" for these reasons. ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Wednesday, 19 October, 2016 10:31:07 AM > Subject: Re: [Hawkular-dev] open shift agent - what to call it? > > I defer to Matt :) > > He seems to have his finger on the pulse of Open Shift and could say if he > thinks there are some specific OS features (current or future features) we > will want to take advantage of. > > ----- Original Message ----- > > How about the forseeable future ? I know it's a big question :) > > > > Thomas > > > > On Wed, Oct 19, 2016 at 4:21 PM, John Mazzitelli wrote: > > > > > > As far as I can tell this agent (as of today, think of the demo I > > > > showed > > > > yesterday) does not use anything Open Shift specific. Its all k8s APIs > > > > we > > > > are calling. But I've only ever run it against an Open Shift > > > installation. > > > > > > Here's the Go imports - notice all k8s - so nothing OS specific: > > > > > > core "k8s.io/client-go/1.4/kubernetes/typed/core/v1" > > > "k8s.io/client-go/1.4/pkg/api" > > > "k8s.io/client-go/1.4/pkg/api/v1" > > > "k8s.io/client-go/1.4/pkg/fields" > > > "k8s.io/client-go/1.4/pkg/watch" > > > _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mwringe at redhat.com Wed Oct 19 16:00:21 2016 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 19 Oct 2016 16:00:21 -0400 (EDT) Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor In-Reply-To: References: <2057740863.5014134.1476884996253.JavaMail.zimbra@redhat.com> Message-ID: <1791809396.5161737.1476907221185.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Jiri Kremser" > To: "Discussions around Hawkular development" > Sent: Wednesday, 19 October, 2016 11:49:26 AM > Subject: Re: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > Hi Matt, > thanks for response. > > PetSets are meant for clustering of pods which have persistent storage. If > this is not your use case, what exactly are you trying to do? There may be > better ways to handle it. > > I am trying to figure out how to monitor wildfly in the openshift. If I am > not mistaken all the metric ids contain the feed id, feed id (at least for > wildfly is autogenerated if it's not provided in the xml config). If > container/pod is killed and re-created its history is lost with the feed id. > That's why I thought the Pet Sets with persistent ids can help. I don't know if this sounds like the right approach. The pods you are monitoring are still "cattle". They may be started, stopped, removed, increase in the number of instances running, decreased in the number of instances running or some other change may occur. I might have a replica set of 3 pods right now, 10 minutes from now due to increased load my replica set may autoscale to 5 pods, and 2 hours from now it may decrease down to 2 pods. If you want to track metrics from individual pods, then you should be setting the feed id to something unique like the namespace plus the pod name. The feed will change when the pod is recreated, but that is what you want, its no longer the same thing being monitored. If you want to keep track of something like a particular replica set, then you need to use inventory or tags and query based on that. > > jk > > On Wed, Oct 19, 2016 at 3:49 PM, Matt Wringe < mwringe at redhat.com > wrote: > > > ----- Original Message ----- > > From: "Jiri Kremser" < jkremser at redhat.com > > > To: "Discussions around Hawkular development" < > > hawkular-dev at lists.jboss.org > > > Sent: Thursday, 13 October, 2016 8:21:10 AM > > Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > > > Hello, > > today, I was on L&L about storage in OpenShift and I learn interesting > > thing. > > I always thought, that everything needs to be immutable and stateless and > > all the state needs to be handled by means of NFS persistent volumes. > > Luckily, there is a feature in Kubernetess (since 1.3) that allows the PODs > > to be treated as pets. It's called PetSet [1] and it assigns a unique ID > > (and persistent DNS record) to a POD that runs in this "mode". > > For OpenShift, we would have moved to using PetSets for our Cassandra pod, > but its not a fully supported feature yet. In the next version we will be > moving over to using it. > > It will make changing the cluster size for Cassandra nodes a lot easier once > we can use this. > > > > > Common use-case for PetSet is a set of pods with a relational DBs that uses > > some kind of master-slave replication and slaves needs to know the master's > > address. But it can be used for anything. We can use the hostname as the > > feed id for instance. > > > > I don't know how much this will be popular because it kind of defeats the > > purpose of immutable infrastructure but it can save us some work with the > > feed identity. And of course we need to support also the "normal" POD > > scenario. > > PetSets are meant for clustering of pods which have persistent storage. If > this is not your use case, what exactly are you trying to do? There may be > better ways to handle it. > > > > > [1]: http://kubernetes.io/docs/user-guide/petset/ > > > > jk > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jshaughn at redhat.com Wed Oct 19 16:53:03 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 19 Oct 2016 16:53:03 -0400 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! Message-ID: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> The Hawkular Alerting team is happy to announce the release of Hawkular Alerting 1.3.0.Final. This is a feature and fix release. * [HWKALERTS-176] - Support conditions on missing events and data o An *exciting* new alerting feature! This introduces *MissingCondition*. MissingConditions let you generate alerts or events when expected data fails to report, or when an expected event does not happen. * [HWKALERTS-174] - Add CORS filters o Cross Origin Resource Sharing support allows for optional request origin validation. * [HWKALERTS-181] - Add clustering information on status endpoint o The /status endpoint now reflects cluster topology! * [HWKALERTS-175] - Improvements on webhook plugins * [HWKALERTS-177] - Add new perf tests to study asynchronous send*() calls Additionally, this is the first Hawkular Alerting release to deliver Alerting features in *3* different distributions: * The alerting engine used inside Hawkular Services and supporting the Middleware Provider in ManageIQ. * As a Standalone alerting engine for general use. * And soon to be released, embedded inside Hawkular Metrics!! For more details for this release: https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12315924&version=12331985 Hawkular Alerting Team Jay Shaughnessy (jshaughn at redhat.com) Lucas Ponce (lponce at redhat.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/106f5877/attachment.html From jdoyle at redhat.com Wed Oct 19 17:16:15 2016 From: jdoyle at redhat.com (John Doyle) Date: Wed, 19 Oct 2016 17:16:15 -0400 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! In-Reply-To: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> References: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> Message-ID: Congrats on the release, sounds like some great features! On Wed, Oct 19, 2016 at 4:53 PM, Jay Shaughnessy wrote: > > The Hawkular Alerting team is happy to announce the release of Hawkular > Alerting 1.3.0.Final. > > This is a feature and fix release. > > - [HWKALERTS-176] - Support conditions on missing events and data > - An *exciting* new alerting feature! This introduces > *MissingCondition*. MissingConditions let you generate alerts or > events when expected data fails to report, or when an expected event does > not happen. > - [HWKALERTS-174] - Add CORS filters > - Cross Origin Resource Sharing support allows for optional request > origin validation. > - [HWKALERTS-181] - Add clustering information on status endpoint > - The /status endpoint now reflects cluster topology! > - [HWKALERTS-175] - Improvements on webhook plugins > - [HWKALERTS-177] - Add new perf tests to study asynchronous send*() > calls > > Additionally, this is the first Hawkular Alerting release to deliver > Alerting features in *3* different distributions: > > - The alerting engine used inside Hawkular Services and supporting the > Middleware Provider in ManageIQ. > - As a Standalone alerting engine for general use. > - And soon to be released, embedded inside Hawkular Metrics!! > > > For more details for this release: https://issues.jboss.org/ > secure/ReleaseNote.jspa?projectId=12315924&version=12331985 > > Hawkular Alerting Team > Jay Shaughnessy (jshaughn at redhat.com) > Lucas Ponce (lponce at redhat.com) > > > > > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161019/cc78eca1/attachment-0001.html From theute at redhat.com Thu Oct 20 03:44:59 2016 From: theute at redhat.com (Thomas Heute) Date: Thu, 20 Oct 2016 09:44:59 +0200 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! In-Reply-To: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> References: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> Message-ID: Congratulations Jay and Lucas, great work, great collaboration ! On Wed, Oct 19, 2016 at 10:53 PM, Jay Shaughnessy wrote: > > The Hawkular Alerting team is happy to announce the release of Hawkular > Alerting 1.3.0.Final. > > This is a feature and fix release. > > - [HWKALERTS-176] - Support conditions on missing events and data > - An *exciting* new alerting feature! This introduces > *MissingCondition*. MissingConditions let you generate alerts or > events when expected data fails to report, or when an expected event does > not happen. > - [HWKALERTS-174] - Add CORS filters > - Cross Origin Resource Sharing support allows for optional request > origin validation. > - [HWKALERTS-181] - Add clustering information on status endpoint > - The /status endpoint now reflects cluster topology! > - [HWKALERTS-175] - Improvements on webhook plugins > - [HWKALERTS-177] - Add new perf tests to study asynchronous send*() > calls > > Additionally, this is the first Hawkular Alerting release to deliver > Alerting features in *3* different distributions: > > - The alerting engine used inside Hawkular Services and supporting the > Middleware Provider in ManageIQ. > - As a Standalone alerting engine for general use. > - And soon to be released, embedded inside Hawkular Metrics!! > > > For more details for this release: https://issues.jboss.org/ > secure/ReleaseNote.jspa?projectId=12315924&version=12331985 > > Hawkular Alerting Team > Jay Shaughnessy (jshaughn at redhat.com) > Lucas Ponce (lponce at redhat.com) > > > > > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161020/f0ee43b9/attachment.html From jtakvori at redhat.com Thu Oct 20 05:14:41 2016 From: jtakvori at redhat.com (Joel Takvorian) Date: Thu, 20 Oct 2016 11:14:41 +0200 Subject: [Hawkular-dev] New repo for various travis scripts? Message-ID: Hi, I just wonder if we should create a new git repo to store the different files that are required for integration tests on the hawkular clients (ruby, java, now dropwizard...). For now there's just 2 required files afaik: ".travis/wait_for_services.rb" and the docker-compose file, but there may be more in the future. So, rather than storing a copy of each file in each client that use that docker-based integration tests, isn't it better to store them in a new repo and download them from travis script? There's also a maven install script that I picked from inventory and copied to the java client repo, that would fit in this scripts repo as well. Joel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161020/f6097ab8/attachment.html From lkrejci at redhat.com Thu Oct 20 06:28:09 2016 From: lkrejci at redhat.com (Lukas Krejci) Date: Thu, 20 Oct 2016 12:28:09 +0200 Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor In-Reply-To: <1791809396.5161737.1476907221185.JavaMail.zimbra@redhat.com> References: <1791809396.5161737.1476907221185.JavaMail.zimbra@redhat.com> Message-ID: <2001419.dEdRJZaRj9@localhost.localdomain> I have hardly any hands-on experience with Openshift or Kubernetes, but I was under the impression the pods can be configured using environment variables. If that is true, couldn't the agent just read its feed id from a well-known env var? Or if that is not possible then surely we can have some wrapper script that would take the env var and pass it on to the agent as a -D system property (which I believe the agent accepts already). On Wednesday, October 19, 2016 4:00:21 PM CEST Matt Wringe wrote: > ----- Original Message ----- > > > From: "Jiri Kremser" > > To: "Discussions around Hawkular development" > > Sent: Wednesday, 19 October, 2016 11:49:26 > > AM > > Subject: Re: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > > > Hi Matt, > > thanks for response. > > > > PetSets are meant for clustering of pods which have persistent storage. If > > this is not your use case, what exactly are you trying to do? There may be > > better ways to handle it. > > > > I am trying to figure out how to monitor wildfly in the openshift. If I am > > not mistaken all the metric ids contain the feed id, feed id (at least for > > wildfly is autogenerated if it's not provided in the xml config). If > > container/pod is killed and re-created its history is lost with the feed > > id. That's why I thought the Pet Sets with persistent ids can help. > > I don't know if this sounds like the right approach. > > The pods you are monitoring are still "cattle". They may be started, > stopped, removed, increase in the number of instances running, decreased in > the number of instances running or some other change may occur. > > I might have a replica set of 3 pods right now, 10 minutes from now due to > increased load my replica set may autoscale to 5 pods, and 2 hours from now > it may decrease down to 2 pods. > > If you want to track metrics from individual pods, then you should be > setting the feed id to something unique like the namespace plus the pod > name. The feed will change when the pod is recreated, but that is what you > want, its no longer the same thing being monitored. > > If you want to keep track of something like a particular replica set, then > you need to use inventory or tags and query based on that. > > jk > > > > On Wed, Oct 19, 2016 at 3:49 PM, Matt Wringe < mwringe at redhat.com > wrote: > > > > > > ----- Original Message ----- > > > > > From: "Jiri Kremser" < jkremser at redhat.com > > > > To: "Discussions around Hawkular development" < > > > hawkular-dev at lists.jboss.org > > > > Sent: Thursday, 13 October, 2016 8:21:10 AM > > > Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > > > > > Hello, > > > today, I was on L&L about storage in OpenShift and I learn interesting > > > thing. > > > I always thought, that everything needs to be immutable and stateless > > > and > > > all the state needs to be handled by means of NFS persistent volumes. > > > Luckily, there is a feature in Kubernetess (since 1.3) that allows the > > > PODs > > > to be treated as pets. It's called PetSet [1] and it assigns a unique ID > > > (and persistent DNS record) to a POD that runs in this "mode". > > > > For OpenShift, we would have moved to using PetSets for our Cassandra pod, > > but its not a fully supported feature yet. In the next version we will be > > moving over to using it. > > > > It will make changing the cluster size for Cassandra nodes a lot easier > > once we can use this. > > > > > Common use-case for PetSet is a set of pods with a relational DBs that > > > uses > > > some kind of master-slave replication and slaves needs to know the > > > master's > > > address. But it can be used for anything. We can use the hostname as the > > > feed id for instance. > > > > > > I don't know how much this will be popular because it kind of defeats > > > the > > > purpose of immutable infrastructure but it can save us some work with > > > the > > > feed identity. And of course we need to support also the "normal" POD > > > scenario. > > > > PetSets are meant for clustering of pods which have persistent storage. If > > this is not your use case, what exactly are you trying to do? There may be > > better ways to handle it. > > > > > [1]: http://kubernetes.io/docs/user-guide/petset/ > > > > > > jk > > > > > > _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Lukas Krejci From mwringe at redhat.com Thu Oct 20 08:41:44 2016 From: mwringe at redhat.com (Matt Wringe) Date: Thu, 20 Oct 2016 08:41:44 -0400 (EDT) Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor In-Reply-To: <2001419.dEdRJZaRj9@localhost.localdomain> References: <1791809396.5161737.1476907221185.JavaMail.zimbra@redhat.com> <2001419.dEdRJZaRj9@localhost.localdomain> Message-ID: <166688671.5291734.1476967304738.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Lukas Krejci" > To: hawkular-dev at lists.jboss.org > Sent: Thursday, 20 October, 2016 6:28:09 AM > Subject: Re: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > I have hardly any hands-on experience with Openshift or Kubernetes, but I was > under the impression the pods can be configured using environment variables. If you are dealing with a specific single pod that you are monitoring, then yes, you could manually set a unique feed id via something like a environment variable, or just give the pod a specified name. Since we can't use petsets for our Cassandra instances yet, we do something similar. Each Cassandra instance in a cluster has its own unique id which has a corresponding persistent volume (eg hawkular-cassandra-node-1, hawkular-cassandra-node-2, etc). But this is not a great solution since we have to manually increase or decrease the cluster size with templates instead of using the tools which are meant to do this (eg replicasets/petsets). Using the proper tools will make things much easier for admins and will be much less prone to errors. The problem is that for the most part you are probably not going to be dealing with single pods. Users are going to be running groups of pods in a replicaset, and the number of pods running there is dynamic. It can increase or decrease, can be all stopped and restarted again, etc. Even if say you know you only want to be running one pod for your application, you may want to do something like bring up a new version of our application in one pod and have it running a bit before shutting down your old pod. This overall would cause some strange values in your collected metrics if you had the same feed id for both. You may want to rethink how to handle this. I would expect each pod to have its own feed and metrics being gathered since they are separate, individual things. I would then expect the console figure out how to manage and group this information in a manner that makes sense. This is how we handle it in OpenShift. Each pod has its own metrics being gathered for it, and we have the metadata for the pods stored as tags. From this I can see the graphs for the currently running pods, as well as get a graph of the overall usage across a replica set (currently running and previously running pods). > > If that is true, couldn't the agent just read its feed id from a well-known > env var? Or if that is not possible then surely we can have some wrapper > script that would take the env var and pass it on to the agent as a -D system > property (which I believe the agent accepts already). > > On Wednesday, October 19, 2016 4:00:21 PM CEST Matt Wringe wrote: > > ----- Original Message ----- > > > > > From: "Jiri Kremser" > > > To: "Discussions around Hawkular development" > > > Sent: Wednesday, 19 October, 2016 11:49:26 > > > AM > > > Subject: Re: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > > > > > Hi Matt, > > > thanks for response. > > > > > > PetSets are meant for clustering of pods which have persistent storage. > > > If > > > this is not your use case, what exactly are you trying to do? There may > > > be > > > better ways to handle it. > > > > > > I am trying to figure out how to monitor wildfly in the openshift. If I > > > am > > > not mistaken all the metric ids contain the feed id, feed id (at least > > > for > > > wildfly is autogenerated if it's not provided in the xml config). If > > > container/pod is killed and re-created its history is lost with the feed > > > id. That's why I thought the Pet Sets with persistent ids can help. > > > > I don't know if this sounds like the right approach. > > > > The pods you are monitoring are still "cattle". They may be started, > > stopped, removed, increase in the number of instances running, decreased in > > the number of instances running or some other change may occur. > > > > I might have a replica set of 3 pods right now, 10 minutes from now due to > > increased load my replica set may autoscale to 5 pods, and 2 hours from now > > it may decrease down to 2 pods. > > > > If you want to track metrics from individual pods, then you should be > > setting the feed id to something unique like the namespace plus the pod > > name. The feed will change when the pod is recreated, but that is what you > > want, its no longer the same thing being monitored. > > > > If you want to keep track of something like a particular replica set, then > > you need to use inventory or tags and query based on that. > > > jk > > > > > > On Wed, Oct 19, 2016 at 3:49 PM, Matt Wringe < mwringe at redhat.com > > > > wrote: > > > > > > > > > ----- Original Message ----- > > > > > > > From: "Jiri Kremser" < jkremser at redhat.com > > > > > To: "Discussions around Hawkular development" < > > > > hawkular-dev at lists.jboss.org > > > > > Sent: Thursday, 13 October, 2016 8:21:10 AM > > > > Subject: [Hawkular-dev] OpenShift Pet vs Cattle metaphor > > > > > > > > Hello, > > > > today, I was on L&L about storage in OpenShift and I learn interesting > > > > thing. > > > > I always thought, that everything needs to be immutable and stateless > > > > and > > > > all the state needs to be handled by means of NFS persistent volumes. > > > > Luckily, there is a feature in Kubernetess (since 1.3) that allows the > > > > PODs > > > > to be treated as pets. It's called PetSet [1] and it assigns a unique > > > > ID > > > > (and persistent DNS record) to a POD that runs in this "mode". > > > > > > For OpenShift, we would have moved to using PetSets for our Cassandra > > > pod, > > > but its not a fully supported feature yet. In the next version we will be > > > moving over to using it. > > > > > > It will make changing the cluster size for Cassandra nodes a lot easier > > > once we can use this. > > > > > > > Common use-case for PetSet is a set of pods with a relational DBs that > > > > uses > > > > some kind of master-slave replication and slaves needs to know the > > > > master's > > > > address. But it can be used for anything. We can use the hostname as > > > > the > > > > feed id for instance. > > > > > > > > I don't know how much this will be popular because it kind of defeats > > > > the > > > > purpose of immutable infrastructure but it can save us some work with > > > > the > > > > feed identity. And of course we need to support also the "normal" POD > > > > scenario. > > > > > > PetSets are meant for clustering of pods which have persistent storage. > > > If > > > this is not your use case, what exactly are you trying to do? There may > > > be > > > better ways to handle it. > > > > > > > [1]: http://kubernetes.io/docs/user-guide/petset/ > > > > > > > > jk > > > > > > > > _______________________________________________ > > > > hawkular-dev mailing list > > > > hawkular-dev at lists.jboss.org > > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > > > > _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -- > Lukas Krejci > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Thu Oct 20 11:10:53 2016 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 20 Oct 2016 11:10:53 -0400 (EDT) Subject: [Hawkular-dev] Hawkular OpenShift Agent is now available In-Reply-To: <1408882624.1212399.1476975043205.JavaMail.zimbra@redhat.com> Message-ID: <27994349.1220434.1476976253324.JavaMail.zimbra@redhat.com> (FYI: we had an "underwhelming" participation in the naming poll. What was decided upon was a more descriptive name rather than some code name anyway) Hawkular OpenShift Agent source has been published on github.com here: https://github.com/hawkular/hawkular-openshift-agent For now, we'll track issues in github until we figure out what we want to do (if anything) in JIRA and where. If interested, read the README - it provides build and run instructions. Currently the Hawkular OpenShift Agent supports the following: 1) Watches OpenShift as pods and configmaps are added, modified, and removed 2) As things change in OpenShift, the agent adjusts what it monitors 3) All metric data is stored in Hawkular Metrics 4) Pods tell the agent what should be monitored via an annotation which names a config map. In that config map is a single YAML configuration that contains all the endpoint information the agent needs in order to monitor it and store its data. Pods can ask for multiple endpoints to be monitored, and multiple pods within the node can be monitored - but only one node is monitored. If you have multiple nodes, you need one agent per node. 5) Each endpoint data can have its data stored in its own tenant (as defined in the config map yaml) 6) The agent can monitor any endpoints you define in the global agent config file also - you don't need to have pods/config maps for this (this is useful if the agent is running outside of Open Shift or there are some things in the agent's node that you want monitored without having to look up pods/configmaps). 7) Currently, Prometheus endpoints are supported (both binary and text protocols). There are many things we need to get done. * Jolokia support is not yet implemented * Secure access (encryption and authentication) to both Hawkular-Metrics and the metric endpoints * Details on how to run the agent within Open Shift (daemon set?) * Tag the metrics being stored (there are no tags being associated with the metrics yet) * Determine the names of the metrics (right now its just using the names of the prometheus metrics as-is) * etc, etc, etc Many thanks to Matt Wringe who got this kicked off with his ideas and Open Shift integration code which was the foundation of the current codebase. --John Mazz From tsegismo at redhat.com Thu Oct 20 11:17:34 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 20 Oct 2016 17:17:34 +0200 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! In-Reply-To: References: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> Message-ID: Awesome! Is it planned to add the Alerts component to Origin-Metrics? 2016-10-20 9:44 GMT+02:00 Thomas Heute : > Congratulations Jay and Lucas, great work, great collaboration ! > > On Wed, Oct 19, 2016 at 10:53 PM, Jay Shaughnessy > wrote: > >> >> The Hawkular Alerting team is happy to announce the release of Hawkular >> Alerting 1.3.0.Final. >> >> This is a feature and fix release. >> >> - [HWKALERTS-176] - Support conditions on missing events and data >> - An *exciting* new alerting feature! This introduces >> *MissingCondition*. MissingConditions let you generate alerts or >> events when expected data fails to report, or when an expected event does >> not happen. >> - [HWKALERTS-174] - Add CORS filters >> - Cross Origin Resource Sharing support allows for optional >> request origin validation. >> - [HWKALERTS-181] - Add clustering information on status endpoint >> - The /status endpoint now reflects cluster topology! >> - [HWKALERTS-175] - Improvements on webhook plugins >> - [HWKALERTS-177] - Add new perf tests to study asynchronous send*() >> calls >> >> Additionally, this is the first Hawkular Alerting release to deliver >> Alerting features in *3* different distributions: >> >> - The alerting engine used inside Hawkular Services and supporting >> the Middleware Provider in ManageIQ. >> - As a Standalone alerting engine for general use. >> - And soon to be released, embedded inside Hawkular Metrics!! >> >> >> For more details for this release: https://issues.jboss.org/secur >> e/ReleaseNote.jspa?projectId=12315924&version=12331985 >> >> Hawkular Alerting Team >> Jay Shaughnessy (jshaughn at redhat.com) >> Lucas Ponce (lponce at redhat.com) >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161020/6502c114/attachment.html From jshaughn at redhat.com Thu Oct 20 11:25:06 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 20 Oct 2016 11:25:06 -0400 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! In-Reply-To: References: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> Message-ID: On 10/20/2016 11:17 AM, Thomas Segismont wrote: > Awesome! Is it planned to add the Alerts component to Origin-Metrics? The forthcoming hawkular metrics release will contain hawkular alerting out of the box (as an EAR). So, both REST APIs will be available, and relevant metric data will be efficiently forwarded to alerting for evaluation. So, basically, wherever Hmetrics is you have Halerting as well. > > 2016-10-20 9:44 GMT+02:00 Thomas Heute >: > > Congratulations Jay and Lucas, great work, great collaboration ! > > On Wed, Oct 19, 2016 at 10:53 PM, Jay Shaughnessy > > wrote: > > > The Hawkular Alerting team is happy to announce the release of > Hawkular Alerting 1.3.0.Final. > > This is a feature and fix release. > > * [HWKALERTS-176] - Support conditions on missing events and > data > o An *exciting* new alerting feature! This introduces > *MissingCondition*. MissingConditions let you generate > alerts or events when expected data fails to report, > or when an expected event does not happen. > * [HWKALERTS-174] - Add CORS filters > o Cross Origin Resource Sharing support allows for > optional request origin validation. > * [HWKALERTS-181] - Add clustering information on status > endpoint > o The /status endpoint now reflects cluster topology! > * [HWKALERTS-175] - Improvements on webhook plugins > * [HWKALERTS-177] - Add new perf tests to study asynchronous > send*() calls > > Additionally, this is the first Hawkular Alerting release to > deliver Alerting features in *3* different distributions: > > * The alerting engine used inside Hawkular Services and > supporting the Middleware Provider in ManageIQ. > * As a Standalone alerting engine for general use. > * And soon to be released, embedded inside Hawkular Metrics!! > > > For more details for this release: > https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12315924&version=12331985 > > > Hawkular Alerting Team > Jay Shaughnessy (jshaughn at redhat.com > ) > Lucas Ponce (lponce at redhat.com ) > > > > > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161020/9059ced4/attachment.html From tsegismo at redhat.com Thu Oct 20 11:35:15 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 20 Oct 2016 17:35:15 +0200 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! In-Reply-To: References: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> Message-ID: So I guess the Origin Metrics project will be updated to consume this EAR instead of the standalone WAR as it does currently. Cool then 2016-10-20 17:25 GMT+02:00 Jay Shaughnessy : > > > On 10/20/2016 11:17 AM, Thomas Segismont wrote: > > Awesome! Is it planned to add the Alerts component to Origin-Metrics? > > > The forthcoming hawkular metrics release will contain hawkular alerting > out of the box (as an EAR). So, both REST APIs will be available, and > relevant metric data will be efficiently forwarded to alerting for > evaluation. So, basically, wherever Hmetrics is you have Halerting as > well. > > > > 2016-10-20 9:44 GMT+02:00 Thomas Heute : > >> Congratulations Jay and Lucas, great work, great collaboration ! >> >> On Wed, Oct 19, 2016 at 10:53 PM, Jay Shaughnessy >> wrote: >> >>> >>> The Hawkular Alerting team is happy to announce the release of Hawkular >>> Alerting 1.3.0.Final. >>> >>> This is a feature and fix release. >>> >>> - [HWKALERTS-176] - Support conditions on missing events and data >>> - An *exciting* new alerting feature! This introduces >>> *MissingCondition*. MissingConditions let you generate alerts >>> or events when expected data fails to report, or when an expected event >>> does not happen. >>> - [HWKALERTS-174] - Add CORS filters >>> - Cross Origin Resource Sharing support allows for optional >>> request origin validation. >>> - [HWKALERTS-181] - Add clustering information on status endpoint >>> - The /status endpoint now reflects cluster topology! >>> - [HWKALERTS-175] - Improvements on webhook plugins >>> - [HWKALERTS-177] - Add new perf tests to study asynchronous send*() >>> calls >>> >>> Additionally, this is the first Hawkular Alerting release to deliver >>> Alerting features in *3* different distributions: >>> >>> - The alerting engine used inside Hawkular Services and supporting >>> the Middleware Provider in ManageIQ. >>> - As a Standalone alerting engine for general use. >>> - And soon to be released, embedded inside Hawkular Metrics!! >>> >>> >>> For more details for this release: https://issues.jboss.org/secur >>> e/ReleaseNote.jspa?projectId=12315924&version=12331985 >>> >>> Hawkular Alerting Team >>> Jay Shaughnessy (jshaughn at redhat.com) >>> Lucas Ponce (lponce at redhat.com) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >>> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> > > > _______________________________________________ > hawkular-dev mailing listhawkular-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161020/bdd41cd3/attachment-0001.html From jshaughn at redhat.com Thu Oct 20 13:15:40 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 20 Oct 2016 13:15:40 -0400 Subject: [Hawkular-dev] Hawkular Alerting 1.3.0.Final has been released! In-Reply-To: References: <96125a74-b330-d316-a8c6-36d0c5707d2b@redhat.com> Message-ID: <938665f3-3d16-3a4c-f70b-9b50d29b1de6@redhat.com> Thomas, that should be the case, yes, sorry for the roundabout answer :) On 10/20/2016 11:35 AM, Thomas Segismont wrote: > So I guess the Origin Metrics project will be updated to consume this > EAR instead of the standalone WAR as it does currently. Cool then > > 2016-10-20 17:25 GMT+02:00 Jay Shaughnessy >: > > > > On 10/20/2016 11:17 AM, Thomas Segismont wrote: >> Awesome! Is it planned to add the Alerts component to Origin-Metrics? > > The forthcoming hawkular metrics release will contain hawkular > alerting out of the box (as an EAR). So, both REST APIs will be > available, and relevant metric data will be efficiently forwarded > to alerting for evaluation. So, basically, wherever Hmetrics is > you have Halerting as well. > > >> >> 2016-10-20 9:44 GMT+02:00 Thomas Heute > >: >> >> Congratulations Jay and Lucas, great work, great collaboration ! >> >> On Wed, Oct 19, 2016 at 10:53 PM, Jay Shaughnessy >> > wrote: >> >> >> The Hawkular Alerting team is happy to announce the >> release of Hawkular Alerting 1.3.0.Final. >> >> This is a feature and fix release. >> >> * [HWKALERTS-176] - Support conditions on missing >> events and data >> o An *exciting* new alerting feature! This >> introduces *MissingCondition*. MissingConditions >> let you generate alerts or events when expected >> data fails to report, or when an expected event >> does not happen. >> * [HWKALERTS-174] - Add CORS filters >> o Cross Origin Resource Sharing support allows for >> optional request origin validation. >> * [HWKALERTS-181] - Add clustering information on >> status endpoint >> o The /status endpoint now reflects cluster topology! >> * [HWKALERTS-175] - Improvements on webhook plugins >> * [HWKALERTS-177] - Add new perf tests to study >> asynchronous send*() calls >> >> Additionally, this is the first Hawkular Alerting release >> to deliver Alerting features in *3* different distributions: >> >> * The alerting engine used inside Hawkular Services and >> supporting the Middleware Provider in ManageIQ. >> * As a Standalone alerting engine for general use. >> * And soon to be released, embedded inside Hawkular >> Metrics!! >> >> >> For more details for this release: >> https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12315924&version=12331985 >> >> >> Hawkular Alerting Team >> Jay Shaughnessy (jshaughn at redhat.com >> ) >> Lucas Ponce (lponce at redhat.com ) >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> >> >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ hawkular-dev > mailing list hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161020/2a24c544/attachment.html From mazz at redhat.com Thu Oct 20 16:37:52 2016 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 20 Oct 2016 16:37:52 -0400 (EDT) Subject: [Hawkular-dev] hawkular openshift agent integration tests In-Reply-To: <768552831.1283284.1476995615163.JavaMail.zimbra@redhat.com> Message-ID: <1150834787.1283419.1476995872396.JavaMail.zimbra@redhat.com> (moving this thread to the mailing list for public consumption) Mazz says, > As for the Hawkular OpenShift Agent, note that there are no integration tests yet. > I have no idea how to do that yet. I've got unit tests throughout, but not itests. > Something we'll need to do eventually. Gotta figure out how to mock an OpenShift environment. Matt says, > For origin metrics we run e2e tests. So we point our test to our OpenShift instance, > it will then deploy our metrics components (with the various different deployment options) > and then check and see that everything deployed properly, there are no errors or restarts > and that we can gather metrics. > > For the agent we might want to do something similar. I don't know how useful it would be > to mock up an OpenShift environment when you can just have it directly use one. > > ... I would look to do this using the kubernetes client more directly and using a proper test framework. Matt - I have a followup question: Does this mean we would not be able to run itests on github commits via Travis? Not that that is a bad thing - I would not be heartbroken if I was told we will not be able to use Travis :) I'm just wondering if this means there will be no CI. From mwringe at redhat.com Fri Oct 21 13:39:39 2016 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 21 Oct 2016 13:39:39 -0400 (EDT) Subject: [Hawkular-dev] hawkular openshift agent integration tests In-Reply-To: <1150834787.1283419.1476995872396.JavaMail.zimbra@redhat.com> References: <1150834787.1283419.1476995872396.JavaMail.zimbra@redhat.com> Message-ID: <465380450.5668599.1477071579902.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Thursday, 20 October, 2016 4:37:52 PM > Subject: [Hawkular-dev] hawkular openshift agent integration tests > > (moving this thread to the mailing list for public consumption) > > Mazz says, > > > As for the Hawkular OpenShift Agent, note that there are no integration > > tests yet. > > I have no idea how to do that yet. I've got unit tests throughout, but not > > itests. > > Something we'll need to do eventually. Gotta figure out how to mock an > > OpenShift environment. > > Matt says, > > > For origin metrics we run e2e tests. So we point our test to our OpenShift > > instance, > > it will then deploy our metrics components (with the various different > > deployment options) > > and then check and see that everything deployed properly, there are no > > errors or restarts > > and that we can gather metrics. > > > > For the agent we might want to do something similar. I don't know how > > useful it would be > > to mock up an OpenShift environment when you can just have it directly use > > one. > > > > ... I would look to do this using the kubernetes client more directly and > > using a proper test framework. > > Matt - I have a followup question: > > Does this mean we would not be able to run itests on github commits via > Travis? Not that that is a bad thing - I would not be heartbroken if I was > told we will not be able to use Travis :) I'm just wondering if this means > there will be no CI. I don't believe you will be able to run this on travis, you may have to use your own jenkins like we do for OpenShift. From mazz at redhat.com Fri Oct 21 17:11:40 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 21 Oct 2016 17:11:40 -0400 (EDT) Subject: [Hawkular-dev] hawkular-openshift-agent now supports jolokia endpoints In-Reply-To: <468214999.1525007.1477084170394.JavaMail.zimbra@redhat.com> Message-ID: <1061389510.1525096.1477084300479.JavaMail.zimbra@redhat.com> Hawkular OpenShift Agent now supports Jolokia endpoints. This means we can now scrape metrics from both Prometheus endpoints and Jolokia endpoints running inside OpenShift pods (it also supports endpoints if the agent is running outside OS, too). This PR needs to be reviewed and merged for this feature to get in: https://github.com/hawkular/hawkular-openshift-agent/pull/10 There are several issues that need to be addressed to further build out this agent. If interested in working on this, see https://github.com/hawkular/hawkular-openshift-agent/issues From mazz at redhat.com Fri Oct 21 23:19:16 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 21 Oct 2016 23:19:16 -0400 (EDT) Subject: [Hawkular-dev] hawkular-openshift-agent can limit what pods are monitored In-Reply-To: <1558874773.1587714.1477106255939.JavaMail.zimbra@redhat.com> Message-ID: <1669358752.1588144.1477106356536.JavaMail.zimbra@redhat.com> The Hawkular OpenShift Agent will have a new feature once the PR is merged - a pod authorization list. You can now tell the agent (via environment variable or in its YAML config) to only authorize a specific set of pods (i.e. only the pods on the list can have their metrics collected) - any other pods not on the list are ignored and will not have metrics collected for them. See: https://github.com/hawkular/hawkular-openshift-agent/issues/6 https://github.com/hawkular/hawkular-openshift-agent/pull/13 From hrupp at redhat.com Sun Oct 23 03:38:03 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Sun, 23 Oct 2016 09:38:03 +0200 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: <9AB7C1BC-C30B-4FCA-8DCB-DDABDC36D288@redhat.com> References: <781ED7FD-EA4C-47A8-BB1C-AC2C72042350@redhat.com> <9AB7C1BC-C30B-4FCA-8DCB-DDABDC36D288@redhat.com> Message-ID: <76E826C0-9B5A-4312-BD89-892FA4C64AD6@redhat.com> Update [ tl;dr: good news ] Lukas provided me with "connection properties" for hsqldb that limit the memory usage and this seems stable. I have been running h-services since Friday afternoon (~40h) with 33 HawkFly feeds. Two heap dumps taken yesterday afternoon and this morning both show a stable heap size of 192M, so I think we are on the safe side here (with hsqldb, see also below). On top of just ingesting metrics, I have created poor mans Chaos Monkey that randomly stops and restarts a container every 5 minutes (parallel on 2 machines where my feeds live) to also stress the initial sync when a feeds is (re) connecting. I also had a (very small) number of alert triggers defined, that were firing when heap on a hawkfly is > 60k, which happened quite often. This was running during the last 36h, so is included in above heap metrics. The MBeans for hawkular metrics show an insertion rate of around 33 metrics/s , so a HawkFly corresponds to 1metric/s. About hsqldb: I am going to shut this down now and test with a h2-based instance, as h2 is already in WildFly and EAP and should be used instead of hsqldb. Some time next week I will also try to instrument HawkFly in a away that it automatically defines some alert triggers on startup. From hrupp at redhat.com Mon Oct 24 03:48:43 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 24 Oct 2016 09:48:43 +0200 Subject: [Hawkular-dev] Memory-usage of Hawkular-services In-Reply-To: <76E826C0-9B5A-4312-BD89-892FA4C64AD6@redhat.com> References: <781ED7FD-EA4C-47A8-BB1C-AC2C72042350@redhat.com> <9AB7C1BC-C30B-4FCA-8DCB-DDABDC36D288@redhat.com> <76E826C0-9B5A-4312-BD89-892FA4C64AD6@redhat.com> Message-ID: <6D587275-7C81-4C5E-83D2-29A16DBF3396@redhat.com> Update with h2 I only ran this for ~20h and with only 30 feeds but memory usage was lower than with hsqldb as the h2 cache was some 20MB smaller than with hsqldb. This was stable during until I added at around 12h from the start some more feeds in parallel. This needs thus more investigation. Heiko On 23 Oct 2016, at 9:38, Heiko W.Rupp wrote: > Update > > [ tl;dr: good news ] > > Lukas provided me with "connection properties" for hsqldb > that limit the memory usage and this seems stable. I have been > running h-services since Friday afternoon (~40h) with 33 HawkFly > feeds. > > Two heap dumps taken yesterday afternoon and this morning > both show a stable heap size of 192M, so I think we are on the > safe side here (with hsqldb, see also below). > > On top of just ingesting metrics, I have created poor mans Chaos > Monkey that randomly stops and restarts a container every 5 minutes > (parallel on 2 machines where my feeds live) to also stress the > initial > sync when a feeds is (re) connecting. > I also had a (very small) number of alert triggers defined, that > were firing when heap on a hawkfly is > 60k, which happened quite > often. > This was running during the last 36h, so is included in above > heap metrics. > > The MBeans for hawkular metrics show an insertion rate of around > 33 metrics/s , so a HawkFly corresponds to 1metric/s. > > About hsqldb: I am going to shut this down now and test with a > h2-based instance, as h2 is already in WildFly and EAP and > should be used instead of hsqldb. > > Some time next week I will also try to instrument HawkFly in a away > that it automatically defines some alert triggers on startup. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander From snegrea at redhat.com Tue Oct 25 17:44:41 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 25 Oct 2016 16:44:41 -0500 Subject: [Hawkular-dev] Hawkular Metrics 0.21.0 - Release Message-ID: Hello Everybody, I am happy to announce release 0.21.0 of Hawkular Metrics. This release is anchored by performance enhancements and general fixes. Here is a list of major changes: - Cassandra - *Cassandra 3.0.9 is now the supported version of Cassandra.* - Note: this is a rollback from previously supported version of 3.7 due to Cassandra community recommendations for stability and production deployment. Cassandra 3.7 or 3.9 are still compatible but development and testing use 3.0.9 release. - Compression - Fixed an issue that allowed duplicate instances of the compression job to get scheduled on server restart (HWKMETRICS-492 ) - Improved the fault tolerance of the compression job (HWKMETRICS-494 ) - Improved the performance of the merge process for reading compressed data (HWKMETRICS-488 ) - Fixed wrong ordering when fetching compressed and uncompressed data (HWKMETRICS-506 ) - Compression job provides back pressure (HWKMETRICS-500 ) - The job scheduler now handles failure scenarios (HWKMETRICS-505 ) - Cassandra Schema - Fixed an issue where the server can fail to start due to Cassalog being in inconsistent state (HWKMETRICS-495 ) - gc_grace_second is set to zero for single node clusters ( HWKMETRICS-381 ) - API Updates - Inserting data points has server side retries to increase the fault tolerance for simple error scenarios (HWKMETRICS-510 ) - fromEarliest parameter is now supported in all query endpoints ( HWKMETRICS-445 ) - Configuration - The configuration options did not have a consistent naming scheme. hawkular-metrics, hawkular.metrics, and hawkular prefixes were used along no prefixes. - In this release the naming schema has been standardized to hawkular.metrics.* for metrics specific configuration and hawkular.* for general configuration. - Here is list of all configuration options currently available: ConfigurationKey - For more details: HWKMETRICS-508 *Hawkular Alerting - included* - Version 1.3.0 - Project details and repository: Github - Documentation: REST API Documentation , Examples , Developer Guide *Hawkular Metrics Clients* - Python: https://github.com/hawkular/hawkular-client-python - Go: https://github.com/hawkular/hawkular-client-go - Ruby: https://github.com/hawkular/hawkular-client-ruby - Java: https://github.com/hawkular/hawkular-client-java *Release Links* Github Release: https://github.com/hawkular/hawkular-metrics/releases/tag/0.21.0 JBoss Nexus Maven artifacts: http://origin-repository.jboss.org/nexus/content/repositorie s/public/org/hawkular/metrics/ Jira release tracker: https://issues.jboss.org/projects/HWKMETRICS/versions/12331718 A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, Joel Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their project contributions. Thank you, Stefan Negrea -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161025/c8ab0fc0/attachment.html From miburman at redhat.com Wed Oct 26 05:39:59 2016 From: miburman at redhat.com (Michael Burman) Date: Wed, 26 Oct 2016 12:39:59 +0300 Subject: [Hawkular-dev] Hawkular Metrics 0.21.0 - Release In-Reply-To: References: Message-ID: <90da81c0-5907-459d-0e5c-db10d98cd7c3@redhat.com> Hi, Missing from API updates, the tagQueryLanguage now supports also defining non-existant tag as one search query: !tag:* Meaning the metrics should not have a tag called "tag". HWKMETRICS-480 - Micke On 10/26/2016 12:44 AM, Stefan Negrea wrote: > > Hello Everybody, > > I am happy to announce release 0.21.0 of Hawkular Metrics. This > release is anchored by performance enhancements and general fixes. > > Here is a list of major changes: > > * > > Cassandra > > o *Cassandra 3.0.9 is now the supported version of Cassandra.* > o Note: this is a rollback from previously supported version of > 3.7 due to Cassandra community recommendations for stability > and production deployment. Cassandra 3.7 or 3.9 are still > compatible but development and testing use 3.0.9 release. > * > > Compression > > o Fixed an issue that allowed duplicate instances of the > compression job to get scheduled on server restart > (HWKMETRICS-492 ) > o Improved the fault tolerance of the compression job > (HWKMETRICS-494 ) > o Improved the performance of the merge process for reading > compressed data (HWKMETRICS-488 > ) > o Fixed wrong ordering when fetching compressed and uncompressed > data (HWKMETRICS-506 > ) > o Compression job provides back pressure (HWKMETRICS-500 > ) > o The job scheduler now handles failure scenarios > (HWKMETRICS-505 ) > * > > Cassandra Schema > > o Fixed an issue where the server can fail to start due to > Cassalog being in inconsistent state (HWKMETRICS-495 > ) > o |gc_grace_second| is set to zero for single node clusters > (HWKMETRICS-381 ) > * > > API Updates > > o Inserting data points has server side retries to increase the > fault tolerance for simple error scenarios (HWKMETRICS-510 > ) > o |fromEarliest| parameter is now supported in all query > endpoints (HWKMETRICS-445 > ) > * > > Configuration > > o The configuration options did not have a consistent naming > scheme. |hawkular-metrics|, |hawkular.metrics, and |hawkular > prefixes were used along no prefixes. > o In this release the naming schema has been standardized to > |hawkular.metrics.*| for metrics specific configuration and > |hawkular.|* for general configuration. > o Here is list of all configuration options currently available: > ConfigurationKey > > > o For more details: HWKMETRICS-508 > > > > *Hawkular Alerting - included > * > > * Version 1.3.0 > > * Project details and repository: Github > > * Documentation: REST API Documentation > , Examples > , > Developer Guide > > > *Hawkular Metrics Clients* > > * Python: https://github.com/hawkular/hawkular-client-python > > * Go: https://github.com/hawkular/hawkular-client-go > > * Ruby: https://github.com/hawkular/hawkular-client-ruby > > * Java: https://github.com/hawkular/hawkular-client-java > > > * > * > > *Release Links* > > Github Release: > https://github.com/hawkular/hawkular-metrics/releases/tag/0.21.0 > > JBoss Nexus Maven artifacts: > http://origin-repository.jboss.org/nexus/content/repositories/public/org/hawkular/metrics/ > > > Jira release tracker: > https://issues.jboss.org/projects/HWKMETRICS/versions/12331718 > > > A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, > Joel Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their > project contributions. > > Thank you, > Stefan Negrea > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From tsegismo at redhat.com Wed Oct 26 06:14:11 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 26 Oct 2016 12:14:11 +0200 Subject: [Hawkular-dev] Hawkular Metrics 0.21.0 - Release In-Reply-To: References: Message-ID: Congrats to the team! With respect to HWKMETRICS-510 , how do you make sure that the server is not overwhelmed with retries stored in memory? Is the client response sent after the retry? Thanks 2016-10-25 23:44 GMT+02:00 Stefan Negrea : > Hello Everybody, > I am happy to announce release 0.21.0 of Hawkular Metrics. This release is > anchored by performance enhancements and general fixes. > > Here is a list of major changes: > > - > > Cassandra > - *Cassandra 3.0.9 is now the supported version of Cassandra.* > - Note: this is a rollback from previously supported version of 3.7 > due to Cassandra community recommendations for stability and production > deployment. Cassandra 3.7 or 3.9 are still compatible but development and > testing use 3.0.9 release. > - > > Compression > - Fixed an issue that allowed duplicate instances of the compression > job to get scheduled on server restart (HWKMETRICS-492 > ) > - Improved the fault tolerance of the compression job ( > HWKMETRICS-494 ) > - Improved the performance of the merge process for reading > compressed data (HWKMETRICS-488 > ) > - Fixed wrong ordering when fetching compressed and uncompressed > data (HWKMETRICS-506 > ) > - Compression job provides back pressure (HWKMETRICS-500 > ) > - The job scheduler now handles failure scenarios (HWKMETRICS-505 > ) > - > > Cassandra Schema > - Fixed an issue where the server can fail to start due to Cassalog > being in inconsistent state (HWKMETRICS-495 > ) > - gc_grace_second is set to zero for single node clusters ( > HWKMETRICS-381 ) > - > > API Updates > - Inserting data points has server side retries to increase the fault > tolerance for simple error scenarios (HWKMETRICS-510 > ) > - fromEarliest parameter is now supported in all query endpoints ( > HWKMETRICS-445 ) > - > > Configuration > - The configuration options did not have a consistent naming scheme. > hawkular-metrics, hawkular.metrics, and hawkular prefixes were used > along no prefixes. > - In this release the naming schema has been standardized to > hawkular.metrics.* for metrics specific configuration and hawkular.* > for general configuration. > - Here is list of all configuration options currently available: > ConfigurationKey > > - For more details: HWKMETRICS-508 > > > > > *Hawkular Alerting - included* > > - Version 1.3.0 > > - Project details and repository: Github > > - Documentation: REST API Documentation > , Examples > , Developer > Guide > > > *Hawkular Metrics Clients* > > - Python: https://github.com/hawkular/hawkular-client-python > - Go: https://github.com/hawkular/hawkular-client-go > - Ruby: https://github.com/hawkular/hawkular-client-ruby > - Java: https://github.com/hawkular/hawkular-client-java > > > *Release Links* > > Github Release: https://github.com/hawkular/hawkular-metrics/ > releases/tag/0.21.0 > > JBoss Nexus Maven artifacts: > http://origin-repository.jboss.org/nexus/content/repositorie > s/public/org/hawkular/metrics/ > > Jira release tracker: > https://issues.jboss.org/projects/HWKMETRICS/versions/12331718 > > A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, Joel > Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their project > contributions. > > Thank you, > Stefan Negrea > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161026/f0021937/attachment-0001.html From jsanda at redhat.com Wed Oct 26 09:01:36 2016 From: jsanda at redhat.com (John Sanda) Date: Wed, 26 Oct 2016 09:01:36 -0400 Subject: [Hawkular-dev] Hawkular Metrics 0.21.0 - Release In-Reply-To: References: Message-ID: <91A62EE2-E171-4A9C-9F61-467014D80140@redhat.com> > On Oct 26, 2016, at 6:14 AM, Thomas Segismont wrote: > > Congrats to the team! > > With respect to HWKMETRICS-510 , how do you make sure that the server is not overwhelmed with retries stored in memory? Is the client response sent after the retry? The number of retries and the delay between retries is configurable. It defaults to 5 retries with a max delay of 30 seconds. The response won?t be sent until after the retries. We need to figure out how best to handle scenarios in which the server gets overwhelmed. It is not addressed in HWKMETRICS-510. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161026/15668c40/attachment.html From tsegismo at redhat.com Wed Oct 26 09:50:09 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 26 Oct 2016 15:50:09 +0200 Subject: [Hawkular-dev] Hawkular Metrics 0.21.0 - Release In-Reply-To: <91A62EE2-E171-4A9C-9F61-467014D80140@redhat.com> References: <91A62EE2-E171-4A9C-9F61-467014D80140@redhat.com> Message-ID: Thanks for the details 2016-10-26 15:01 GMT+02:00 John Sanda : > > On Oct 26, 2016, at 6:14 AM, Thomas Segismont wrote: > > Congrats to the team! > > With respect to HWKMETRICS-510 > , how do you make sure > that the server is not overwhelmed with retries stored in memory? Is the > client response sent after the retry? > > > The number of retries and the delay between retries is configurable. It > defaults to 5 retries with a max delay of 30 seconds. The response won?t be > sent until after the retries. We need to figure out how best to handle > scenarios in which the server gets overwhelmed. It is not addressed in > HWKMETRICS-510. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161026/d2c5cc2d/attachment.html From theute at redhat.com Fri Oct 28 06:11:51 2016 From: theute at redhat.com (Thomas Heute) Date: Fri, 28 Oct 2016 12:11:51 +0200 Subject: [Hawkular-dev] Hawkular Inventory Storage and Queries In a Nutshell In-Reply-To: <1484493.1SUJ1YA7jL@localhost.localdomain> References: <1484493.1SUJ1YA7jL@localhost.localdomain> Message-ID: I know it's a relatively old doc, but I finally took the time to read it and would encourage people to read it, it's well written and explain a lot about inventory "internals" through examples On Mon, Sep 12, 2016 at 9:41 PM, Lukas Krejci wrote: > Hi all, > > given the recent discussion about the backend we should be using for > inventory, I've quickly put together a brief explanation of what inventory > actually stores and how it queries things. > > This is expected to raise more questions than to provide answers though > and is > meant as a starter for the discussion that I hope will happen in the > comments > section of the google doc [1]. > > Thanks, > > [1] https://docs.google.com/document/d/ > 1Z1pIQS4EML7_0tuSyHUPX6jzNvaeEqGm4XAiYcRGbvQ/edit?usp=sharing > > -- > Lukas Krejci > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161028/823ccc94/attachment.html From mazz at redhat.com Sat Oct 29 00:51:42 2016 From: mazz at redhat.com (John Mazzitelli) Date: Sat, 29 Oct 2016 00:51:42 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics go client - need assistance for a couple issues In-Reply-To: <1237382365.2109247.1477716292392.JavaMail.zimbra@redhat.com> Message-ID: <1464344723.2109263.1477716702936.JavaMail.zimbra@redhat.com> I found two bugs in the Hawkular Go Client and I need assistance: 1. https://github.com/hawkular/hawkular-client-go/issues/10 2. https://github.com/hawkular/hawkular-client-go/issues/12 The first one I have a PR written that appears to fix it (link to PR in the issue). I would need someone to peer review it and merge it if it is OK. Frankly, I don't see how it ever worked - but I suspect that Hawkular Metrics changed the REST API recently in this area and that's why it wasn't seen before. The second one I do not have a PR, but in the issue I mention a quick and dirty fix, but I don't think it is the proper way to fix it. I need someone to look at that issue and recommend a proper fix. Just need a way or a GoLang API that encodes a string for a URL *path* (not for a URL query string, which is happening here) due to the different ways spaces are encoded. I'd do it myself, but it's really late, and I have to get up early to hopefully meet up with a turkey that is "dying" to make it on my dinner table. Bu Wa Ha... BU WA HAHAHAHAHAHA! :) That is all. --John Mazz From miburman at redhat.com Sun Oct 30 06:08:48 2016 From: miburman at redhat.com (Michael Burman) Date: Sun, 30 Oct 2016 12:08:48 +0200 Subject: [Hawkular-dev] hawkular metrics go client - need assistance for a couple issues In-Reply-To: <1464344723.2109263.1477716702936.JavaMail.zimbra@redhat.com> References: <1464344723.2109263.1477716702936.JavaMail.zimbra@redhat.com> Message-ID: Hi, I replied to both of your issues. Short story here: 1. The old behavior should work, as we promised so in the HWKMETRICS-385. If it does not, then there's a bug in the Hawkular-Metrics, not in the client. We can change this method signature in new versions, but that old API must work (and the tests against HWKMETRICS work also, so I couldn't replicate your bug yet) 2. There's no proper fix for this Golang bug. There's only workarounds and the one you suggested is the one that's used in almost every Golang application that can have whitespace in the URLs. Golang's net/url package is just .. crap and can't follow many standards. - Micke On 10/29/2016 07:51 AM, John Mazzitelli wrote: > I found two bugs in the Hawkular Go Client and I need assistance: > > 1. https://github.com/hawkular/hawkular-client-go/issues/10 > > 2. https://github.com/hawkular/hawkular-client-go/issues/12 > > The first one I have a PR written that appears to fix it (link to PR in the issue). I would need someone to peer review it and merge it if it is OK. Frankly, I don't see how it ever worked - but I suspect that Hawkular Metrics changed the REST API recently in this area and that's why it wasn't seen before. > > The second one I do not have a PR, but in the issue I mention a quick and dirty fix, but I don't think it is the proper way to fix it. I need someone to look at that issue and recommend a proper fix. Just need a way or a GoLang API that encodes a string for a URL *path* (not for a URL query string, which is happening here) due to the different ways spaces are encoded. I'd do it myself, but it's really late, and I have to get up early to hopefully meet up with a turkey that is "dying" to make it on my dinner table. Bu Wa Ha... BU WA HAHAHAHAHAHA! :) > > That is all. > > --John Mazz > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From mazz at redhat.com Sun Oct 30 08:22:58 2016 From: mazz at redhat.com (John Mazzitelli) Date: Sun, 30 Oct 2016 08:22:58 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics go client - need assistance for a couple issues In-Reply-To: References: <1464344723.2109263.1477716702936.JavaMail.zimbra@redhat.com> Message-ID: <19021533.2268896.1477830178603.JavaMail.zimbra@redhat.com> > 1. The old behavior should work, as we promised so in the > HWKMETRICS-385. If it does not, then there's a bug in the > Hawkular-Metrics, not in the client. We can change this method signature > in new versions, but that old API must work (and the tests against > HWKMETRICS work also, so I couldn't replicate your bug yet) OK. I noticed the tests in client worked for both. I couldn't understand why. Now I know :) Didn't know about that JIRA. I'll try to replicate my problem and see what's going on - I was definitely getting an error code from H-Metrics until I ran with my PR code. > 2. There's no proper fix for this Golang bug. There's only workarounds > and the one you suggested is the one that's used in almost every Golang > application that can have whitespace in the URLs. Golang's net/url > package is just .. crap and can't follow many standards. That is --- unfortunate. I'd implement my stupid fix but, what happens if there is a literal "+" in the string? Bah... "just don't use spaces in tag names" seems extreme though. From mazz at redhat.com Sun Oct 30 22:34:55 2016 From: mazz at redhat.com (John Mazzitelli) Date: Sun, 30 Oct 2016 22:34:55 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics go client - need assistance for a couple issues In-Reply-To: <19021533.2268896.1477830178603.JavaMail.zimbra@redhat.com> References: <1464344723.2109263.1477716702936.JavaMail.zimbra@redhat.com> <19021533.2268896.1477830178603.JavaMail.zimbra@redhat.com> Message-ID: <1237249360.2343820.1477881295836.JavaMail.zimbra@redhat.com> > > 1. The old behavior should work, as we promised so in the > > HWKMETRICS-385. If it does not, then there's a bug in the > > Hawkular-Metrics, not in the client. We can change this method signature > > in new versions, but that old API must work (and the tests against > > HWKMETRICS work also, so I couldn't replicate your bug yet) > > OK. I noticed the tests in client worked for both. I couldn't understand why. > Now I know :) Didn't know about that JIRA. > > I'll try to replicate my problem and see what's going on - I was definitely > getting an error code from H-Metrics until I ran with my PR code. I replicated the problem. It is a bug in the hawkular Go client - see: https://github.com/hawkular/hawkular-client-go/issues/10 Short story - if tags have spaces in their values, Go client builds a bad DELETE URL and the delete fails with a 400 I submitted a second PR - but I don't know if that would be backward compatible either. So there are two PRs that fix this problem, but I don't know if either one is good enough. But somehow we need to fix this, or else we cannot support tags with spaces in their values (this is different than the issue where spaces are in their NAMES). > > 2. There's no proper fix for this Golang bug. There's only workarounds > > and the one you suggested is the one that's used in almost every Golang > > application that can have whitespace in the URLs. Golang's net/url > > package is just .. crap and can't follow many standards. > > That is --- unfortunate. I'd implement my stupid fix but, what happens if > there is a literal "+" in the string? Bah... "just don't use spaces in tag > names" seems extreme though. Just to be clear, this is different than the above because here we are talking about spaces in the tag NAMES. So, as of right now, the Go client cannot support tag names OR values with spaces in them. That's going to be a limitation we need to document, unless we fix these two issues. I'll leave that up to Micke how best to solve these - because this involves backward compatibility issues with older Hawkular Metrics servers and he'd know best what to do there. I have two PRs for the first one, and a suggested fix documented in the github issue for the second - but I don't know if any are good enough to push to master. From mazz at redhat.com Sun Oct 30 22:41:31 2016 From: mazz at redhat.com (John Mazzitelli) Date: Sun, 30 Oct 2016 22:41:31 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics go client - need assistance for a couple issues In-Reply-To: <1237249360.2343820.1477881295836.JavaMail.zimbra@redhat.com> References: <1464344723.2109263.1477716702936.JavaMail.zimbra@redhat.com> <19021533.2268896.1477830178603.JavaMail.zimbra@redhat.com> <1237249360.2343820.1477881295836.JavaMail.zimbra@redhat.com> Message-ID: <1464476167.2344548.1477881691080.JavaMail.zimbra@redhat.com> > Just to be clear, this is different than the above because here we are > talking about spaces in the tag NAMES. > > So, as of right now, the Go client cannot support tag names OR values with > spaces in them. That's going to be a limitation we need to document, unless > we fix these two issues. Sorry - I misspoke. It is more than spaces in tag names, the problem is when spaces are in METRIC IDs. I suspect spaces in tag names will also fail, but for sure, spaces in METRIC IDs cause the failure. From hrupp at redhat.com Mon Oct 31 03:39:46 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 31 Oct 2016 08:39:46 +0100 Subject: [Hawkular-dev] Labeling needs ? Message-ID: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> Hey, we have labels in Hawkular-metrics right now, but apparently there are use cases that are not yet covered (and I know Matt has more) - Listing keys of tags. Currently one has to know the available keys to be able to list the available values - tag values are currently a comma-separated string and not an array, which may have implications on allowed characters and escaping of separators - Post-tagging of data points (tagging could be provoked by another system that e.g. parses log files and sees an anomaly) The idea behind it is that one should be able to create tags at certain points in time for a single metric or a list of metrics to uniquely identify that point in time for queries (relative performance of two versions of a deployment). What else? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161031/61f5169a/attachment.html From jtakvori at redhat.com Mon Oct 31 04:29:09 2016 From: jtakvori at redhat.com (Joel Takvorian) Date: Mon, 31 Oct 2016 09:29:09 +0100 Subject: [Hawkular-dev] Labeling needs ? In-Reply-To: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> References: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> Message-ID: +1 for listing keys when we don't know the metric ids: as you say we need to know them, or to fetch all metrics definitions, extract and aggregate tag names; so it's not impossible but a more user friendly method would be appreciated. On Mon, Oct 31, 2016 at 8:39 AM, Heiko W.Rupp wrote: > Hey, > > we have labels in Hawkular-metrics right now, but apparently there are > use cases that are not yet covered (and I know Matt has more) > > - Listing keys of tags. Currently one has to know the available keys > to be able to list the available values > - tag values are currently a comma-separated string and not an array, > which may have implications on allowed characters and escaping of separators > - Post-tagging of data points (tagging could be provoked by another > system that e.g. parses log files and sees an anomaly) The idea behind it > is that one should be able to create tags at certain points in time for a > single metric or a list of metrics to uniquely identify that point in time > for queries (relative performance of two versions of a deployment). > > What else? > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161031/c70c011a/attachment.html From jsanda at redhat.com Mon Oct 31 10:22:19 2016 From: jsanda at redhat.com (John Sanda) Date: Mon, 31 Oct 2016 10:22:19 -0400 Subject: [Hawkular-dev] Labeling needs ? In-Reply-To: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> References: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> Message-ID: > On Oct 31, 2016, at 3:39 AM, Heiko W.Rupp wrote: > > Hey, > > we have labels in Hawkular-metrics right now, but apparently there are > use cases that are not yet covered (and I know Matt has more) > > Listing keys of tags. Currently one has to know the available keys to be able to list the available values > tag values are currently a comma-separated string and not an array, which may have implications on allowed characters and escaping of separators > Post-tagging of data points (tagging could be provoked by another system that e.g. parses log files and sees an anomaly) The idea behind it is that one should be able to create tags at certain points in time for a single metric or a list of metrics to uniquely identify that point in time for queries (relative performance of two versions of a deployment). Data points are immutable by design. Once a row is written it is never updated. This allows for optimizations with compaction and deletions. We need to take that into consideration with post-tagging. Using a different table or tables might be a better option. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161031/ff4e29be/attachment.html From mwringe at redhat.com Mon Oct 31 17:30:47 2016 From: mwringe at redhat.com (Matt Wringe) Date: Mon, 31 Oct 2016 17:30:47 -0400 (EDT) Subject: [Hawkular-dev] Labeling needs ? In-Reply-To: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> References: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> Message-ID: <728118784.11512773.1477949447663.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Heiko W.Rupp" > To: "Discussions around Hawkular development" > Sent: Monday, 31 October, 2016 3:39:46 AM > Subject: [Hawkular-dev] Labeling needs ? > > > > Hey, > > we have labels in Hawkular-metrics right now, but apparently there are > use cases that are not yet covered (and I know Matt has more) > > * Listing keys of tags. Currently one has to know the available keys to > be able to list the available values > * tag values are currently a comma-separated string and not an array, > which may have implications on allowed characters and escaping of > separators > * Post-tagging of data points (tagging could be provoked by another > system that e.g. parses log files and sees an anomaly) The idea behind > it is that one should be able to create tags at certain points in time > for a single metric or a list of metrics to uniquely identify that point > in time for queries (relative performance of two versions of a > deployment). > > > What else? > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mwringe at redhat.com Mon Oct 31 17:59:16 2016 From: mwringe at redhat.com (Matt Wringe) Date: Mon, 31 Oct 2016 17:59:16 -0400 (EDT) Subject: [Hawkular-dev] Labeling needs ? In-Reply-To: References: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> Message-ID: <1109667729.11515636.1477951156706.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "John Sanda" > To: "Discussions around Hawkular development" > Sent: Monday, 31 October, 2016 10:22:19 AM > Subject: Re: [Hawkular-dev] Labeling needs ? > > > > > > On Oct 31, 2016, at 3:39 AM, Heiko W.Rupp < hrupp at redhat.com > wrote: > > > > Hey, > > we have labels in Hawkular-metrics right now, but apparently there are > use cases that are not yet covered (and I know Matt has more) > > * Listing keys of tags. Currently one has to know the available keys to > be able to list the available values hmm, interesting, we can list all the values values of a key, but not get the list of the key itself? > * tag values are currently a comma-separated string and not an array, > which may have implications on allowed characters and escaping of > separators The problem with OpenShift is that we have hierarchical data that we would like to store as tags so that we can perform queries on them. Most of our data that we store as tags are simple key:value pairs, where the value is a string. This works nicely with how tags work. But the more important tag we want to keep track of is labels, and in this case the value is an array of key:value pairs, which doesn't work too nicely. We store this as a comma separated string and its possible to use regex in this case, but it starts to be really messy to write and is prone to errors. Its really not a great solution and looks really bad when people have to write these complex queries which should be really easy. > * Post-tagging of data points (tagging could be provoked by another > system that e.g. parses log files and sees an anomaly) The idea behind > it is that one should be able to create tags at certain points in time > for a single metric or a list of metrics to uniquely identify that point > in time for queries (relative performance of two versions of a > deployment). This feels like an 'event' what could be marked. I don't know if it needs to be applied to a datapoint directly, or stored in a separate list, or if it even needs to be stored in Hawkular Metrics. The main thing I think is to be able to query the list of events which have occurred over a specific time frame. OpenShift has a lot of events it can gather, it could be really cool if we could display these events as marks in the graphs. I don't know if we can use Hawkular Metric string metrics to store these, or something in Alerts to do it. Heapster has a component to gather these events for us (https://github.com/kubernetes/heapster/tree/master/events). > Data points are immutable by design. Once a row is written it is never > updated. This allows for optimizations with compaction and deletions. We > need to take that into consideration with post-tagging. Using a different > table or tables might be a better option. Do individual data points have tags applied to them? Or only the overall metric? We may want to keep track of historic changes to tags. I know in OpenShift we store values in tags which can change at any point in time but I don't believe we can query what the values was at time X. From jsanda at redhat.com Mon Oct 31 23:11:17 2016 From: jsanda at redhat.com (John Sanda) Date: Mon, 31 Oct 2016 23:11:17 -0400 Subject: [Hawkular-dev] Labeling needs ? In-Reply-To: <1109667729.11515636.1477951156706.JavaMail.zimbra@redhat.com> References: <92C3A812-05CA-4C88-80D4-EC9D3AB4ADDE@redhat.com> <1109667729.11515636.1477951156706.JavaMail.zimbra@redhat.com> Message-ID: > On Oct 31, 2016, at 5:59 PM, Matt Wringe wrote: > > ----- Original Message ----- >> From: "John Sanda" >> To: "Discussions around Hawkular development" >> Sent: Monday, 31 October, 2016 10:22:19 AM >> Subject: Re: [Hawkular-dev] Labeling needs ? >> >> >> >> >> >> On Oct 31, 2016, at 3:39 AM, Heiko W.Rupp < hrupp at redhat.com > wrote: >> >> >> >> Hey, >> >> we have labels in Hawkular-metrics right now, but apparently there are >> use cases that are not yet covered (and I know Matt has more) >> >> * Listing keys of tags. Currently one has to know the available keys to >> be able to list the available values > > hmm, interesting, we can list all the values values of a key, but not get the list of the key itself? That is correct. We simply have not had a need for it thus far. Adding endpoints to query tags is straightforward. In fact, it would be a great ticket to work on for anyone looking to get involved in the project. > >> * tag values are currently a comma-separated string and not an array, >> which may have implications on allowed characters and escaping of >> separators > > The problem with OpenShift is that we have hierarchical data that we would like to store as tags so that we can perform queries on them. > > Most of our data that we store as tags are simple key:value pairs, where the value is a string. This works nicely with how tags work. > > But the more important tag we want to keep track of is labels, and in this case the value is an array of key:value pairs, which doesn't work too nicely. We store this as a comma separated string and its possible to use regex in this case, but it starts to be really messy to write and is prone to errors. Its really not a great solution and looks really bad when people have to write these complex queries which should be really easy. > >> * Post-tagging of data points (tagging could be provoked by another >> system that e.g. parses log files and sees an anomaly) The idea behind >> it is that one should be able to create tags at certain points in time >> for a single metric or a list of metrics to uniquely identify that point >> in time for queries (relative performance of two versions of a >> deployment). > > This feels like an 'event' what could be marked. I don't know if it needs to be applied to a datapoint directly, or stored in a separate list, or if it even needs to be stored in Hawkular Metrics. > > The main thing I think is to be able to query the list of events which have occurred over a specific time frame. > > OpenShift has a lot of events it can gather, it could be really cool if we could display these events as marks in the graphs. I don't know if we can use Hawkular Metric string metrics to store these, or something in Alerts to do it. Heapster has a component to gather these events for us (https://github.com/kubernetes/heapster/tree/master/events ). > >> Data points are immutable by design. Once a row is written it is never >> updated. This allows for optimizations with compaction and deletions. We >> need to take that into consideration with post-tagging. Using a different >> table or tables might be a better option. > > Do individual data points have tags applied to them? Or only the overall metric? > > We may want to keep track of historic changes to tags. I know in OpenShift we store values in tags which can change at any point in time but I don't believe we can query what the values was at time X. > When the oVirt team was looking at Hawkular Metrics, they too were asking for a history of tag changes. Would this be a generally useful feature? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20161031/c8495989/attachment-0001.html