From theute at redhat.com Wed Apr 1 09:15:42 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 01 Apr 2015 14:15:42 +0100 Subject: [Hawkular-dev] Public Hawkular Instances In-Reply-To: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> References: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> Message-ID: <551BEF7E.7090605@redhat.com> Very cool, thanks :) I guess both instances are using embedded Cassandra ? Once we have multitenant and expiration/aggregation working we should use a separate Cassandra to keep the data as long as we can. Thomas On 03/31/2015 03:38 PM, Matthew Mahoney wrote: > A couple notes on the Public Hawkular instance: > > 1) Improvements have been made to the Public Hawkular instance > (http://209.132.178.218:18080 ) smoke > test, as to mitigate the number of false failures that were happening. > > /We invite & encourage community use of this instance./ > > 2) A new Public Hawkular instance (http://209.132.178.218:18090 > ) has been created which is intended to > be used by Development/community for those use-cases such as Demos where > you need to ensure that the instance will not be randomly restarted (for > reasons such as when new Hawkular images are created, test cases git > pushes, infrastructure improvements, etc). > This instance is an on-demand build only. Please ping/email me > (mmahoney at redhat.com ) if you want access to > the Jenkins Hawkular Dev build job. > > Thanks, > > Matt > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mmahoney at redhat.com Wed Apr 1 11:17:34 2015 From: mmahoney at redhat.com (Matthew Mahoney) Date: Wed, 1 Apr 2015 11:17:34 -0400 (EDT) Subject: [Hawkular-dev] Public Hawkular Instances In-Reply-To: <551BEF7E.7090605@redhat.com> References: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> <551BEF7E.7090605@redhat.com> Message-ID: <387763148.10182892.1427901454219.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Thomas Heute" > To: "Discussions around Hawkular development" > Sent: Wednesday, April 1, 2015 9:15:42 AM > Subject: Re: [Hawkular-dev] Public Hawkular Instances > > Very cool, thanks :) > > I guess both instances are using embedded Cassandra ? > That is correct. > Once we have multitenant and expiration/aggregation working we should > use a separate Cassandra to keep the data as long as we can. > Sure - let's discuss when a separate Cassandra is ready. > Thomas > > On 03/31/2015 03:38 PM, Matthew Mahoney wrote: > > A couple notes on the Public Hawkular instance: > > > > 1) Improvements have been made to the Public Hawkular instance > > (http://209.132.178.218:18080 ) smoke > > test, as to mitigate the number of false failures that were happening. > > > > /We invite & encourage community use of this instance./ > > > > 2) A new Public Hawkular instance (http://209.132.178.218:18090 > > ) has been created which is intended to > > be used by Development/community for those use-cases such as Demos where > > you need to ensure that the instance will not be randomly restarted (for > > reasons such as when new Hawkular images are created, test cases git > > pushes, infrastructure improvements, etc). > > This instance is an on-demand build only. Please ping/email me > > (mmahoney at redhat.com ) if you want access to > > the Jenkins Hawkular Dev build job. > > > > Thanks, > > > > Matt > > > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From theute at redhat.com Wed Apr 1 13:01:56 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 01 Apr 2015 18:01:56 +0100 Subject: [Hawkular-dev] Public Hawkular Instances In-Reply-To: <387763148.10182892.1427901454219.JavaMail.zimbra@redhat.com> References: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> <551BEF7E.7090605@redhat.com> <387763148.10182892.1427901454219.JavaMail.zimbra@redhat.com> Message-ID: <551C2484.7070206@redhat.com> Matt has a separate Cassandra docker container we could use I guess, but I think the main problem is to not run out of disk space. Thomas On 04/01/2015 04:17 PM, Matthew Mahoney wrote: > > > ----- Original Message ----- >> From: "Thomas Heute" >> To: "Discussions around Hawkular development" >> Sent: Wednesday, April 1, 2015 9:15:42 AM >> Subject: Re: [Hawkular-dev] Public Hawkular Instances >> >> Very cool, thanks :) >> >> I guess both instances are using embedded Cassandra ? >> > > That is correct. > >> Once we have multitenant and expiration/aggregation working we should >> use a separate Cassandra to keep the data as long as we can. >> > > Sure - let's discuss when a separate Cassandra is ready. > >> Thomas >> >> On 03/31/2015 03:38 PM, Matthew Mahoney wrote: >>> A couple notes on the Public Hawkular instance: >>> >>> 1) Improvements have been made to the Public Hawkular instance >>> (http://209.132.178.218:18080 ) smoke >>> test, as to mitigate the number of false failures that were happening. >>> >>> /We invite & encourage community use of this instance./ >>> >>> 2) A new Public Hawkular instance (http://209.132.178.218:18090 >>> ) has been created which is intended to >>> be used by Development/community for those use-cases such as Demos where >>> you need to ensure that the instance will not be randomly restarted (for >>> reasons such as when new Hawkular images are created, test cases git >>> pushes, infrastructure improvements, etc). >>> This instance is an on-demand build only. Please ping/email me >>> (mmahoney at redhat.com ) if you want access to >>> the Jenkins Hawkular Dev build job. >>> >>> Thanks, >>> >>> Matt >>> >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jsanda at redhat.com Wed Apr 1 13:16:16 2015 From: jsanda at redhat.com (John Sanda) Date: Wed, 1 Apr 2015 13:16:16 -0400 Subject: [Hawkular-dev] Public Hawkular Instances In-Reply-To: <551C2484.7070206@redhat.com> References: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> <551BEF7E.7090605@redhat.com> <387763148.10182892.1427901454219.JavaMail.zimbra@redhat.com> <551C2484.7070206@redhat.com> Message-ID: We already have support for setting data retention. It currently defaults to 7 days. > On Apr 1, 2015, at 1:01 PM, Thomas Heute wrote: > > Matt has a separate Cassandra docker container we could use I guess, but > I think the main problem is to not run out of disk space. > > Thomas > > On 04/01/2015 04:17 PM, Matthew Mahoney wrote: >> >> >> ----- Original Message ----- >>> From: "Thomas Heute" >>> To: "Discussions around Hawkular development" >>> Sent: Wednesday, April 1, 2015 9:15:42 AM >>> Subject: Re: [Hawkular-dev] Public Hawkular Instances >>> >>> Very cool, thanks :) >>> >>> I guess both instances are using embedded Cassandra ? >>> >> >> That is correct. >> >>> Once we have multitenant and expiration/aggregation working we should >>> use a separate Cassandra to keep the data as long as we can. >>> >> >> Sure - let's discuss when a separate Cassandra is ready. >> >>> Thomas >>> >>> On 03/31/2015 03:38 PM, Matthew Mahoney wrote: >>>> A couple notes on the Public Hawkular instance: >>>> >>>> 1) Improvements have been made to the Public Hawkular instance >>>> (http://209.132.178.218:18080 ) smoke >>>> test, as to mitigate the number of false failures that were happening. >>>> >>>> /We invite & encourage community use of this instance./ >>>> >>>> 2) A new Public Hawkular instance (http://209.132.178.218:18090 >>>> ) has been created which is intended to >>>> be used by Development/community for those use-cases such as Demos where >>>> you need to ensure that the instance will not be randomly restarted (for >>>> reasons such as when new Hawkular images are created, test cases git >>>> pushes, infrastructure improvements, etc). >>>> This instance is an on-demand build only. Please ping/email me >>>> (mmahoney at redhat.com ) if you want access to >>>> the Jenkins Hawkular Dev build job. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Thu Apr 2 07:06:25 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 02 Apr 2015 12:06:25 +0100 Subject: [Hawkular-dev] Public Hawkular Instances In-Reply-To: <551C2484.7070206@redhat.com> References: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> <551BEF7E.7090605@redhat.com> <387763148.10182892.1427901454219.JavaMail.zimbra@redhat.com> <551C2484.7070206@redhat.com> Message-ID: On 1 Apr 2015, at 18:01, Thomas Heute wrote: > Matt has a separate Cassandra docker container we could use I guess, > but > I think the main problem is to not run out of disk space. The other issue to look into is the remainder of $wildfly/standalone/data, as this will (at least for some time) also contain data like alert trigger definitions. From vrockai at redhat.com Thu Apr 2 11:20:44 2015 From: vrockai at redhat.com (Viliam Rockai) Date: Thu, 02 Apr 2015 17:20:44 +0200 Subject: [Hawkular-dev] Kettle build profiles Message-ID: <1427988044.2024.5.camel@vrockai-laptop> Hello team, TL;DR: Feel free to use -Pdev profile during the kettle build again. It should not break the console stuff anymore. There was a lot of confusion about the -Pdev profile in the hawkular/hawkular project. Originally this was created just to provide caching of bower packages and node.js stuff during the console build. But then, there was some new functionality added to the profile, more people started to use it and caching caused issues. To fix that, I've renamed the "caching" profile (from "dev") to "cache". So now we have three profiles available during the kettle build: 1. -Pdev: Useful for everyone and safe. It unzips the assembly, so it can be started directly from /target directory and creates a default user. 2. -Pcache: Useful for everyone but potentionaly dangerous. Using this profile only makes sense with the "clean" lifecycle. It basically just excludes the node.js stuff from the clean task, so those files are not deleted => they are not downloaded again during the following build. However, if there are changes in JS packages/JS build files used by the console, you will still see the old/cached ones or even won't be able to build the console. If you're using the -Pcache profile and the console build fails, try to rebuild without using this profile. 3. -Plink: Useful for UI developers. This will use the local version of the hawkular-ui-components. More info: https://github.com/hawkular/hawkular/tree/master/ui#hawkular-ui-components-development Viliam From mazz at redhat.com Thu Apr 2 17:51:31 2015 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 2 Apr 2015 17:51:31 -0400 (EDT) Subject: [Hawkular-dev] start of message bus REST client In-Reply-To: <409661762.9245311.1428011168651.JavaMail.zimbra@redhat.com> Message-ID: <1086799654.9246235.1428011491691.JavaMail.zimbra@redhat.com> I created a very simple REST client that you can use to send any json string as a bus message. It requires no dependencies other than apache httpclient and jboss logging. See: https://github.com/hawkular/hawkular-bus/tree/master/hawkular-bus-rest-client new org.hawkular.bus.restclient.RestClient(url).postTopicMessage(topicName, jsonPayload, headers); My agent uses it to send metrics to the hawkular bus's metric topic. I use REST rather than the bus-common API because (AFAIK) the thinking is this agent can be deployed in something like WildFly Core where JMS API isn't even available. So right now, this agent should be sending data just like the Pinger - its sending metrics directly to Metrics REST interface for storage and sending those same metrics to the message bus's metric topic so other things like alerts can process it. Unfortunately, the kettle UI can't show any of it since it is tightly related to the Pinger's metrics and nothing else. But I think its working :) From lkrejci at redhat.com Fri Apr 3 05:26:52 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Fri, 03 Apr 2015 11:26:52 +0200 Subject: [Hawkular-dev] start of message bus REST client In-Reply-To: <1086799654.9246235.1428011491691.JavaMail.zimbra@redhat.com> References: <1086799654.9246235.1428011491691.JavaMail.zimbra@redhat.com> Message-ID: <2329135.JUMlvSkmPs@localhost.localdomain> Ok, with the Jinn out of the lamp, let's move the discussion we've had privately amongst the team-leads to the public forum :) With the ability to send rest messages to the bus we need to solve 1 fundamental question: *Who is supposed be sending what kind of messages?* It is clear, that any interested party should be able to listen to any message. But I am concerned about the "source" of messages. For example I think it is wrong to send the message about metric being collected to the bus prior to it being stored. It could happen that the system reacts to something that we will have no record about later on. But with more "involved" components that do more work than just store data upon arrival, the consequences of being out of sync between what's stored and what's being reacted upon are potentially worse. As an example, consider this: * someone sends a "resource inventoried" message to the bus - what should inventory think about it? Did the author make sure to do everything exactly like inventory would if it sent the message itself? * someone sends an "alert fired" message to the bus - what should alerts do about it? * someone DDoSes the the bus I may be old fashioned in this, but I prefer consistency over eventual consistency where possible. I don't think it would be too much of a performance problem to architect the message flow like this: 1) anyone can listen on the bus 2) only Hawkular components can send messages to the bus (or rather the hawkular glue code built atop the standalone components) 3) 3rd parties (including feeds) can send data only to the component endpoints (which will upon processing generate appropriate messages on the bus) I am no expert in either distributed architectures or in microservices, so this view may be naive and I'd love to be proven wrong about it, but I think the concerns I raise are real and we should have a solid answer to them whatever architecture we choose. Cheers, Lukas On Thursday, April 02, 2015 17:51:31 John Mazzitelli wrote: > I created a very simple REST client that you can use to send any json string > as a bus message. > > It requires no dependencies other than apache httpclient and jboss logging. > > See: > https://github.com/hawkular/hawkular-bus/tree/master/hawkular-bus-rest-clie > nt > > new > org.hawkular.bus.restclient.RestClient(url).postTopicMessage(topicName, > jsonPayload, headers); > > My agent uses it to send metrics to the hawkular bus's metric topic. I use > REST rather than the bus-common API because (AFAIK) the thinking is this > agent can be deployed in something like WildFly Core where JMS API isn't > even available. > > So right now, this agent should be sending data just like the Pinger - its > sending metrics directly to Metrics REST interface for storage and sending > those same metrics to the message bus's metric topic so other things like > alerts can process it. Unfortunately, the kettle UI can't show any of it > since it is tightly related to the Pinger's metrics and nothing else. But I > think its working :) _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From mazz at redhat.com Fri Apr 3 10:32:59 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 3 Apr 2015 10:32:59 -0400 (EDT) Subject: [Hawkular-dev] start of message bus REST client In-Reply-To: <2329135.JUMlvSkmPs@localhost.localdomain> References: <1086799654.9246235.1428011491691.JavaMail.zimbra@redhat.com> <2329135.JUMlvSkmPs@localhost.localdomain> Message-ID: <229827221.9445883.1428071579447.JavaMail.zimbra@redhat.com> (I'm in PTO-mode, so I don't have an intelligible response to this yet but...) Just for the record, the only reason why I coded up the wildfly agent to both send over the metrics REST API and put a metric message on the bus was because (and only because) that's what the Pinger does today and since the Pinger is the thing that is "working" today, I just wanted to copy what it does so the agent can "work" too. I by no means am advocating that we have feeds send both messages (I don't think anyone is saying that's the long term solution) - this is just following Heiko's mantra "just get something to work" :) As to whether feeds can or should be able to send messages on the bus, I can be convinced either way. ----- Original Message ----- > Ok, with the Jinn out of the lamp, let's move the discussion we've had > privately amongst the team-leads to the public forum :) > > With the ability to send rest messages to the bus we need to solve 1 > fundamental question: > > *Who is supposed be sending what kind of messages?* > > It is clear, that any interested party should be able to listen to any > message. > > But I am concerned about the "source" of messages. > > For example I think it is wrong to send the message about metric being > collected to the bus prior to it being stored. It could happen that the > system > reacts to something that we will have no record about later on. > > But with more "involved" components that do more work than just store data > upon arrival, the consequences of being out of sync between what's stored and > what's being reacted upon are potentially worse. > > As an example, consider this: > * someone sends a "resource inventoried" message to the bus - what should > inventory think about it? Did the author make sure to do everything exactly > like inventory would if it sent the message itself? > * someone sends an "alert fired" message to the bus - what should alerts do > about it? > * someone DDoSes the the bus > > I may be old fashioned in this, but I prefer consistency over eventual > consistency where possible. I don't think it would be too much of a > performance problem to architect the message flow like this: > > 1) anyone can listen on the bus > 2) only Hawkular components can send messages to the bus (or rather the > hawkular glue code built atop the standalone components) > 3) 3rd parties (including feeds) can send data only to the component > endpoints > (which will upon processing generate appropriate messages on the bus) > > I am no expert in either distributed architectures or in microservices, so > this view may be naive and I'd love to be proven wrong about it, but I think > the concerns I raise are real and we should have a solid answer to them > whatever architecture we choose. > > Cheers, > > Lukas > > On Thursday, April 02, 2015 17:51:31 John Mazzitelli wrote: > > I created a very simple REST client that you can use to send any json > > string > > as a bus message. > > > > It requires no dependencies other than apache httpclient and jboss logging. > > > > See: > > https://github.com/hawkular/hawkular-bus/tree/master/hawkular-bus-rest-clie > > nt > > > > new > > org.hawkular.bus.restclient.RestClient(url).postTopicMessage(topicName, > > jsonPayload, headers); > > > > My agent uses it to send metrics to the hawkular bus's metric topic. I use > > REST rather than the bus-common API because (AFAIK) the thinking is this > > agent can be deployed in something like WildFly Core where JMS API isn't > > even available. > > > > So right now, this agent should be sending data just like the Pinger - its > > sending metrics directly to Metrics REST interface for storage and sending > > those same metrics to the message bus's metric topic so other things like > > alerts can process it. Unfortunately, the kettle UI can't show any of it > > since it is tightly related to the Pinger's metrics and nothing else. But I > > think its working :) _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From lponce at redhat.com Mon Apr 6 03:19:12 2015 From: lponce at redhat.com (Lucas Ponce) Date: Mon, 6 Apr 2015 03:19:12 -0400 (EDT) Subject: [Hawkular-dev] Public Hawkular Instances In-Reply-To: References: <594183060.9221248.1427812686912.JavaMail.zimbra@redhat.com> <551BEF7E.7090605@redhat.com> <387763148.10182892.1427901454219.JavaMail.zimbra@redhat.com> <551C2484.7070206@redhat.com> Message-ID: <559094017.9680669.1428304752559.JavaMail.zimbra@redhat.com> Hi, > > On 1 Apr 2015, at 18:01, Thomas Heute wrote: > > > Matt has a separate Cassandra docker container we could use I guess, > > but > > I think the main problem is to not run out of disk space. > > The other issue to look into is the remainder of > $wildfly/standalone/data, > as this will (at least for some time) also contain data like alert > trigger definitions. hawkular-alerts use the $wildfly/standalone/data to pre-populate triggers definitions. This is used just for demo purposes, if no data is found simply there are no definitions pre-populated but the alerts component works without any restriction. These init files was setted just for demo/poc reason and we can move them to a proper way. Is there any suggestion about which is preferred location for it ? Thanks, Lucas From gbrown at redhat.com Tue Apr 7 03:54:05 2015 From: gbrown at redhat.com (Gary Brown) Date: Tue, 7 Apr 2015 03:54:05 -0400 (EDT) Subject: [Hawkular-dev] start of message bus REST client In-Reply-To: <229827221.9445883.1428071579447.JavaMail.zimbra@redhat.com> References: <1086799654.9246235.1428011491691.JavaMail.zimbra@redhat.com> <2329135.JUMlvSkmPs@localhost.localdomain> <229827221.9445883.1428071579447.JavaMail.zimbra@redhat.com> Message-ID: <2124223250.10010240.1428393245229.JavaMail.zimbra@redhat.com> I agree the feed sending both messages is not ideal, so eventually it would be good if the component is responsible for sending out its "this has occurred" message. However it would be good if the bus could still be a general resource that any app could make use of. So a possible compromise - have certain 'reserved' destinations that only hawkular components can publish too - allowing any app to publish to other topics? To enable any app to publish an event that will be consumed by the alert engine, I would assume the destination used by the alert engine would not be restricted. Regards Gary ----- Original Message ----- > (I'm in PTO-mode, so I don't have an intelligible response to this yet > but...) > > Just for the record, the only reason why I coded up the wildfly agent to both > send over the metrics REST API and put a metric message on the bus was > because (and only because) that's what the Pinger does today and since the > Pinger is the thing that is "working" today, I just wanted to copy what it > does so the agent can "work" too. > > I by no means am advocating that we have feeds send both messages (I don't > think anyone is saying that's the long term solution) - this is just > following Heiko's mantra "just get something to work" :) > > As to whether feeds can or should be able to send messages on the bus, I can > be convinced either way. > > ----- Original Message ----- > > Ok, with the Jinn out of the lamp, let's move the discussion we've had > > privately amongst the team-leads to the public forum :) > > > > With the ability to send rest messages to the bus we need to solve 1 > > fundamental question: > > > > *Who is supposed be sending what kind of messages?* > > > > It is clear, that any interested party should be able to listen to any > > message. > > > > But I am concerned about the "source" of messages. > > > > For example I think it is wrong to send the message about metric being > > collected to the bus prior to it being stored. It could happen that the > > system > > reacts to something that we will have no record about later on. > > > > But with more "involved" components that do more work than just store data > > upon arrival, the consequences of being out of sync between what's stored > > and > > what's being reacted upon are potentially worse. > > > > As an example, consider this: > > * someone sends a "resource inventoried" message to the bus - what should > > inventory think about it? Did the author make sure to do everything exactly > > like inventory would if it sent the message itself? > > * someone sends an "alert fired" message to the bus - what should alerts do > > about it? > > * someone DDoSes the the bus > > > > I may be old fashioned in this, but I prefer consistency over eventual > > consistency where possible. I don't think it would be too much of a > > performance problem to architect the message flow like this: > > > > 1) anyone can listen on the bus > > 2) only Hawkular components can send messages to the bus (or rather the > > hawkular glue code built atop the standalone components) > > 3) 3rd parties (including feeds) can send data only to the component > > endpoints > > (which will upon processing generate appropriate messages on the bus) > > > > I am no expert in either distributed architectures or in microservices, so > > this view may be naive and I'd love to be proven wrong about it, but I > > think > > the concerns I raise are real and we should have a solid answer to them > > whatever architecture we choose. > > > > Cheers, > > > > Lukas > > > > On Thursday, April 02, 2015 17:51:31 John Mazzitelli wrote: > > > I created a very simple REST client that you can use to send any json > > > string > > > as a bus message. > > > > > > It requires no dependencies other than apache httpclient and jboss > > > logging. > > > > > > See: > > > https://github.com/hawkular/hawkular-bus/tree/master/hawkular-bus-rest-clie > > > nt > > > > > > new > > > org.hawkular.bus.restclient.RestClient(url).postTopicMessage(topicName, > > > jsonPayload, headers); > > > > > > My agent uses it to send metrics to the hawkular bus's metric topic. I > > > use > > > REST rather than the bus-common API because (AFAIK) the thinking is this > > > agent can be deployed in something like WildFly Core where JMS API isn't > > > even available. > > > > > > So right now, this agent should be sending data just like the Pinger - > > > its > > > sending metrics directly to Metrics REST interface for storage and > > > sending > > > those same metrics to the message bus's metric topic so other things like > > > alerts can process it. Unfortunately, the kettle UI can't show any of it > > > since it is tightly related to the Pinger's metrics and nothing else. But > > > I > > > think its working :) _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mithomps at redhat.com Tue Apr 7 13:40:55 2015 From: mithomps at redhat.com (mike thompson) Date: Tue, 7 Apr 2015 10:40:55 -0700 Subject: [Hawkular-dev] Another perspective on Availability Message-ID: <54221624-5F4B-4CD7-9794-05E7E928A72E@redhat.com> I found this view of availability interesting from open source monitoring project: CachetHQ(https://cachethq.io/ ) Perhaps we might want to incorporate these ideas of component status versus incident status. Component Status https://docs.cachethq.io/v1.0/docs/component-statuses Status Name Description 1 Operational The component is working. 2 Performance Issues The component is experiencing some slowness. 3 Partial Outage The component may not be working for everybody. This could be a geographical issue for example. 4 Major Outage The component is not working for anybody. Incident Status https://docs.cachethq.io/v1.0/docs/incident-statuses Status Name Description 0 Scheduled This status is used for a scheduled status. 1 Investigating You have reports of a problem and you're currently looking into them. 2 Identified You've found the issue and you're working on a fix. 3 Watching You've since deployed a fix and you're currently watching the situation. 4 Fixed The fix has worked, you're happy to close the incident. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150407/d69e455b/attachment-0001.html From snegrea at redhat.com Tue Apr 7 15:35:16 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 7 Apr 2015 15:35:16 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - 0.3.1 In-Reply-To: <1969397940.5917124.1423168997249.JavaMail.zimbra@redhat.com> Message-ID: <1555745278.11199263.1428435316002.JavaMail.zimbra@redhat.com> Hello Everybody, I am happy to announce a big milestone for the Hawkular Metrics project. We are releasing today for the first time from the repository hosted by the Hawkular organization. Major changes: 1) UI - The core UI has been migrated to Hawkular UI related projects (hawkular-ui-components, hawkular, and hawkular-ui-services). The explorer project is still available for testing purposes. 2) REST - Consistent error reporting and status codes usage across endpoints - Added Availability statistics (https://issues.jboss.org/browse/HWKMETRICS-35): * Total downtime duration * Last downtime * Percentage of uptime * Number of downtimes - New Numeric Metric statistics (https://issues.jboss.org/browse/HWKMETRICS-34): * Average * Median * 95th Percentile - The REST implementation has been decoupled from the actual core logic, which paves the way for alternate REST implementations 3) Core - Large refactoring of the core classes and packages, the domain related logic has been pushed to the core layer - The Memory storage engine has been dropped from the project. Cassandra (embedded or standalone) is the exclusive storage engine. 4) InfluxDB Integration - Influx Java client supports sending and reading data (it was not possible before because of the endoint URI differences) to/from Hawkular Metrics. Other clients are not tested but should work as well. 5) PTrans - Configurable list of listeners (previously all collectd, ganglia, ... etc listeners were started) - Bug fix: send data to Metrics server if the buffer is full or no data was received recently (previously data could sit in the buffer and wait for the buffer to be full before being sent) Github Release: https://github.com/hawkular/hawkular-metrics/releases/tag/0.3.1 JBoss Nexus Maven artifacts: http://origin-repository.jboss.org/nexus/content/repositories/public/org/hawkular/metrics/ A big "Thank you" goes to John Sanda, Thomas Segismont, Mike Thompson, and Heiko Rupp for their project contributions. Special mentions go to Jeeva Kandasamy, Jirka Kremser, and Michael Burman for their project help. I am happy to announce that Matt Wringe joined the Hawkular Metrics team with a focus on containers and project integrations. We are looking forward to his contributions. Any discussion, suggestions or contributions are more than welcomed; so feel free to reply to this email or join #hawkular on freenode. Thank you, Stefan Negrea Software Engineer From snegrea at redhat.com Tue Apr 7 16:31:34 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 7 Apr 2015 16:31:34 -0400 (EDT) Subject: [Hawkular-dev] Hawkular 0.3.2 & Beyond - Release Planning In-Reply-To: <1365152606.11220737.1428438363509.JavaMail.zimbra@redhat.com> Message-ID: <830388823.11222407.1428438694029.JavaMail.zimbra@redhat.com> Hello Everybody, Here is a list of major features and enhancements planned for 0.3.2 and beyond: 1) REST API - the end-points will be separated based on the metric type. Current availability and numeric, will be joined by counter and gauges metrics. 2) Counters and gauges - add two new specialized metric types to allow more flexibility for clients. 3) Numeric Aggregates - Will allow long-term storage of numeric metrics at the expense of losing some fidelity. The design is already in progress. Because this is a really complex feature, the expectation is to start the work in 0.3.2 and publish a mini-roadmap. 4) Enhanced Availability - while the current model works, the goal would be expand this to cover most of the use cases brought up in the community threads. We will start the design in 0.3.2 with the implementation expected in future releases. 4) Go client - will help integrating with third party metrics collection framework. This work is the foundation for the Heapster sink. 5) Public Java API - following the work done in 0.3.1 in the core of the project, the goal is to separate the implementation from a public API and make that available to clients 6) Update REST testing - the current set of tests is a good gauge for regressions, but the overall coverage is still low. The plan for 0.3.2 is to increase coverage. Any discussion, suggestions or contributions are more than welcomed; so feel free to reply to this email or join #hawkular on freenode. Thank you, Stefan Negrea Software Engineer From snegrea at redhat.com Tue Apr 7 16:32:16 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 7 Apr 2015 16:32:16 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - 0.3.2 & Beyond - Release Planning In-Reply-To: <830388823.11222407.1428438694029.JavaMail.zimbra@redhat.com> Message-ID: <1888603226.11222745.1428438736211.JavaMail.zimbra@redhat.com> Hello Everybody, Here is a list of major features and enhancements planned for Hawkular Metrics 0.3.2 and beyond: 1) REST API - the end-points will be separated based on the metric type. Current availability and numeric, will be joined by counter and gauges metrics. 2) Counters and gauges - add two new specialized metric types to allow more flexibility for clients. 3) Numeric Aggregates - Will allow long-term storage of numeric metrics at the expense of losing some fidelity. The design is already in progress. Because this is a really complex feature, the expectation is to start the work in 0.3.2 and publish a mini-roadmap. 4) Enhanced Availability - while the current model works, the goal would be expand this to cover most of the use cases brought up in the community threads. We will start the design in 0.3.2 with the implementation expected in future releases. 4) Go client - will help integrating with third party metrics collection framework. This work is the foundation for the Heapster sink. 5) Public Java API - following the work done in 0.3.1 in the core of the project, the goal is to separate the implementation from a public API and make that available to clients 6) Update REST testing - the current set of tests is a good gauge for regressions, but the overall coverage is still low. The plan for 0.3.2 is to increase coverage. Any discussion, suggestions or contributions are more than welcomed; so feel free to reply to this email or join #hawkular on freenode. Thank you, Stefan Negrea Software Engineer From snegrea at redhat.com Tue Apr 7 16:33:16 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 7 Apr 2015 16:33:16 -0400 (EDT) Subject: [Hawkular-dev] Hawkular 0.3.2 & Beyond - Release Planning In-Reply-To: <830388823.11222407.1428438694029.JavaMail.zimbra@redhat.com> References: <830388823.11222407.1428438694029.JavaMail.zimbra@redhat.com> Message-ID: <396183907.11223060.1428438796763.JavaMail.zimbra@redhat.com> Really sorry but I missed the Metrics word in the title. This is the release planning for Hawkular Metrics. I resent the email with the proper subject, so we can continue the discussion in that context. Thank you, Stefan ----- Original Message ----- > From: "Stefan Negrea" > To: "Discussions around Hawkular development" > Sent: Tuesday, April 7, 2015 3:31:34 PM > Subject: [Hawkular-dev] Hawkular 0.3.2 & Beyond - Release Planning > > Hello Everybody, > > > Here is a list of major features and enhancements planned for 0.3.2 and > beyond: > > 1) REST API - the end-points will be separated based on the metric type. > Current availability and numeric, will be joined by counter and gauges > metrics. > > 2) Counters and gauges - add two new specialized metric types to allow more > flexibility for clients. > > 3) Numeric Aggregates - Will allow long-term storage of numeric metrics at > the expense of losing some fidelity. The design is already in progress. > Because this is a really complex feature, the expectation is to start the > work in 0.3.2 and publish a mini-roadmap. > > 4) Enhanced Availability - while the current model works, the goal would be > expand this to cover most of the use cases brought up in the community > threads. We will start the design in 0.3.2 with the implementation expected > in future releases. > > 4) Go client - will help integrating with third party metrics collection > framework. This work is the foundation for the Heapster sink. > > 5) Public Java API - following the work done in 0.3.1 in the core of the > project, the goal is to separate the implementation from a public API and > make that available to clients > > 6) Update REST testing - the current set of tests is a good gauge for > regressions, but the overall coverage is still low. The plan for 0.3.2 is to > increase coverage. > > > Any discussion, suggestions or contributions are more than welcomed; so feel > free to reply to this email or join #hawkular on freenode. > > > Thank you, > Stefan Negrea > > Software Engineer > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mithomps at redhat.com Tue Apr 7 18:00:02 2015 From: mithomps at redhat.com (mike thompson) Date: Tue, 7 Apr 2015 15:00:02 -0700 Subject: [Hawkular-dev] Which Maven version do we support? Message-ID: <22A112D9-A54F-4DB7-9BC7-4C884B9984DA@redhat.com> Lukas found an issue with the latest maven 3.3.1 running the project: https://jira.codehaus.org/browse/MNG-5787 (thank Jirka for finding that) It seems the solution is either: to downgrade to maven 3.2.5 or upgrade the frontend-maven-plugin to 0.0.23 when using maven 3.3.1 (which I am) Sooo it begs the question, ?What version of Maven are we supporting?" https://github.com/hawkular/hawkular/pull/70 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150407/3e545a68/attachment.html From tsegismo at redhat.com Wed Apr 8 03:01:04 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 08 Apr 2015 09:01:04 +0200 Subject: [Hawkular-dev] Which Maven version do we support? In-Reply-To: <22A112D9-A54F-4DB7-9BC7-4C884B9984DA@redhat.com> References: <22A112D9-A54F-4DB7-9BC7-4C884B9984DA@redhat.com> Message-ID: <5524D230.4000007@redhat.com> Le 08/04/2015 00:00, mike thompson a ?crit : > Lukas found an issue with the latest maven 3.3.1 running the project: > https://jira.codehaus.org/browse/MNG-5787 (thank Jirka for finding that) > > It seems the solution is either: > to downgrade to maven 3.2.5 or upgrade the frontend-maven-plugin to > 0.0.23 when using maven 3.3.1 (which I am) > > > Sooo it begs the question, ?What version of Maven are we supporting?" > > https://github.com/hawkular/hawkular/pull/70 > If we upgrade frontend-maven-plugin to 0.0.23, it works with both Maven 3.2.5 and 3.3.1, correct? Does 0.0.23 cause any regression? From miburman at redhat.com Wed Apr 8 10:07:17 2015 From: miburman at redhat.com (Michael Burman) Date: Wed, 8 Apr 2015 10:07:17 -0400 (EDT) Subject: [Hawkular-dev] Metrics, tags and search capabilities In-Reply-To: <1578391240.13425399.1428499924906.JavaMail.zimbra@redhat.com> Message-ID: <1666011628.13455219.1428502037048.JavaMail.zimbra@redhat.com> Hi, Earlier today I created HWKMETRICS-54, but I later thought about it a bit more and to me it looks like we're not sure what the tags should really do and how the system should be used. Lets assume the following scenario: 3 machines, each one running an agent that provides data for memory.free, memory.cached, cpu.idle, cpu.user, cpu.system, disk.free, disk.used Now what user might want to do in these cases is: a) I want to get all the statistics affecting the host 1 b) I want to get all the memory.free statistics from each host Think about the data modeling in our current hawkular-metrics for a moment. The user starts: I. Model everything with machine1.memory.free, machine2.memory.free etc a) How to query machine1.* ? Can't be done. c) How to get *.memory.free? Can't be done. How would the query succeed with our current format? By creating tags for every occasion on metric creation: create machine1.memory.free (tags: machine='machine1', category='memory') create machine2.memory.free (tags: machine='machine2', category='memory') What the user finds out is that "why on earth do I have the metricId at all" ? It's a good question. In our current structure we should remove metricId and just invent something random for better Cassandra partitioning. What it probably should look like (and this is how I assumed it was to be done until I checked the unit tests and find out that there's nothing pointing to this OpenTSDB familiar method): II. memory.free, cpu.idle are the metricIds and I'll define it has a parameter 'machine'. When pushing a metric I set a tag with a value, such as machine='machine2'. Now when I fetch the metric "memory.free", I can get all the memory.free valuess with 'machine'-tag indicating which machine it was gathered from. If I need to search for all machine-statistics, then I could use the tag-searching. If I wanted only machine1 memory.free, I would add a filter: tag machine='machine1'. Or how are we supposed to model real-world-use-cases? The current model is quite cumbersome and not even necessarily doable in many cases (am I supposed to query for a metric definition before pushing any metric - because a new container could give me new set of parameters or a new machine new set of machine parameters and I need to remember to register them instead of pre-defined types which I might know). - Micke From gbrown at redhat.com Wed Apr 8 10:25:59 2015 From: gbrown at redhat.com (Gary Brown) Date: Wed, 8 Apr 2015 10:25:59 -0400 (EDT) Subject: [Hawkular-dev] Metrics, tags and search capabilities In-Reply-To: <1666011628.13455219.1428502037048.JavaMail.zimbra@redhat.com> References: <1666011628.13455219.1428502037048.JavaMail.zimbra@redhat.com> Message-ID: <1464613919.10849757.1428503159387.JavaMail.zimbra@redhat.com> Hi I think use of tags to provide greater flexibility would be good, to support future requirements, such as metrics related to services, apps, queries, customers, etc. So maybe a generic 'type' tag/field would define the type of metric, e.g. response time, size, availability, etc - but other information defined using appropriate tags. Although is there any standard naming convention for referring to resources in inventory - i.e. so where metrics relate to a resource in inventory, then perhaps a tag/field could be used to link them? Regards Gary ----- Original Message ----- > Hi, > > Earlier today I created HWKMETRICS-54, but I later thought about it a bit > more and to me it looks like we're not sure what the tags should really do > and how the system should be used. Lets assume the following scenario: > > 3 machines, each one running an agent that provides data for memory.free, > memory.cached, cpu.idle, cpu.user, cpu.system, disk.free, disk.used > > Now what user might want to do in these cases is: > > a) I want to get all the statistics affecting the host 1 > b) I want to get all the memory.free statistics from each host > > Think about the data modeling in our current hawkular-metrics for a moment. > The user starts: > > I. Model everything with machine1.memory.free, machine2.memory.free etc > a) How to query machine1.* ? Can't be done. c) How to get *.memory.free? > Can't be done. > > How would the query succeed with our current format? By creating tags for > every occasion on metric creation: > > create machine1.memory.free (tags: machine='machine1', category='memory') > create machine2.memory.free (tags: machine='machine2', category='memory') > > What the user finds out is that "why on earth do I have the metricId at all" > ? It's a good question. In our current structure we should remove metricId > and just invent something random for better Cassandra partitioning. > > What it probably should look like (and this is how I assumed it was to be > done until I checked the unit tests and find out that there's nothing > pointing to this OpenTSDB familiar method): > > II. memory.free, cpu.idle are the metricIds and I'll define it has a > parameter 'machine'. When pushing a metric I set a tag with a value, such as > machine='machine2'. > > Now when I fetch the metric "memory.free", I can get all the memory.free > valuess with 'machine'-tag indicating which machine it was gathered from. If > I need to search for all machine-statistics, then I could use the > tag-searching. If I wanted only machine1 memory.free, I would add a filter: > tag machine='machine1'. > > Or how are we supposed to model real-world-use-cases? The current model is > quite cumbersome and not even necessarily doable in many cases (am I > supposed to query for a metric definition before pushing any metric - > because a new container could give me new set of parameters or a new machine > new set of machine parameters and I need to remember to register them > instead of pre-defined types which I might know). > > - Micke > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jsanda at redhat.com Wed Apr 8 10:51:28 2015 From: jsanda at redhat.com (John Sanda) Date: Wed, 8 Apr 2015 10:51:28 -0400 Subject: [Hawkular-dev] Metrics, tags and search capabilities In-Reply-To: <1666011628.13455219.1428502037048.JavaMail.zimbra@redhat.com> References: <1666011628.13455219.1428502037048.JavaMail.zimbra@redhat.com> Message-ID: <4AE0856F-FFE5-474A-842F-2DD204D95312@redhat.com> > On Apr 8, 2015, at 10:07 AM, Michael Burman wrote: > > Hi, > > Earlier today I created HWKMETRICS-54, but I later thought about it a bit more and to me it looks like we're not sure what the tags should really do and how the system should be used. Lets assume the following scenario: You are correct to some degree about tags. We do know that we want to support querying/filtering by tags. We also plan to use tags for configuring things like data retention and aggregation. Right now we have limited support for querying for data points by tags. We do not yet have support for querying by metric tags. > > 3 machines, each one running an agent that provides data for memory.free, memory.cached, cpu.idle, cpu.user, cpu.system, disk.free, disk.used > > Now what user might want to do in these cases is: > > a) I want to get all the statistics affecting the host 1 > b) I want to get all the memory.free statistics from each host > > Think about the data modeling in our current hawkular-metrics for a moment. The user starts: > > I. Model everything with machine1.memory.free, machine2.memory.free etc > a) How to query machine1.* ? Can't be done. c) How to get *.memory.free? Can't be done. It cannot be done yet because we only have support right now for filtering individual data points by tag(s). Assuming each metric from machine1 has the tag, machine=machine1, then I think we should support filtering like, machine = machine1 ?> retrieve data points for all metrics on machine1 machine = * ?> retrieve data points for all metrics on machines 1, 2, 3 machine = [machine1, machine2] ?> retrieve data points from machines 1, 2 machine = regular expression ?> retrieve data points for metrics with machine tag where values matches regex I think this would handle both querying machine1 metrics and querying memory.free metrics. > > How would the query succeed with our current format? By creating tags for every occasion on metric creation: > > create machine1.memory.free (tags: machine='machine1', category='memory') > create machine2.memory.free (tags: machine='machine2', category='memory') > > What the user finds out is that "why on earth do I have the metricId at all" ? It's a good question. In our current structure we should remove metricId and just invent something random for better Cassandra partitioning. > > What it probably should look like (and this is how I assumed it was to be done until I checked the unit tests and find out that there's nothing pointing to this OpenTSDB familiar method): > > II. memory.free, cpu.idle are the metricIds and I'll define it has a parameter 'machine'. When pushing a metric I set a tag with a value, such as machine='machine2?. You have strayed into the topic of adding more explicit grouping of metrics, something I have thought about as well. Remember that metric id is basically the primary key, and the PK consists of the partition key and any number of optional clustering columns. Partition keys have to be unique. In your example the tag machine=machine2 would really have to be a special tag that winds up being part of the partition key; otherwise, our keys will not be unique. I think that I tend to favor more explicit grouping. Tags can be changed, and the partition key cannot change. So let?s say we have an explicit (and maybe optional) grouping parameter. We can use host. Then the actual partition key still winds up being machine1.memory.free and so on. And sure, the client can query for all metrics belonging to machine1 just by specifying host = machine1. Isn?t this just a special case of filtering by tags? > > Now when I fetch the metric "memory.free", I can get all the memory.free valuess with 'machine'-tag indicating which machine it was gathered from. If I need to search for all machine-statistics, then I could use the tag-searching. If I wanted only machine1 memory.free, I would add a filter: tag machine='machine1'. > > Or how are we supposed to model real-world-use-cases? The current model is quite cumbersome and not even necessarily doable in many cases (am I supposed to query for a metric definition before pushing any metric - because a new container could give me new set of parameters or a new machine new set of machine parameters and I need to remember to register them instead of pre-defined types which I might know). > > - Micke > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From mazz at redhat.com Wed Apr 8 12:42:37 2015 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 8 Apr 2015 12:42:37 -0400 (EDT) Subject: [Hawkular-dev] ${project.version} in dependency versions considered harmful In-Reply-To: <55255878.3080709@redhat.com> References: <55255878.3080709@redhat.com> Message-ID: <955245385.11315805.1428511357502.JavaMail.zimbra@redhat.com> FYI: ----- Forwarded Message ----- Subject: ${project.version} in dependency versions considered harmful Hey everyone, I just wanted to give a little PSA about this insidious little expression (see $subject). Using that expression in dependency declarations seems like a shortcut, but it can go wrong in SO MANY WAYS. By far the most common problem is the use of ${project.version} in a BOM or parent POM. If anyone inherits from that parent POM or imports that BOM in an external project, that external project's version will be used in place of the one that the parent POM / BOM intended, and all of your carefully managed dependencies will be wrong. Example: jboss-as-console-bom-2.5.5.Final-redhat-1.pom This declares org.jboss.as:console-spi:sources:${project.version}:jar. Then, the Teiid build imports that BOM and uses it when it builds against the console-core library. The above sources jar listed as a second-level dependency (coming in via console-core) uses the Teiid project version, and everything grinds to a halt. Please, if you find dependency declarations using ${project.version}, fix it! If there are many, many references, simply switch to using a property (eg. ${consoleVersion} in place of ${project.version})...and DON'T use ${project.version} as the value for that property. From jsanda at redhat.com Wed Apr 8 14:40:06 2015 From: jsanda at redhat.com (John Sanda) Date: Wed, 8 Apr 2015 14:40:06 -0400 Subject: [Hawkular-dev] Metrics, tags and search capabilities In-Reply-To: <4AE0856F-FFE5-474A-842F-2DD204D95312@redhat.com> References: <1666011628.13455219.1428502037048.JavaMail.zimbra@redhat.com> <4AE0856F-FFE5-474A-842F-2DD204D95312@redhat.com> Message-ID: I added a comment to HWKMETRICS-54 that is worth repeating here. We support tagging at two different levels - 1) the metric or times as a whole and 2) individual data points. Tags applied to individual data points will expire alongside the data. Metric tags on the other hand do not expire. I anticipate using metric tags most frequently for query filtering as discussed below, providing meta data that might be used by other services such as inventory or alerting, and configuring things like aggregation and data retention. > On Apr 8, 2015, at 10:51 AM, John Sanda wrote: > >> >> On Apr 8, 2015, at 10:07 AM, Michael Burman > wrote: >> >> Hi, >> >> Earlier today I created HWKMETRICS-54, but I later thought about it a bit more and to me it looks like we're not sure what the tags should really do and how the system should be used. Lets assume the following scenario: > > You are correct to some degree about tags. We do know that we want to support querying/filtering by tags. We also plan to use tags for configuring things like data retention and aggregation. Right now we have limited support for querying for data points by tags. We do not yet have support for querying by metric tags. > >> >> 3 machines, each one running an agent that provides data for memory.free, memory.cached, cpu.idle, cpu.user, cpu.system, disk.free, disk.used >> >> Now what user might want to do in these cases is: >> >> a) I want to get all the statistics affecting the host 1 >> b) I want to get all the memory.free statistics from each host >> >> Think about the data modeling in our current hawkular-metrics for a moment. The user starts: >> >> I. Model everything with machine1.memory.free, machine2.memory.free etc >> a) How to query machine1.* ? Can't be done. c) How to get *.memory.free? Can't be done. > > It cannot be done yet because we only have support right now for filtering individual data points by tag(s). Assuming each metric from machine1 has the tag, machine=machine1, then I think we should support filtering like, > > machine = machine1 ?> retrieve data points for all metrics on machine1 > machine = * ?> retrieve data points for all metrics on machines 1, 2, 3 > machine = [machine1, machine2] ?> retrieve data points from machines 1, 2 > machine = regular expression ?> retrieve data points for metrics with machine tag where values matches regex > > I think this would handle both querying machine1 metrics and querying memory.free metrics. > >> >> How would the query succeed with our current format? By creating tags for every occasion on metric creation: >> >> create machine1.memory.free (tags: machine='machine1', category='memory') >> create machine2.memory.free (tags: machine='machine2', category='memory') >> >> What the user finds out is that "why on earth do I have the metricId at all" ? It's a good question. In our current structure we should remove metricId and just invent something random for better Cassandra partitioning. >> >> What it probably should look like (and this is how I assumed it was to be done until I checked the unit tests and find out that there's nothing pointing to this OpenTSDB familiar method): >> >> II. memory.free, cpu.idle are the metricIds and I'll define it has a parameter 'machine'. When pushing a metric I set a tag with a value, such as machine='machine2?. > > You have strayed into the topic of adding more explicit grouping of metrics, something I have thought about as well. Remember that metric id is basically the primary key, and the PK consists of the partition key and any number of optional clustering columns. Partition keys have to be unique. In your example the tag machine=machine2 would really have to be a special tag that winds up being part of the partition key; otherwise, our keys will not be unique. > > I think that I tend to favor more explicit grouping. Tags can be changed, and the partition key cannot change. So let?s say we have an explicit (and maybe optional) grouping parameter. We can use host. Then the actual partition key still winds up being machine1.memory.free and so on. And sure, the client can query for all metrics belonging to machine1 just by specifying host = machine1. Isn?t this just a special case of filtering by tags? > >> >> Now when I fetch the metric "memory.free", I can get all the memory.free valuess with 'machine'-tag indicating which machine it was gathered from. If I need to search for all machine-statistics, then I could use the tag-searching. If I wanted only machine1 memory.free, I would add a filter: tag machine='machine1'. >> >> Or how are we supposed to model real-world-use-cases? The current model is quite cumbersome and not even necessarily doable in many cases (am I supposed to query for a metric definition before pushing any metric - because a new container could give me new set of parameters or a new machine new set of machine parameters and I need to remember to register them instead of pre-defined types which I might know). >> >> - Micke >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150408/90e0dfca/attachment-0001.html From snegrea at redhat.com Wed Apr 8 16:16:06 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Wed, 8 Apr 2015 16:16:06 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - REST API Remake Message-ID: <1803454035.12019533.1428524166120.JavaMail.zimbra@redhat.com> The following is a new meeting request: Subject: Hawkular Metrics - REST API Remake Organizer: "Stefan Negrea" Location: Conf - 398-055-2127 & BlueJeans Time: Monday, April 13, 2015, 9:00:00 AM - 10:00:00 AM GMT -06:00 US/Canada Central Invitees: jsanda at redhat.com; hawkular-dev at lists.jboss.org *~*~*~*~*~*~*~*~*~* Hello Everybody, I would like to schedule a meeting to discuss a proposal for changes to Hawkular Metrics REST API. The agenda also includes recent proposals from the channel and mailing list around tags and raw vs aggregated metrics from an end-point perspective. Here is a preliminary simple spreadsheet for this remake: https://docs.google.com/a/redhat.com/spreadsheets/d/1CQBfAenNlWbQENoK_Rms9pHooK2e7yGf15bm5avaN9A/edit#gid=329319157 Thank you, Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2072 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150408/fabe8028/attachment.bin From snegrea at redhat.com Wed Apr 8 16:25:15 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Wed, 8 Apr 2015 16:25:15 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - Aggregate Numeric Metrics Message-ID: <1657646868.12034119.1428524715847.JavaMail.zimbra@redhat.com> The following is a new meeting request: Subject: Hawkular Metrics - Aggregate Numeric Metrics Organizer: "Stefan Negrea" Location: Conf - 398-055-2127 & BlueJeans Time: Monday, April 20, 2015, 9:00:00 AM - 10:00:00 AM GMT -06:00 US/Canada Central Invitees: jsanda at redhat.com; hawkular-dev at lists.jboss.org *~*~*~*~*~*~*~*~*~* Hello Everybody, I would like to schedule a review meeting for the architecture & design of aggregate numeric metrics for Hawkular Metrics. John Sanda created a very detailed document that will serve as basis for the meeting. Here is the document: https://docs.google.com/a/redhat.com/document/d/1vQkj_rR_wUHwGnalDij8XXGKXXPTm9El3gQoEA3NCtU/edit?pli=1# Thank you, Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2005 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150408/5f633943/attachment.bin From mithomps at redhat.com Wed Apr 8 18:25:00 2015 From: mithomps at redhat.com (mike thompson) Date: Wed, 8 Apr 2015 15:25:00 -0700 Subject: [Hawkular-dev] Which Maven version do we support? In-Reply-To: <5524D230.4000007@redhat.com> References: <22A112D9-A54F-4DB7-9BC7-4C884B9984DA@redhat.com> <5524D230.4000007@redhat.com> Message-ID: > On 8 Apr 2015, at 00:01, Thomas Segismont wrote: > > Le 08/04/2015 00:00, mike thompson a ?crit : >> Lukas found an issue with the latest maven 3.3.1 running the project: >> https://jira.codehaus.org/browse/MNG-5787 (thank Jirka for finding that) >> >> It seems the solution is either: >> to downgrade to maven 3.2.5 or upgrade the frontend-maven-plugin to >> 0.0.23 when using maven 3.3.1 (which I am) >> >> >> Sooo it begs the question, ?What version of Maven are we supporting?" >> >> https://github.com/hawkular/hawkular/pull/70 >> > > If we upgrade frontend-maven-plugin to 0.0.23, it works with both Maven > 3.2.5 and 3.3.1, correct? I don?t think so. Waiting for Lukas to confirm. > > Does 0.0.23 cause any regression? No. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From tsegismo at redhat.com Thu Apr 9 03:30:39 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 9 Apr 2015 03:30:39 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - REST API Remake Message-ID: <1349108042.17153819.1428564639884.JavaMail.zimbra@redhat.com> I'll have to drop at 16h25. ----- Mail original ----- > The following is a new meeting request: > Subject: Hawkular Metrics - REST API Remake > Organizer: "Stefan Negrea" > Location: Conf - 398-055-2127 & BlueJeans > Time: Monday, April 13, 2015, 9:00:00 AM - 10:00:00 AM GMT -06:00 US/Canada > Central > Invitees: jsanda at redhat.com; hawkular-dev at lists.jboss.org > *~*~*~*~*~*~*~*~*~* > Hello Everybody, > I would like to schedule a meeting to discuss a proposal for changes to > Hawkular Metrics REST API. The agenda also includes recent proposals from > the channel and mailing list around tags and raw vs aggregated metrics from > an end-point perspective. > Here is a preliminary simple spreadsheet for this remake: > https://docs.google.com/a/redhat.com/spreadsheets/d/1CQBfAenNlWbQENoK_Rms9pHooK2e7yGf15bm5avaN9A/edit#gid=329319157 > Thank you, > Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150409/aae548a4/attachment.html From tsegismo at redhat.com Thu Apr 9 03:32:13 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 09 Apr 2015 09:32:13 +0200 Subject: [Hawkular-dev] Hawkular Metrics - REST API Remake In-Reply-To: <1349108042.17153819.1428564639884.JavaMail.zimbra@redhat.com> References: <1349108042.17153819.1428564639884.JavaMail.zimbra@redhat.com> Message-ID: <55262AFD.6090308@redhat.com> Sorry, answered in my TZ and time format :) I'll have to drop 25 minutes after the start. Le 09/04/2015 09:30, Thomas Segismont a ?crit : > I'll have to drop at 16h25. > > ------------------------------------------------------------------------ > > The following is a new meeting request: > > Subject: Hawkular Metrics - REST API Remake > Organizer: "Stefan Negrea" > > Location: Conf - 398-055-2127 & BlueJeans > Time: Monday, April 13, 2015, 9:00:00 AM - 10:00:00 AM GMT -06:00 > US/Canada Central > > Invitees: jsanda at redhat.com; hawkular-dev at lists.jboss.org > > > *~*~*~*~*~*~*~*~*~* > > Hello Everybody, > > I would like to schedule a meeting to discuss a proposal for changes > to Hawkular Metrics REST API. The agenda also includes recent > proposals from the channel and mailing list around tags and raw vs > aggregated metrics from an end-point perspective. > > Here is a preliminary simple spreadsheet for this remake: > https://docs.google.com/a/redhat.com/spreadsheets/d/1CQBfAenNlWbQENoK_Rms9pHooK2e7yGf15bm5avaN9A/edit#gid=329319157 > > > > Thank you, > Stefan > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From miburman at redhat.com Thu Apr 9 06:03:38 2015 From: miburman at redhat.com (Michael Burman) Date: Thu, 9 Apr 2015 06:03:38 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - REST API Remake In-Reply-To: <55262AFD.6090308@redhat.com> References: <1349108042.17153819.1428564639884.JavaMail.zimbra@redhat.com> <55262AFD.6090308@redhat.com> Message-ID: <1634053522.14077412.1428573818436.JavaMail.zimbra@redhat.com> Hi, Yups, I can't make it at all on Mondays that time. - Micke ----- Original Message ----- From: "Thomas Segismont" To: hawkular-dev at lists.jboss.org Sent: Thursday, April 9, 2015 10:32:13 AM Subject: Re: [Hawkular-dev] Hawkular Metrics - REST API Remake Sorry, answered in my TZ and time format :) I'll have to drop 25 minutes after the start. Le 09/04/2015 09:30, Thomas Segismont a ?crit : > I'll have to drop at 16h25. > > ------------------------------------------------------------------------ > > The following is a new meeting request: > > Subject: Hawkular Metrics - REST API Remake > Organizer: "Stefan Negrea" > > Location: Conf - 398-055-2127 & BlueJeans > Time: Monday, April 13, 2015, 9:00:00 AM - 10:00:00 AM GMT -06:00 > US/Canada Central > > Invitees: jsanda at redhat.com; hawkular-dev at lists.jboss.org > > > *~*~*~*~*~*~*~*~*~* > > Hello Everybody, > > I would like to schedule a meeting to discuss a proposal for changes > to Hawkular Metrics REST API. The agenda also includes recent > proposals from the channel and mailing list around tags and raw vs > aggregated metrics from an end-point perspective. > > Here is a preliminary simple spreadsheet for this remake: > https://docs.google.com/a/redhat.com/spreadsheets/d/1CQBfAenNlWbQENoK_Rms9pHooK2e7yGf15bm5avaN9A/edit#gid=329319157 > > > > Thank you, > Stefan > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From mazz at redhat.com Thu Apr 9 08:58:24 2015 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 9 Apr 2015 08:58:24 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics avail API In-Reply-To: <788472262.11692525.1428584301167.JavaMail.zimbra@redhat.com> Message-ID: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> Can someone point me to some code that stores availability data into hawkular metrics via REST and/or bus? The hawkular monitor agent is in a state where I can finally add this code and get it to start storing avail data (its already doing metrics). Once I get that done, I'm off to implement JMX/Jolokia metrics/avail collection and storage. From tsegismo at redhat.com Thu Apr 9 10:12:08 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 09 Apr 2015 16:12:08 +0200 Subject: [Hawkular-dev] hawkular metrics avail API In-Reply-To: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> References: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> Message-ID: <552688B8.1010001@redhat.com> Doesn't the pinger send avail data to metrics? Le 09/04/2015 14:58, John Mazzitelli a ?crit : > Can someone point me to some code that stores availability data into hawkular metrics via REST and/or bus? > > The hawkular monitor agent is in a state where I can finally add this code and get it to start storing avail data (its already doing metrics). > > Once I get that done, I'm off to implement JMX/Jolokia metrics/avail collection and storage. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Thu Apr 9 15:18:58 2015 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 9 Apr 2015 15:18:58 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics avail API In-Reply-To: <552688B8.1010001@redhat.com> References: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> <552688B8.1010001@redhat.com> Message-ID: <679666672.11965216.1428607138004.JavaMail.zimbra@redhat.com> > Doesn't the pinger send avail data to metrics? I thought so, but I don't see it anywhere in here: https://github.com/hawkular/hawkular/tree/master/modules/pinger/src/main/java/org/hawkular/component/pinger Anyone know where the code is that shows the pinger sending avail data? Is there any example code elsewhere (doesn't have to be the pinger) that sends avail data? ----- Original Message ----- > Doesn't the pinger send avail data to metrics? > > Le 09/04/2015 14:58, John Mazzitelli a ?crit : > > Can someone point me to some code that stores availability data into > > hawkular metrics via REST and/or bus? > > > > The hawkular monitor agent is in a state where I can finally add this code > > and get it to start storing avail data (its already doing metrics). > > > > Once I get that done, I'm off to implement JMX/Jolokia metrics/avail > > collection and storage. > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mithomps at redhat.com Thu Apr 9 15:22:42 2015 From: mithomps at redhat.com (mike thompson) Date: Thu, 9 Apr 2015 12:22:42 -0700 Subject: [Hawkular-dev] hawkular metrics avail API In-Reply-To: <679666672.11965216.1428607138004.JavaMail.zimbra@redhat.com> References: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> <552688B8.1010001@redhat.com> <679666672.11965216.1428607138004.JavaMail.zimbra@redhat.com> Message-ID: > On 9 Apr 2015, at 12:18, John Mazzitelli wrote: > >> Doesn't the pinger send avail data to metrics? > > I thought so, but I don't see it anywhere in here: > > https://github.com/hawkular/hawkular/tree/master/modules/pinger/src/main/java/org/hawkular/component/pinger > > Anyone know where the code is that shows the pinger sending avail data? It is the avail_creator module in hawkular repo https://github.com/hawkular/hawkular/pull/61 AvailPublisher.java > > Is there any example code elsewhere (doesn't have to be the pinger) that sends avail data? modules/avail_creator/src/main/java/org/hawkular/component/availcreator/AvailPublisher.java > > ----- Original Message ----- >> Doesn't the pinger send avail data to metrics? >> >> Le 09/04/2015 14:58, John Mazzitelli a ?crit : >>> Can someone point me to some code that stores availability data into >>> hawkular metrics via REST and/or bus? >>> >>> The hawkular monitor agent is in a state where I can finally add this code >>> and get it to start storing avail data (its already doing metrics). >>> >>> Once I get that done, I'm off to implement JMX/Jolokia metrics/avail >>> collection and storage. >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150409/12aa2b87/attachment.html From vnguyen at redhat.com Thu Apr 9 15:57:54 2015 From: vnguyen at redhat.com (Viet Nguyen) Date: Thu, 9 Apr 2015 15:57:54 -0400 (EDT) Subject: [Hawkular-dev] hawkular metrics avail API In-Reply-To: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> References: <692965069.11692556.1428584304743.JavaMail.zimbra@redhat.com> Message-ID: <660828174.9062889.1428609474966.JavaMail.zimbra@redhat.com> Mazz, Maybe this helps? https://github.com/Hawkular-QE/hawkular-java-client/blob/master/src/test/java/org/hawkular/client/test/AvailabilityMetricTest.java Viet ----- Original Message ----- From: "John Mazzitelli" To: "Discussions around Hawkular development" Sent: Thursday, April 9, 2015 5:58:24 AM Subject: [Hawkular-dev] hawkular metrics avail API Can someone point me to some code that stores availability data into hawkular metrics via REST and/or bus? The hawkular monitor agent is in a state where I can finally add this code and get it to start storing avail data (its already doing metrics). Once I get that done, I'm off to implement JMX/Jolokia metrics/avail collection and storage. _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From mithomps at redhat.com Thu Apr 9 22:24:45 2015 From: mithomps at redhat.com (mike thompson) Date: Thu, 9 Apr 2015 19:24:45 -0700 Subject: [Hawkular-dev] Ways to Test Availability Down? Message-ID: I guess this question is mainly targeted to Hawkular QE. So all of my testing (especially on a dev box) shows availability 100% (as most sites will). So while I can mock this in code to show downtime and unknown. It appears that we need a consistent way to demonstrate up/down/unknown/whatever else. Is there a QE way to do this currently? I?m thinking a mock site that changes states every 5 or 10 minutes that is publicly available. This way we could all link our integration tests to this site to provide full coverage of states(and there could be other availability states like I have mentioned in other Hawkular ML mails). The use case I want to add is as a developer I want to test all availability states within a half hour (or whatever timeframe, but as everything waits on tests passing less is better) by pulling in certain site(s). Ideas? ? Mike T From mithomps at redhat.com Thu Apr 9 22:33:47 2015 From: mithomps at redhat.com (mike thompson) Date: Thu, 9 Apr 2015 19:33:47 -0700 Subject: [Hawkular-dev] Ways to Test Availability Down? In-Reply-To: References: Message-ID: > On 9 Apr 2015, at 19:24, mike thompson wrote: > > I guess this question is mainly targeted to Hawkular QE. So all of my testing (especially on a dev box) shows availability 100% (as most sites will). So while I can mock this in code to show downtime and unknown. It appears that we need a consistent way to demonstrate up/down/unknown/whatever else. Is there a QE way to do this currently? > > I?m thinking a mock site that changes states every 5 or 10 minutes that is publicly available. This way we could all link our integration tests to this site to provide full coverage of states(and there could be other availability states like I have mentioned in other Hawkular ML mails). > > The use case I want to add is as a developer I want to test all availability states within a half hour (or whatever timeframe, but as everything waits on tests passing less is better) by pulling in certain site(s). > > Ideas? This can be as simple as having a servlet that cycles through response codes 2xx, 404, 5xx in a predictable manner. (content is irrelevant on status codes matter so should be easy). This way we could expect http status codes to change every 5 minutes: '12:00' -> ?up?, 12:05 -> ?down?, ?12:10? -> ?unknown?. So that it changes in a predictable way. Anyway, just an idea? Obviously, if this idea is legit, it can expanded in may different ways down in the future. > > ? Mike T > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jsanda at redhat.com Thu Apr 9 23:41:43 2015 From: jsanda at redhat.com (John Sanda) Date: Thu, 9 Apr 2015 23:41:43 -0400 Subject: [Hawkular-dev] Ways to Test Availability Down? In-Reply-To: References:

Message-ID: > On Apr 9, 2015, at 10:33 PM, mike thompson wrote: > >> >> On 9 Apr 2015, at 19:24, mike thompson wrote: >> >> I guess this question is mainly targeted to Hawkular QE. So all of my testing (especially on a dev box) shows availability 100% (as most sites will). So while I can mock this in code to show downtime and unknown. It appears that we need a consistent way to demonstrate up/down/unknown/whatever else. Is there a QE way to do this currently? >> >> I?m thinking a mock site that changes states every 5 or 10 minutes that is publicly available. This way we could all link our integration tests to this site to provide full coverage of states(and there could be other availability states like I have mentioned in other Hawkular ML mails). >> >> The use case I want to add is as a developer I want to test all availability states within a half hour (or whatever timeframe, but as everything waits on tests passing less is better) by pulling in certain site(s). >> >> Ideas? > This can be as simple as having a servlet that cycles through response codes 2xx, 404, 5xx in a predictable manner. (content is irrelevant on status codes matter so should be easy). This way we could expect http status codes to change every 5 minutes: '12:00' -> ?up?, 12:05 -> ?down?, ?12:10? -> ?unknown?. So that it changes in a predictable way. > > Anyway, just an idea? > > Obviously, if this idea is legit, it can expanded in may different ways down in the future. I think there are a couple different concerns here. The first is simulating a resource that returns the various response codes. That should be easy enough. It is just a matter of starting up an HTTP server in a different thread that returns status codes in a known, predictable order. The second involving time is more challenging in my opinion. This was a big challenge for me when I was trying to write automated tests for aggregation in RHQ. The aggregation job ran hourly and computed hourly rollups, 6 hr rollups, and 24 hr rollups. Having tests run for 24 hrs in order to produce a 24 hr rollup was not practical. I encapsulated most of the date/time functionality in a service that could easily be stubbed or mocked. I also avoided calls like System.currentTimeMillis() or DateTime.now() in production code. Instead I used DateTimeService.currentTime() which I could easily override with test stubs. With these and other similar types of strategies, I think you should be able to test availability in milliseconds or seconds, not minutes. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150409/0010fb6d/attachment.html From jsanda at redhat.com Fri Apr 10 00:11:19 2015 From: jsanda at redhat.com (John Sanda) Date: Fri, 10 Apr 2015 00:11:19 -0400 Subject: [Hawkular-dev] Ways to Test Availability Down? In-Reply-To: References:

Message-ID: <33424691-7CE3-4B07-AF36-CBCE7D557B21@redhat.com> > On Apr 9, 2015, at 11:41 PM, John Sanda wrote: > >> >> On Apr 9, 2015, at 10:33 PM, mike thompson > wrote: >> >>> >>> On 9 Apr 2015, at 19:24, mike thompson > wrote: >>> >>> I guess this question is mainly targeted to Hawkular QE. So all of my testing (especially on a dev box) shows availability 100% (as most sites will). So while I can mock this in code to show downtime and unknown. It appears that we need a consistent way to demonstrate up/down/unknown/whatever else. Is there a QE way to do this currently? >>> >>> I?m thinking a mock site that changes states every 5 or 10 minutes that is publicly available. This way we could all link our integration tests to this site to provide full coverage of states(and there could be other availability states like I have mentioned in other Hawkular ML mails). >>> >>> The use case I want to add is as a developer I want to test all availability states within a half hour (or whatever timeframe, but as everything waits on tests passing less is better) by pulling in certain site(s). >>> >>> Ideas? >> This can be as simple as having a servlet that cycles through response codes 2xx, 404, 5xx in a predictable manner. (content is irrelevant on status codes matter so should be easy). This way we could expect http status codes to change every 5 minutes: '12:00' -> ?up?, 12:05 -> ?down?, ?12:10? -> ?unknown?. So that it changes in a predictable way. >> >> Anyway, just an idea? >> >> Obviously, if this idea is legit, it can expanded in may different ways down in the future. > > I think there are a couple different concerns here. The first is simulating a resource that returns the various response codes. That should be easy enough. It is just a matter of starting up an HTTP server in a different thread that returns status codes in a known, predictable order. The second involving time is more challenging in my opinion. This was a big challenge for me when I was trying to write automated tests for aggregation in RHQ. The aggregation job ran hourly and computed hourly rollups, 6 hr rollups, and 24 hr rollups. Having tests run for 24 hrs in order to produce a 24 hr rollup was not practical. I encapsulated most of the date/time functionality in a service that could easily be stubbed or mocked. I also avoided calls like System.currentTimeMillis() or DateTime.now() in production code. Instead I used DateTimeService.currentTime() which I could easily override with test stubs. With these and other similar types of strategies, I think you should be able to test availability in milliseconds or seconds, not minutes._______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev For an HTTP server you want something that is lightweight and easily embeddable, two criteria that most servlet containers do not satisfy. If tests are being written in Java, I would look at things like Jetty, Vert.x, Undertow, or RxNetty if you want to go the reactive route. And if tests aren?t being written in Java but even staying on the JVM, you might have even more options. With Clojure for example, http-kit has no dependencies other than the core Clojure runtime. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150410/f22f95e3/attachment-0001.html From mfoley at redhat.com Fri Apr 10 08:28:04 2015 From: mfoley at redhat.com (Michael Foley) Date: Fri, 10 Apr 2015 08:28:04 -0400 (EDT) Subject: [Hawkular-dev] Ways to Test Availability Down? In-Reply-To: References: Message-ID: <504366061.13495729.1428668884779.JavaMail.zimbra@redhat.com> Hi Michael, This is a great question ... thank you for raising it. So you are asking about a mock resource ... or test fixture ... to facilitate integration testing of Hawkular? I think that is a great idea. In fact, I can envision a small portfolio or quiver of mock resources or test fixutres: * availability up * availability down * availability unknown * generating metrics low volume * generating metrics high volume * etc ... * to cover absolutely every primary use-case & resource that Hawkular would me monitoring I agree if we had a set, quiver, portfolio, or inventory of test fixtures it would facilitate integration testing. Next question ... might be "where"? * on the JON QE Bladecenter ... * in a private Docker registry * developers and QE could do a 'docker pull' and get whatever fixture they needed * this would de-couple the fixtures from the bladecenter ...and allow the test fixtures to be manipulated more easily by the developers and QE * in a public Docker registry * this would decouple from the bladecenter * allow developers in the community to use these test fixtures Something I saw recently ... is this thing called Arqullian Cube ... which can be used to control the lifecycle of Docker containers as part of integration tests. Some info here ... --->> http://blog.arungupta.me/run-javaee-tests-wildfly-docker-arquillian-cube/ So with this approach possibly ... integration tests could be written where test fixtures for availability up/down/unknown are programmatically spun up in Docker and automated integration tests run. Also ... another possible approach ... is to programmatically control these test fixtures in our Bladecenter via a REST API. So we/QE built a REST API to wrap the RHEV/Foreman infrastructure in our Bladecenter. So anything that can be done thru the UI can be done programmatically. So you can imagine an integration test for availability... where in @BeforeClass ... the automated test makes a REST call to spin up ...say ... an availability unknown test fixture ...and some tests run. Let's please talk more about this. I would like to learn more about your mock site idea where the availability state changes on a schedule. I'll set up a Bluejeans time and an etherpad where we can discuss this ...requirements, approaches, use-cases, etc .... Regards, Michael Foley JON and Lumbini QE Supervisor ----- Original Message ----- From: "mike thompson" To: "Discussions around Hawkular development" Cc: "Michael Foley" , "Armine Hovsepyan" Sent: Thursday, April 9, 2015 10:24:45 PM Subject: Ways to Test Availability Down? I guess this question is mainly targeted to Hawkular QE. So all of my testing (especially on a dev box) shows availability 100% (as most sites will). So while I can mock this in code to show downtime and unknown. It appears that we need a consistent way to demonstrate up/down/unknown/whatever else. Is there a QE way to do this currently? I?m thinking a mock site that changes states every 5 or 10 minutes that is publicly available. This way we could all link our integration tests to this site to provide full coverage of states(and there could be other availability states like I have mentioned in other Hawkular ML mails). The use case I want to add is as a developer I want to test all availability states within a half hour (or whatever timeframe, but as everything waits on tests passing less is better) by pulling in certain site(s). Ideas? ? Mike T -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150410/fdc0ce8b/attachment.html From mazz at redhat.com Sat Apr 11 00:08:49 2015 From: mazz at redhat.com (John Mazzitelli) Date: Sat, 11 Apr 2015 00:08:49 -0400 (EDT) Subject: [Hawkular-dev] hawkular monitor agent now sending avail to h-metrics rest In-Reply-To: <1333740959.12655975.1428724944639.JavaMail.zimbra@redhat.com> Message-ID: <224516634.12656183.1428725329306.JavaMail.zimbra@redhat.com> I have the hawkular monitor agent sending avail data successfully to h-metrics over its REST interface. I cannot get the bus message to flow successfully - I'm sure the JSON just isn't right (getting a NPE on the consumer end). Have I mentioned we need some strongly-typed Java client API jars? :) To see it work, build hawkular-agent repo: https://github.com/hawkular/hawkular-agent and install the subsystem in a wildfly instance. I have been installing it directly into a kettle instance like this: mvn -Dorg.hawkular.wildfly.home=/source/hawkular/kettle/target/wildfly-8.2.0.Final/ clean install wildfly-extension:deploy You define what metrics to collect and what availability to check via standalone.xml config of the agent subsystem. See this file as an example: https://github.com/hawkular/hawkular-agent/blob/master/hawkular-wildfly-monitor/src/main/assembly/subsystem.xml Next up, integrating JMX metrics/avail. (well, I guess I should get the avail bus message working first - anyone that can tell me what the avail bus message should look like, fill me in please). From tsegismo at redhat.com Mon Apr 13 04:44:19 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 13 Apr 2015 10:44:19 +0200 Subject: [Hawkular-dev] updated design doc for computing aggregate metrics In-Reply-To: References: Message-ID: <552B81E3.3000508@redhat.com> I've started to read the doc but I can't comment it. GDoc says it's read-only. Le 01/04/2015 03:36, John Sanda a ?crit : > I have updated update the design doc[1] for aggregate metrics to include a brief summary of a few different approaches in terms of overall architecture. I have also included details on how work can be distributed and coordinated among instances of hawkular-metrics. The changes I am proposing will also have an impact on the REST API as has been discussed in some other recent threads. Thanks in advance for any feedback. > > [1] http://bit.ly/1BAfF8d > > - John > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jsanda at redhat.com Mon Apr 13 07:25:47 2015 From: jsanda at redhat.com (John Sanda) Date: Mon, 13 Apr 2015 07:25:47 -0400 Subject: [Hawkular-dev] updated design doc for computing aggregate metrics In-Reply-To: <552B81E3.3000508@redhat.com> References: <552B81E3.3000508@redhat.com> Message-ID: <533B98EB-CC66-4F2E-832B-1BD7478187D9@redhat.com> Maybe I pasted the wrong link. Try this one. https://docs.google.com/document/d/1vQkj_rR_wUHwGnalDij8XXGKXXPTm9El3gQoEA3NCtU/edit?usp=sharing > On Apr 13, 2015, at 4:44 AM, Thomas Segismont wrote: > > I've started to read the doc but I can't comment it. GDoc says it's > read-only. > > Le 01/04/2015 03:36, John Sanda a ?crit : >> I have updated update the design doc[1] for aggregate metrics to include a brief summary of a few different approaches in terms of overall architecture. I have also included details on how work can be distributed and coordinated among instances of hawkular-metrics. The changes I am proposing will also have an impact on the REST API as has been discussed in some other recent threads. Thanks in advance for any feedback. >> >> [1] http://bit.ly/1BAfF8d >> >> - John >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150413/13f545b2/attachment.html From crobson at redhat.com Mon Apr 13 08:42:24 2015 From: crobson at redhat.com (Catherine Robson) Date: Mon, 13 Apr 2015 08:42:24 -0400 Subject: [Hawkular-dev] UX Meeting today - CANCELED Message-ID: <552BB9B0.9000205@redhat.com> Hi all - Due to the REST API meeting and some other conflicts, we'd like to cancel the UX meeting today. Let's plan to reconnect Thursday. Feel free to reach out if you need anything in the meantime. - Catherine -- Catherine Robson User Experience Design Red Hat JBoss Middleware c: 978-944-3825 From tsegismo at redhat.com Mon Apr 13 08:49:20 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 13 Apr 2015 14:49:20 +0200 Subject: [Hawkular-dev] updated design doc for computing aggregate metrics In-Reply-To: <533B98EB-CC66-4F2E-832B-1BD7478187D9@redhat.com> References: <552B81E3.3000508@redhat.com> <533B98EB-CC66-4F2E-832B-1BD7478187D9@redhat.com> Message-ID: <552BBB50.8060000@redhat.com> Le 13/04/2015 13:25, John Sanda a ?crit : > Maybe I pasted the wrong link. Try this one. > > https://docs.google.com/document/d/1vQkj_rR_wUHwGnalDij8XXGKXXPTm9El3gQoEA3NCtU/edit?usp=sharing Yep, this one works. > >> On Apr 13, 2015, at 4:44 AM, Thomas Segismont > > wrote: >> >> I've started to read the doc but I can't comment it. GDoc says it's >> read-only. >> >> Le 01/04/2015 03:36, John Sanda a ?crit : >>> I have updated update the design doc[1] for aggregate metrics to >>> include a brief summary of a few different approaches in terms of >>> overall architecture. I have also included details on how work can be >>> distributed and coordinated among instances of hawkular-metrics. The >>> changes I am proposing will also have an impact on the REST API as >>> has been discussed in some other recent threads. Thanks in advance >>> for any feedback. >>> >>> [1] http://bit.ly/1BAfF8d >>> >>> - John >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From theute at redhat.com Mon Apr 13 09:28:53 2015 From: theute at redhat.com (Thomas Heute) Date: Mon, 13 Apr 2015 09:28:53 -0400 (EDT) Subject: [Hawkular-dev] UX Meeting today - CANCELED In-Reply-To: <552BB9B0.9000205@redhat.com> References: <552BB9B0.9000205@redhat.com> Message-ID: <892245367.14353042.1428931733902.JavaMail.zimbra@redhat.com> Works for me ----- Original Message ----- From: "Catherine Robson" To: "Thomas Heute" , "mike thompson" , "Liz Clayton" , "Gabriel Cardoso" Cc: hawkular-dev at lists.jboss.org Sent: Monday, April 13, 2015 2:42:24 PM Subject: UX Meeting today - CANCELED Hi all - Due to the REST API meeting and some other conflicts, we'd like to cancel the UX meeting today. Let's plan to reconnect Thursday. Feel free to reach out if you need anything in the meantime. - Catherine -- Catherine Robson User Experience Design Red Hat JBoss Middleware c: 978-944-3825 From snegrea at redhat.com Mon Apr 13 09:49:01 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Mon, 13 Apr 2015 09:49:01 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - REST API Remake Message-ID: <1390849748.18348127.1428932941134.JavaMail.zimbra@redhat.com> The following meeting has been modified: Subject: Hawkular Metrics - REST API Remake Organizer: "Stefan Negrea" Location: Conf - 398-055-2127 & BlueJeans Time: Monday, April 13, 2015, 9:00:00 AM - 10:00:00 AM GMT -06:00 US/Canada Central Invitees: jsanda at redhat.com; hawkular-dev at lists.jboss.org; lclayton at redhat.com; tsegismo at redhat.com *~*~*~*~*~*~*~*~*~* Hello Everybody, I would like to schedule a meeting to discuss a proposal for changes to Hawkular Metrics REST API. The agenda also includes recent proposals from the channel and mailing list around tags and raw vs aggregated metrics from an end-point perspective. Here is a preliminary simple spreadsheet for this remake: https://docs.google.com/a/redhat.com/spreadsheets/d/1CQBfAenNlWbQENoK_Rms9pHooK2e7yGf15bm5avaN9A/edit#gid=329319157 Etherpad: http://jbosson.etherpad.corp.redhat.com/250 Thank you, Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2391 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150413/7fabccb8/attachment.bin From jshaughn at redhat.com Mon Apr 13 17:11:19 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Mon, 13 Apr 2015 17:11:19 -0400 Subject: [Hawkular-dev] Hawkular-113 Message-ID: <552C30F7.9060802@redhat.com> Just a note about a workaround pushed to master for https://issues.jboss.org/browse/HAWKULAR-113. The root cause may or may not be the same as for HWKMETRICS-47, it's not clear, but the behavior is similar. After a lot of research we can see that invoking concurrent, asynchronous EJB methods, that perform rest posts to metrics, can hang the threads in WFly 8.2 (used by Kettle). So, beware the @Asynchronous annotation on pooled EJBs, if the method is performing posts. With no obvious fix, the only choice I could see was to workaround the issue, and make the method synchronous. This was the issue where if you were to add multiple urls to Hawkular, you would quickly end up with all URLs reporting DOWN and basically the application was locked up, with a bunch of hanging threads. Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150413/2658babd/attachment.html From jkremser at redhat.com Mon Apr 13 19:41:59 2015 From: jkremser at redhat.com (Jiri Kremser) Date: Mon, 13 Apr 2015 19:41:59 -0400 (EDT) Subject: [Hawkular-dev] migration to the new inventory In-Reply-To: <182570064.10830927.1428968274957.JavaMail.zimbra@redhat.com> Message-ID: <1233525955.10831154.1428968519392.JavaMail.zimbra@redhat.com> Hi, we've merged all the pull requests so the new inventory is there. There are still some minor issues though. In UI you may see couple of errors, but the pinger seems to be working after all. Everything should be buildable of course. Hopefully I'll resolve the rest of the issues tomorrow, jk From jsanda at redhat.com Mon Apr 13 21:22:12 2015 From: jsanda at redhat.com (John Sanda) Date: Mon, 13 Apr 2015 21:22:12 -0400 Subject: [Hawkular-dev] Jira work log Message-ID: <777CE834-0195-4DC9-B24E-4DD1C7CF2383@redhat.com> I see that there is a work log tab for tickets. Is there a way to set it up so that when commits are pushed, the ticket can automatically be updated? I am pushing commits to my personal repo, and it would be nice for the ticket to indicate this so that it is easy to see where work is being done. I could just add a comment with the info, but I figured it would be worth checking to see if there is an automagic way for the ticket to be updated with that info. - John From hrupp at redhat.com Tue Apr 14 08:26:15 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 14 Apr 2015 14:26:15 +0200 Subject: [Hawkular-dev] Jira work log In-Reply-To: <777CE834-0195-4DC9-B24E-4DD1C7CF2383@redhat.com> References: <777CE834-0195-4DC9-B24E-4DD1C7CF2383@redhat.com> Message-ID: <349BE2C4-AB90-4281-AAE9-5A7F25B817E0@redhat.com> On 14 Apr 2015, at 3:22, John Sanda wrote: > I see that there is a work log tab for tickets. Is there a way to set > it up so that when commits are pushed, the ticket can automatically be > updated? I am pushing commits to my personal repo, and it would be GitHub provides web-hooks and services for this purpose. The service that is available is pretty odd. And I am not even convinced that it is meant for this purpose. Perhaps someone has solved that problem already. I am also not convinced that a webhook/service on organization org hawkular/* level would apply to your personal clones of the hawkular/* repos. What should be doable though, is that in the IDE you set up Jira as a context provider and have the IDE push comments to Jira when you push commits. Inside IJ, this is hidden under Tools->Task&Contexts. I am unfortunately lacking information about the correct setup for the JBoss Jira instance. From mazz at redhat.com Tue Apr 14 10:59:56 2015 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 14 Apr 2015 10:59:56 -0400 (EDT) Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <1854657651.14213676.1429023317733.JavaMail.zimbra@redhat.com> Message-ID: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> Can someone describe to me what needs to be done to see the Hawkular-Metrics UI (the explorer)? If I store metric and avail data into kettle via the Metrics REST API, I can't use the kettle UI because it is geared to the pinger. Is there a explorer UI that comes with kettle outside of the main hawkular UI? If not, is there something I have to do to enable that explorer webapp? From hrupp at redhat.com Tue Apr 14 12:12:19 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 14 Apr 2015 18:12:19 +0200 Subject: [Hawkular-dev] Availability In-Reply-To: References: <5B9D3EC5-2BBB-4DBA-B0B6-C07AF61CEAA4@redhat.com> <1997792.nRBWP4ZPt4@localhost.localdomain> <920594E5-9EF1-4C9D-B1C3-D9C1C42184AB@redhat.com> <8CE3BD16-97AD-44B9-9B04-F2EBB41AEB46@redhat.com> Message-ID: <6D94D3E6-3441-44A9-AF69-B5B9596317BF@redhat.com> On 20 Mar 2015, at 17:42, John Sanda wrote: > Great point about multiple resources. I have to therefore slightly > refine my previous statement. I think that the functionality belongs > on the agent. As I said in the agent discussions, I think that there > are use cases for different types of agents - embedded, c-located, and > remote. For the example of monitoring availability for a resource (or > resources) spanning several machines, I think it should be the job of > a remote agent. Maybe that remote agent is running inside the hawkular > server. I am not sure. They key though is that the approach is > consistent in terms of who is applying that availability function. I have been talking about "cheap alert pre-computation" on the agent in the past, and this certainly makes sense. But it also opens a can of worms^w complexity, that we should not tackle in the first place. The other question is what do we do with raw-avail-data-sources, that are not delivered by an agent we write, but e.g. via curl? For now I propose to do the computation on the server side. Attached image illustrates this. All incoming "raw" data (that should be used for alerting / computed resource state) is * stored in e.g. Hawkular-Metrics * run through the function that e.g. determines that a http status code 500 means DOWN or a duration > 300ms means WARN * the result of the function is then also stored and forwarded to alerting and other users. We would of course need to provide default-functions, but with the possibility to fine-tune them. -------------- next part -------------- A non-text attachment was scrubbed... Name: avail_function.png Type: image/png Size: 62163 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150414/1783579c/attachment-0001.png From mithomps at redhat.com Tue Apr 14 14:13:40 2015 From: mithomps at redhat.com (mike thompson) Date: Tue, 14 Apr 2015 11:13:40 -0700 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> Message-ID: <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> > On 14 Apr 2015, at 07:59, John Mazzitelli wrote: > > Can someone describe to me what needs to be done to see the Hawkular-Metrics UI (the explorer)? If I store metric and avail data into kettle via the Metrics REST API, I can't use the kettle UI because it is geared to the pinger. > > Is there a explorer UI that comes with kettle outside of the main hawkular UI? If not, is there something I have to do to enable that explorer webapp? A month or so ago I would have said "The explorer gets deployed with hawkular-metrics so just http://localhost:8080/explorer ?. However, there is no start.sh now, and deployment is deferred to Hawkular kettle. So there is a javascript app with no war for deployment. Also the REST urls have changed and will change again soon. In short, it looks the explorer needs a considerable amount of attention to make it work with latest hawkular kettle. This effort will need to be prioritised with the normal hawkular UI work. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150414/5eb04883/attachment.html From mazz at redhat.com Tue Apr 14 14:32:33 2015 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 14 Apr 2015 14:32:33 -0400 (EDT) Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> Message-ID: <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> How can I request it receive higher priority? :) If we are going to start allowing external feeds and agents to send in metric/avail data, we are going to need to have at least a minimal UI to just see the data (even if its a little HTML with the data in it). Anything to at least verify the data is getting in there and is what we expect it to be. Right now, AFAIK, we are limited to seeing metrics and avail for URL pings... that is going to get boring after a while :) If we had a generic metrics UI, we'd be able to see data for any subsystem in Wildfly. ----- Original Message ----- > > > On 14 Apr 2015, at 07:59, John Mazzitelli wrote: > > > > Can someone describe to me what needs to be done to see the > > Hawkular-Metrics UI (the explorer)? If I store metric and avail data into > > kettle via the Metrics REST API, I can't use the kettle UI because it is > > geared to the pinger. > > > > Is there a explorer UI that comes with kettle outside of the main hawkular > > UI? If not, is there something I have to do to enable that explorer > > webapp? > A month or so ago I would have said "The explorer gets deployed with > hawkular-metrics so just http://localhost:8080/explorer > ?. > > However, there is no start.sh now, and deployment is deferred to Hawkular > kettle. So there is a javascript app with no war for deployment. Also the > REST urls have changed and will change again soon. > > In short, it looks the explorer needs a considerable amount of attention to > make it work with latest hawkular kettle. This effort will need to be > prioritised with the normal hawkular UI work. > > > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > From tsegismo at redhat.com Tue Apr 14 16:19:29 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 14 Apr 2015 22:19:29 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> Message-ID: <552D7651.3020302@redhat.com> I'd be happy to share efforts to build a Grafana driver for Hawkular Metrics. Le 14/04/2015 20:32, John Mazzitelli a ?crit : > How can I request it receive higher priority? :) > > If we are going to start allowing external feeds and agents to send in metric/avail data, we are going to need to have at least a minimal UI to just see the data (even if its a little HTML

with the data in it). Anything to at least verify the data is getting in there and is what we expect it to be. > > Right now, AFAIK, we are limited to seeing metrics and avail for URL pings... that is going to get boring after a while :) If we had a generic metrics UI, we'd be able to see data for any subsystem in Wildfly. > > ----- Original Message ----- >> >>> On 14 Apr 2015, at 07:59, John Mazzitelli wrote: >>> >>> Can someone describe to me what needs to be done to see the >>> Hawkular-Metrics UI (the explorer)? If I store metric and avail data into >>> kettle via the Metrics REST API, I can't use the kettle UI because it is >>> geared to the pinger. >>> >>> Is there a explorer UI that comes with kettle outside of the main hawkular >>> UI? If not, is there something I have to do to enable that explorer >>> webapp? >> A month or so ago I would have said "The explorer gets deployed with >> hawkular-metrics so just http://localhost:8080/explorer >> ?. >> >> However, there is no start.sh now, and deployment is deferred to Hawkular >> kettle. So there is a javascript app with no war for deployment. Also the >> REST urls have changed and will change again soon. >> >> In short, it looks the explorer needs a considerable amount of attention to >> make it work with latest hawkular kettle. This effort will need to be >> prioritised with the normal hawkular UI work. >> >> >> >> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From miburman at redhat.com Tue Apr 14 17:52:13 2015 From: miburman at redhat.com (Michael Burman) Date: Tue, 14 Apr 2015 17:52:13 -0400 (EDT) Subject: [Hawkular-dev] Stronger typing of metrics In-Reply-To: <1630650056.17389606.1429048067427.JavaMail.zimbra@redhat.com> Message-ID: <689825787.17391240.1429048333625.JavaMail.zimbra@redhat.com> Hi, With our metric definitions, I'd like to see stronger definition of what sort of data we're storing and how it could be processed in the future. And with this I mean the same sort of stuff we had in the RHQ, such as "cumulative / gauge / trendsup / etc", so that we could give better post processing capabilities when fetching the data such as transforming the data between deltas and cumulative (depending on the user needs). While this could definitely be done with the tags, using such as "units", "type", we don't have any defined names for these options that we could depend on later. I guess this goes to the other tags discussion also, but I assume our tags are designed to work for searching capabilities as well as for definitions? - Micke From jsanda at redhat.com Tue Apr 14 20:24:52 2015 From: jsanda at redhat.com (John Sanda) Date: Tue, 14 Apr 2015 20:24:52 -0400 Subject: [Hawkular-dev] Stronger typing of metrics In-Reply-To: <689825787.17391240.1429048333625.JavaMail.zimbra@redhat.com> References: <689825787.17391240.1429048333625.JavaMail.zimbra@redhat.com> Message-ID: <4B451BE3-CB61-4249-8042-660AF3CA406F@redhat.com> We will be providing stronger typing with the changes for pre-computed aggregate metrics. We will be supporting gauges and counters and possibly other types as well. This will be reflected in the REST API. The changes were discussed quite a bit in the meeting that Stefan held on Monday. > On Apr 14, 2015, at 5:52 PM, Michael Burman wrote: > > Hi, > > With our metric definitions, I'd like to see stronger definition of what sort of data we're storing and how it could be processed in the future. And with this I mean the same sort of stuff we had in the RHQ, such as "cumulative / gauge / trendsup / etc", so that we could give better post processing capabilities when fetching the data such as transforming the data between deltas and cumulative (depending on the user needs). > > While this could definitely be done with the tags, using such as "units", "type", we don't have any defined names for these options that we could depend on later. I guess this goes to the other tags discussion also, but I assume our tags are designed to work for searching capabilities as well as for definitions? > > - Micke > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Wed Apr 15 02:55:52 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 15 Apr 2015 08:55:52 +0200 Subject: [Hawkular-dev] Another perspective on Availability In-Reply-To: <54221624-5F4B-4CD7-9794-05E7E928A72E@redhat.com> References: <54221624-5F4B-4CD7-9794-05E7E928A72E@redhat.com> Message-ID: <1228CB96-EB73-40CA-B0CE-B00ACBEF8609@redhat.com> On 7 Apr 2015, at 19:40, mike thompson wrote: > I found this view of availability interesting from open source > monitoring project: CachetHQ(https://cachethq.io/ > ) > > Perhaps we might want to incorporate these ideas of component status > versus incident status. I think this is another example of computed resource state. Where some function analysis incoming data and then computes those. > Operational This probably corresponds to "UP" in out terms. > Performance Issues > > The component is experiencing some slowness. > > 3 > > Partial Outage Both are sort of "WARN", that we do not directly have on the radar for a single resource yet, but more in the sense of mixed state. I agree, that it could be good to give the admin an idea of some sub-variants of "WARN" to convey more information directly in the state. > Incident Status This is completely orthogonal to the above. As e.g. "scheduled" sort of "overrides" in the sense that we know that a resource may be down of affected during maintenance. > Scheduled => Maintenance mode > Investigating > > You have reports of a problem and you're currently looking into them. Sub-division of "acknowledged" > > 2 > > Identified > > You've found the issue and you're working on a fix. Another sub-variant of acknowledged > 3 > > Watching > > You've since deployed a fix and you're currently watching the > situation. Dito to me > Fixed > > The fix has worked, you're happy to close the incident. This is probably more like "closed" -- and could be even triggered by recovery alerts (forgot how we call them now - safety mode?) From hrupp at redhat.com Wed Apr 15 03:00:21 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 15 Apr 2015 09:00:21 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <552D7651.3020302@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> Message-ID: <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> On 14 Apr 2015, at 22:19, Thomas Segismont wrote: > I'd be happy to share efforts to build a Grafana driver for Hawkular > Metrics. Why not use the influx endpoint as described here http://pilhuhn.blogspot.de/2014/06/rhq-metrics-and-grafana.html ? From tsegismo at redhat.com Wed Apr 15 03:40:34 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 15 Apr 2015 09:40:34 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> Message-ID: <552E15F2.4000906@redhat.com> Le 15/04/2015 09:00, Heiko W.Rupp a ?crit : > On 14 Apr 2015, at 22:19, Thomas Segismont wrote: > >> I'd be happy to share efforts to build a Grafana driver for Hawkular >> Metrics. > > Why not use the influx endpoint as described here > http://pilhuhn.blogspot.de/2014/06/rhq-metrics-and-grafana.html ? Because I don't think we can maintain it in the future: http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000594.html From tsegismo at redhat.com Wed Apr 15 03:58:14 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 15 Apr 2015 09:58:14 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <552E15F2.4000906@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> <552E15F2.4000906@redhat.com> Message-ID: <552E1A16.7070804@redhat.com> Le 15/04/2015 09:40, Thomas Segismont a ?crit : > Le 15/04/2015 09:00, Heiko W.Rupp a ?crit : >> On 14 Apr 2015, at 22:19, Thomas Segismont wrote: >> >>> I'd be happy to share efforts to build a Grafana driver for Hawkular >>> Metrics. >> >> Why not use the influx endpoint as described here >> http://pilhuhn.blogspot.de/2014/06/rhq-metrics-and-grafana.html ? > > Because I don't think we can maintain it in the future: > http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000594.html I realize I haven't answered correctly. Mazz, if you're working with a master build you can use the Influx endpoint with Grafana. When metric types (gauges and counters) will land in master, I'm not sure it will still work. From hrupp at redhat.com Wed Apr 15 04:28:10 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 15 Apr 2015 10:28:10 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <552E15F2.4000906@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> <552E15F2.4000906@redhat.com> Message-ID: <90E58160-C7D0-4AE2-B9E5-A2D22B6E7002@redhat.com> On 15 Apr 2015, at 9:40, Thomas Segismont wrote: > Le 15/04/2015 09:00, Heiko W.Rupp a ?crit : >> On 14 Apr 2015, at 22:19, Thomas Segismont wrote: >> >>> I'd be happy to share efforts to build a Grafana driver for Hawkular >>> Metrics. >> >> Why not use the influx endpoint as described here >> http://pilhuhn.blogspot.de/2014/06/rhq-metrics-and-grafana.html ? > > Because I don't think we can maintain it in the future: > http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000594.html Do we have any idea of the effort to create *and* maintain such a Grafana driver? And what is the likeliness that it may be accepted into Grafana? I think it is sad to see this (current) integration go. From tsegismo at redhat.com Wed Apr 15 05:45:06 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 15 Apr 2015 11:45:06 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <90E58160-C7D0-4AE2-B9E5-A2D22B6E7002@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> <552E15F2.4000906@redhat.com> <90E58160-C7D0-4AE2-B9E5-A2D22B6E7002@redhat.com> Message-ID: <552E3322.8020606@redhat.com> Le 15/04/2015 10:28, Heiko W.Rupp a ?crit : > On 15 Apr 2015, at 9:40, Thomas Segismont wrote: > >> Le 15/04/2015 09:00, Heiko W.Rupp a ?crit : >>> On 14 Apr 2015, at 22:19, Thomas Segismont wrote: >>> >>>> I'd be happy to share efforts to build a Grafana driver for Hawkular >>>> Metrics. >>> >>> Why not use the influx endpoint as described here >>> http://pilhuhn.blogspot.de/2014/06/rhq-metrics-and-grafana.html ? >> >> Because I don't think we can maintain it in the future: >> http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000594.html > > Do we have any idea of the effort to create *and* maintain such a Grafana No idea, I haven't looked at it yet. > driver? And what is the likeliness that it may be accepted into Grafana? I couldn't say. We need to ask them. > > I think it is sad to see this (current) integration go. It wouldn't go away strictly speaking. But it would work with gauges only. From theute at redhat.com Wed Apr 15 07:54:47 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 15 Apr 2015 13:54:47 +0200 Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> Message-ID: <552E5187.5000203@redhat.com> Catching up on old emails... On 03/31/2015 02:53 PM, Gary Brown wrote: > Ok thanks for the info. > > Just to be clear - so as components are dynamically deployed/undeployed from a server, these should be reflected in the Inventory - so it represents a current view of the environment being managed? This is an important point. When a resource in deployed / undeployed / deployed (or redeployed), you want to know if it's there but you also want to keep historical data (say response time history before the redeployment). It also has an impact on alerts, if an app is undeployed legitimately you don't want to receive 'non-available' alerts but you want them back once the app is deployed... > Are there any plans to represent docker images in Inventory, associated with the servers that have been launched using them? It should become available in the UI, but will it come directly from Kubernetes (when available) ? Synched with Kubernetes ? I don't know, we'll tackle this later. Thomas > > Regards > Gary > > ----- Original Message ----- >> ----- Original Message ----- >>> From: "Gary Brown" >>> To: "Discussions around Hawkular development" >>> >>> Sent: Tuesday, 31 March, 2015 10:37:53 AM >>> Subject: [Hawkular-dev] Business app/services representation in Inventory >>> >>> Hi >>> >>> Before going too far down the BTM road, I just wanted to confirm whether or >>> not we want the business app, their components services, and their >>> relationships to IT resources they use, stored in Hawkular Inventory? >>> >> >> Inventory definitely is the right place to store such information. >> >>> An alternative approach would be to derive the structure and relationships >>> dynamically from the business transaction instance information. >>> >> >> Deriving the structure and relationships dynamically is basically >> a "discovery" as called in ye olde RHQ days. That is a capability >> which we'd very much like to keep. >> >> The new inventory is (so far) unaware of special "discovery" step - >> everything >> from resource creation to establishing relationships is done through 1 public >> API that "anyone" can use. >> >>> The benefit of storing in Inventory is it enables end users to navigate >>> through the inventory to understand the relationships to the business >>> apps/services, as well as allow other tooling (e.g. impact analysis) to >>> determine the effect of IT resource downtime on business apps. >>> >> >> +1. I know Brett will object that that's what Artificer is for, too, but I >> personally see the difference in Inventory's focus on relationships, while >> Artificer is more geared towards managing content. >> >>> Thoughts? >>> >>> Regards >>> Gary >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From theute at redhat.com Wed Apr 15 08:04:34 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 15 Apr 2015 14:04:34 +0200 Subject: [Hawkular-dev] Hawkular Metrics - 0.3.1 In-Reply-To: <1555745278.11199263.1428435316002.JavaMail.zimbra@redhat.com> References: <1555745278.11199263.1428435316002.JavaMail.zimbra@redhat.com> Message-ID: <552E53D2.1040708@redhat.com> On 04/07/2015 09:35 PM, Stefan Negrea wrote: > Hello Everybody, > > I am happy to announce a big milestone for the Hawkular Metrics project. We are releasing today for the first time from the repository hosted by the Hawkular organization. > > Major changes: > 1) UI > - The core UI has been migrated to Hawkular UI related projects (hawkular-ui-components, hawkular, and hawkular-ui-services). The explorer project is still available for testing purposes. > 2) REST > - Consistent error reporting and status codes usage across endpoints > - Added Availability statistics (https://issues.jboss.org/browse/HWKMETRICS-35): > * Total downtime duration > * Last downtime > * Percentage of uptime > * Number of downtimes > - New Numeric Metric statistics (https://issues.jboss.org/browse/HWKMETRICS-34): > * Average > * Median > * 95th Percentile Is that precalculated ? I guess you can't query for 99th percentile ? > - The REST implementation has been decoupled from the actual core logic, which paves the way for alternate REST implementations > 3) Core > - Large refactoring of the core classes and packages, the domain related logic has been pushed to the core layer > - The Memory storage engine has been dropped from the project. Cassandra (embedded or standalone) is the exclusive storage engine. > 4) InfluxDB Integration > - Influx Java client supports sending and reading data (it was not possible before because of the endoint URI differences) to/from Hawkular Metrics. Other clients are not tested but should work as well. > 5) PTrans > - Configurable list of listeners (previously all collectd, ganglia, ... etc listeners were started) > - Bug fix: send data to Metrics server if the buffer is full or no data was received recently (previously data could sit in the buffer and wait for the buffer to be full before being sent) > > > Github Release: > https://github.com/hawkular/hawkular-metrics/releases/tag/0.3.1 > > JBoss Nexus Maven artifacts: > http://origin-repository.jboss.org/nexus/content/repositories/public/org/hawkular/metrics/ > > > A big "Thank you" goes to John Sanda, Thomas Segismont, Mike Thompson, and Heiko Rupp for their project contributions. Special mentions go to Jeeva Kandasamy, Jirka Kremser, and Michael Burman for their project help. > > I am happy to announce that Matt Wringe joined the Hawkular Metrics team with a focus on containers and project integrations. We are looking forward to his contributions. > > Any discussion, suggestions or contributions are more than welcomed; so feel free to reply to this email or join #hawkular on freenode. > > > Thank you, > Stefan Negrea > > Software Engineer > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jshaughn at redhat.com Wed Apr 15 11:22:04 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 15 Apr 2015 11:22:04 -0400 Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <552E5187.5000203@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> <552E5187.5000203@redhat.com> Message-ID: <552E821C.2010507@redhat.com> On 4/15/2015 7:54 AM, Thomas Heute wrote: > Catching up on old emails... > > On 03/31/2015 02:53 PM, Gary Brown wrote: >> Ok thanks for the info. >> >> Just to be clear - so as components are dynamically deployed/undeployed from a server, these should be reflected in the Inventory - so it represents a current view of the environment being managed? > This is an important point. > > When a resource in deployed / undeployed / deployed (or redeployed), you > want to know if it's there but you also want to keep historical data > (say response time history before the redeployment). > > It also has an impact on alerts, if an app is undeployed legitimately > you don't want to receive 'non-available' alerts but you want them back > once the app is deployed... This could be relevant if avail is reported from an external monitor, like the pinger. It either needs to be aware of the redeploy, or aware of some maintenance window, I think, in order to not report DOWN avail. Maybe easier is to let the down avail be reported but instead mute the Triggers. If Triggers are logically connected to Resources then we may want to disable the triggers during certain resource-level events, like a maintenance window or a scheduled redeploy operation. In RHQ I think people liked to see down avail reported for a resource when it was down for a redeploy. If we could optionally disable alerting then that would be nice. Another possibility could maybe be to add some sort of validation action on a trigger, or maybe some sort of callback, such that persistence and further actions are ignored if validation does not pass. So, instead of preventing the triggering up front, we could mute it in a post-processing way. From gbrown at redhat.com Wed Apr 15 11:58:40 2015 From: gbrown at redhat.com (Gary Brown) Date: Wed, 15 Apr 2015 11:58:40 -0400 (EDT) Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <552E821C.2010507@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> <552E5187.5000203@redhat.com> <552E821C.2010507@redhat.com> Message-ID: <243003922.305370.1429113520769.JavaMail.zimbra@redhat.com> Couldn't the deployment status be added to the trigger when relevant, e.g. avail = DOWN && deployment_status = DEPLOYED - if user wants to ignore DOWN notifications when in maintenance mode. Reason being that something checking the end to end availability of a linked (dependent) set of resources would want to know if a resource was down, regardless of whether it was in maintenance, as it impacts the higher level business app. So it may be on a case by case basis - so possibly best left to the trigger definition to determine if deployment status is relevant? Regards Gary ----- Original Message ----- > > > On 4/15/2015 7:54 AM, Thomas Heute wrote: > > Catching up on old emails... > > > > On 03/31/2015 02:53 PM, Gary Brown wrote: > >> Ok thanks for the info. > >> > >> Just to be clear - so as components are dynamically deployed/undeployed > >> from a server, these should be reflected in the Inventory - so it > >> represents a current view of the environment being managed? > > This is an important point. > > > > When a resource in deployed / undeployed / deployed (or redeployed), you > > want to know if it's there but you also want to keep historical data > > (say response time history before the redeployment). > > > > It also has an impact on alerts, if an app is undeployed legitimately > > you don't want to receive 'non-available' alerts but you want them back > > once the app is deployed... > > This could be relevant if avail is reported from an external monitor, > like the pinger. It either needs to be aware of the redeploy, or aware > of some maintenance window, I think, in order to not report DOWN avail. > Maybe easier is to let the down avail be reported but instead mute the > Triggers. If Triggers are logically connected to Resources then we may > want to disable the triggers during certain resource-level events, like > a maintenance window or a scheduled redeploy operation. In RHQ I think > people liked to see down avail reported for a resource when it was down > for a redeploy. If we could optionally disable alerting then that would > be nice. > > Another possibility could maybe be to add some sort of validation action > on a trigger, or maybe some sort of callback, such that persistence and > further actions are ignored if validation does not pass. So, instead of > preventing the triggering up front, we could mute it in a > post-processing way. > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jshaughn at redhat.com Wed Apr 15 14:18:35 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 15 Apr 2015 14:18:35 -0400 Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <243003922.305370.1429113520769.JavaMail.zimbra@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> <552E5187.5000203@redhat.com> <552E821C.2010507@redhat.com> <243003922.305370.1429113520769.JavaMail.zimbra@redhat.com> Message-ID: <552EAB7B.1030903@redhat.com> It certainly could be. As I mentioned below, I agree that users want to see DOWN avail during redeploy, they just don't want to see an alert. If we incorporate deployment_status into the data being fed to alerting then we can do this sort of alerting. Although, I'm not positive that we want to report deployment_status in that way. I think given reporting intervals we may encounter some edge cases. On 4/15/2015 11:58 AM, Gary Brown wrote: > Couldn't the deployment status be added to the trigger when relevant, e.g. avail = DOWN && deployment_status = DEPLOYED - if user wants to ignore DOWN notifications when in maintenance mode. > > Reason being that something checking the end to end availability of a linked (dependent) set of resources would want to know if a resource was down, regardless of whether it was in maintenance, as it impacts the higher level business app. So it may be on a case by case basis - so possibly best left to the trigger definition to determine if deployment status is relevant? > > Regards > Gary > > ----- Original Message ----- >> >> On 4/15/2015 7:54 AM, Thomas Heute wrote: >>> Catching up on old emails... >>> >>> On 03/31/2015 02:53 PM, Gary Brown wrote: >>>> Ok thanks for the info. >>>> >>>> Just to be clear - so as components are dynamically deployed/undeployed >>>> from a server, these should be reflected in the Inventory - so it >>>> represents a current view of the environment being managed? >>> This is an important point. >>> >>> When a resource in deployed / undeployed / deployed (or redeployed), you >>> want to know if it's there but you also want to keep historical data >>> (say response time history before the redeployment). >>> >>> It also has an impact on alerts, if an app is undeployed legitimately >>> you don't want to receive 'non-available' alerts but you want them back >>> once the app is deployed... >> This could be relevant if avail is reported from an external monitor, >> like the pinger. It either needs to be aware of the redeploy, or aware >> of some maintenance window, I think, in order to not report DOWN avail. >> Maybe easier is to let the down avail be reported but instead mute the >> Triggers. If Triggers are logically connected to Resources then we may >> want to disable the triggers during certain resource-level events, like >> a maintenance window or a scheduled redeploy operation. In RHQ I think >> people liked to see down avail reported for a resource when it was down >> for a redeploy. If we could optionally disable alerting then that would >> be nice. >> >> Another possibility could maybe be to add some sort of validation action >> on a trigger, or maybe some sort of callback, such that persistence and >> further actions are ignored if validation does not pass. So, instead of >> preventing the triggering up front, we could mute it in a >> post-processing way. >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150415/104e0ba5/attachment.html From gbrown at redhat.com Thu Apr 16 03:26:14 2015 From: gbrown at redhat.com (Gary Brown) Date: Thu, 16 Apr 2015 03:26:14 -0400 (EDT) Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <552EAB7B.1030903@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> <552E5187.5000203@redhat.com> <552E821C.2010507@redhat.com> <243003922.305370.1429113520769.JavaMail.zimbra@redhat.com> <552EAB7B.1030903@redhat.com> Message-ID: <867876190.733992.1429169174987.JavaMail.zimbra@redhat.com> Hi ----- Original Message ----- > > It certainly could be. As I mentioned below, I agree that users want to see > DOWN avail during redeploy, they just don't want to see an alert. If we > incorporate deployment_status into the data being fed to alerting then we > can do this sort of alerting. Although, I'm not positive that we want to > report deployment_status in that way. I think given reporting intervals we > may encounter some edge cases. I was thinking that the deployment status for the expression could be retrieved from inventory/cached data rather than be part of the metric. Is that possible? Regards Gary > > On 4/15/2015 11:58 AM, Gary Brown wrote: > > > > Couldn't the deployment status be added to the trigger when relevant, e.g. > avail = DOWN && deployment_status = DEPLOYED - if user wants to ignore DOWN > notifications when in maintenance mode. > > Reason being that something checking the end to end availability of a linked > (dependent) set of resources would want to know if a resource was down, > regardless of whether it was in maintenance, as it impacts the higher level > business app. So it may be on a case by case basis - so possibly best left > to the trigger definition to determine if deployment status is relevant? > > Regards > Gary > > ----- Original Message ----- > > > > On 4/15/2015 7:54 AM, Thomas Heute wrote: > > > > Catching up on old emails... > > On 03/31/2015 02:53 PM, Gary Brown wrote: > > > > Ok thanks for the info. > > Just to be clear - so as components are dynamically deployed/undeployed > from a server, these should be reflected in the Inventory - so it > represents a current view of the environment being managed? > This is an important point. > > When a resource in deployed / undeployed / deployed (or redeployed), you > want to know if it's there but you also want to keep historical data > (say response time history before the redeployment). > > It also has an impact on alerts, if an app is undeployed legitimately > you don't want to receive 'non-available' alerts but you want them back > once the app is deployed... > This could be relevant if avail is reported from an external monitor, > like the pinger. It either needs to be aware of the redeploy, or aware > of some maintenance window, I think, in order to not report DOWN avail. > Maybe easier is to let the down avail be reported but instead mute the > Triggers. If Triggers are logically connected to Resources then we may > want to disable the triggers during certain resource-level events, like > a maintenance window or a scheduled redeploy operation. In RHQ I think > people liked to see down avail reported for a resource when it was down > for a redeploy. If we could optionally disable alerting then that would > be nice. > > Another possibility could maybe be to add some sort of validation action > on a trigger, or maybe some sort of callback, such that persistence and > further actions are ignored if validation does not pass. So, instead of > preventing the triggering up front, we could mute it in a > post-processing way. > > > _______________________________________________ > hawkular-dev mailing list hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From hrupp at redhat.com Thu Apr 16 09:33:32 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 16 Apr 2015 15:33:32 +0200 Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> Message-ID: Hi, On 31 Mar 2015, at 10:37, Gary Brown wrote: > whether or not we want the business app, their components services, > and their relationships to IT resources they use, stored in Hawkular > Inventory? Yes. We need to still decide though what kind of resources we want to store and represent. One of the confusing aspects of RHQ (UI) was that we basically did our job too good and showed too much. From hrupp at redhat.com Thu Apr 16 09:36:39 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 16 Apr 2015 15:36:39 +0200 Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <1122568546.10127626.1427827146205.JavaMail.zimbra@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1122568546.10127626.1427827146205.JavaMail.zimbra@redhat.com> Message-ID: On 31 Mar 2015, at 20:39, John Doyle wrote: >> Deriving the structure and relationships dynamically is basically >> a "discovery" as called in ye olde RHQ days. That is a capability >> which we'd very much like to keep. > > +1 Very powerful and important in getting a great user experience. Just to make this a bit more explicit: for Hawkular we want not only discover and represent the classic parent-child relations like "WildFly running on JVM on RHEL", but also discover that e.g. a couple of WildFly servers (or apps) use the same JDBC url, so they are most likely to form an application that is clustered. Similarly for joint Caches / JGroups / messaging infrastructure. From hrupp at redhat.com Thu Apr 16 10:03:37 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 16 Apr 2015 16:03:37 +0200 Subject: [Hawkular-dev] Business app/services representation in Inventory In-Reply-To: <552EAB7B.1030903@redhat.com> References: <647407669.6594118.1427791073691.JavaMail.zimbra@redhat.com> <1837689158.7959369.1427805015434.JavaMail.zimbra@redhat.com> <1735048842.6798059.1427806415238.JavaMail.zimbra@redhat.com> <552E5187.5000203@redhat.com> <552E821C.2010507@redhat.com> <243003922.305370.1429113520769.JavaMail.zimbra@redhat.com> <552EAB7B.1030903@redhat.com> Message-ID: On 15 Apr 2015, at 20:18, Jay Shaughnessy wrote: > It certainly could be. As I mentioned below, I agree that users want > to see DOWN avail during redeploy, they just don't want to see an > alert. If we incorporate deployment_status into the data being fed to > alerting then we can do this sort of alerting. Although, I'm not > positive that we want to report deployment_status in that way. I > think given reporting intervals we may encounter some edge cases This "deployment_status" could be another value in the resource state like "maintenance mode" - perhaps "re-deploying" or such. See also "Computed resource state" email(s) http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000413.html From tsegismo at redhat.com Thu Apr 16 12:44:52 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 16 Apr 2015 18:44:52 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <552E3322.8020606@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> <552E15F2.4000906@redhat.com> <90E58160-C7D0-4AE2-B9E5-A2D22B6E7002@redhat.com> <552E3322.8020606@redhat.com> Message-ID: <552FE704.2050006@redhat.com> Le 15/04/2015 11:45, Thomas Segismont a ?crit : > Le 15/04/2015 10:28, Heiko W.Rupp a ?crit : >> On 15 Apr 2015, at 9:40, Thomas Segismont wrote: >> >>> Le 15/04/2015 09:00, Heiko W.Rupp a ?crit : >>>> On 14 Apr 2015, at 22:19, Thomas Segismont wrote: >>>> >>>>> I'd be happy to share efforts to build a Grafana driver for Hawkular >>>>> Metrics. >>>> >>>> Why not use the influx endpoint as described here >>>> http://pilhuhn.blogspot.de/2014/06/rhq-metrics-and-grafana.html ? >>> >>> Because I don't think we can maintain it in the future: >>> http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000594.html >> >> Do we have any idea of the effort to create *and* maintain such a Grafana > > No idea, I haven't looked at it yet. > >> driver? And what is the likeliness that it may be accepted into Grafana? > > I couldn't say. We need to ask them. > >> >> I think it is sad to see this (current) integration go. > > It wouldn't go away strictly speaking. But it would work with gauges only. We had a metrics meeting yesterday about data visualization and here's the outcome for Influx support: * we'll update the "list series" handler so that it returns gauges with the "gauge." prefix and counters with the "counter." prefix * we'll update the query handler so that it looks at the series name for a known prefix ("gauge." or "counter."); if there's no prefix, it will assume it's a gauge With such changes we should still be able to use Grafana to visualize Metrics data. Is there a parent JIRA for gauge/counter changes? I'd like to add children tasks for the Influx API. From theute at redhat.com Thu Apr 16 13:20:25 2015 From: theute at redhat.com (Thomas Heute) Date: Thu, 16 Apr 2015 19:20:25 +0200 Subject: [Hawkular-dev] metrics explorer UI In-Reply-To: <552FE704.2050006@redhat.com> References: <1749835078.14216547.1429023596153.JavaMail.zimbra@redhat.com> <6E6EB20E-D262-4CB3-9000-3EFEEF082B9A@redhat.com> <1378513309.14322464.1429036353190.JavaMail.zimbra@redhat.com> <552D7651.3020302@redhat.com> <81B264C1-0FC7-4D1D-AD5B-85F50DADF0B3@redhat.com> <552E15F2.4000906@redhat.com> <90E58160-C7D0-4AE2-B9E5-A2D22B6E7002@redhat.com> <552E3322.8020606@redhat.com> <552FE704.2050006@redhat.com> Message-ID: <552FEF59.2080105@redhat.com> On 04/16/2015 06:44 PM, Thomas Segismont wrote: > With such changes we should still be able to use Grafana to visualize > Metrics data. Awesome :) From mazz at redhat.com Thu Apr 16 17:21:07 2015 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 16 Apr 2015 17:21:07 -0400 (EDT) Subject: [Hawkular-dev] hawkular monitor agent - inventory In-Reply-To: <929295931.1645197.1429218221254.JavaMail.zimbra@redhat.com> Message-ID: <696123875.1657322.1429219267038.JavaMail.zimbra@redhat.com> The Hawkular Monitor Agent (the thing that runs inside of WildFly as a subsystem) can now monitor both its own WildFly instance as well as any remote WildFly instance. It can collect metrics and perform availability checks on any attribute in any subsystem within WildFly and can store that data in Hawkular-Metrics or Hawkular ecosystem. (so if there is any product that runs inside of WildFly, we can now monitor it). I am at a point where I can integrate it into kettle. I need to now talk to Lukas and gang about the inventory stuff. I have no idea where to start when it comes to integrating with inventory. Lukas - can you setup a call of some sort where you guys can tell me what I have to do and I can ask questions? Hopefully this can help us figure out what we need from inventory from a client perspective. From lkrejci at redhat.com Fri Apr 17 04:17:12 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Fri, 17 Apr 2015 04:17:12 -0400 (EDT) Subject: [Hawkular-dev] hawkular monitor agent - inventory Message-ID: <405209618.1590915.1429258632731.JavaMail.zimbra@redhat.com> The following is a new meeting request: Subject: [Hawkular-dev] hawkular monitor agent - inventory Organiser: "Lukas Krejci" Time: Friday, 17 April, 2015, 4:00:00 PM - 5:00:00 PM GMT +01:00 Belgrade, Bratislava, Budapest, Ljubljana, Prague Invitees: mazz at redhat.com; hawkular-dev at lists.jboss.org *~*~*~*~*~*~*~*~*~* Agenda: Discuss how to set up and represent wildfly management model entities in hawkular inventory. To join the Meeting: https://bluejeans.com/691016054 To join via Browser: https://bluejeans.com/691016054/browser To join with Lync: https://bluejeans.com/691016054/lync To join via Room System: Video Conferencing System: bjn.vc -or- 199.48.152.152 Meeting ID: 691016054 To join via Phone: 1) Dial: +442035746870 (see all numbers - https://www.intercallonline.com/listNumbersByCode.action?confCode=8169978803) 2) Enter Conference ID: 8169978803 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2176 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150417/76c268d5/attachment.bin From mazz at redhat.com Fri Apr 17 18:03:01 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 17 Apr 2015 18:03:01 -0400 (EDT) Subject: [Hawkular-dev] hawkular monitor agent integrated in kettle In-Reply-To: <212473500.2196087.1429308088193.JavaMail.zimbra@redhat.com> Message-ID: <1794040262.2196163.1429308181136.JavaMail.zimbra@redhat.com> The Hawkular Monitor agent is now integrated in kettle. When you build kettle, along with all the other stuff, you will get a new subsystem extension installed which will collect metrics from the kettle instance itself. It stores data directly to Hawkular-Metrics and sends a bus message so alerts can do stuff with it. You will see nothing in the UI regarding this stuff - but every 5 minutes you should see some diagnostics in your log file. And nothing has yet been done wrt inventory integration. From mwringe at redhat.com Fri Apr 17 18:44:12 2015 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 17 Apr 2015 18:44:12 -0400 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers Message-ID: <55318CBC.8010105@redhat.com> I have a new subproject in Hawkular Metrics which sets up creating components for Openshift/Fabric8 (https://github.com/hawkular/hawkular-metrics/pull/200). There are 3 main parts Cassandra: creates a custom seed provider to support ReplicationControllers in Kubernetes, creates a folder/zip archive which can be used to generate a Docker image. It may make sense to move the Cassandra parts out to a separate project. Hawkular Metrics: creates a folder/zip archive which can be used to generate a Docker image Kubernetes: pulls everything together into a single kubernetes application. Can be used to deploy an application zip into fabric8 (via drag and drop in the web console or via the maven plugin) or deploy all the components into Openshift via the kubernetes.json configuration file. The docker images are not created and deployed to a docker registry as part of the build, it will just create a folder where you can run the docker build from. None of the maven docker plugins I looked at seemed to really work properly, so its still a manual process to do the build (and push to a registry). Its something which needs to be improved. The Cassandra service currently only supports adding new nodes to a cluster and not removing them via the ReplicationController. This is due to the replication factor being set to be 1 by default (which means when a node is removed, so is the data it contained). I believe the docker subproject of hawkular metrics is obsolete and can be removed (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but someone please correct me if I am wrong. It's scripts are referring to the console which no longer exists as part of the project. - Matt From rhauch at redhat.com Sat Apr 18 11:10:52 2015 From: rhauch at redhat.com (Randall Hauch) Date: Sat, 18 Apr 2015 10:10:52 -0500 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <55318CBC.8010105@redhat.com> References: <55318CBC.8010105@redhat.com> Message-ID: I?m using com.spotify:docker-maven-plugin (https://github.com/spotify/docker-maven-plugin ) in a project other than Hawkular, and I think it works quite well. Yes, it is easier to use if you build an assembly with the layout, and then simply ADD the archive via the Dockerfile. (Recall that ADD will extract the contents of a ZIP or TAR archive, whereas COPY just copies files.) The Maven plugin can easily build the image, register it locally, and optionally push to DockerHub. > On Apr 17, 2015, at 5:44 PM, Matt Wringe wrote: > > The docker images are not created and deployed to a docker registry as > part of the build, it will just create a folder where you can run the > docker build from. None of the maven docker plugins I looked at seemed > to really work properly, so its still a manual process to do the build > (and push to a registry). Its something which needs to be improved. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150418/a67899b4/attachment.html From mwringe at redhat.com Mon Apr 20 09:32:03 2015 From: mwringe at redhat.com (Matt Wringe) Date: Mon, 20 Apr 2015 09:32:03 -0400 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: References: <55318CBC.8010105@redhat.com> Message-ID: <5534FFD3.9080802@redhat.com> On 18/04/15 11:10 AM, Randall Hauch wrote: > I?m using com.spotify:docker-maven-plugin > (https://github.com/spotify/docker-maven-plugin) in a project other > than Hawkular, and I think it works quite well. Yes, it is easier to > use if you build an assembly with the layout, and then simply ADD the > archive via the Dockerfile. (Recall that ADD will extract the contents > of a ZIP or TAR archive, whereas COPY just copies files.) The Maven > plugin can easily build the image, register it locally, and optionally > push to DockerHub. The main issue I had with the spotify and jolokia maven plugins is that they don't support unix sockets (eg for local builds). The docker daemon defaults to unix sockets, which is the more secure option. So to use these plugins, you need to manually configure your system to use tcp instead of the unix sockets. Which if you do it the easy way, opens up your machine to an easy root vulnerability. Or, to do it the proper way, manually setup all your certificates and handle things over ssl. But, I guess I can give it another look to see just how difficult it is to manually set it all up. Overall I really got the impression that you gain more control over just building it manually yourself and then using a script to push out the images after. > > >> On Apr 17, 2015, at 5:44 PM, Matt Wringe > > wrote: >> >> The docker images are not created and deployed to a docker registry as >> part of the build, it will just create a folder where you can run the >> docker build from. None of the maven docker plugins I looked at seemed >> to really work properly, so its still a manual process to do the build >> (and push to a registry). Its something which needs to be improved. > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150420/e34686e2/attachment.html From vnguyen at redhat.com Mon Apr 20 09:57:51 2015 From: vnguyen at redhat.com (Viet Nguyen) Date: Mon, 20 Apr 2015 09:57:51 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <55318CBC.8010105@redhat.com> References: <55318CBC.8010105@redhat.com> Message-ID: <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> Matt, I was the last person who worked on that Docker build for Hawkular-metrics. Yes, it is obsolete and can be removed. FYI there's an automated Docker build of Kettle in DockerHub and a continuous deployment to a Kubernetes cluster Docker Hub: https://registry.hub.docker.com/u/hawkular/hawkular/ CI Pipeline: https://docs.google.com/presentation/d/14zqOHOvXAB4uN7W_fWOub3ZD6txiS_Vp9cjUNIlj2yQ/edit?usp=sharing Viet ----- Original Message ----- From: "Matt Wringe" To: hawkular-dev at lists.jboss.org Sent: Friday, April 17, 2015 3:44:12 PM Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers I have a new subproject in Hawkular Metrics which sets up creating components for Openshift/Fabric8 (https://github.com/hawkular/hawkular-metrics/pull/200). There are 3 main parts Cassandra: creates a custom seed provider to support ReplicationControllers in Kubernetes, creates a folder/zip archive which can be used to generate a Docker image. It may make sense to move the Cassandra parts out to a separate project. Hawkular Metrics: creates a folder/zip archive which can be used to generate a Docker image Kubernetes: pulls everything together into a single kubernetes application. Can be used to deploy an application zip into fabric8 (via drag and drop in the web console or via the maven plugin) or deploy all the components into Openshift via the kubernetes.json configuration file. The docker images are not created and deployed to a docker registry as part of the build, it will just create a folder where you can run the docker build from. None of the maven docker plugins I looked at seemed to really work properly, so its still a manual process to do the build (and push to a registry). Its something which needs to be improved. The Cassandra service currently only supports adding new nodes to a cluster and not removing them via the ReplicationController. This is due to the replication factor being set to be 1 by default (which means when a node is removed, so is the data it contained). I believe the docker subproject of hawkular metrics is obsolete and can be removed (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but someone please correct me if I am wrong. It's scripts are referring to the console which no longer exists as part of the project. - Matt _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From mazz at redhat.com Mon Apr 20 15:43:26 2015 From: mazz at redhat.com (John Mazzitelli) Date: Mon, 20 Apr 2015 15:43:26 -0400 (EDT) Subject: [Hawkular-dev] bus rest client In-Reply-To: <1250726378.3042637.1429558935825.JavaMail.zimbra@redhat.com> Message-ID: <674211617.3042787.1429559006136.JavaMail.zimbra@redhat.com> Did I mention this yet? There is a REST client for sending bus messages. Single jar, only two dependencies (jboss logging, apache HTTP client). To send a bus message (in this case, to the alerts topic): new RestClient(new URL("http://localhost:8080")).postTopicMessage("HawkularAlertData", yourJsonPayload, null); That "null" can be a map of headers if you need to send headers. The client is here: https://github.com/hawkular/hawkular-bus/tree/master/hawkular-bus-rest-client and the maven artifact is org.hawkular.bus:hawkular-bus-rest-client - its just a jar. From mazz at redhat.com Mon Apr 20 15:56:32 2015 From: mazz at redhat.com (John Mazzitelli) Date: Mon, 20 Apr 2015 15:56:32 -0400 (EDT) Subject: [Hawkular-dev] inventory hierarchy In-Reply-To: <1132954904.3031863.1429557035972.JavaMail.zimbra@redhat.com> Message-ID: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> Well, after staring at things today, I am coming to the conclusion I probably should re-arrange my agent configuration before I get too far ahead. Need to know what people think. The current hawkular monitor agent defines things like this - you define your metric definitions and then you define where your servers are (host/port) and what metrics to collect from those servers. A very small agent config would be something like:

But after looking at the kinds of metrics we want to collect, I think we need to introduce some intermediate data - specifically, resource hierarchy. For example, if you look at Kettle (which has a pretty interesting and non-trivial DMR hierarchy) there are things like this: Alerts EAR (/deployment=hawkular-alerts-ear-1.0.ear) | \-- Alerts REST WAR (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest.war/) | \-- Undertow Subsystem (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest.war/subsystem=undertow/) | \-- METRIC: active-sessions \-- METRIC: sessions-created | \-- Servlet (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest.war/subsystem=undertow/servlet=org.hawkular.alerts.rest.HawkularAlertsApp/) | \-- METRIC: max-request-time \-- METRIC: min-request-time \-- METRIC: request-count There's also EJB3 subdeployments under the EAR which have singleton-beans that have execution-time and other invocation metrics. So, clearly, I think there has to be something like managed entities in between those endpoint servers and metrics in the agent configuration. Essentially, we need to define resources in the config. I was thinking of renaming "managed-resources" to "managed-servers" and then have "managed-resources" in between servers and metrics:

One difference between this and RHQ is I don't define those low-low level resources as separate resources (in RHQ I would have to define the undertow subsystem as a child under the WAR and then a servlet under the undertow resource). There is only the EAR and the WAR resource with the low level metrics from undertow and the servlet being "assigned" or linked to the WAR resouce. So the Hawkular inventory hierachy would be this (compare with the Wildfly hierarchy): Alerts EAR | \-- Alerts WAR | \-- METRICS: Active Sessions (coming from undertow) \-- METRICS: Servlet Requests (coming from the servlet inside undertow) From tsegismo at redhat.com Tue Apr 21 03:37:19 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 21 Apr 2015 09:37:19 +0200 Subject: [Hawkular-dev] inventory hierarchy In-Reply-To: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> References: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> Message-ID: <5535FE2F.3010205@redhat.com> Le 20/04/2015 21:56, John Mazzitelli a ?crit : > Well, after staring at things today, I am coming to the conclusion I probably should re-arrange my agent configuration before I get too far ahead. Need to know what people think. > > The current hawkular monitor agent defines things like this - you define your metric definitions and then you define where your servers are (host/port) and what metrics to collect from those servers. A very small agent config would be something like: > > > > > > > > > But after looking at the kinds of metrics we want to collect, I think we need to introduce some intermediate data - specifically, resource hierarchy. > > For example, if you look at Kettle (which has a pretty interesting and non-trivial DMR hierarchy) there are things like this: > > Alerts EAR (/deployment=hawkular-alerts-ear-1.0.ear) > | > \-- Alerts REST WAR (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest.war/) > | > \-- Undertow Subsystem (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest.war/subsystem=undertow/) > | > \-- METRIC: active-sessions > \-- METRIC: sessions-created > | > \-- Servlet (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest.war/subsystem=undertow/servlet=org.hawkular.alerts.rest.HawkularAlertsApp/) > | > \-- METRIC: max-request-time > \-- METRIC: min-request-time > \-- METRIC: request-count > > There's also EJB3 subdeployments under the EAR which have singleton-beans that have execution-time and other invocation metrics. > > So, clearly, I think there has to be something like managed entities in between those endpoint servers and metrics in the agent configuration. Essentially, we need to define resources in the config. > > I was thinking of renaming "managed-resources" to "managed-servers" and then have "managed-resources" in between servers and metrics: > > > > > > > > > path="/deployment=hawkular-alerts-ear-1.0.ear" > parent="Alerts EAR" > path="/subdeployment=hawkular-alerts-rest.war" > metricSets="my-metrics"/> > > > > > > > One difference between this and RHQ is I don't define those low-low level resources as separate resources (in RHQ I would have to define the undertow subsystem as a child under the WAR and then a servlet under the undertow resource). There is only the EAR and the WAR resource with the low level metrics from undertow and the servlet being "assigned" or linked to the WAR resouce. > > So the Hawkular inventory hierachy would be this (compare with the Wildfly hierarchy): > > Alerts EAR > | > \-- Alerts WAR > | > \-- METRICS: Active Sessions (coming from undertow) > \-- METRICS: Servlet Requests (coming from the servlet inside undertow) It surely makes sense when the sub-resource is unique, but in this case precisely, there could be more than one servlet, correct? We may want to distinguish the metrics for each end point. From tsegismo at redhat.com Tue Apr 21 04:06:08 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 21 Apr 2015 10:06:08 +0200 Subject: [Hawkular-dev] Hawkular implementation of the Vert.x Metrics SPI Message-ID: <553604F0.4000201@redhat.com> Hi everyone, I'm writing to let you know that I've started an Hawkular [1] implementation of the Vert.x Metrics SPI, called vertx-monitor [2]. It's quite limited at this point [3], and subject to a lot of future changes, but hopefully it will become a great companion of Vert.x applications. Regards, Thomas [1] http://www.hawkular.org/ [2] https://github.com/tsegismont/vertx-monitor [3] https://github.com/tsegismont/vertx-monitor#limitations From hrupp at redhat.com Tue Apr 21 04:10:01 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 21 Apr 2015 10:10:01 +0200 Subject: [Hawkular-dev] inventory hierarchy In-Reply-To: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> References: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> Message-ID: <35E8E4A9-7F38-4608-9BD4-0CD091E1173E@redhat.com> On 20 Apr 2015, at 21:56, John Mazzitelli wrote: > But after looking at the kinds of metrics we want to collect, I think > we need to introduce some intermediate data - specifically, resource > hierarchy. Yes. Didn't we talk about that on the meeting with Lukas last week? > resource="/subsystem=undertow/servlet=org.hawkular.alerts.rest.HawkularAlertsApp" > attribute="request-count" /> >

We need wildcard support here resource="/subsystem=undertow/servlet=*" as we could have more than one servlet per app > > > path="/deployment=hawkular-alerts-ear-1.0.ear" > parent="Alerts EAR" > path="/subdeployment=hawkular-alerts-rest.war" Same here as the outer EAR may have many WARs inside. > So the Hawkular inventory hierachy would be this (compare with the > Wildfly hierarchy): > > Alerts EAR > | > \-- Alerts WAR > | > \-- METRICS: Active Sessions (coming from undertow) > \-- METRICS: Servlet Requests (coming from the servlet inside > undertow) Makes certainly sense to gather them in one place if they logically belong together as you show them. This is certainly an issue in RHQ right now. From jpkroehling at redhat.com Tue Apr 21 05:54:37 2015 From: jpkroehling at redhat.com (=?UTF-8?B?SnVyYWNpIFBhaXjDo28gS3LDtmhsaW5n?=) Date: Tue, 21 Apr 2015 11:54:37 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts Message-ID: <55361E5D.7090302@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 All, Hawkular Accounts have reached a point where it's possible for other components to integrate with it. As such, I would like to ask if there's a project that would volunteer for it, to be the first with this integration, so that I can fix possible integration bugs that might come from this experience. I know that you all are busy with the MVP, but if you think you can afford to spend, say, a day on this, I'll be ready to guide you. Those are the items that I would like to see being used during this first integration: - - Securing the backend with Keycloak - - Consuming the tenant information (Persona) - - Protecting resources based on the Persona's role Needless to say, I'll support you in achieving this and will promptly fix the issues that may arise. If you need more information before you volunteer, you can take a look at the readme file from the accounts module: https://github.com/hawkular/hawkular-accounts - - Juca. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVNh5dAAoJECKM1e+fkPrXiL8H/jGBE9/SDTD19GNPcdvsueKm qBKkR2bKX0FP+hKJjzDFgnEIq3HYV9UXCPyPC2pODUXnWIDUqHMoRUz+eF2rCtLT Q93tfuC3gE1Kn6fY1ZPs8l3fL8RZXjZWLx4q9Qnxey3roIJtDUFWKX777O0eBb+q EkO4zWbl42qwZCv0pWMdlUusWHDm9A1X6+G70cjGjl8nfRChNBm9oU2LpBWDo0jq Fkw9dToobLFt5O3MmjSBZo4xYF65WXIDsGdrfXJilERvtJJgWsiq8/sgEWMaNewv 0HitfhhkUbbW8XTYAqwDtYB2v4EllIge2klekPqu3vrBulOHLnYXVibwRWE0zdY= =6ath -----END PGP SIGNATURE----- From lkrejci at redhat.com Tue Apr 21 07:31:58 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 21 Apr 2015 13:31:58 +0200 Subject: [Hawkular-dev] inventory hierarchy In-Reply-To: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> References: <1789246184.3044985.1429559792419.JavaMail.zimbra@redhat.com> Message-ID: <2050335.9yNP6m5b09@localhost.localdomain> IMHO, technically, you don't need the hierarchy to report the metrics in your examples. Now if you want to report the resources, too, then yes, hierarchy is needed. But there is a distinction between metrics and resources and I think we should keep them as separate concepts - metrics maybe logically included in resources but they should exist as standalone entities not inherently tied to resources. If for nothing else then for the fact that we somehow have to report the metrics to hawkular-metrics, where there is no concept of a resource (and I think there never should be). Now to the way how you would define a resource hierarchy. IMHO the way we've done it in RHQ is more or less right. You define the types of the resources, the possible "hierarchizations" (in RHQ specified either implicitly by the nesting of the definitions or explicitly by "runs-inside") and then you discover the resources as "instances" of those types. Also if you rather think of types than the concrete instances, the problem of the "wildcards" that Heiko is highlighting finds a comfortable place in the design. With that in mind, I think something along the lines of the following could work: - metric sets are basically equivalent to hawkular inventory resource types:

${path.1}-${path.2}

your-metrics, his-metrics - managed resource sets are I think no longer needed - managed servers reference resource types they want managed on the servers. The resource type definition above basically combines the resource type def as it existed in RHQ (of sorts) with a simple discovery by querying the management model and matching it with the regular expr. On Monday, April 20, 2015 15:56:32 John Mazzitelli wrote: > Well, after staring at things today, I am coming to the conclusion I > probably should re-arrange my agent configuration before I get too far > ahead. Need to know what people think. > > The current hawkular monitor agent defines things like this - you define > your metric definitions and then you define where your servers are > (host/port) and what metrics to collect from those servers. A very small > agent config would be something like: > > > resource="/core-service=platform-mbean/type=threading" > attribute="thread-count" > > > metricSets="my-metrics" /> > > But after looking at the kinds of metrics we want to collect, I think we > need to introduce some intermediate data - specifically, resource > hierarchy. > > For example, if you look at Kettle (which has a pretty interesting and > non-trivial DMR hierarchy) there are things like this: > > Alerts EAR (/deployment=hawkular-alerts-ear-1.0.ear) > > \-- Alerts REST WAR > (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest > .war/) > > \-- Undertow Subsystem > (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest > .war/subsystem=undertow/) > > \-- METRIC: active-sessions > \-- METRIC: sessions-created > > \-- Servlet > (/deployment=hawkular-alerts-ear-1.0.ear/subdeployment=hawkular-alerts-rest > .war/subsystem=undertow/servlet=org.hawkular.alerts.rest.HawkularAlertsApp/) > > \-- METRIC: max-request-time > \-- METRIC: min-request-time > \-- METRIC: request-count > > There's also EJB3 subdeployments under the EAR which have singleton-beans > that have execution-time and other invocation metrics. > > So, clearly, I think there has to be something like managed entities in > between those endpoint servers and metrics in the agent configuration. > Essentially, we need to define resources in the config. > > I was thinking of renaming "managed-resources" to "managed-servers" and then > have "managed-resources" in between servers and metrics: > > > >

resource="/subsystem=undertow/servlet=org.hawkular.alerts.rest.HawkularAler > tsApp" attribute="request-count" />

> > > path="/deployment=hawkular-alerts-ear-1.0.ear" > parent="Alerts EAR" > path="/subdeployment=hawkular-alerts-rest.war" > metricSets="my-metrics"/> > > > > resourceSets="Alerts Application" /> > > One difference between this and RHQ is I don't define those low-low level > resources as separate resources (in RHQ I would have to define the undertow > subsystem as a child under the WAR and then a servlet under the undertow > resource). There is only the EAR and the WAR resource with the low level > metrics from undertow and the servlet being "assigned" or linked to the WAR > resouce. > > So the Hawkular inventory hierachy would be this (compare with the Wildfly > hierarchy): > > Alerts EAR > > \-- Alerts WAR > > \-- METRICS: Active Sessions (coming from undertow) > \-- METRICS: Servlet Requests (coming from the servlet inside > undertow) _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From lkrejci at redhat.com Tue Apr 21 08:07:46 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 21 Apr 2015 14:07:46 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <55361E5D.7090302@redhat.com> References: <55361E5D.7090302@redhat.com> Message-ID: <4269691.JhrSYD0rPu@localhost.localdomain> In inventory, we have a simple concept of tenants, environments and resources, where tenants are unique and share no information and environments model the usual split of the infrastructure into dev, testing, production, et al. How would you change that model to play nicely with the accounts concepts of users and organizations? IMHO replacing inventory's tenants with accounts' organizations might be a way forward but I am not sure. If it was the case, how would you envision syncing the organization info between accounts and inventory? Thanks, Lukas On Tuesday, April 21, 2015 11:54:37 Juraci Paix?o Kr?hling wrote: > All, > > Hawkular Accounts have reached a point where it's possible for other > components to integrate with it. As such, I would like to ask if > there's a project that would volunteer for it, to be the first with > this integration, so that I can fix possible integration bugs that > might come from this experience. > > I know that you all are busy with the MVP, but if you think you can > afford to spend, say, a day on this, I'll be ready to guide you. > > Those are the items that I would like to see being used during this > first integration: > > - Securing the backend with Keycloak > - Consuming the tenant information (Persona) > - Protecting resources based on the Persona's role > > Needless to say, I'll support you in achieving this and will promptly > fix the issues that may arise. > > If you need more information before you volunteer, you can take a look > at the readme file from the accounts module: > > https://github.com/hawkular/hawkular-accounts > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Tue Apr 21 08:27:23 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 21 Apr 2015 14:27:23 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <4269691.JhrSYD0rPu@localhost.localdomain> References: <55361E5D.7090302@redhat.com> <4269691.JhrSYD0rPu@localhost.localdomain> Message-ID: <0B5672B5-CFE9-47EF-B5A9-4F736458CBC0@redhat.com> On 21 Apr 2015, at 14:07, Lukas Krejci wrote: > In inventory, we have a simple concept of tenants, environments and > resources, > where tenants are unique and share no information and environments > model the > usual split of the infrastructure into dev, testing, production, et > al. > > How would you change that model to play nicely with the accounts > concepts of > users and organizations? I think users and organizations can both be tenants. Either the org is the tenant, then the users "inherit" from the org, i.e. can see all data from the org (*) or the users are the tenants and then they only see their own stuff. *) Additional access control may apply. From jpkroehling at redhat.com Tue Apr 21 08:29:28 2015 From: jpkroehling at redhat.com (=?UTF-8?B?SnVyYWNpIFBhaXjDo28gS3LDtmhsaW5n?=) Date: Tue, 21 Apr 2015 14:29:28 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <4269691.JhrSYD0rPu@localhost.localdomain> References: <55361E5D.7090302@redhat.com> <4269691.JhrSYD0rPu@localhost.localdomain> Message-ID: <553642A8.7030803@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/21/2015 02:07 PM, Lukas Krejci wrote: > In inventory, we have a simple concept of tenants, environments and > resources, where tenants are unique and share no information and > environments model the usual split of the infrastructure into dev, > testing, production, et al. > > How would you change that model to play nicely with the accounts > concepts of users and organizations? The "persona" model was made with *our* concept of tenant in mind. So, a tenant could be either a real person or an organization, depending on which persona is currently selected on the UI (or which header was sent via curl). So, you could safely take a persona as the tenant. If you also use the "Resource" concept in Account's side, you could also have a way to display resources from a sub-tenant to the parent (ie: from "Acme Financial" to "Acme, Inc"). > IMHO replacing inventory's tenants with accounts' organizations > might be a way forward but I am not sure. If it was the case, how > would you envision syncing the organization info between accounts > and inventory? I think the inventory needs only to know what's the current tenant, and leave the permissions part to the accounts API. If there's something you might need to accomplish a particular task, we could add this as a feature to the accounts part. - - Juca. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVNkKoAAoJECKM1e+fkPrXtOcH/0LQiJv13ImuDSQF1+jkNbll P5t0zgxkCYs/2jsHD5KVGCc6lsq0HKfDjNv6/oqurNZyrUP0u5ATZxt2HDYxtrjm IYugq7Kg1P64eLwjXaVD7/WFaOaR8np4EXhd6OtKjfz8DyD1GSl8fTRQPGelK+ff gi+PEFbhMEkHkzG3TL2zhciBvueeS+TT+LH/bOnHX49gC826ABjK5LDN+YROPE/a ezytelXVkFyzyKBZjA31baHCAyr3G1qgA7Rq5Dg24VHUYgIRdg2O/ado2VhRTmPL Hw5hk+5BpS5m1tkEw9sFQPiCXKvPD0okJZcd6KzATyu0pJYsMNOxXfc42fOkOZI= =85NR -----END PGP SIGNATURE----- From jpkroehling at redhat.com Tue Apr 21 08:55:25 2015 From: jpkroehling at redhat.com (=?windows-1252?Q?Juraci_Paix=E3o_Kr=F6hling?=) Date: Tue, 21 Apr 2015 14:55:25 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <0B5672B5-CFE9-47EF-B5A9-4F736458CBC0@redhat.com> References: <55361E5D.7090302@redhat.com> <4269691.JhrSYD0rPu@localhost.localdomain> <0B5672B5-CFE9-47EF-B5A9-4F736458CBC0@redhat.com> Message-ID: <553648BD.1020805@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/21/2015 02:27 PM, Heiko W.Rupp wrote: > I think users and organizations can both be tenants. > > Either the org is the tenant, then the users "inherit" from the > org, i.e. can see all data from the org (*) or the users are the > tenants and then they only see their own stuff. > > *) Additional access control may apply. Exactly, that's how I tried to model it. The "additional access control" part is also there, with the relationship between Resources, Operations and Roles, similar to the PicketLink Permission API (ie: persona with role "x" can perform the operation "y" on resource "z"). - - Juca. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVNki9AAoJECKM1e+fkPrXeDIH/00wQCAPhBWVDljCZkS5qlX1 N1M7ULPaZJVzQK6WBLzFpETucUBTBcaXlVRhFk8iSYUiUdPTzXiwWjCyWZH5P2T/ M1Jum8xVgqSnRIE8V2Z17m7BR9LxS+We2gJXJq6jtN4/1c+wND55IbqLx1B1aE2e iVnACXPy2hiUveNHFhF5q6i/srWCb1D/IhlSbHOy7/m/Nr72lMvLdIrXjUrMSFBh QYR7CQJVuKlkhA75WLWIercFG6Pruq0yIXZ/Rah+sQ28Kz4JeTJymyLfHn3IcdXg ZwQU3FpeLyfKcw2SMcDGP1GpVJ/WrkD9Y3Y5NqjhTruWapRISkaj40X4HzpN0zQ= =HilD -----END PGP SIGNATURE----- From lkrejci at redhat.com Tue Apr 21 09:29:35 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 21 Apr 2015 09:29:35 -0400 (EDT) Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <553642A8.7030803@redhat.com> References: <55361E5D.7090302@redhat.com> <4269691.JhrSYD0rPu@localhost.localdomain> <553642A8.7030803@redhat.com> Message-ID: <142543204.4914633.1429622975024.JavaMail.zimbra@redhat.com> Sorry for a long and slightly confused email - I am probably too ingrained with the current design to understand how this should work. ----- Original Message ----- > From: "Juraci Paix?o Kr?hling" > To: "Lukas Krejci" , "Discussions around Hawkular development" > Sent: Tuesday, 21 April, 2015 2:29:28 PM > Subject: Re: [Hawkular-dev] Looking for a first project to integrate accounts > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 04/21/2015 02:07 PM, Lukas Krejci wrote: > > In inventory, we have a simple concept of tenants, environments and > > resources, where tenants are unique and share no information and > > environments model the usual split of the infrastructure into dev, > > testing, production, et al. > > > > How would you change that model to play nicely with the accounts > > concepts of users and organizations? > > The "persona" model was made with *our* concept of tenant in mind. So, > a tenant could be either a real person or an organization, depending > on which persona is currently selected on the UI (or which header was > sent via curl). So, you could safely take a persona as the tenant. > Is sharing of information possible between personas? It is not possible in inventory between tenants currently so this is the part where I struggle a bit. You can imagine the inventory's tenants as "access points" to disjoint subgraphs in the inventory. You only can access any data if you know your tenant. Also, and possibly more importantly, no entity in inventory is required to have a "globally" unique identifier. The only thing unique is the path to the entity, starting at the tenant, then down the environment, etc. So if a user is a tenant they will not be able to even reference data from other tenants. I guess the my main struggle is that the inventory is modelled after a "physical" layout of the monitoring data - data belongs to some tenant and can be further classified/narrowed down by the environment, etc. In accounts, users/orgs/personas are very fluid concepts that can inherit from each other, etc. while the classification of the data they access is not that fluid. Or am I completely missing the point here? (given my confusion right now, that's more than likely :) ). > If you also use the "Resource" concept in Account's side, you could > also have a way to display resources from a sub-tenant to the parent > (ie: from "Acme Financial" to "Acme, Inc"). > > > IMHO replacing inventory's tenants with accounts' organizations > > might be a way forward but I am not sure. If it was the case, how > > would you envision syncing the organization info between accounts > > and inventory? > > I think the inventory needs only to know what's the current tenant, > and leave the permissions part to the accounts API. If there's > something you might need to accomplish a particular task, we could add > this as a feature to the accounts part. > Inventory's tenant is a crucial part to be able to address any entity. So "knowing a current tenant" to check for permissions is quite ok, but I am not sure I can "just know the current tenant" to be able to address the data. I.e. 2 users are supposed to access data from agent "A" in environment "staging" for "Acme, Inc.", while a third user is supposed to access data from agent "A" in environment "staging" for "Redhat, Inc." I suppose that our model should allow for more organizations (in a SaaS sense, not accounts sense) to have the environment called "staging" and that their agents should be independent. Remember that inventory doesn't have a unique ID of the "staging" environments for the particular organizations. Currently this is resolved by having a set of tenants that are required to have unique names and that you can use to start composing your path to the agents, i.e. "Acme/staging/A" or "Redhat/staging/A". If persona is a tenant, how would that work? > - - Juca. > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQEcBAEBCAAGBQJVNkKoAAoJECKM1e+fkPrXtOcH/0LQiJv13ImuDSQF1+jkNbll > P5t0zgxkCYs/2jsHD5KVGCc6lsq0HKfDjNv6/oqurNZyrUP0u5ATZxt2HDYxtrjm > IYugq7Kg1P64eLwjXaVD7/WFaOaR8np4EXhd6OtKjfz8DyD1GSl8fTRQPGelK+ff > gi+PEFbhMEkHkzG3TL2zhciBvueeS+TT+LH/bOnHX49gC826ABjK5LDN+YROPE/a > ezytelXVkFyzyKBZjA31baHCAyr3G1qgA7Rq5Dg24VHUYgIRdg2O/ado2VhRTmPL > Hw5hk+5BpS5m1tkEw9sFQPiCXKvPD0okJZcd6KzATyu0pJYsMNOxXfc42fOkOZI= > =85NR > -----END PGP SIGNATURE----- > From jpkroehling at redhat.com Tue Apr 21 10:04:11 2015 From: jpkroehling at redhat.com (=?UTF-8?B?SnVyYWNpIFBhaXjDo28gS3LDtmhsaW5n?=) Date: Tue, 21 Apr 2015 16:04:11 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <142543204.4914633.1429622975024.JavaMail.zimbra@redhat.com> References: <55361E5D.7090302@redhat.com> <4269691.JhrSYD0rPu@localhost.localdomain> <553642A8.7030803@redhat.com> <142543204.4914633.1429622975024.JavaMail.zimbra@redhat.com> Message-ID: <553658DB.9040105@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/21/2015 03:29 PM, Lukas Krejci wrote: >> Is sharing of information possible between personas? It is not >> possible in inventory between tenants currently so this is the >> part where I struggle a bit. - From accounts perspective, yes. See below. >> You can imagine the inventory's tenants as "access points" to >> disjoint subgraphs in the inventory. You only can access any data >> if you know your tenant. Also, and possibly more importantly, no >> entity in inventory is required to have a "globally" unique >> identifier. The only thing unique is the path to the entity, >> starting at the tenant, then down the environment, etc. > >> So if a user is a tenant they will not be able to even reference >> data from other tenants. There's a scenario that was discussed on demos and, I believe, on previous threads. Those discussions impacted the modeling of the tenants concept on the accounts side and one of the use cases is: - - Acme, Inc is an organization - - "Marketing" is an organization whose parent is "Acme, Inc" - - "Finance" is an organization whose parent is "Acme, Inc" - - "Finance" cannot see resources from "Marketing" or vice-versa - - "Operations" is an organization whose parent is "Acme, Inc" - - "Operations" should be able to monitor its own resources plus resources from finance and operations, all in one (or more) dashboards. >> I guess the my main struggle is that the inventory is modelled >> after a "physical" layout of the monitoring data - data belongs >> to some tenant and can be further classified/narrowed down by the >> environment, etc. As far as I can see, it can be kept this way. The data can be stored in any way you prefer, while the access control part has different storage and different ways of resolving who has access to what. Note that there's a "resource" concept on Accounts which stores the owner information, and that's independent from the one in, say, inventory. >> In accounts, users/orgs/personas are very fluid concepts that can >> inherit from each other, etc. while the classification of the >> data they access is not that fluid. Note that "user/orgs" abstracts to "persona". So, inventory would only care about "persona" and would ask accounts if the current persona has access to this resource. It should not matter to inventory if the current persona (user or org) is different than the owner of the resource, although inventory can keep track of this information for other reasons, if needed. >> Inventory's tenant is a crucial part to be able to address any >> entity. You mean, the tenant being part of the identifier? If so, that's irrelevant for accounts. As long as you can provide me with a stable identifier for the resource, I'm able to tell if you if the current persona (user/org) has access to the resource. >> So "knowing a current tenant" to check for permissions is quite >> ok, but I am not sure I can "just know the current tenant" to be >> able to address the data. I.e. 2 users are supposed to access >> data from agent "A" in environment "staging" for "Acme, Inc.", >> while a third user is supposed to access data from agent "A" in >> environment "staging" for "Redhat, Inc." Forgetting a bit about the "staging" part, this is how I see it: - - User "jdoe" creates the organization "Acme, Inc", he is therefore "Super User" of it. - - User "jsmith" is added to "Acme, Inc" as "Super User" - - Agent "a" is a resource that belongs to "Acme, Inc" In that situation, user "jdoe" makes a request to the backend, which Accounts translates to as being the persona "Acme, Inc". So, inventory knows that it's related to the persona/tenant "Acme, Inc". If you absolutely need to know who is the actual user for the request, you can request it to the accounts API. In the usual case, however, knowing the current "persona" should be enough. >> I suppose that our model should allow for more organizations (in >> a SaaS sense, not accounts sense) to have the environment called >> "staging" and that their agents should be independent. - From the accounts perspective, this is achievable. >> Remember that inventory doesn't have a unique ID of the >> "staging" environments for the particular organizations. >> Currently this is resolved by having a set of tenants that are >> required to have unique names and that you can use to start >> composing your path to the agents, i.e. "Acme/staging/A" or >> "Redhat/staging/A". As far as I see it, "Acme" is an organization, "staging" is an organization owned by "Acme" and "A" is a resource in "staging". From Account's perspective, the owner is "staging" (from the Acme organization), so, users who are "Administrators" on this organization have access to this resource. But do you mean that you don't have a unique ID for a specific resource for a specific tenant? This is something I would need. Note that it's not needed to be an UUID, I just need a value that you are able to consistently provide to the accounts API when checking for the permission. On the above example, it could very well be "Acme/staging/A" . - - Juca. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVNljbAAoJECKM1e+fkPrXQpEH/AjYbHyLFfs7IBrpEwwAfZwZ IFzgX5cdXktO9loEvtxnH7nkSag+iWCIH2EJWVg2ql8YXr81UdyGlDSUl2NP+EtL PFy3l8b7FbYp6SPby7CvyXHtmhsRe1SEUTAOcU5dn3ufPbwkRliPrEhyKL3hqOIH Fz4kENeRugJyn8X48GCRP5EI4KSXUzN9F2zRFNP2Ub37QFN9njxt5e0E3J+ZDUeD HC0g8o2nLkai37gZJX9eWm5NPJT83w7ynNrZKDTS3rkq3of+gfTlEv2yqWc/brgp HFII4jp1VDQeRX+B/AUugzn879sLF+psJaRMq4Mls1NntznDVpDoO5bNOHr0T5c= =k82w -----END PGP SIGNATURE----- From sebastian.laskawiec at gmail.com Wed Apr 22 04:11:48 2015 From: sebastian.laskawiec at gmail.com (=?UTF-8?Q?Sebastian_=C5=81askawiec?=) Date: Wed, 22 Apr 2015 08:11:48 +0000 Subject: [Hawkular-dev] Looking for the opportunities code Message-ID: Hey! My name is Sebastian and I'm a remote productization engineer for JBoss Data Grid from Poland. I also contribute to the project New Castle (this is a replacement for Brew) and I'm really interested in Hawkular as our monitoring approach in the future. I would love to help you with the coding in my free slots. Personally, I'm very interested in cloud-related technologies, so I think I would fit in Metrics Storage or Analytics. Could you please give me a hint where I could start and how I could help? I have also a small question regarding to the project's assumptions - do we plan to analyze logs with Hawkular? This feature together with alerting might create a lot of new use cases. Thanks! Sebastian -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150422/7d4d14e3/attachment-0001.html From theute at redhat.com Wed Apr 22 04:56:20 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 22 Apr 2015 10:56:20 +0200 Subject: [Hawkular-dev] Looking for the opportunities code In-Reply-To: References: Message-ID: <55376234.40000@redhat.com> On 04/22/2015 10:11 AM, Sebastian ?askawiec wrote: > Hey! > > My name is Sebastian and I'm a remote productization engineer for JBoss > Data Grid from Poland. I also contribute to the project New Castle (this > is a replacement for Brew) and I'm really interested in Hawkular as our > monitoring approach in the future. Great, thanks for approaching us ! > I would love to help you with the coding in my free slots. Personally, > I'm very interested in cloud-related technologies, so I think I would > fit in Metrics Storage or Analytics. Could you please give me a hint > where I could start and how I could help? What in specific do you enjoy or are you good at ? We have few JIRAs in https://issues.jboss.org/browse/HWKMETRICS and https://issues.jboss.org/browse/HAWKULAR Feel free to join #hawkular on Freenode IRC if you wish to discuss > I have also a small question regarding to the project's assumptions - do > we plan to analyze logs with Hawkular? This feature together with > alerting might create a lot of new use cases. Yes, that will come later, likely a solution such as http://fabric8.io/guide/logging.html (or that works well with it) Thomas > > Thanks! > Sebastian > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From theute at redhat.com Wed Apr 22 05:29:00 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 22 Apr 2015 11:29:00 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <55361E5D.7090302@redhat.com> References: <55361E5D.7090302@redhat.com> Message-ID: <553769DC.3090305@redhat.com> On 04/21/2015 11:54 AM, Juraci Paix?o Kr?hling wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > All, > > Hawkular Accounts have reached a point where it's possible for other > components to integrate with it. As such, I would like to ask if > there's a project that would volunteer for it, to be the first with > this integration, so that I can fix possible integration bugs that > might come from this experience. > > I know that you all are busy with the MVP, but if you think you can > afford to spend, say, a day on this, I'll be ready to guide you. Having a secured solution is part of an MVP so it's not a distraction and should be a priority. Multi-tenancy is also needed as this is assumed from the UI Tomas > > Those are the items that I would like to see being used during this > first integration: > > - - Securing the backend with Keycloak > - - Consuming the tenant information (Persona) > - - Protecting resources based on the Persona's role > > Needless to say, I'll support you in achieving this and will promptly > fix the issues that may arise. > > If you need more information before you volunteer, you can take a look > at the readme file from the accounts module: > > https://github.com/hawkular/hawkular-accounts > > - - Juca. > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQEcBAEBCAAGBQJVNh5dAAoJECKM1e+fkPrXiL8H/jGBE9/SDTD19GNPcdvsueKm > qBKkR2bKX0FP+hKJjzDFgnEIq3HYV9UXCPyPC2pODUXnWIDUqHMoRUz+eF2rCtLT > Q93tfuC3gE1Kn6fY1ZPs8l3fL8RZXjZWLx4q9Qnxey3roIJtDUFWKX777O0eBb+q > EkO4zWbl42qwZCv0pWMdlUusWHDm9A1X6+G70cjGjl8nfRChNBm9oU2LpBWDo0jq > Fkw9dToobLFt5O3MmjSBZo4xYF65WXIDsGdrfXJilERvtJJgWsiq8/sgEWMaNewv > 0HitfhhkUbbW8XTYAqwDtYB2v4EllIge2klekPqu3vrBulOHLnYXVibwRWE0zdY= > =6ath > -----END PGP SIGNATURE----- > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From gbrown at redhat.com Wed Apr 22 06:47:29 2015 From: gbrown at redhat.com (Gary Brown) Date: Wed, 22 Apr 2015 06:47:29 -0400 (EDT) Subject: [Hawkular-dev] What is Business Transaction Management (prep for F2F) In-Reply-To: <1831068056.4645641.1429698309904.JavaMail.zimbra@redhat.com> Message-ID: <1443849909.4662889.1429699649666.JavaMail.zimbra@redhat.com> Hi all As suggested recently by others, we want to focus the F2F sessions on discussions rather than presentations. So in that spirit, I thought it would be good to get the "What is Business Transaction Management" discussion out of the way as a ML thread, so that the F2F session can discuss what and how to build BTM in a Hawkular context. Taking some excerpts from Wikipedia: "Business transaction management (BTM), is the practice of managing information technology (IT) from a business transaction perspective. It provides a tool for tracking the flow of transactions across IT infrastructure, in addition to detection, alerting, and correction of unexpected changes in business or technical conditions. BTM provides visibility into the flow of transactions across infrastructure tiers, including a dynamic mapping of the application topology. Using BTM, application support teams are able to search for transactions based on message context and content ? for instance, time of arrival or message type ? providing a way to isolate causes for common issues such as application exceptions, stalled transactions, and lower-level issues such as incorrect data values. The ultimate goal of BTM is to improve service quality for users conducting business transactions while improving the effectiveness of the IT applications and infrastructure across which those transactions execute. The main benefit of BTM is its capacity to identify precisely where transactions are delayed within the IT infrastructure. BTM also aims to provide proactive problem prevention and the generation of business service intelligence for optimization of resource provisioning and virtualization." Some of the applications of BTM listed are: "BTM solutions capture all of the transaction instances in the production environment and as such can be used for monitoring as well as for analysis and planning. Some applications include: * Outage avoidance and problem isolation: Identification and isolation of tier-specific performance and availability issues. * Service level management: Monitoring of SLAs and alerting of threshold breaches both at the end-user and infrastructure tier level. * Infrastructure optimization: Modification of the configuration of data center infrastructure to maximize utilization and improve performance. * Capacity planning: Analysis of usage and performance trends in order to estimate future capacity requirements. * Change management: Analysis of the impact of change on transaction execution. * Cloud management: Track the end-to-end transaction flow across both cloud (private, hybrid, public) and dedicated (on-premises, off-premises) infrastructure." Obviously we need to focus our efforts on the monitoring/alerting aspects initially, and this is what I am expecting the F2F discussion will be focused on, but a couple of these areas may be of interest in the future. Any views on the above appreciated. Regards Gary From mwringe at redhat.com Wed Apr 22 09:18:19 2015 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 22 Apr 2015 09:18:19 -0400 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> References: <55318CBC.8010105@redhat.com> <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> Message-ID: <55379F9B.4010307@redhat.com> On 20/04/15 09:57 AM, Viet Nguyen wrote: > Matt, > I was the last person who worked on that Docker build for Hawkular-metrics. Yes, it is obsolete and can be removed. > > FYI there's an automated Docker build of Kettle in DockerHub and a continuous deployment to a Kubernetes cluster > > Docker Hub: > https://registry.hub.docker.com/u/hawkular/hawkular/ > > CI Pipeline: > https://docs.google.com/presentation/d/14zqOHOvXAB4uN7W_fWOub3ZD6txiS_Vp9cjUNIlj2yQ/edit?usp=sharing I went through and have the docker images being built using a docker maven plugin. So during the maven build, you can create the image and push it out to a registry. It works great on a local machine, one maven build and I have the docker images and kubernetes application zip created. But looking at how the current CI is done, it might not have been the best solution (it would require the credentials of the docker hub user to be added into the CI setup somewhere). It might make more sense to do it the way the current CI is setup with the kettle builds. Anyone have any preferences? > > > Viet > > > > > ----- Original Message ----- > From: "Matt Wringe" > To: hawkular-dev at lists.jboss.org > Sent: Friday, April 17, 2015 3:44:12 PM > Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers > > I have a new subproject in Hawkular Metrics which sets up creating > components for Openshift/Fabric8 > (https://github.com/hawkular/hawkular-metrics/pull/200). > > There are 3 main parts > > Cassandra: creates a custom seed provider to support > ReplicationControllers in Kubernetes, creates a folder/zip archive which > can be used to generate a Docker image. It may make sense to move the > Cassandra parts out to a separate project. > > Hawkular Metrics: creates a folder/zip archive which can be used to > generate a Docker image > > Kubernetes: pulls everything together into a single kubernetes > application. Can be used to deploy an application zip into fabric8 (via > drag and drop in the web console or via the maven plugin) or deploy all > the components into Openshift via the kubernetes.json configuration file. > > The docker images are not created and deployed to a docker registry as > part of the build, it will just create a folder where you can run the > docker build from. None of the maven docker plugins I looked at seemed > to really work properly, so its still a manual process to do the build > (and push to a registry). Its something which needs to be improved. > > The Cassandra service currently only supports adding new nodes to a > cluster and not removing them via the ReplicationController. This is due > to the replication factor being set to be 1 by default (which means when > a node is removed, so is the data it contained). > > I believe the docker subproject of hawkular metrics is obsolete and can > be removed > (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but > someone please correct me if I am wrong. It's scripts are referring to > the console which no longer exists as part of the project. > > - Matt > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From mwringe at redhat.com Wed Apr 22 10:16:15 2015 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 22 Apr 2015 10:16:15 -0400 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <55379F9B.4010307@redhat.com> References: <55318CBC.8010105@redhat.com> <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> <55379F9B.4010307@redhat.com> Message-ID: <5537AD2F.5040806@redhat.com> On 22/04/15 09:18 AM, Matt Wringe wrote: > On 20/04/15 09:57 AM, Viet Nguyen wrote: >> Matt, >> I was the last person who worked on that Docker build for Hawkular-metrics. Yes, it is obsolete and can be removed. >> >> FYI there's an automated Docker build of Kettle in DockerHub and a continuous deployment to a Kubernetes cluster >> >> Docker Hub: >> https://registry.hub.docker.com/u/hawkular/hawkular/ >> >> CI Pipeline: >> https://docs.google.com/presentation/d/14zqOHOvXAB4uN7W_fWOub3ZD6txiS_Vp9cjUNIlj2yQ/edit?usp=sharing > I went through and have the docker images being built using a docker > maven plugin. So during the maven build, you can create the image and > push it out to a registry. > > It works great on a local machine, one maven build and I have the docker > images and kubernetes application zip created. But looking at how the > current CI is done, it might not have been the best solution (it would > require the credentials of the docker hub user to be added into the CI > setup somewhere). > > It might make more sense to do it the way the current CI is setup with > the kettle builds. Anyone have any preferences? Either way, the CI build way is also easy to setup. You would just need to sets these up with the official accounts and add in the automated build hooks: https://registry.hub.docker.com/u/mwringe/docker-hawkular-cassandra/ https://github.com/mwringe/docker-hawkular-cassandra https://registry.hub.docker.com/u/mwringe/docker-hawkular-metrics/ https://github.com/mwringe/docker-hawkular-metrics > >> >> Viet >> >> >> >> >> ----- Original Message ----- >> From: "Matt Wringe" >> To: hawkular-dev at lists.jboss.org >> Sent: Friday, April 17, 2015 3:44:12 PM >> Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers >> >> I have a new subproject in Hawkular Metrics which sets up creating >> components for Openshift/Fabric8 >> (https://github.com/hawkular/hawkular-metrics/pull/200). >> >> There are 3 main parts >> >> Cassandra: creates a custom seed provider to support >> ReplicationControllers in Kubernetes, creates a folder/zip archive which >> can be used to generate a Docker image. It may make sense to move the >> Cassandra parts out to a separate project. >> >> Hawkular Metrics: creates a folder/zip archive which can be used to >> generate a Docker image >> >> Kubernetes: pulls everything together into a single kubernetes >> application. Can be used to deploy an application zip into fabric8 (via >> drag and drop in the web console or via the maven plugin) or deploy all >> the components into Openshift via the kubernetes.json configuration file. >> >> The docker images are not created and deployed to a docker registry as >> part of the build, it will just create a folder where you can run the >> docker build from. None of the maven docker plugins I looked at seemed >> to really work properly, so its still a manual process to do the build >> (and push to a registry). Its something which needs to be improved. >> >> The Cassandra service currently only supports adding new nodes to a >> cluster and not removing them via the ReplicationController. This is due >> to the replication factor being set to be 1 by default (which means when >> a node is removed, so is the data it contained). >> >> I believe the docker subproject of hawkular metrics is obsolete and can >> be removed >> (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but >> someone please correct me if I am wrong. It's scripts are referring to >> the console which no longer exists as part of the project. >> >> - Matt >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jkremser at redhat.com Thu Apr 23 07:19:23 2015 From: jkremser at redhat.com (Jiri Kremser) Date: Thu, 23 Apr 2015 07:19:23 -0400 (EDT) Subject: [Hawkular-dev] Serialization format for inventory relations In-Reply-To: <1950611038.3536663.1429786481859.JavaMail.zimbra@redhat.com> Message-ID: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> Hello devs, currently, the inventory REST API can't answer requests like 'give me all related entities to this entity' or similar. I am working on exposing the relationships. Although, it's not probably the use case the REST was designed for (relations are not resources), I would like to stick with standards as much as possible. I suggest using the JSON-LD[1] standard as the underlying serialization format. pros: * w3c standard * can be converted into RDF, RDFa, Turtle, etc. and stored in arbitrary triplestore (might be useful for some offline analysis, querying with SPARQL, etc) * HATEOASS ready, because the ids are URIs cons: * not as concise as plain JSON Here is an example of 1 relation/edge: { "@context": { "inv": "http://hawkular.org/inventory/0.0.1", "baseUrl": "http://127.0.0.1:8080/hawkular/inventory/" }, "@id": "baseUrl:relationships/1337", "inv:shortId": "2", "@type": "inv:Relationship", "inv:source": { "@id": "baseUrl:tenants/acme", "@type": "inv:Tenant", "inv:shortId": "acme" }, "inv:label": "contains", "inv:target": { "@id": "baseUrl:acme/resourceTypes/URL", "@type": "inv:ResourceType", "inv:shortId": "URL" }, "inv:properties": { "created": "2000-01-01", "strength": 0.8 } } (inv:properties can be omitted) The work is almost done in here[2] [1]: http://www.w3.org/TR/json-ld/ [2]: https://github.com/hawkular/hawkular-inventory/pull/55 wdyt? From hrupp at redhat.com Thu Apr 23 08:28:20 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 23 Apr 2015 14:28:20 +0200 Subject: [Hawkular-dev] Serialization format for inventory relations In-Reply-To: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> References: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> Message-ID: <6DC369AD-7972-4BB9-9F83-D88C624684D3@redhat.com> On 23 Apr 2015, at 13:19, Jiri Kremser wrote: > exposing the relationships. Although, it's not probably the use case > the REST was designed for (relations are not resources), I would like No, but relations are links Link: rel="child", href="http://bla/bla" Are you talking here about exposing the relationships only? If you don't plan to include the resources in the result, would be a list of links not enough? > Here is an example of 1 relation/edge: > > { > "@context": { > "inv": "http://hawkular.org/inventory/0.0.1", > "baseUrl": "http://127.0.0.1:8080/hawkular/inventory/" > }, > "@id": "baseUrl:relationships/1337", > "inv:shortId": "2", > "@type": "inv:Relationship", > "inv:source": { > "@id": "baseUrl:tenants/acme", > "@type": "inv:Tenant", > "inv:shortId": "acme" > }, > "inv:label": "contains", > "inv:target": { > "@id": "baseUrl:acme/resourceTypes/URL", > "@type": "inv:ResourceType", > "inv:shortId": "URL" > }, This doesn't really show me its strengths at the moment and looks bloated. But perhaps the strength is that the relationship target has its type supplied? Do we expect to use some existing clients to browse and visualize our inventory using JSON-LD? The other question is if the query is 'give me all related entities to this entity' as you describe above, should we really "only" return the links? Or how will we treat the fetching of the target entities? Can they optionally be embedded in the answer? From mwringe at redhat.com Thu Apr 23 09:35:24 2015 From: mwringe at redhat.com (Matt Wringe) Date: Thu, 23 Apr 2015 09:35:24 -0400 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <5537AD2F.5040806@redhat.com> References: <55318CBC.8010105@redhat.com> <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> <55379F9B.4010307@redhat.com> <5537AD2F.5040806@redhat.com> Message-ID: <5538F51C.4040807@redhat.com> On 22/04/15 10:16 AM, Matt Wringe wrote: > > On 22/04/15 09:18 AM, Matt Wringe wrote: >> On 20/04/15 09:57 AM, Viet Nguyen wrote: >>> Matt, >>> I was the last person who worked on that Docker build for Hawkular-metrics. Yes, it is obsolete and can be removed. >>> >>> FYI there's an automated Docker build of Kettle in DockerHub and a continuous deployment to a Kubernetes cluster >>> >>> Docker Hub: >>> https://registry.hub.docker.com/u/hawkular/hawkular/ >>> >>> CI Pipeline: >>> https://docs.google.com/presentation/d/14zqOHOvXAB4uN7W_fWOub3ZD6txiS_Vp9cjUNIlj2yQ/edit?usp=sharing >> I went through and have the docker images being built using a docker >> maven plugin. So during the maven build, you can create the image and >> push it out to a registry. >> >> It works great on a local machine, one maven build and I have the docker >> images and kubernetes application zip created. But looking at how the >> current CI is done, it might not have been the best solution (it would >> require the credentials of the docker hub user to be added into the CI >> setup somewhere). >> >> It might make more sense to do it the way the current CI is setup with >> the kettle builds. Anyone have any preferences? > Either way, the CI build way is also easy to setup. You would just need > to sets these up with the official accounts and add in the automated > build hooks: > > https://registry.hub.docker.com/u/mwringe/docker-hawkular-cassandra/ > https://github.com/mwringe/docker-hawkular-cassandra > > https://registry.hub.docker.com/u/mwringe/docker-hawkular-metrics/ > https://github.com/mwringe/docker-hawkular-metrics We really need to get this setup as part of the build so that we can hand off the containers and kubernetes application. I can't set this up myself as we need whomever is able to access the CI builds and docker hub account to perform the setup. I have proposed two different setups, and they both should work for the CI builds: We can either build it via the maven plugin in the CI builds, or we can use the existing docker hub automated builds (like what is done with the kettle docker images). The only thing about the maven build is that we need to store the user credentials in the CI configuration somewhere and need access to a machine capable of performing docker image builds. Everything else in the maven build is handled much better (everything kept in one place, versions are automatically updated, names are kept in sync between the docker images and kubernetes applications, can easier push out to different docker registries, much better developer experience for local builds, etc). The docker hub automated builds are probably faster to setup right now though since we already know we have everything working for it. Note: the maven docker setup is not going away, we need that for developer builds (either locally or pushing out to developer own docker hub account). The docker hub automated builds are essentially useless from a developer's perspective, but they work fine for CI builds. >>> Viet >>> >>> >>> >>> >>> ----- Original Message ----- >>> From: "Matt Wringe" >>> To: hawkular-dev at lists.jboss.org >>> Sent: Friday, April 17, 2015 3:44:12 PM >>> Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers >>> >>> I have a new subproject in Hawkular Metrics which sets up creating >>> components for Openshift/Fabric8 >>> (https://github.com/hawkular/hawkular-metrics/pull/200). >>> >>> There are 3 main parts >>> >>> Cassandra: creates a custom seed provider to support >>> ReplicationControllers in Kubernetes, creates a folder/zip archive which >>> can be used to generate a Docker image. It may make sense to move the >>> Cassandra parts out to a separate project. >>> >>> Hawkular Metrics: creates a folder/zip archive which can be used to >>> generate a Docker image >>> >>> Kubernetes: pulls everything together into a single kubernetes >>> application. Can be used to deploy an application zip into fabric8 (via >>> drag and drop in the web console or via the maven plugin) or deploy all >>> the components into Openshift via the kubernetes.json configuration file. >>> >>> The docker images are not created and deployed to a docker registry as >>> part of the build, it will just create a folder where you can run the >>> docker build from. None of the maven docker plugins I looked at seemed >>> to really work properly, so its still a manual process to do the build >>> (and push to a registry). Its something which needs to be improved. >>> >>> The Cassandra service currently only supports adding new nodes to a >>> cluster and not removing them via the ReplicationController. This is due >>> to the replication factor being set to be 1 by default (which means when >>> a node is removed, so is the data it contained). >>> >>> I believe the docker subproject of hawkular metrics is obsolete and can >>> be removed >>> (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but >>> someone please correct me if I am wrong. It's scripts are referring to >>> the console which no longer exists as part of the project. >>> >>> - Matt >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From tsegismo at redhat.com Thu Apr 23 10:10:00 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 23 Apr 2015 16:10:00 +0200 Subject: [Hawkular-dev] Serialization format for inventory relations In-Reply-To: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> References: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> Message-ID: <5538FD38.607@redhat.com> Le 23/04/2015 13:19, Jiri Kremser a ?crit : > cons: > * not as concise as plain JSON Will it be possible to switch to plain JSON representation? From jkremser at redhat.com Thu Apr 23 10:13:48 2015 From: jkremser at redhat.com (Jiri Kremser) Date: Thu, 23 Apr 2015 10:13:48 -0400 (EDT) Subject: [Hawkular-dev] Serialization format for inventory relations In-Reply-To: <6DC369AD-7972-4BB9-9F83-D88C624684D3@redhat.com> References: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> <6DC369AD-7972-4BB9-9F83-D88C624684D3@redhat.com> Message-ID: <355819095.3615953.1429798428065.JavaMail.zimbra@redhat.com> hi, | Link: rel="child", href="http://bla/bla" | | | Are you talking here about exposing the relationships only? | If you don't plan to include the resources in the result, | would be a list of links not enough? The source and the target of the relationship are also present in the data. This is how it works: send GET to /hawkular/inventory/.../relationships and you'll get all the relationships of the ... so for instance /hawkular/inventory/acme/prod/resources/host42/relationships will return all the relationships (and the basic info of the other side of this relationship, i.e. where host42 functions as a target and those where host42 acts as a source) I guess "Link: rel="child", href="http://bla/bla" is a good way how to address one resource from another one, but we need to expose the information about the relationship itself, it can have properties. | > Here is an example of 1 relation/edge: | > | > { | > "@context": { | > "inv": "http://hawkular.org/inventory/0.0.1", | > "baseUrl": "http://127.0.0.1:8080/hawkular/inventory/" | > }, | > "@id": "baseUrl:relationships/1337", | > "inv:shortId": "2", | > "@type": "inv:Relationship", | > "inv:source": { | > "@id": "baseUrl:tenants/acme", | > "@type": "inv:Tenant", | > "inv:shortId": "acme" | > }, | > "inv:label": "contains", | > "inv:target": { | > "@id": "baseUrl:acme/resourceTypes/URL", | > "@type": "inv:ResourceType", | > "inv:shortId": "URL" | > }, | | This doesn't really show me its strengths at the moment and looks | bloated. But perhaps the strength is that the relationship target | has its type supplied? in fact it could look like this and it'll be still json-ld: { "@context": "http://hawkular.org/inventory/0.0.1/relationship.jsonld", "@id": "baseUrl:relationships/1337", "shortId": "1337", "source": "baseUrl:tenants/acme", "label": "contains", "target": "baseUrl:acme/resourceTypes/URL" } It's up to us how much information we expose, almost everything can be remotely referenced. But I think the entity type is a good thing to have in the data. | Do we expect to use some existing clients to browse and visualize | our inventory using JSON-LD? it's one of the standards for storing the triples and it can be converted [1] to RDF, so yes, there are tools out there for visualizing the structure | The other question is if the query is 'give me all related entities to | this entity' | as you describe above, should we really "only" return the links? definitely, this can be tweak once we find out what we want from the UI perspective. There can be a path/query parameter that will embed the entities (with all the fields including the property hashmap) into the response instead of the link. | Or how | will we treat the fetching of the target entities? Can they optionally | be embedded in the answer? yes, json-ld supports embedding [2]. Currently, the source and target contain the name (shortId), type and the URL if we need to fetch more/less information. [1]: http://www.w3.org/TR/json-ld-api/#deserialize-json-ld-to-rdf-algorithm [2]: https://www.youtube.com/watch?v=UmvWk_TQ30A&t=3m05s From jkremser at redhat.com Thu Apr 23 10:16:32 2015 From: jkremser at redhat.com (Jiri Kremser) Date: Thu, 23 Apr 2015 10:16:32 -0400 (EDT) Subject: [Hawkular-dev] Serialization format for inventory relations In-Reply-To: <5538FD38.607@redhat.com> References: <950401333.3544653.1429787963840.JavaMail.zimbra@redhat.com> <5538FD38.607@redhat.com> Message-ID: <1480096154.3618343.1429798592415.JavaMail.zimbra@redhat.com> | Will it be possible to switch to plain JSON representation? Well, it's easy to do that, but I am not sure whether it is a good idea to have two serialization formats. It is a JSON, so it's easily consumable in JS From vnguyen at redhat.com Thu Apr 23 12:28:39 2015 From: vnguyen at redhat.com (Viet Nguyen) Date: Thu, 23 Apr 2015 12:28:39 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <5538F51C.4040807@redhat.com> References: <55318CBC.8010105@redhat.com> <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> <55379F9B.4010307@redhat.com> <5537AD2F.5040806@redhat.com> <5538F51C.4040807@redhat.com> Message-ID: <1499876691.3669790.1429806519900.JavaMail.zimbra@redhat.com> Matt, I have not been intimately involved with hawkular dev due to other obligations. I'm curious about your use case, running metrics as an independent app... as I always thought metrics, inventory, alert, etc are to meant to be assembled together. I would be happy to assist with the Dockerhub build steps. Let's have a chat early next week? Viet ----- Original Message ----- From: "Matt Wringe" To: hawkular-dev at lists.jboss.org Sent: Thursday, April 23, 2015 6:35:24 AM Subject: Re: [Hawkular-dev] Hawkular Metrics Openshift Containers On 22/04/15 10:16 AM, Matt Wringe wrote: > > On 22/04/15 09:18 AM, Matt Wringe wrote: >> On 20/04/15 09:57 AM, Viet Nguyen wrote: >>> Matt, >>> I was the last person who worked on that Docker build for Hawkular-metrics. Yes, it is obsolete and can be removed. >>> >>> FYI there's an automated Docker build of Kettle in DockerHub and a continuous deployment to a Kubernetes cluster >>> >>> Docker Hub: >>> https://registry.hub.docker.com/u/hawkular/hawkular/ >>> >>> CI Pipeline: >>> https://docs.google.com/presentation/d/14zqOHOvXAB4uN7W_fWOub3ZD6txiS_Vp9cjUNIlj2yQ/edit?usp=sharing >> I went through and have the docker images being built using a docker >> maven plugin. So during the maven build, you can create the image and >> push it out to a registry. >> >> It works great on a local machine, one maven build and I have the docker >> images and kubernetes application zip created. But looking at how the >> current CI is done, it might not have been the best solution (it would >> require the credentials of the docker hub user to be added into the CI >> setup somewhere). >> >> It might make more sense to do it the way the current CI is setup with >> the kettle builds. Anyone have any preferences? > Either way, the CI build way is also easy to setup. You would just need > to sets these up with the official accounts and add in the automated > build hooks: > > https://registry.hub.docker.com/u/mwringe/docker-hawkular-cassandra/ > https://github.com/mwringe/docker-hawkular-cassandra > > https://registry.hub.docker.com/u/mwringe/docker-hawkular-metrics/ > https://github.com/mwringe/docker-hawkular-metrics We really need to get this setup as part of the build so that we can hand off the containers and kubernetes application. I can't set this up myself as we need whomever is able to access the CI builds and docker hub account to perform the setup. I have proposed two different setups, and they both should work for the CI builds: We can either build it via the maven plugin in the CI builds, or we can use the existing docker hub automated builds (like what is done with the kettle docker images). The only thing about the maven build is that we need to store the user credentials in the CI configuration somewhere and need access to a machine capable of performing docker image builds. Everything else in the maven build is handled much better (everything kept in one place, versions are automatically updated, names are kept in sync between the docker images and kubernetes applications, can easier push out to different docker registries, much better developer experience for local builds, etc). The docker hub automated builds are probably faster to setup right now though since we already know we have everything working for it. Note: the maven docker setup is not going away, we need that for developer builds (either locally or pushing out to developer own docker hub account). The docker hub automated builds are essentially useless from a developer's perspective, but they work fine for CI builds. >>> Viet >>> >>> >>> >>> >>> ----- Original Message ----- >>> From: "Matt Wringe" >>> To: hawkular-dev at lists.jboss.org >>> Sent: Friday, April 17, 2015 3:44:12 PM >>> Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers >>> >>> I have a new subproject in Hawkular Metrics which sets up creating >>> components for Openshift/Fabric8 >>> (https://github.com/hawkular/hawkular-metrics/pull/200). >>> >>> There are 3 main parts >>> >>> Cassandra: creates a custom seed provider to support >>> ReplicationControllers in Kubernetes, creates a folder/zip archive which >>> can be used to generate a Docker image. It may make sense to move the >>> Cassandra parts out to a separate project. >>> >>> Hawkular Metrics: creates a folder/zip archive which can be used to >>> generate a Docker image >>> >>> Kubernetes: pulls everything together into a single kubernetes >>> application. Can be used to deploy an application zip into fabric8 (via >>> drag and drop in the web console or via the maven plugin) or deploy all >>> the components into Openshift via the kubernetes.json configuration file. >>> >>> The docker images are not created and deployed to a docker registry as >>> part of the build, it will just create a folder where you can run the >>> docker build from. None of the maven docker plugins I looked at seemed >>> to really work properly, so its still a manual process to do the build >>> (and push to a registry). Its something which needs to be improved. >>> >>> The Cassandra service currently only supports adding new nodes to a >>> cluster and not removing them via the ReplicationController. This is due >>> to the replication factor being set to be 1 by default (which means when >>> a node is removed, so is the data it contained). >>> >>> I believe the docker subproject of hawkular metrics is obsolete and can >>> be removed >>> (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but >>> someone please correct me if I am wrong. It's scripts are referring to >>> the console which no longer exists as part of the project. >>> >>> - Matt >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From jshaughn at redhat.com Fri Apr 24 09:15:09 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Fri, 24 Apr 2015 09:15:09 -0400 Subject: [Hawkular-dev] What is Business Transaction Management (prep for F2F) In-Reply-To: <1443849909.4662889.1429699649666.JavaMail.zimbra@redhat.com> References: <1443849909.4662889.1429699649666.JavaMail.zimbra@redhat.com> Message-ID: <553A41DD.1040109@redhat.com> Thanks for the jump start, Gary, Is there any active BTM going on in the RH Middleware product set today? Anything collecting information that might feed into Hawkular? How about in a containerized deployment env? Sorry if these are naive questions, just trying to get handle on the sort of data we may be looking at, and how it gets correlated. Or, would BTM be more interested in just using H Alerting to generate alerts and take automated actions? On 4/22/2015 6:47 AM, Gary Brown wrote: > Hi all > > As suggested recently by others, we want to focus the F2F sessions on discussions rather than presentations. So in that spirit, I thought it would be good to get the "What is Business Transaction Management" discussion out of the way as a ML thread, so that the F2F session can discuss what and how to build BTM in a Hawkular context. > > Taking some excerpts from Wikipedia: > > "Business transaction management (BTM), is the practice of managing information technology (IT) from a business transaction perspective. It provides a tool for tracking the flow of transactions across IT infrastructure, in addition to detection, alerting, and correction of unexpected changes in business or technical conditions. BTM provides visibility into the flow of transactions across infrastructure tiers, including a dynamic mapping of the application topology. > > Using BTM, application support teams are able to search for transactions based on message context and content ? for instance, time of arrival or message type ? providing a way to isolate causes for common issues such as application exceptions, stalled transactions, and lower-level issues such as incorrect data values. > > The ultimate goal of BTM is to improve service quality for users conducting business transactions while improving the effectiveness of the IT applications and infrastructure across which those transactions execute. The main benefit of BTM is its capacity to identify precisely where transactions are delayed within the IT infrastructure. BTM also aims to provide proactive problem prevention and the generation of business service intelligence for optimization of resource provisioning and virtualization." > > Some of the applications of BTM listed are: > > "BTM solutions capture all of the transaction instances in the production environment and as such can be used for monitoring as well as for analysis and planning. Some applications include: > > * Outage avoidance and problem isolation: Identification and isolation of tier-specific performance and availability issues. > * Service level management: Monitoring of SLAs and alerting of threshold breaches both at the end-user and infrastructure tier level. > * Infrastructure optimization: Modification of the configuration of data center infrastructure to maximize utilization and improve performance. > * Capacity planning: Analysis of usage and performance trends in order to estimate future capacity requirements. > * Change management: Analysis of the impact of change on transaction execution. > * Cloud management: Track the end-to-end transaction flow across both cloud (private, hybrid, public) and dedicated (on-premises, off-premises) infrastructure." > > Obviously we need to focus our efforts on the monitoring/alerting aspects initially, and this is what I am expecting the F2F discussion will be focused on, but a couple of these areas may be of interest in the future. > > Any views on the above appreciated. > > Regards > Gary > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150424/f1a7fc34/attachment-0001.html From gbrown at redhat.com Fri Apr 24 09:23:42 2015 From: gbrown at redhat.com (Gary Brown) Date: Fri, 24 Apr 2015 09:23:42 -0400 (EDT) Subject: [Hawkular-dev] What is Business Transaction Management (prep for F2F) In-Reply-To: <553A41DD.1040109@redhat.com> References: <1443849909.4662889.1429699649666.JavaMail.zimbra@redhat.com> <553A41DD.1040109@redhat.com> Message-ID: <3002676.6132995.1429881822552.JavaMail.zimbra@redhat.com> Hi Jay Currently I think BTM in RH is limited - RTGov has the ability to provide a call trace, but correlation across servers was not properly supported, and I think Fuse has the ability to trace activities, but again within a single VM. So one of the main focuses of this new project is providing proper tracability of business transactions across servers and work well across on-premises and cloud environments. The current thought is the trace fragments from individual servers would be captured and correlated, generating relevant metrics to H Metrics which then would feed into H Alerts to take actions. Regards Gary ----- Original Message ----- > > Thanks for the jump start, Gary, > > Is there any active BTM going on in the RH Middleware product set today? > Anything collecting information that might feed into Hawkular? How about in > a containerized deployment env? Sorry if these are naive questions, just > trying to get handle on the sort of data we may be looking at, and how it > gets correlated. Or, would BTM be more interested in just using H Alerting > to generate alerts and take automated actions? > > > On 4/22/2015 6:47 AM, Gary Brown wrote: > > > > Hi all > > As suggested recently by others, we want to focus the F2F sessions on > discussions rather than presentations. So in that spirit, I thought it would > be good to get the "What is Business Transaction Management" discussion out > of the way as a ML thread, so that the F2F session can discuss what and how > to build BTM in a Hawkular context. > > Taking some excerpts from Wikipedia: > > "Business transaction management (BTM), is the practice of managing > information technology (IT) from a business transaction perspective. It > provides a tool for tracking the flow of transactions across IT > infrastructure, in addition to detection, alerting, and correction of > unexpected changes in business or technical conditions. BTM provides > visibility into the flow of transactions across infrastructure tiers, > including a dynamic mapping of the application topology. > > Using BTM, application support teams are able to search for transactions > based on message context and content ? for instance, time of arrival or > message type ? providing a way to isolate causes for common issues such as > application exceptions, stalled transactions, and lower-level issues such as > incorrect data values. > > The ultimate goal of BTM is to improve service quality for users conducting > business transactions while improving the effectiveness of the IT > applications and infrastructure across which those transactions execute. The > main benefit of BTM is its capacity to identify precisely where transactions > are delayed within the IT infrastructure. BTM also aims to provide proactive > problem prevention and the generation of business service intelligence for > optimization of resource provisioning and virtualization." > > Some of the applications of BTM listed are: > > "BTM solutions capture all of the transaction instances in the production > environment and as such can be used for monitoring as well as for analysis > and planning. Some applications include: > > * Outage avoidance and problem isolation: Identification and isolation of > tier-specific performance and availability issues. > * Service level management: Monitoring of SLAs and alerting of threshold > breaches both at the end-user and infrastructure tier level. > * Infrastructure optimization: Modification of the configuration of data > center infrastructure to maximize utilization and improve performance. > * Capacity planning: Analysis of usage and performance trends in order to > estimate future capacity requirements. > * Change management: Analysis of the impact of change on transaction > execution. > * Cloud management: Track the end-to-end transaction flow across both > cloud (private, hybrid, public) and dedicated (on-premises, > off-premises) infrastructure." > > Obviously we need to focus our efforts on the monitoring/alerting aspects > initially, and this is what I am expecting the F2F discussion will be > focused on, but a couple of these areas may be of interest in the future. > > Any views on the above appreciated. > > Regards > Gary > > _______________________________________________ > hawkular-dev mailing list hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Fri Apr 24 10:05:29 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 24 Apr 2015 10:05:29 -0400 (EDT) Subject: [Hawkular-dev] WildFly NoSQL integration prototype In-Reply-To: <553A4CA6.70905@redhat.com> References: <553A4CA6.70905@redhat.com> Message-ID: <1368169286.5864494.1429884329920.JavaMail.zimbra@redhat.com> cross-posting from the wildfly-dev list - could be of interest to us. ----- Forwarded Message ----- To: "WildFly Dev List" Sent: Friday, April 24, 2015 10:01:10 AM Subject: [wildfly-dev] WildFly NoSQL integration prototype Are you interested in allowing NoSQL databases access from WildFly application deployments? This email is about an integration effort to allow WildFly applications to use NoSQL. Feedback is welcome on this effort, as well as help in improving [1]. Some basic unit tests are already added that show a session bean reading/writing MongoDB [2] + Cassandra [3] databases. In order for the tests to pass, the local machine must already be running MongoDB or Cassandra databases. 1. Things that currently (seems to be) working in the prototype: * During WildFly startup, MongoDB/Cassandra databases are connected to based on settings in their respective subsystems. See the configuration example [4]. * Applications can access native MongoDB/Cassandra objects that represent database connections (with internal native connection pooling). See @Resource example [2][3]. Will see how the requirements evolve going forward and whether @Resource is the right way and/or whether other annotations are needed. 2. Currently not working in the prototype: * Multiple hosts/ports cannot be specified yet for target database. * Protection against applications closing pooled connections. * NoSQL drivers currently may create threads in EE application threads which could leak ClassLoaders/AccessControlContexts. One solution might be to contribute a patch that allows WildFly to do the thread creation in some fashion for the NoSQL drivers. * We have not (yet) tried using (Java) security manager support with the NoSQL driver clients. * Additional NoSQL connection attributes need to be added to the NoSQL subsystems. * Native NoSQL class objects are bound to JNDI currently (e.g. MongoClient). We might want to bind wrapper or proxy objects so that we can extend the NoSQL classes or in some cases, prevent certain actions (e.g. prevent calls to MongoClient.close()). Perhaps we will end up with a mixed approach, where we could extend the NoSQL driver if that is the only way to manage it, or contribute a listener patch for WildFly to get control during certain events (e.g. for ignoring close of pooled database connections). * The prototype currently gives all (WildFly) deployments access to the Cassandra/MongoDB driver module classloaders. This is likely to change but not yet sure to what. 3. The Weld (CDI) project is also looking at NoSQL enhancements, as is the Narayana project. There is also the Hibernate OGM project that is pushing on JPA integration and will also help contribute changes to the NoSQL drivers that are needed for WildFly integration (e.g. introduce alternative way for NoSQL drivers manage thread creation for background task execution). 4. We will need a place to track issues for NoSQL integration. If the NoSQL integration changes are merged directly into WildFly, perhaps we could have a nosql category under https://issues.jboss.org/browse/WFLY. 5. You can view outstanding issues in the MongoDB server [5], Java driver [6] to get feel for problems that others have run into (just like you would with WildFly). You can view outstanding issues in the Cassandra server [7] and Java driver [8] to get a feel for problems as well. 6. Infinispan [9] integration in WildFly is still going strong. Infinispan is still the backbone of WildFly clustering and also available for applications to use as a datasource. 7. The standalone.xml settings [4] will soon change (would like to eliminate the "name=default", add more attributes and get the multiple host/ports wired in). 8. If the NoSQL unit tests do stay in the WildFly repo, they will need to be disabled by default, as most WildFly developers will not have a NoSQL database running. Speaking of which, we need to wire the unit tests to update the standalone.xml to contain the MongoDB/Cassandra subsystem settings [4]. 9. What version of NoSQL databases will work with the WildFly NoSQL integration? At this point, we will only work with one version of each NoSQL database that is integrated with. Because we are likely to need some changes in the NoSQL client drivers, we will work with the upstream communities to ensure the NoSQL driver code can run in an EE container thread, without causing leaks. First we have to identity the changes that we need (e.g. find some actual leaks that I only suspect will happen at this point and propose some changes). The Hibernate OGM team is going to help with the driver patches (thanks Hibernate OGM team! :-) 10. Going forward, how can WildFly extend the NoSQL (client driver side) capabilities to improve the different application life cycles through development, test, production? Scott [1] https://github.com/scottmarlow/wildfly/tree/nosql-dev [2] https://github.com/scottmarlow/wildfly/blob/nosql-dev/testsuite/compat/src/test/java/org/jboss/as/test/compat/nosql/mongodb/StatefulTestBean.java#L23 [3] https://github.com/scottmarlow/wildfly/blob/nosql-dev/testsuite/compat/src/test/java/org/jboss/as/test/compat/nosql/cassandra/StatefulTestBean.java#L19 [4] https://gist.github.com/scottmarlow/b8196bdc56431bb171c8 [5] https://jira.mongodb.org/browse/SERVER [6] https://jira.mongodb.org/browse/JAVA [7] https://issues.apache.org/jira/browse/CASSANDRA [8] https://datastax-oss.atlassian.net/browse/JAVA [9] http://infinispan.org _______________________________________________ wildfly-dev mailing list wildfly-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/wildfly-dev From hrupp at redhat.com Fri Apr 24 11:08:56 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 24 Apr 2015 17:08:56 +0200 Subject: [Hawkular-dev] REST-api pagination Message-ID: Hey, triggered by https://issues.jboss.org/browse/HWKINVENT-33 and https://issues.jboss.org/browse/HAWKULAR-125 we should make sure that all the Hawkular-subprojects use the same pagination mechanism in the REST-endpoints. == Why pagination? Pagination serves two main purposes * Limit the work the server has to do by not returning all results at once * Limit the work a client has to do with a huge list of results. The limiting of work not only applies to CPU time, but also to memory consumption and in the case of remote clients also to load time. Smaller lists just load faster. == What is there? Inseide RHQ we use both: Link-headers and paging information inside the returned objects There are 'page', 'ps' for page number and ps for page size, have links to 'prev', 'next','current' and 'last' (if applicable) And an additional header "X-collection-size" for the total number of items See https://github.com/rhq-project/rhq/blob/master/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/rest/AbstractRestBean.java#L454 and https://github.com/rhq-project/rhq/blob/master/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/rest/AbstractRestBean.java#L505 There is RFC5988 that talks about linking headers and which defines a number of link types ( http://tools.ietf.org/html/rfc5988#section-6.2.2 ), from where the RHQ ones were taken. This url http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api#pagination has some "best practices" (defined by whom?) that are basically like the RHQ link headers, but instead of using 'ps' for the page size, they use 'per_page', which is apparently what GitHub does. Also the total collection count is 'X-Total-Count' and not RHQ's 'X-collection-size'. We had a discussion about the style (link/body) in the past: http://pilhuhn.blogspot.de/2013/02/best-practice-for-paging-in-restful-apis.html and also the objections from the JS side, who have issues getting at the http-headers, but can easily digest body elements), which led to support both paging in body and headers depending on the media type requested ( see https://github.com/rhq-project/rhq/blob/master/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/rest/AbstractRestBean.java#L413 ) For convenience I suggest to copy over the methods from RHQ and re-use them. From jpkroehling at redhat.com Mon Apr 27 04:35:28 2015 From: jpkroehling at redhat.com (=?windows-1252?Q?Juraci_Paix=E3o_Kr=F6hling?=) Date: Mon, 27 Apr 2015 10:35:28 +0200 Subject: [Hawkular-dev] Looking for a first project to integrate accounts In-Reply-To: <55361E5D.7090302@redhat.com> References: <55361E5D.7090302@redhat.com> Message-ID: <553DF4D0.1080709@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 All, I would love to get one component first and help implement this integration, so that problems and missing features would be already fixed at the time the other components start doing the integration. If you are interested in having your component integrated, let me know and we can work something out. In case you are unsure about what needs to be done, there are a couple of documents that could be helpful: https://github.com/hawkular/hawkular-accounts -- Readme file http://www.hawkular.org/docs/dev/accounts.html Thanks! - - Juca. On 04/21/2015 11:54 AM, Juraci Paix?o Kr?hling wrote: > All, > > Hawkular Accounts have reached a point where it's possible for > other components to integrate with it. As such, I would like to ask > if there's a project that would volunteer for it, to be the first > with this integration, so that I can fix possible integration bugs > that might come from this experience. > > I know that you all are busy with the MVP, but if you think you > can afford to spend, say, a day on this, I'll be ready to guide > you. > > Those are the items that I would like to see being used during > this first integration: > > - Securing the backend with Keycloak - Consuming the tenant > information (Persona) - Protecting resources based on the Persona's > role > > Needless to say, I'll support you in achieving this and will > promptly fix the issues that may arise. > > If you need more information before you volunteer, you can take a > look at the readme file from the accounts module: > > https://github.com/hawkular/hawkular-accounts > > - Juca. _______________________________________________ > hawkular-dev mailing list hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVPfTQAAoJECKM1e+fkPrXm6QH/3FmzCvrDzwrzZr3OqL/Tnoc 0EwFO+XXvwWsKTCNz2yZGGM++SlwyGW0YkynHXYaJkqqCvntshN0G/V2oPZCc1NN tyGSFX9v7pVfb3OATgU+Oabk8+Q/VMSPtHy6ktiV+/5jq0WrKrESJHi8+GJRBLoE GtLx0lKXO/iI6QC8e87sOjAhJm+/Nvfn5fZrvMWP9R4vuj+m2O+zX7dZw34ot3op G9qPhQw+9+cQX1FpgRuaUNbcITO4pHD7dtYv/hcjKt5MDhwuCxVIy8UUK2qrZj9A oA60BsBgAii1CIjJU93AA295rjJl8Nxz4zRFM1s+QlmpbvGtme7mmXXDzDvgDhE= =VUnM -----END PGP SIGNATURE----- From vrockai at redhat.com Mon Apr 27 06:59:01 2015 From: vrockai at redhat.com (Viliam Rockai) Date: Mon, 27 Apr 2015 12:59:01 +0200 Subject: [Hawkular-dev] REST-api pagination In-Reply-To: References: Message-ID: <1430132341.17833.3.camel@vrockai-laptop> On Fri, 2015-04-24 at 17:08 +0200, Heiko W.Rupp wrote: > Hey, > > triggered by https://issues.jboss.org/browse/HWKINVENT-33 > and https://issues.jboss.org/browse/HAWKULAR-125 > > we should make sure that all the Hawkular-subprojects use the > same pagination mechanism in the REST-endpoints. > > == Why pagination? > > Pagination serves two main purposes > > * Limit the work the server has to do by not returning all results at > once > * Limit the work a client has to do with a huge list of results. > > The limiting of work not only applies to CPU time, but also to memory > consumption > and in the case of remote clients also to load time. Smaller lists just > load faster. > > == What is there? > > Inseide RHQ we use both: Link-headers and paging information > inside the returned objects > > There are 'page', 'ps' for page number and ps for page size, > have links to 'prev', 'next','current' and 'last' (if applicable) > > And an additional header "X-collection-size" for the total number of > items > > See > https://github.com/rhq-project/rhq/blob/master/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/rest/AbstractRestBean.java#L454 > and > https://github.com/rhq-project/rhq/blob/master/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/rest/AbstractRestBean.java#L505 > > > There is RFC5988 that talks about linking headers and which defines a > number of link types ( > http://tools.ietf.org/html/rfc5988#section-6.2.2 ), from where the RHQ > ones were taken. > > This url > http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api#pagination > has some "best practices" (defined by whom?) that are basically like the > RHQ link headers, > but instead of using 'ps' for the page size, they use 'per_page', which > is apparently what GitHub > does. Also the total collection count is 'X-Total-Count' and not RHQ's > 'X-collection-size'. > > We had a discussion about the style (link/body) in the past: > http://pilhuhn.blogspot.de/2013/02/best-practice-for-paging-in-restful-apis.html > and also the objections from the JS side, who have issues getting at the > http-headers, but can easily digest body elements), which led to support > both paging in body and headers depending on the media type requested ( > see > https://github.com/rhq-project/rhq/blob/master/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/rest/AbstractRestBean.java#L413 > ) This should not be a problem anymore, since in the current AngularJS $resource object which we use as the REST "client", we can access the headers directly the same way as the json data: "Success callback is called with (value, responseHeaders) arguments." https://docs.angularjs.org/api/ngResource/service/$resource > > For convenience I suggest to copy over the methods from RHQ and re-use > them. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jshaughn at redhat.com Mon Apr 27 16:07:01 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Mon, 27 Apr 2015 16:07:01 -0400 Subject: [Hawkular-dev] Hawkular Alerts - 0.0.1-Snapshot pushed to master Message-ID: <553E96E5.2060803@redhat.com> Hawkular Alerts has been updated in master. Given our latest Hawkular convention for versioning, the versioning has gone backwards. We pushed Version 0.0.1-SNAPSHOT, changed from Version 1.0.0-SNAPSHOT. We maintain snapshot releases for use with Kettle. But the version has changed to reflect API changes. Going forward we'll maintain the version of 0.0.1 unless we have an API change, at which point we'll up the micro-version. Eventually this will settle down and we can move to versioned components, and avoid breaking API changes. We've begun supplying developer documentation, here: http://www.hawkular.org/docs/dev/alerts.html What's NEW! * Alert life-cycle support o Alerts now support Open, Ack and Resolved states. o This gives us the ability to store and display full start and end times for an alerting "event". * Trigger AutoResolve o This replaces and builds on the former "trigger safety" feature. o Optionally provide AutoResolve conditions and dampening to toggle the Trigger from Firing mode to AutoResolve mode. o Optionally have a Trigger's alerts set Resolved automatically, when the trigger's AutoResolve conditions are satisfied. * Trigger AutoDisable o Optionally have a Trigger disable itself after firing. o Mutes a trigger without the need to define AutoResolve. * Rest Api control of the new features. * Twilio SMS sender action * Aerogear sender action * Performance testing infrastructure * Improved Rest API documentation And for Hawkular Kettle! * Data Filtering o Only forward relevant metric and availability data for alert processing. o Remove the metrics and availability not used in any trigger. And of course several fixes and improvements. Coming Up: * Moving the persistence back-end to Cassandra to leverage the Metrics cluster (soon). * PagerDuty action integration. * Integration of the new 0.0.1 features into Hawkular. Hawkular Alerts Team Jay Shaughnessy Lucas Ponce Thomas Segismont -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150427/92c404ac/attachment-0001.html From jmorgan at redhat.com Mon Apr 27 20:38:47 2015 From: jmorgan at redhat.com (Jared MORGAN) Date: Mon, 27 Apr 2015 20:38:47 -0400 (EDT) Subject: [Hawkular-dev] Coming to Parent: publishing JavaDoc and sources via Maven In-Reply-To: <1073938809.6663690.1427748801386.JavaMail.zimbra@redhat.com> References: <5519518D.7030308@redhat.com> <55196003.9090202@redhat.com> <5519A53D.2020005@redhat.com> <1073938809.6663690.1427748801386.JavaMail.zimbra@redhat.com> Message-ID: <583269998.7134666.1430181527605.JavaMail.zimbra@redhat.com> I'm coming late to the party here, so my apologies for that. I understand that docs for Hawkular will be done in Asciidoc (great move IMHO). Why not extend that to Javadocs: https://github.com/asciidoctor/asciidoclet Compatible with Maven. No need to try and kludge HTML to make tables work. Easy to read directly in code as well as looking pretty when compiled. Thoughts? J ----- Original Message ----- > > Hi Jay, there is just a couple of them in Bus - see the attached file. -- P > > If everyone writes as good as javadocs as I do, I'm all for the javadoc > checker :-D > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From artur.dryomov at gmail.com Tue Apr 28 03:29:20 2015 From: artur.dryomov at gmail.com (Artur Dryomov) Date: Tue, 28 Apr 2015 10:29:20 +0300 Subject: [Hawkular-dev] [GSoC] Hawkular Android Client Message-ID: Hi everyone, This year I will be working on the Hawkular Android client application as part of the Google Summer of Code 2015 program. The application itself will use Hawkular API and AeroGear SDK. In coming days I?ll research these areas, especially documentation, and will try to create some sort of architecture and basic design. Thank you all for this opportunity! Artur. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150428/fd028bba/attachment.html From lkrejci at redhat.com Tue Apr 28 05:43:36 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 28 Apr 2015 11:43:36 +0200 Subject: [Hawkular-dev] Inventory REST now has result sets pagination Message-ID: <2159243.DYWF8jYFne@localhost.localdomain> Hi, I went ahead and prototyped the pagination support both in the backend and the REST API within the boundaries of inventory. Because this is a generic feature, it's been agreed that this stuff would deserve a place of its own so that it can be shared amongst the various hawkular components. There are 2 aspects of the paging: * support in backend - This basically only revolves around the specification of the paging requirements and a list "wrapper" that adds paging context to the results. The way how the paging will be implemented will differ in each component. https://github.com/hawkular/hawkular-inventory/tree/master/api/src/main/java/org/hawkular/inventory/api/paging * support in the REST API - This means to translate the query parameters with the paging specification into paging java API and transform the results from the backend into the set of headers attached to the response. The support for paging in REST in inventory is contained in these classes: https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/RequestUtil.java https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/ResponseUtil.java https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/json/Link.java https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/json/LinkDeserializer.java https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/json/LinkSerializer.java Both the backend and the REST support were either taken from or were heavily inspired by the paging support in RHQ. Where do you think these should live? Do we have a repo for common stuff? I imagine we will have to have 2 separate common modules so that we don't drag jaxrs deps into core components. Cheers, Lukas From matzew at apache.org Tue Apr 28 07:13:56 2015 From: matzew at apache.org (Matthias Wessendorf) Date: Tue, 28 Apr 2015 13:13:56 +0200 Subject: [Hawkular-dev] [aerogear-dev] [GSoC] Hawkular Android Client In-Reply-To: References: Message-ID: Hello Artur! welcome to our community/communities and congratulations on getting a seat at the GSoC program! We are looking forward to your contributions to our mailing lists and code repos :-) Good luck! On Tue, Apr 28, 2015 at 9:29 AM, Artur Dryomov wrote: > Hi everyone, > > This year I will be working on the Hawkular Android client application as > part of the Google Summer of Code 2015 program. > > The application itself will use Hawkular API and AeroGear SDK. In coming > days I?ll research these areas, especially documentation, and will try to > create some sort of architecture and basic design. > > Thank you all for this opportunity! > Artur. > > _______________________________________________ > aerogear-dev mailing list > aerogear-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev > -- Matthias Wessendorf blog: http://matthiaswessendorf.wordpress.com/ sessions: http://www.slideshare.net/mwessendorf twitter: http://twitter.com/mwessendorf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150428/aad77144/attachment.html From lkrejci at redhat.com Tue Apr 28 07:50:47 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 28 Apr 2015 13:50:47 +0200 Subject: [Hawkular-dev] Coming to Parent: publishing JavaDoc and sources via Maven In-Reply-To: <583269998.7134666.1430181527605.JavaMail.zimbra@redhat.com> References: <5519518D.7030308@redhat.com> <1073938809.6663690.1427748801386.JavaMail.zimbra@redhat.com> <583269998.7134666.1430181527605.JavaMail.zimbra@redhat.com> Message-ID: <1833327.3eVncBMr9F@localhost.localdomain> What about IDE support for Asciidoc docs? Me, I like being able to click on links in javadoc popups. On Monday, April 27, 2015 20:38:47 Jared MORGAN wrote: > I'm coming late to the party here, so my apologies for that. > > I understand that docs for Hawkular will be done in Asciidoc (great move > IMHO). > > Why not extend that to Javadocs: https://github.com/asciidoctor/asciidoclet > > Compatible with Maven. No need to try and kludge HTML to make tables work. > Easy to read directly in code as well as looking pretty when compiled. > > Thoughts? > > J > > ----- Original Message ----- > > > > Hi Jay, there is just a couple of them in Bus - see the attached file. > > > -- P > > > > If everyone writes as good as javadocs as I do, I'm all for the javadoc > > checker :-D > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jbalunas at redhat.com Tue Apr 28 08:41:54 2015 From: jbalunas at redhat.com (Jay Balunas) Date: Tue, 28 Apr 2015 08:41:54 -0400 Subject: [Hawkular-dev] [aerogear-dev] [GSoC] Hawkular Android Client In-Reply-To: References: Message-ID: +1, Great to have you and can't wait to see your contributions and ideas! On Tue, Apr 28, 2015 at 7:13 AM, Matthias Wessendorf wrote: > Hello Artur! > > welcome to our community/communities and congratulations on getting a seat > at the GSoC program! > > We are looking forward to your contributions to our mailing lists and code > repos :-) > > Good luck! > > On Tue, Apr 28, 2015 at 9:29 AM, Artur Dryomov > wrote: > >> Hi everyone, >> >> This year I will be working on the Hawkular Android client application as >> part of the Google Summer of Code 2015 program. >> >> The application itself will use Hawkular API and AeroGear SDK. In coming >> days I?ll research these areas, especially documentation, and will try to >> create some sort of architecture and basic design. >> >> Thank you all for this opportunity! >> Artur. >> >> _______________________________________________ >> aerogear-dev mailing list >> aerogear-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/aerogear-dev >> > > > > -- > Matthias Wessendorf > > blog: http://matthiaswessendorf.wordpress.com/ > sessions: http://www.slideshare.net/mwessendorf > twitter: http://twitter.com/mwessendorf > > _______________________________________________ > aerogear-dev mailing list > aerogear-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150428/74ef7fe4/attachment.html From supittma at redhat.com Tue Apr 28 09:27:49 2015 From: supittma at redhat.com (Summers Pittman) Date: Tue, 28 Apr 2015 09:27:49 -0400 Subject: [Hawkular-dev] [aerogear-dev] [GSoC] Hawkular Android Client In-Reply-To: References:

Message-ID: Hi, Arur and welcome. Look for me (summersp) and passos (passos) in IRC. We are the two main Android developers. Abstractj and qmx tend to help out a lot too. Summers On Tue, Apr 28, 2015 at 8:41 AM, Jay Balunas wrote: > +1, Great to have you and can't wait to see your contributions and ideas! > > > On Tue, Apr 28, 2015 at 7:13 AM, Matthias Wessendorf > wrote: > >> Hello Artur! >> >> welcome to our community/communities and congratulations on getting a >> seat at the GSoC program! >> >> We are looking forward to your contributions to our mailing lists and >> code repos :-) >> >> Good luck! >> >> On Tue, Apr 28, 2015 at 9:29 AM, Artur Dryomov >> wrote: >> >>> Hi everyone, >>> >>> This year I will be working on the Hawkular Android client application >>> as part of the Google Summer of Code 2015 program. >>> >>> The application itself will use Hawkular API and AeroGear SDK. In coming >>> days I?ll research these areas, especially documentation, and will try to >>> create some sort of architecture and basic design. >>> >>> Thank you all for this opportunity! >>> Artur. >>> >>> _______________________________________________ >>> aerogear-dev mailing list >>> aerogear-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/aerogear-dev >>> >> >> >> >> -- >> Matthias Wessendorf >> >> blog: http://matthiaswessendorf.wordpress.com/ >> sessions: http://www.slideshare.net/mwessendorf >> twitter: http://twitter.com/mwessendorf >> >> _______________________________________________ >> aerogear-dev mailing list >> aerogear-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/aerogear-dev >> > > > _______________________________________________ > aerogear-dev mailing list > aerogear-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150428/a290435b/attachment-0001.html From snegrea at redhat.com Tue Apr 28 11:44:56 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 28 Apr 2015 11:44:56 -0400 (EDT) Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <591057080.8494667.1430233780909.JavaMail.zimbra@redhat.com> Message-ID: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> Hello Everybody, I've been working on a PR for the upcoming Hawkular Metrics release that will remove the tenant id from the end-point URLs. The tenant id will be moved to either a header parameter or a query parameter. The query parameter is in place for cases (such as curl) where setting a header is not possible, difficult, or inconvenient. Here is an example of the change: Existing URL: /{tenantId}/gauge/{metricId}/data New URL: /gauge/{metricId}/data Tenant id set via: 1) header - tenantId 2) query parameter - tenantId There are two exceptions to this rule, /tenants and /db/{tenantid}/series. The /tenants end-point will be changed into something different in the upcoming releases since it is mostly a management type API that does not belong in the same place with the regular metrics endpoint. And /db/{tenantid}/series end-point is needed in this exact format for compatibility with Influxdb compatible services. Now, to the merits of this change. The tenant id is volatile, can change any time, and changes to it should be expected; but the rest of the URL is fixed. The second issue is that the tenant id is a security concern. So we were limited in design choices since a security concern was leaking as part of the URL. So removing the tenant id from the URL will give us permanent & consistent addresses for resources (metrics and metric data points). And we will gain a lot of flexibility on the security side. In the future, users could authenticate with a user/pass combo and the backend would transform that into a tenant id to be used on the request. If the same user later decides to use a tenant id to pass along the request, the URL of the resources would not change. Another expectation is that tenant id is not sufficient, it is typically a combo of id + secret; so we would have resorted to a header or query param for the second piece of information (the secret). This change will give us the flexibility to adjust the security model (the meaning of tenant ids and ways to validate them) without compromising the URL structure. This will help Hawkular Metrics as it gets integrated into more and more projects and products. Here are the links to the JIRA and the PR for this change: https://github.com/hawkular/hawkular-metrics/pull/202 https://issues.jboss.org/browse/HWKMETRICS-68 Thank you, Stefan Negrea Software Engineer From lkrejci at redhat.com Tue Apr 28 15:58:47 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 28 Apr 2015 15:58:47 -0400 (EDT) Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> Message-ID: <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> How do metrics' tenants fit into the hawkular accounts and its persona concept? ----- Original Message ----- > From: "Stefan Negrea" > To: "Discussions" > Sent: Tuesday, 28 April, 2015 5:44:56 PM > Subject: [Hawkular-dev] Tenant Id - Not Part of URL > > Hello Everybody, > > I've been working on a PR for the upcoming Hawkular Metrics release that will > remove the tenant id from the end-point URLs. The tenant id will be moved to > either a header parameter or a query parameter. The query parameter is in > place for cases (such as curl) where setting a header is not possible, > difficult, or inconvenient. > > Here is an example of the change: > > Existing URL: > /{tenantId}/gauge/{metricId}/data > > New URL: > /gauge/{metricId}/data > > Tenant id set via: > 1) header - tenantId > 2) query parameter - tenantId > > > There are two exceptions to this rule, /tenants and /db/{tenantid}/series. > The /tenants end-point will be changed into something different in the > upcoming releases since it is mostly a management type API that does not > belong in the same place with the regular metrics endpoint. And > /db/{tenantid}/series end-point is needed in this exact format for > compatibility with Influxdb compatible services. > > > Now, to the merits of this change. The tenant id is volatile, can change any > time, and changes to it should be expected; but the rest of the URL is > fixed. The second issue is that the tenant id is a security concern. So we > were limited in design choices since a security concern was leaking as part > of the URL. > > So removing the tenant id from the URL will give us permanent & consistent > addresses for resources (metrics and metric data points). And we will gain a > lot of flexibility on the security side. In the future, users could > authenticate with a user/pass combo and the backend would transform that > into a tenant id to be used on the request. If the same user later decides > to use a tenant id to pass along the request, the URL of the resources would > not change. Another expectation is that tenant id is not sufficient, it is > typically a combo of id + secret; so we would have resorted to a header or > query param for the second piece of information (the secret). > > This change will give us the flexibility to adjust the security model (the > meaning of tenant ids and ways to validate them) without compromising the > URL structure. This will help Hawkular Metrics as it gets integrated into > more and more projects and products. > > Here are the links to the JIRA and the PR for this change: > https://github.com/hawkular/hawkular-metrics/pull/202 > https://issues.jboss.org/browse/HWKMETRICS-68 > > > > Thank you, > Stefan Negrea > > Software Engineer > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From agalante at redhat.com Tue Apr 28 22:14:06 2015 From: agalante at redhat.com (Andres Galante) Date: Tue, 28 Apr 2015 22:14:06 -0400 Subject: [Hawkular-dev] [aerogear-dev] [GSoC] Hawkular Android Client In-Reply-To: References:

Message-ID: And if you are looking for design help you can ping Gabriel gcardoso at redhat.com he is the UX dedicated to Hawkular and a great designer. On Tue, Apr 28, 2015 at 9:27 AM, Summers Pittman wrote: > Hi, Arur and welcome. > > Look for me (summersp) and passos (passos) in IRC. We are the two main > Android developers. Abstractj and qmx tend to help out a lot too. > > Summers > > On Tue, Apr 28, 2015 at 8:41 AM, Jay Balunas wrote: > >> +1, Great to have you and can't wait to see your contributions and ideas! >> >> >> On Tue, Apr 28, 2015 at 7:13 AM, Matthias Wessendorf >> wrote: >> >>> Hello Artur! >>> >>> welcome to our community/communities and congratulations on getting a >>> seat at the GSoC program! >>> >>> We are looking forward to your contributions to our mailing lists and >>> code repos :-) >>> >>> Good luck! >>> >>> On Tue, Apr 28, 2015 at 9:29 AM, Artur Dryomov >>> wrote: >>> >>>> Hi everyone, >>>> >>>> This year I will be working on the Hawkular Android client application >>>> as part of the Google Summer of Code 2015 program. >>>> >>>> The application itself will use Hawkular API and AeroGear SDK. In >>>> coming days I?ll research these areas, especially documentation, and will >>>> try to create some sort of architecture and basic design. >>>> >>>> Thank you all for this opportunity! >>>> Artur. >>>> >>>> _______________________________________________ >>>> aerogear-dev mailing list >>>> aerogear-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev >>>> >>> >>> >>> >>> -- >>> Matthias Wessendorf >>> >>> blog: http://matthiaswessendorf.wordpress.com/ >>> sessions: http://www.slideshare.net/mwessendorf >>> twitter: http://twitter.com/mwessendorf >>> >>> _______________________________________________ >>> aerogear-dev mailing list >>> aerogear-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/aerogear-dev >>> >> >> >> _______________________________________________ >> aerogear-dev mailing list >> aerogear-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/aerogear-dev >> > > > _______________________________________________ > aerogear-dev mailing list > aerogear-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150428/d895f97c/attachment.html From ppalaga at redhat.com Wed Apr 29 05:26:05 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Wed, 29 Apr 2015 11:26:05 +0200 Subject: [Hawkular-dev] Coming to Parent: publishing JavaDoc and sources via Maven In-Reply-To: <1833327.3eVncBMr9F@localhost.localdomain> References: <5519518D.7030308@redhat.com> <1073938809.6663690.1427748801386.JavaMail.zimbra@redhat.com> <583269998.7134666.1430181527605.JavaMail.zimbra@redhat.com> <1833327.3eVncBMr9F@localhost.localdomain> Message-ID: <5540A3AD.7030905@redhat.com> On 2015-04-28 13:50, Lukas Krejci wrote: > What about IDE support for Asciidoc docs? Me, I like being able to click on > links in javadoc popups. +1 for the question about the IDE support. Jared, do you know how does it work in Eclipse & Co? Regardless of the above, I do not see the need to introduce asciidoclet. There is very little plain HTML in typical good JavaDoc and this small amount is not worth the trouble, IMO. -- P > On Monday, April 27, 2015 20:38:47 Jared MORGAN wrote: >> I'm coming late to the party here, so my apologies for that. >> >> I understand that docs for Hawkular will be done in Asciidoc (great move >> IMHO). >> >> Why not extend that to Javadocs: https://github.com/asciidoctor/asciidoclet >> >> Compatible with Maven. No need to try and kludge HTML to make tables work. >> Easy to read directly in code as well as looking pretty when compiled. >> >> Thoughts? >> >> J >> >> ----- Original Message ----- >> >>>> Hi Jay, there is just a couple of them in Bus - see the attached file. >>>> -- P >>> >>> If everyone writes as good as javadocs as I do, I'm all for the javadoc >>> checker :-D >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jshaughn at redhat.com Wed Apr 29 09:16:37 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 29 Apr 2015 09:16:37 -0400 Subject: [Hawkular-dev] Coming to Parent: publishing JavaDoc and sources via Maven In-Reply-To: <5540A3AD.7030905@redhat.com> References: <5519518D.7030308@redhat.com> <1073938809.6663690.1427748801386.JavaMail.zimbra@redhat.com> <583269998.7134666.1430181527605.JavaMail.zimbra@redhat.com> <1833327.3eVncBMr9F@localhost.localdomain> <5540A3AD.7030905@redhat.com> Message-ID: <5540D9B5.8070608@redhat.com> Yes, good question. And I have concerns about how existing, or submitted jdoc, would be handled. Can you mix the two approaches? I agree with ppalaga, perhaps not worth the change. On 4/29/2015 5:26 AM, Peter Palaga wrote: > On 2015-04-28 13:50, Lukas Krejci wrote: >> What about IDE support for Asciidoc docs? Me, I like being able to click on >> links in javadoc popups. > +1 for the question about the IDE support. Jared, do you know how does > it work in Eclipse & Co? > > Regardless of the above, I do not see the need to introduce asciidoclet. > There is very little plain HTML in typical good JavaDoc and this small > amount is not worth the trouble, IMO. > > -- P > >> On Monday, April 27, 2015 20:38:47 Jared MORGAN wrote: >>> I'm coming late to the party here, so my apologies for that. >>> >>> I understand that docs for Hawkular will be done in Asciidoc (great move >>> IMHO). >>> >>> Why not extend that to Javadocs: https://github.com/asciidoctor/asciidoclet >>> >>> Compatible with Maven. No need to try and kludge HTML to make tables work. >>> Easy to read directly in code as well as looking pretty when compiled. >>> >>> Thoughts? >>> >>> J >>> >>> ----- Original Message ----- >>> >>>>> Hi Jay, there is just a couple of them in Bus - see the attached file. >>>>> -- P >>>> If everyone writes as good as javadocs as I do, I'm all for the javadoc >>>> checker :-D >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150429/d57452ad/attachment-0001.html From mwringe at redhat.com Wed Apr 29 12:05:16 2015 From: mwringe at redhat.com (Matt Wringe) Date: Wed, 29 Apr 2015 12:05:16 -0400 Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers In-Reply-To: <1499876691.3669790.1429806519900.JavaMail.zimbra@redhat.com> References: <55318CBC.8010105@redhat.com> <1890420879.2259162.1429538271466.JavaMail.zimbra@redhat.com> <55379F9B.4010307@redhat.com> <5537AD2F.5040806@redhat.com> <5538F51C.4040807@redhat.com> <1499876691.3669790.1429806519900.JavaMail.zimbra@redhat.com> Message-ID: <5541013C.3040609@redhat.com> On 23/04/15 12:28 PM, Viet Nguyen wrote: > Matt, > I have not been intimately involved with hawkular dev due to other obligations. I'm curious about your use case, running metrics as an independent app... as I always thought metrics, inventory, alert, etc are to meant to be assembled together. Ah, sorry I forgot to respond to this last week. All that we need in OpenShift for now is just the Hawkular Metrics part, we don't need the other components (at least not yet). There is also the problem where the other components are not setup to be run in a cluster yet. In terms of how things are meant to be run in the future, I am not sure if we would have a single application running all of Hawkular, or have it more modular with multiple pods running different components. > > I would be happy to assist with the Dockerhub build steps. Let's have a chat early next week? Basically I just need whomever is in control of the CI build process and in charge of the Hawkular docker hub account to help out with this since I don't have access to those accounts. Ideally we should just use the maven build setup and not the current docker hub automated build but if the docker hub automated build setup is easier to get things up and running then it might be good to use that for now. I will see if I can ping you on irc about all of this. - Matt > > > Viet > > > > ----- Original Message ----- > From: "Matt Wringe" > To: hawkular-dev at lists.jboss.org > Sent: Thursday, April 23, 2015 6:35:24 AM > Subject: Re: [Hawkular-dev] Hawkular Metrics Openshift Containers > > On 22/04/15 10:16 AM, Matt Wringe wrote: >> On 22/04/15 09:18 AM, Matt Wringe wrote: >>> On 20/04/15 09:57 AM, Viet Nguyen wrote: >>>> Matt, >>>> I was the last person who worked on that Docker build for Hawkular-metrics. Yes, it is obsolete and can be removed. >>>> >>>> FYI there's an automated Docker build of Kettle in DockerHub and a continuous deployment to a Kubernetes cluster >>>> >>>> Docker Hub: >>>> https://registry.hub.docker.com/u/hawkular/hawkular/ >>>> >>>> CI Pipeline: >>>> https://docs.google.com/presentation/d/14zqOHOvXAB4uN7W_fWOub3ZD6txiS_Vp9cjUNIlj2yQ/edit?usp=sharing >>> I went through and have the docker images being built using a docker >>> maven plugin. So during the maven build, you can create the image and >>> push it out to a registry. >>> >>> It works great on a local machine, one maven build and I have the docker >>> images and kubernetes application zip created. But looking at how the >>> current CI is done, it might not have been the best solution (it would >>> require the credentials of the docker hub user to be added into the CI >>> setup somewhere). >>> >>> It might make more sense to do it the way the current CI is setup with >>> the kettle builds. Anyone have any preferences? >> Either way, the CI build way is also easy to setup. You would just need >> to sets these up with the official accounts and add in the automated >> build hooks: >> >> https://registry.hub.docker.com/u/mwringe/docker-hawkular-cassandra/ >> https://github.com/mwringe/docker-hawkular-cassandra >> >> https://registry.hub.docker.com/u/mwringe/docker-hawkular-metrics/ >> https://github.com/mwringe/docker-hawkular-metrics > We really need to get this setup as part of the build so that we can > hand off the containers and kubernetes application. I can't set this up > myself as we need whomever is able to access the CI builds and docker > hub account to perform the setup. > > I have proposed two different setups, and they both should work for the > CI builds: We can either build it via the maven plugin in the CI builds, > or we can use the existing docker hub automated builds (like what is > done with the kettle docker images). > > The only thing about the maven build is that we need to store the user > credentials in the CI configuration somewhere and need access to a > machine capable of performing docker image builds. > > Everything else in the maven build is handled much better (everything > kept in one place, versions are automatically updated, names are kept in > sync between the docker images and kubernetes applications, can easier > push out to different docker registries, much better developer > experience for local builds, etc). > > The docker hub automated builds are probably faster to setup right now > though since we already know we have everything working for it. > > Note: the maven docker setup is not going away, we need that for > developer builds (either locally or pushing out to developer own docker > hub account). The docker hub automated builds are essentially useless > from a developer's perspective, but they work fine for CI builds. > >>>> Viet >>>> >>>> >>>> >>>> >>>> ----- Original Message ----- >>>> From: "Matt Wringe" >>>> To: hawkular-dev at lists.jboss.org >>>> Sent: Friday, April 17, 2015 3:44:12 PM >>>> Subject: [Hawkular-dev] Hawkular Metrics Openshift Containers >>>> >>>> I have a new subproject in Hawkular Metrics which sets up creating >>>> components for Openshift/Fabric8 >>>> (https://github.com/hawkular/hawkular-metrics/pull/200). >>>> >>>> There are 3 main parts >>>> >>>> Cassandra: creates a custom seed provider to support >>>> ReplicationControllers in Kubernetes, creates a folder/zip archive which >>>> can be used to generate a Docker image. It may make sense to move the >>>> Cassandra parts out to a separate project. >>>> >>>> Hawkular Metrics: creates a folder/zip archive which can be used to >>>> generate a Docker image >>>> >>>> Kubernetes: pulls everything together into a single kubernetes >>>> application. Can be used to deploy an application zip into fabric8 (via >>>> drag and drop in the web console or via the maven plugin) or deploy all >>>> the components into Openshift via the kubernetes.json configuration file. >>>> >>>> The docker images are not created and deployed to a docker registry as >>>> part of the build, it will just create a folder where you can run the >>>> docker build from. None of the maven docker plugins I looked at seemed >>>> to really work properly, so its still a manual process to do the build >>>> (and push to a registry). Its something which needs to be improved. >>>> >>>> The Cassandra service currently only supports adding new nodes to a >>>> cluster and not removing them via the ReplicationController. This is due >>>> to the replication factor being set to be 1 by default (which means when >>>> a node is removed, so is the data it contained). >>>> >>>> I believe the docker subproject of hawkular metrics is obsolete and can >>>> be removed >>>> (https://github.com/hawkular/hawkular-metrics/tree/master/docker), but >>>> someone please correct me if I am wrong. It's scripts are referring to >>>> the console which no longer exists as part of the project. >>>> >>>> - Matt >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jsanda at redhat.com Wed Apr 29 12:28:50 2015 From: jsanda at redhat.com (John Sanda) Date: Wed, 29 Apr 2015 12:28:50 -0400 Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> Message-ID: Will the REST APIs for other hawkular services take a similar approach? This seems like an area where we want to be consistent across APIs. > On Apr 28, 2015, at 3:58 PM, Lukas Krejci wrote: > > How do metrics' tenants fit into the hawkular accounts and its persona concept? > > ----- Original Message ----- >> From: "Stefan Negrea" >> To: "Discussions" >> Sent: Tuesday, 28 April, 2015 5:44:56 PM >> Subject: [Hawkular-dev] Tenant Id - Not Part of URL >> >> Hello Everybody, >> >> I've been working on a PR for the upcoming Hawkular Metrics release that will >> remove the tenant id from the end-point URLs. The tenant id will be moved to >> either a header parameter or a query parameter. The query parameter is in >> place for cases (such as curl) where setting a header is not possible, >> difficult, or inconvenient. >> >> Here is an example of the change: >> >> Existing URL: >> /{tenantId}/gauge/{metricId}/data >> >> New URL: >> /gauge/{metricId}/data >> >> Tenant id set via: >> 1) header - tenantId >> 2) query parameter - tenantId >> >> >> There are two exceptions to this rule, /tenants and /db/{tenantid}/series. >> The /tenants end-point will be changed into something different in the >> upcoming releases since it is mostly a management type API that does not >> belong in the same place with the regular metrics endpoint. And >> /db/{tenantid}/series end-point is needed in this exact format for >> compatibility with Influxdb compatible services. >> >> >> Now, to the merits of this change. The tenant id is volatile, can change any >> time, and changes to it should be expected; but the rest of the URL is >> fixed. The second issue is that the tenant id is a security concern. So we >> were limited in design choices since a security concern was leaking as part >> of the URL. >> >> So removing the tenant id from the URL will give us permanent & consistent >> addresses for resources (metrics and metric data points). And we will gain a >> lot of flexibility on the security side. In the future, users could >> authenticate with a user/pass combo and the backend would transform that >> into a tenant id to be used on the request. If the same user later decides >> to use a tenant id to pass along the request, the URL of the resources would >> not change. Another expectation is that tenant id is not sufficient, it is >> typically a combo of id + secret; so we would have resorted to a header or >> query param for the second piece of information (the secret). >> >> This change will give us the flexibility to adjust the security model (the >> meaning of tenant ids and ways to validate them) without compromising the >> URL structure. This will help Hawkular Metrics as it gets integrated into >> more and more projects and products. >> >> Here are the links to the JIRA and the PR for this change: >> https://github.com/hawkular/hawkular-metrics/pull/202 >> https://issues.jboss.org/browse/HWKMETRICS-68 >> >> >> >> Thank you, >> Stefan Negrea >> >> Software Engineer >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From lkrejci at redhat.com Wed Apr 29 12:46:33 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Wed, 29 Apr 2015 18:46:33 +0200 Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> Message-ID: <2426339.6kgTx6A8MY@localhost.localdomain> Well, for inventory, we're considering dropping the context of a tenant altogether. Accounts don't work with a tenant in a usual meaning of that word very well - in accounts there is nothing like an isolated "island" that cannot be accessed by anyone outside of that island. Instead, accounts work with a hierarchy of organizations, of which users can be members of (and have different roles in) and impersonate them. In inventory, we're using tenants rather loosely - while they form a root of the path to any other entity in inventory, it is not disallowed to link entities from different tenants. A single tenant as a root of the hierarchy doesn't correspond well to the hierarchical nature of accounts organizations. So instead, to offer similar, if disconnected, approach to structuring things, we are thinking about replacing a single tenant with a hierarchy of "partitions" or "locations" (the nomenclature is not settled as isn't the concept itself ;) ). The thinking behind it is that while orgs model the departmental and security structure of a company, the partitions/locations would/could model the physical location of machines in the company's infrastructure (/geo5/datacenterA/...) or they could model a logical structure of the machines (/middleware/client5/...), or they even could mirror the security structure of orgs. In any case, given the fact that either tenant or any of its successors are iherent part of a an ID of any entity (entities in inventory are identified by their path, not by a unique ID), I think we will have to keep them in the URI's, too, as it would not make sense to address things like: http://.../EAP-5?tenantId=x because http://.../EAP-5?tenantId=y is most probably an utterly different thing that is completely unrelated to the first one. On Wednesday, April 29, 2015 12:28:50 John Sanda wrote: > Will the REST APIs for other hawkular services take a similar approach? This > seems like an area where we want to be consistent across APIs. > > On Apr 28, 2015, at 3:58 PM, Lukas Krejci wrote: > > > > How do metrics' tenants fit into the hawkular accounts and its persona > > concept? > > > > ----- Original Message ----- > > > >> From: "Stefan Negrea" > >> To: "Discussions" > >> Sent: Tuesday, 28 April, 2015 5:44:56 PM > >> Subject: [Hawkular-dev] Tenant Id - Not Part of URL > >> > >> Hello Everybody, > >> > >> I've been working on a PR for the upcoming Hawkular Metrics release that > >> will remove the tenant id from the end-point URLs. The tenant id will be > >> moved to either a header parameter or a query parameter. The query > >> parameter is in place for cases (such as curl) where setting a header is > >> not possible, difficult, or inconvenient. > >> > >> Here is an example of the change: > >> > >> Existing URL: > >> /{tenantId}/gauge/{metricId}/data > >> > >> New URL: > >> /gauge/{metricId}/data > >> > >> Tenant id set via: > >> 1) header - tenantId > >> 2) query parameter - tenantId > >> > >> > >> There are two exceptions to this rule, /tenants and > >> /db/{tenantid}/series. > >> The /tenants end-point will be changed into something different in the > >> upcoming releases since it is mostly a management type API that does not > >> belong in the same place with the regular metrics endpoint. And > >> /db/{tenantid}/series end-point is needed in this exact format for > >> compatibility with Influxdb compatible services. > >> > >> > >> Now, to the merits of this change. The tenant id is volatile, can change > >> any time, and changes to it should be expected; but the rest of the URL > >> is fixed. The second issue is that the tenant id is a security concern. > >> So we were limited in design choices since a security concern was > >> leaking as part of the URL. > >> > >> So removing the tenant id from the URL will give us permanent & > >> consistent > >> addresses for resources (metrics and metric data points). And we will > >> gain a lot of flexibility on the security side. In the future, users > >> could authenticate with a user/pass combo and the backend would > >> transform that into a tenant id to be used on the request. If the same > >> user later decides to use a tenant id to pass along the request, the URL > >> of the resources would not change. Another expectation is that tenant id > >> is not sufficient, it is typically a combo of id + secret; so we would > >> have resorted to a header or query param for the second piece of > >> information (the secret). > >> > >> This change will give us the flexibility to adjust the security model > >> (the > >> meaning of tenant ids and ways to validate them) without compromising the > >> URL structure. This will help Hawkular Metrics as it gets integrated into > >> more and more projects and products. > >> > >> Here are the links to the JIRA and the PR for this change: > >> https://github.com/hawkular/hawkular-metrics/pull/202 > >> https://issues.jboss.org/browse/HWKMETRICS-68 > >> > >> > >> > >> Thank you, > >> Stefan Negrea > >> > >> Software Engineer > >> > >> _______________________________________________ > >> hawkular-dev mailing list > >> hawkular-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jpkroehling at redhat.com Wed Apr 29 13:01:48 2015 From: jpkroehling at redhat.com (=?windows-1252?Q?Juraci_Paix=E3o_Kr=F6hling?=) Date: Wed, 29 Apr 2015 19:01:48 +0200 Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <2426339.6kgTx6A8MY@localhost.localdomain> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> <2426339.6kgTx6A8MY@localhost.localdomain> Message-ID: <55410E7C.6030906@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/29/2015 06:46 PM, Lukas Krejci wrote: > Well, for inventory, we're considering dropping the context of a > tenant altogether. > > Accounts don't work with a tenant in a usual meaning of that word > very well - in accounts there is nothing like an isolated "island" > that cannot be accessed by anyone outside of that island. > > Instead, accounts work with a hierarchy of organizations, of which > users can be members of (and have different roles in) and > impersonate them. Actually, this is a requirement we got based on the demos and discussions that followed them. Accounts can be changed to fit whatever definition of tenants that we might want. The use case that led to the current model is: - - Company "Acme, Inc" has the following departments: - -- finances - -- marketing - -- operations Finances cannot see the data from Marketing and vice-versa. Neither finances nor marketing can see data from operations, but operations should be able to view and/or manage data from both, including its own. As we also have "users", the concept of tenant in accounts is then an abstraction of "users" or "organizations" (company, department, ...). It's called "persona" because an user might be impersonating a company when doing some action (ie: jdoe is creating an EAP instance on behalf of "Acme, inc"). So, as far as the end component (metrics, inventory, ...) is concerned, all operations belonging to "Acme, Inc" are done by "Acme, Inc" (who the final user is shouldn't be relevant to the component). > In inventory, we're using tenants rather loosely - while they form > a root of the path to any other entity in inventory, it is not > disallowed to link entities from different tenants. I believe you know this better than I do, but just make sure that this model would fit the use case above. - - Juca. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVQQ58AAoJECKM1e+fkPrXhd8IAIOGChSGBaWbDcEBjbK1Hr97 yMbUV8PYDO2b8dtJ0x5B7yntMk/eD03NsSWHJQuglljA+V3m5gkgiUTgzZUgV9/j h25z6HrtbevMGwEE275Jva9/UiePNY02dShN6X9gk/3I3AXz8NnJ7m6ZM3jIt4tB AlD/t7Z9/eOlnNpjpIAvwwXh/aSUPkWzbH2gWVIkCxQVUNplpsThwyPcLmtqbvMd 2H5mp7WkToUK3XFxvN+bBHYAAdBHOJ+sPRrgJGg87Jg/KXbr+7WhZETzDn9euqoa m3Iu3B7Itd+UcHGAaTlVHrQMzBx61Jn6TrPeZRt5ekx9j92tKSGX+3kOcF0KyaQ= =xZJy -----END PGP SIGNATURE----- From vrockai at redhat.com Wed Apr 29 13:09:54 2015 From: vrockai at redhat.com (Viliam Rockai) Date: Wed, 29 Apr 2015 19:09:54 +0200 Subject: [Hawkular-dev] Inventory REST now has result sets pagination In-Reply-To: <2159243.DYWF8jYFne@localhost.localdomain> References: <2159243.DYWF8jYFne@localhost.localdomain> Message-ID: <1430327394.11457.3.camel@vrockai-laptop> Hi Lukas, It seems you're returning multiple "Link:" headers with URLs. I believe a single Link header with a list is better option. For more info check-out the [update] section here: http://pilhuhn.blogspot.de/2013/02/best-practice-for-paging-in-restful-apis.html IIRC we do get access to headers in angular $resource as to a map object. So angular devs may have the same issue as stated in the blog (only single value returned once "Link" key is accessed). Viliam On Tue, 2015-04-28 at 11:43 +0200, Lukas Krejci wrote: > Hi, > > I went ahead and prototyped the pagination support both in the backend and the > REST API within the boundaries of inventory. > > Because this is a generic feature, it's been agreed that this stuff would > deserve a place of its own so that it can be shared amongst the various > hawkular components. > > There are 2 aspects of the paging: > > * support in backend - This basically only revolves around the specification > of the paging requirements and a list "wrapper" that adds paging context to > the results. The way how the paging will be implemented will differ in each > component. > https://github.com/hawkular/hawkular-inventory/tree/master/api/src/main/java/org/hawkular/inventory/api/paging > > * support in the REST API - This means to translate the query parameters with > the paging specification into paging java API and transform the results from > the backend into the set of headers attached to the response. > > The support for paging in REST in inventory is contained in these classes: > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/RequestUtil.java > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/ResponseUtil.java > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/json/Link.java > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/json/LinkDeserializer.java > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/src/main/java/org/hawkular/inventory/rest/json/LinkSerializer.java > > Both the backend and the REST support were either taken from or were heavily > inspired by the paging support in RHQ. > > Where do you think these should live? Do we have a repo for common stuff? I > imagine we will have to have 2 separate common modules so that we don't drag > jaxrs deps into core components. > > Cheers, > > Lukas > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From lkrejci at redhat.com Wed Apr 29 15:06:15 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Wed, 29 Apr 2015 21:06:15 +0200 Subject: [Hawkular-dev] Inventory REST now has result sets pagination In-Reply-To: <1430327394.11457.3.camel@vrockai-laptop> References: <2159243.DYWF8jYFne@localhost.localdomain> <1430327394.11457.3.camel@vrockai-laptop> Message-ID: <130032455.vQh4D99tty@localhost.localdomain> I merely copied the approach we used in RHQ, but yes, we definitely should look into making it better. I'll change that to be a single Link header with a list of URIs+rels as outlined in https://tools.ietf.org/html/rfc5988#section-5.5 Thanks for bringing that up! On Wednesday, April 29, 2015 19:09:54 Viliam Rockai wrote: > Hi Lukas, > > It seems you're returning multiple "Link:" headers with URLs. I believe > a single Link header with a list is better option. For more info > check-out the [update] section here: > http://pilhuhn.blogspot.de/2013/02/best-practice-for-paging-in-restful-apis. > html > > IIRC we do get access to headers in angular $resource as to a map > object. So angular devs may have the same issue as stated in the blog > (only single value returned once "Link" key is accessed). > > Viliam > > On Tue, 2015-04-28 at 11:43 +0200, Lukas Krejci wrote: > > Hi, > > > > I went ahead and prototyped the pagination support both in the backend and > > the REST API within the boundaries of inventory. > > > > Because this is a generic feature, it's been agreed that this stuff would > > deserve a place of its own so that it can be shared amongst the various > > hawkular components. > > > > There are 2 aspects of the paging: > > > > * support in backend - This basically only revolves around the > > specification of the paging requirements and a list "wrapper" that adds > > paging context to the results. The way how the paging will be implemented > > will differ in each component. > > https://github.com/hawkular/hawkular-inventory/tree/master/api/src/main/ja > > va/org/hawkular/inventory/api/paging > > > > * support in the REST API - This means to translate the query parameters > > with the paging specification into paging java API and transform the > > results from the backend into the set of headers attached to the > > response. > > > > The support for paging in REST in inventory is contained in these classes: > > > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/sr > > c/main/java/org/hawkular/inventory/rest/RequestUtil.java > > > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/sr > > c/main/java/org/hawkular/inventory/rest/ResponseUtil.java > > > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/sr > > c/main/java/org/hawkular/inventory/rest/json/Link.java > > > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/sr > > c/main/java/org/hawkular/inventory/rest/json/LinkDeserializer.java > > > > https://github.com/hawkular/hawkular-inventory/blob/master/rest-servlet/sr > > c/main/java/org/hawkular/inventory/rest/json/LinkSerializer.java > > > > Both the backend and the REST support were either taken from or were > > heavily inspired by the paging support in RHQ. > > > > Where do you think these should live? Do we have a repo for common stuff? > > I > > imagine we will have to have 2 separate common modules so that we don't > > drag jaxrs deps into core components. > > > > Cheers, > > > > Lukas > > > > > > > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jshaughn at redhat.com Wed Apr 29 17:30:35 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 29 Apr 2015 17:30:35 -0400 Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> Message-ID: <55414D7B.3040206@redhat.com> And for Alerting we do not yet have an idea of the accounts/authz we even need. As such, we have no tenants in the REST API nor any accounts integration. Any thoughts appreciated, It may be something we can start to figure out at the F2F. On 4/29/2015 12:28 PM, John Sanda wrote: > Will the REST APIs for other hawkular services take a similar approach? This seems like an area where we want to be consistent across APIs. > >> On Apr 28, 2015, at 3:58 PM, Lukas Krejci wrote: >> >> How do metrics' tenants fit into the hawkular accounts and its persona concept? >> >> ----- Original Message ----- >>> From: "Stefan Negrea" >>> To: "Discussions" >>> Sent: Tuesday, 28 April, 2015 5:44:56 PM >>> Subject: [Hawkular-dev] Tenant Id - Not Part of URL >>> >>> Hello Everybody, >>> >>> I've been working on a PR for the upcoming Hawkular Metrics release that will >>> remove the tenant id from the end-point URLs. The tenant id will be moved to >>> either a header parameter or a query parameter. The query parameter is in >>> place for cases (such as curl) where setting a header is not possible, >>> difficult, or inconvenient. >>> >>> Here is an example of the change: >>> >>> Existing URL: >>> /{tenantId}/gauge/{metricId}/data >>> >>> New URL: >>> /gauge/{metricId}/data >>> >>> Tenant id set via: >>> 1) header - tenantId >>> 2) query parameter - tenantId >>> >>> >>> There are two exceptions to this rule, /tenants and /db/{tenantid}/series. >>> The /tenants end-point will be changed into something different in the >>> upcoming releases since it is mostly a management type API that does not >>> belong in the same place with the regular metrics endpoint. And >>> /db/{tenantid}/series end-point is needed in this exact format for >>> compatibility with Influxdb compatible services. >>> >>> >>> Now, to the merits of this change. The tenant id is volatile, can change any >>> time, and changes to it should be expected; but the rest of the URL is >>> fixed. The second issue is that the tenant id is a security concern. So we >>> were limited in design choices since a security concern was leaking as part >>> of the URL. >>> >>> So removing the tenant id from the URL will give us permanent & consistent >>> addresses for resources (metrics and metric data points). And we will gain a >>> lot of flexibility on the security side. In the future, users could >>> authenticate with a user/pass combo and the backend would transform that >>> into a tenant id to be used on the request. If the same user later decides >>> to use a tenant id to pass along the request, the URL of the resources would >>> not change. Another expectation is that tenant id is not sufficient, it is >>> typically a combo of id + secret; so we would have resorted to a header or >>> query param for the second piece of information (the secret). >>> >>> This change will give us the flexibility to adjust the security model (the >>> meaning of tenant ids and ways to validate them) without compromising the >>> URL structure. This will help Hawkular Metrics as it gets integrated into >>> more and more projects and products. >>> >>> Here are the links to the JIRA and the PR for this change: >>> https://github.com/hawkular/hawkular-metrics/pull/202 >>> https://issues.jboss.org/browse/HWKMETRICS-68 >>> >>> >>> >>> Thank you, >>> Stefan Negrea >>> >>> Software Engineer >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150429/72a53ca9/attachment.html From snegrea at redhat.com Wed Apr 29 19:12:04 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Wed, 29 Apr 2015 19:12:04 -0400 (EDT) Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> Message-ID: <396872469.9554315.1430349124044.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Lukas Krejci" > To: "Discussions around Hawkular development" > Sent: Tuesday, April 28, 2015 2:58:47 PM > Subject: Re: [Hawkular-dev] Tenant Id - Not Part of URL > > How do metrics' tenants fit into the hawkular accounts and its persona > concept? This change will get Hawkular Metrics in a position to answer that hard question later; no decision needs to be made before the actual integration. Hawkular Metrics will never drop the idea of a tenant internally. It is essential to the way metrics are partitioned and uniquely identified. However, the mapping between external services and internal tenant ids should not leak in the URL of a resource. That kind of leak would be limiting the design choices for integration. This is a general answer, not just for the Hawkular Accounts integration. Now, specifically to your question, I see personas to be mapped directly to tenant ids. Hawkular Metrics does not need a very complex authorization model. Anybody with the right credentials for a tenant id gets access to all the metrics for that tenant id. When the integration with Hawkular Accounts happens, Metrics would just need to write code to delegate the authentication to Hawkular Accounts, obtain the personas details, and then map personas to internal tenant ids. And this is done without affecting URLs (completely outside of resource addressing). Thank you, Stefan > > ----- Original Message ----- > > From: "Stefan Negrea" > > To: "Discussions" > > Sent: Tuesday, 28 April, 2015 5:44:56 PM > > Subject: [Hawkular-dev] Tenant Id - Not Part of URL > > > > Hello Everybody, > > > > I've been working on a PR for the upcoming Hawkular Metrics release that > > will > > remove the tenant id from the end-point URLs. The tenant id will be moved > > to > > either a header parameter or a query parameter. The query parameter is in > > place for cases (such as curl) where setting a header is not possible, > > difficult, or inconvenient. > > > > Here is an example of the change: > > > > Existing URL: > > /{tenantId}/gauge/{metricId}/data > > > > New URL: > > /gauge/{metricId}/data > > > > Tenant id set via: > > 1) header - tenantId > > 2) query parameter - tenantId > > > > > > There are two exceptions to this rule, /tenants and /db/{tenantid}/series. > > The /tenants end-point will be changed into something different in the > > upcoming releases since it is mostly a management type API that does not > > belong in the same place with the regular metrics endpoint. And > > /db/{tenantid}/series end-point is needed in this exact format for > > compatibility with Influxdb compatible services. > > > > > > Now, to the merits of this change. The tenant id is volatile, can change > > any > > time, and changes to it should be expected; but the rest of the URL is > > fixed. The second issue is that the tenant id is a security concern. So we > > were limited in design choices since a security concern was leaking as part > > of the URL. > > > > So removing the tenant id from the URL will give us permanent & consistent > > addresses for resources (metrics and metric data points). And we will gain > > a > > lot of flexibility on the security side. In the future, users could > > authenticate with a user/pass combo and the backend would transform that > > into a tenant id to be used on the request. If the same user later decides > > to use a tenant id to pass along the request, the URL of the resources > > would > > not change. Another expectation is that tenant id is not sufficient, it is > > typically a combo of id + secret; so we would have resorted to a header or > > query param for the second piece of information (the secret). > > > > This change will give us the flexibility to adjust the security model (the > > meaning of tenant ids and ways to validate them) without compromising the > > URL structure. This will help Hawkular Metrics as it gets integrated into > > more and more projects and products. > > > > Here are the links to the JIRA and the PR for this change: > > https://github.com/hawkular/hawkular-metrics/pull/202 > > https://issues.jboss.org/browse/HWKMETRICS-68 > > > > > > > > Thank you, > > Stefan Negrea > > > > Software Engineer > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From theute at redhat.com Thu Apr 30 02:23:09 2015 From: theute at redhat.com (Thomas Heute) Date: Thu, 30 Apr 2015 02:23:09 -0400 (EDT) Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <396872469.9554315.1430349124044.JavaMail.zimbra@redhat.com> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> <396872469.9554315.1430349124044.JavaMail.zimbra@redhat.com> Message-ID: <724166753.10695424.1430374989776.JavaMail.zimbra@redhat.com> Just a note to say that Account integration needs to happen sooner rather than later, this is blocker for the MVP due ... last week Also let's make sure this is consistent. - All components need to support the same multitenancy model. We may also need to adapt it to various places where Hawkular will be used (with OpenShift, "standalone", with FeedHenry...) so would help if we can keep each service relatively dumb about the model and keep the logic in Account. - URL vs Header, I kinda lean toward header, but more importantly we need consistency so component leads need to agree on 1 way. Thomas ----- Original Message ----- From: "Stefan Negrea" To: "Discussions around Hawkular development" Sent: Thursday, April 30, 2015 1:12:04 AM Subject: Re: [Hawkular-dev] Tenant Id - Not Part of URL ----- Original Message ----- > From: "Lukas Krejci" > To: "Discussions around Hawkular development" > Sent: Tuesday, April 28, 2015 2:58:47 PM > Subject: Re: [Hawkular-dev] Tenant Id - Not Part of URL > > How do metrics' tenants fit into the hawkular accounts and its persona > concept? This change will get Hawkular Metrics in a position to answer that hard question later; no decision needs to be made before the actual integration. Hawkular Metrics will never drop the idea of a tenant internally. It is essential to the way metrics are partitioned and uniquely identified. However, the mapping between external services and internal tenant ids should not leak in the URL of a resource. That kind of leak would be limiting the design choices for integration. This is a general answer, not just for the Hawkular Accounts integration. Now, specifically to your question, I see personas to be mapped directly to tenant ids. Hawkular Metrics does not need a very complex authorization model. Anybody with the right credentials for a tenant id gets access to all the metrics for that tenant id. When the integration with Hawkular Accounts happens, Metrics would just need to write code to delegate the authentication to Hawkular Accounts, obtain the personas details, and then map personas to internal tenant ids. And this is done without affecting URLs (completely outside of resource addressing). Thank you, Stefan > > ----- Original Message ----- > > From: "Stefan Negrea" > > To: "Discussions" > > Sent: Tuesday, 28 April, 2015 5:44:56 PM > > Subject: [Hawkular-dev] Tenant Id - Not Part of URL > > > > Hello Everybody, > > > > I've been working on a PR for the upcoming Hawkular Metrics release that > > will > > remove the tenant id from the end-point URLs. The tenant id will be moved > > to > > either a header parameter or a query parameter. The query parameter is in > > place for cases (such as curl) where setting a header is not possible, > > difficult, or inconvenient. > > > > Here is an example of the change: > > > > Existing URL: > > /{tenantId}/gauge/{metricId}/data > > > > New URL: > > /gauge/{metricId}/data > > > > Tenant id set via: > > 1) header - tenantId > > 2) query parameter - tenantId > > > > > > There are two exceptions to this rule, /tenants and /db/{tenantid}/series. > > The /tenants end-point will be changed into something different in the > > upcoming releases since it is mostly a management type API that does not > > belong in the same place with the regular metrics endpoint. And > > /db/{tenantid}/series end-point is needed in this exact format for > > compatibility with Influxdb compatible services. > > > > > > Now, to the merits of this change. The tenant id is volatile, can change > > any > > time, and changes to it should be expected; but the rest of the URL is > > fixed. The second issue is that the tenant id is a security concern. So we > > were limited in design choices since a security concern was leaking as part > > of the URL. > > > > So removing the tenant id from the URL will give us permanent & consistent > > addresses for resources (metrics and metric data points). And we will gain > > a > > lot of flexibility on the security side. In the future, users could > > authenticate with a user/pass combo and the backend would transform that > > into a tenant id to be used on the request. If the same user later decides > > to use a tenant id to pass along the request, the URL of the resources > > would > > not change. Another expectation is that tenant id is not sufficient, it is > > typically a combo of id + secret; so we would have resorted to a header or > > query param for the second piece of information (the secret). > > > > This change will give us the flexibility to adjust the security model (the > > meaning of tenant ids and ways to validate them) without compromising the > > URL structure. This will help Hawkular Metrics as it gets integrated into > > more and more projects and products. > > > > Here are the links to the JIRA and the PR for this change: > > https://github.com/hawkular/hawkular-metrics/pull/202 > > https://issues.jboss.org/browse/HWKMETRICS-68 > > > > > > > > Thank you, > > Stefan Negrea > > > > Software Engineer > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From jpkroehling at redhat.com Thu Apr 30 02:58:04 2015 From: jpkroehling at redhat.com (=?windows-1252?Q?Juraci_Paix=E3o_Kr=F6hling?=) Date: Thu, 30 Apr 2015 08:58:04 +0200 Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <724166753.10695424.1430374989776.JavaMail.zimbra@redhat.com> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <555395784.9520692.1430251127156.JavaMail.zimbra@redhat.com> <396872469.9554315.1430349124044.JavaMail.zimbra@redhat.com> <724166753.10695424.1430374989776.JavaMail.zimbra@redhat.com> Message-ID: <5541D27C.7000005@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/30/2015 08:23 AM, Thomas Heute wrote: > Just a note to say that Account integration needs to happen sooner > rather than later, this is blocker for the MVP due ... last week > > Also let's make sure this is consistent. - All components need to > support the same multitenancy model. We may also need to adapt it > to various places where Hawkular will be used (with OpenShift, > "standalone", with FeedHenry...) so would help if we can keep each > service relatively dumb about the model and keep the logic in > Account. - URL vs Header, I kinda lean toward header, but more > importantly we need consistency so component leads need to agree on > 1 way. With that in mind, the individual components need not to worry on whether the tenant information is coming from path or header, as accounts is able to inject a Persona instance into your REST service/managed bean via CDI. As it is right now, Accounts decides on the Persona (tenant) based on the data from the request already, and this might either be the currently logged in user (from Keycloak) or an information from HTTP header (for instance, if the user selected an "organization" on the account switcher). - - Juca. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVQdJ8AAoJECKM1e+fkPrXioIH+wSAGUIkykiIBaDY2Jxl1bQI imVdRe/51g4ie+/t2bfMTdA5tV3SghNdXYiAG8vxUw8y0iZIwOh4/ZpXm2PdHyMo FbTwfkGrl6YpVMFlcFV3xEJig4QQa3bVRykkbDNXawPIA3VLDkA70e6xx+ji/xfO BO7++So1pfwMzTT5QCZOoBiAoETX7n1fMIK/Q/ZYuFb42mQGHvMwm3lXH5CZQIAN 18bk2KYIa4gB30YddB5JxY0wbj1Z3WpGRPTT6MWluMJ9+A2DtM7weN2bMqJKOfz8 0wYlewpzzoxB9Udubf5Hwl6ThwqF2VkgZgyfpsdGi5Me3s+KXx8ZqmlF5AysVEk= =yLC/ -----END PGP SIGNATURE----- From ppalaga at redhat.com Thu Apr 30 03:27:58 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Thu, 30 Apr 2015 09:27:58 +0200 Subject: [Hawkular-dev] Coming to Parent: publishing JavaDoc and sources via Maven In-Reply-To: <5519A810.6000901@redhat.com> References: <5519518D.7030308@redhat.com> <55196003.9090202@redhat.com> <5519A53D.2020005@redhat.com> <5519A810.6000901@redhat.com> Message-ID: <5541D97E.50308@redhat.com> Hi *, I have checked the impact of https://github.com/hawkular/hawkular-parent-pom/pull/20 in Alerts, Inventory, Agent, Accounts and Metrics. It works everywhere, producing just a few warnings that can be fixed later. Therefore, I merged the above PR. -- P On 2015-03-30 21:46, Peter Palaga wrote: > On 03/30/2015 09:34 PM, Peter Palaga wrote: >> Hi Jay, there is just a couple of them in Bus - see the attached file. -- P > > There is no single warning about missing JavaDoc. All the warnings are > about inconsistencies and broken syntax inside JavaDoc which indeed is > worth fixing. -- P > >> On 03/30/2015 04:38 PM, Jay Shaughnessy wrote: >>> >>> Hi Peter, to get an idea of the type and volume of issues this will >>> report, can you list the current bus warnings you're seeing? >>> >>> >>> On 3/30/2015 9:37 AM, Peter Palaga wrote: >>>> Hi *, there is a question about handling JavaDoc checks violations at >>>> the bottom. >>>> >>>> I have added maven-javadoc-plugin and maven-source-plugin to the new >>>> "release" profile in Hawkular Parent. >>>> https://github.com/hawkular/hawkular-parent-pom/pull/20 >>>> >>>> I have tested the above only with Hawkular Bus. Therefore, please test >>>> the setup with your Hawkular component: >>>> * Make whatever is appropriate to make your component use the pom from >>>> https://github.com/hawkular/hawkular-parent-pom/pull/20 >>>> * Invoke >>>> >>>> mvn clean install -Prelease >>>> >>>> * Figure out, how many "[WARNING] Javadoc Warnings" are there. >>>> * Look into target folders if *-javadoc.jar and *-sources.jar files were >>>> generated >>>> >>>> >>>> == Handling JavaDoc checks violations >>>> >>>> The present setup turns all javadoc violations into warnings using >>>> -Xdoclint:none [1]. I can only say that there *are* violations in Bus. >>>> >>>> I think that we should aim at having no violations at all - i.e. that >>>> the components should be given a week or two to fix and that we should >>>> switch to -Xdoclint:all after that. WDYT? >>>> >>>> More about -Xdoclint can be found in [2]. >>>> >>>> Thanks, >>>> >>>> Peter >>>> >>>> [1] >>>> https://github.com/hawkular/hawkular-parent-pom/commit/d54a8d03b4ef251d594f1cc4ff3fadfa4a1d4dd3#diff-600376dffeb79835ede4a0b285078036R110 >>>> >>>> >>>> [2]http://docs.oracle.com/javase/8/docs/technotes/tools/unix/javac.html >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>>> >>>> >>> >>> >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From lkrejci at redhat.com Thu Apr 30 05:17:12 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Thu, 30 Apr 2015 11:17:12 +0200 Subject: [Hawkular-dev] Tenant Id - Not Part of URL In-Reply-To: <724166753.10695424.1430374989776.JavaMail.zimbra@redhat.com> References: <1799616218.8541357.1430235896155.JavaMail.zimbra@redhat.com> <396872469.9554315.1430349124044.JavaMail.zimbra@redhat.com> <724166753.10695424.1430374989776.JavaMail.zimbra@redhat.com> Message-ID: <5203459.ly8QEiXKIM@localhost.localdomain> >From what I understood from the discussions, we essentially hand over all the "multitenancy" decisions to accounts which can emulate the tenant separation based on orgs (and at the same time support visibility across tenants for SaaS operator type of personas). As such, inventory basically gave up on supporting multitenancy in and of itself and rather leaves multitenancy only as an authorization concept. I can't comment on metrics' need for having a "tenant" if it is not used for addressing individual metrics. On Thursday, April 30, 2015 02:23:09 Thomas Heute wrote: > Just a note to say that Account integration needs to happen sooner rather > than later, this is blocker for the MVP due ... last week > > Also let's make sure this is consistent. > - All components need to support the same multitenancy model. We may also > need to adapt it to various places where Hawkular will be used (with > OpenShift, "standalone", with FeedHenry...) so would help if we can keep > each service relatively dumb about the model and keep the logic in Account. > - URL vs Header, I kinda lean toward header, but more importantly we need > consistency so component leads need to agree on 1 way. > > Thomas > > > ----- Original Message ----- > From: "Stefan Negrea" > To: "Discussions around Hawkular development" > Sent: Thursday, April 30, 2015 1:12:04 AM > Subject: Re: [Hawkular-dev] Tenant Id - Not Part of URL > > > > ----- Original Message ----- > > > From: "Lukas Krejci" > > To: "Discussions around Hawkular development" > > Sent: Tuesday, April 28, 2015 2:58:47 PM > > Subject: Re: [Hawkular-dev] Tenant Id - Not Part of URL > > > > How do metrics' tenants fit into the hawkular accounts and its persona > > concept? > > This change will get Hawkular Metrics in a position to answer that hard > question later; no decision needs to be made before the actual integration. > Hawkular Metrics will never drop the idea of a tenant internally. It is > essential to the way metrics are partitioned and uniquely identified. > However, the mapping between external services and internal tenant ids > should not leak in the URL of a resource. That kind of leak would be > limiting the design choices for integration. This is a general answer, not > just for the Hawkular Accounts integration. > > Now, specifically to your question, I see personas to be mapped directly to > tenant ids. Hawkular Metrics does not need a very complex authorization > model. Anybody with the right credentials for a tenant id gets access to > all the metrics for that tenant id. When the integration with Hawkular > Accounts happens, Metrics would just need to write code to delegate the > authentication to Hawkular Accounts, obtain the personas details, and then > map personas to internal tenant ids. And this is done without affecting > URLs (completely outside of resource addressing). > > > Thank you, > Stefan > > > ----- Original Message ----- > > > > > From: "Stefan Negrea" > > > To: "Discussions" > > > Sent: Tuesday, 28 April, 2015 5:44:56 PM > > > Subject: [Hawkular-dev] Tenant Id - Not Part of URL > > > > > > Hello Everybody, > > > > > > I've been working on a PR for the upcoming Hawkular Metrics release that > > > will > > > remove the tenant id from the end-point URLs. The tenant id will be > > > moved > > > to > > > either a header parameter or a query parameter. The query parameter is > > > in > > > place for cases (such as curl) where setting a header is not possible, > > > difficult, or inconvenient. > > > > > > Here is an example of the change: > > > > > > Existing URL: > > > /{tenantId}/gauge/{metricId}/data > > > > > > New URL: > > > /gauge/{metricId}/data > > > > > > Tenant id set via: > > > 1) header - tenantId > > > 2) query parameter - tenantId > > > > > > > > > There are two exceptions to this rule, /tenants and > > > /db/{tenantid}/series. > > > The /tenants end-point will be changed into something different in the > > > upcoming releases since it is mostly a management type API that does not > > > belong in the same place with the regular metrics endpoint. And > > > /db/{tenantid}/series end-point is needed in this exact format for > > > compatibility with Influxdb compatible services. > > > > > > > > > Now, to the merits of this change. The tenant id is volatile, can change > > > any > > > time, and changes to it should be expected; but the rest of the URL is > > > fixed. The second issue is that the tenant id is a security concern. So > > > we > > > were limited in design choices since a security concern was leaking as > > > part > > > of the URL. > > > > > > So removing the tenant id from the URL will give us permanent & > > > consistent > > > addresses for resources (metrics and metric data points). And we will > > > gain > > > a > > > lot of flexibility on the security side. In the future, users could > > > authenticate with a user/pass combo and the backend would transform that > > > into a tenant id to be used on the request. If the same user later > > > decides > > > to use a tenant id to pass along the request, the URL of the resources > > > would > > > not change. Another expectation is that tenant id is not sufficient, it > > > is > > > typically a combo of id + secret; so we would have resorted to a header > > > or > > > query param for the second piece of information (the secret). > > > > > > This change will give us the flexibility to adjust the security model > > > (the > > > meaning of tenant ids and ways to validate them) without compromising > > > the > > > URL structure. This will help Hawkular Metrics as it gets integrated > > > into > > > more and more projects and products. > > > > > > Here are the links to the JIRA and the PR for this change: > > > https://github.com/hawkular/hawkular-metrics/pull/202 > > > https://issues.jboss.org/browse/HWKMETRICS-68 > > > > > > > > > > > > Thank you, > > > Stefan Negrea > > > > > > Software Engineer > > > > > > _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From gbrown at redhat.com Thu Apr 30 09:17:34 2015 From: gbrown at redhat.com (Gary Brown) Date: Thu, 30 Apr 2015 09:17:34 -0400 (EDT) Subject: [Hawkular-dev] Maven swagger plugin In-Reply-To: <21645870.9571433.1430399376140.JavaMail.zimbra@redhat.com> Message-ID: <1728625498.9575594.1430399854829.JavaMail.zimbra@redhat.com> Hi The current version of the maven-swagger-plugin used by hawkular doesn't include derived types in the generated docs. I've tried updating to 2.3.4 and it still does not work. From some basic tests with 3.0.0, it seems to produce the correct content in the json schema - but has slightly different attributes to the plugin configuration, which when updated does not seem to run the template correctly. The other issue is some API changes when updating the com.wordnik:swagger-* dependencies in line with this maven plugin, as there have been some GAV and API changes. Its not an urgent issue, but it looks like we will be impacted by changes when we need to upgrade, so thought I would raise the issue. Regards Gary From lponce at redhat.com Thu Apr 30 10:36:05 2015 From: lponce at redhat.com (Lucas Ponce) Date: Thu, 30 Apr 2015 10:36:05 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Alerts - 0.0.1-Snapshot pushed to master In-Reply-To: <553E96E5.2060803@redhat.com> References: <553E96E5.2060803@redhat.com> Message-ID: <1324367531.9126076.1430404565233.JavaMail.zimbra@redhat.com> Hello, We have moved the Cassandra persitence backend into master for Hawkular Alerts in v0.0.1. Thanks, Lucas ----- Original Message ----- > From: "Jay Shaughnessy" > To: hawkular-dev at lists.jboss.org > Sent: Monday, April 27, 2015 10:07:01 PM > Subject: [Hawkular-dev] Hawkular Alerts - 0.0.1-Snapshot pushed to master > > > Hawkular Alerts has been updated in master. > > Given our latest Hawkular convention for versioning, the versioning has gone > backwards. We pushed Version 0.0.1-SNAPSHOT, changed from Version > 1.0.0-SNAPSHOT. We maintain snapshot releases for use with Kettle. But the > version has changed to reflect API changes. Going forward we'll maintain the > version of 0.0.1 unless we have an API change, at which point we'll up the > micro-version. Eventually this will settle down and we can move to versioned > components, and avoid breaking API changes. > > We've begun supplying developer documentation, here: > > http://www.hawkular.org/docs/dev/alerts.html > > What's NEW! > > > * Alert life-cycle support > > > * Alerts now support Open, Ack and Resolved states. > * This gives us the ability to store and display full start and end > times for an alerting "event". > * Trigger AutoResolve > > * This replaces and builds on the former "trigger safety" feature. > * Optionally provide AutoResolve conditions and dampening to toggle > the Trigger from Firing mode to AutoResolve mode. > * Optionally have a Trigger's alerts set Resolved automatically, when > the trigger's AutoResolve conditions are satisfied. > * Trigger AutoDisable > > * Optionally have a Trigger disable itself after firing. > * Mutes a trigger without the need to define AutoResolve. > * Rest Api control of the new features. > * Twilio SMS sender action > * Aerogear sender action > * Performance testing infrastructure > * Improved Rest API documentation > > > And for Hawkular Kettle! > > > * Data Filtering > > > * Only forward relevant metric and availability data for alert > processing. > * Remove the metrics and availability not used in any trigger. > > And of course several fixes and improvements. > > Coming Up: > > > * Moving the persistence back-end to Cassandra to leverage the Metrics > cluster (soon). > * PagerDuty action integration. > * Integration of the new 0.0.1 features into Hawkular. > > > > > > Hawkular Alerts Team > Jay Shaughnessy > Lucas Ponce > Thomas Segismont > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mmahoney at redhat.com Thu Apr 30 16:33:29 2015 From: mmahoney at redhat.com (Matthew Mahoney) Date: Thu, 30 Apr 2015 16:33:29 -0400 (EDT) Subject: [Hawkular-dev] Public Hawkular Instances - Feedback Please In-Reply-To: <2041921250.10664134.1430425335461.JavaMail.zimbra@redhat.com> Message-ID: <739646864.10670849.1430426009913.JavaMail.zimbra@redhat.com> Hawkular Community, The JON QE team is querying feedback regarding the availability & usefulness of the Public Hawkular Instances: http://209.132.178.218:18080 (Try it out - " kick the tires ") http://209.132.178.218:18090 (On-demand build for Development / Integration usage) * Are either of these instances being used, and if so for what purpose? * What can we do to make these instances more usable / useful? Thanks! Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150430/e74b3107/attachment.html From snegrea at redhat.com Thu Apr 30 18:28:14 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Thu, 30 Apr 2015 18:28:14 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics 0.3.3 - Release & Beyond In-Reply-To: <1345058757.10138787.1430432449139.JavaMail.zimbra@redhat.com> Message-ID: <563346197.10140584.1430432894081.JavaMail.zimbra@redhat.com> Hello Everybody, I am happy to announce release 0.3.3 of Hawkular Metrics. The release is anchored by three major and notable changes: REST API reorganization, tenant id changes, and Docker + Kubernetes work. 1) REST API reoganization - The API has been reorganized to follow a more traditional REST structure. - The numeric metrics have been renamed to Gauage. Going forward the term number will be used for an overarching category of metrics (gauge, counter, histograms, etc.). - The root of the project has been updated to be consistent with the rest of the Hawkular projects. The new root is http://host:port/hawkular/metrics The attached pdf document has the comparison for the API changes between 0.3.1 and 0.3.3. 2) Tenant Id The tenant id has been removed from the URL to either a header parameter or a query parameter. The query parameter can be used in cases (such as curl) where setting a header is not possible, difficult, or inconvenient. The concept of tenant will be an integral part of the project going forward. But we wanted more flexibility in the way the tenant id for a request is derived. So removing the tenant id from the URL will give us permanent & consistent resource addresses (metrics and metric data points); and will gain a lot of flexibility on the security side. In the future, users could authenticate with a user/pass combo and the backend would transform that into a tenant id. If the same user later decides to use the tenant id header, the URL of the resources accessed with the user/pass combo would not change. Another expectation is that tenant id is not sufficient, it is typically a combo of id + secret; so we would have resorted to a header or query param for the second piece of information (the secret) in the long run anyway. Here is an example of the change: Existing URL: /{tenantId}/gauge/{metricId}/data New URL: /gauge/{metricId}/data Tenant id set via: 1) header - tenantId 2) query parameter - tenantId There are two exceptions to this rule, /tenants and /db/{tenantid}/series. The /tenants end-point will be changed into something different in the upcoming releases since it is mostly a management type API that does not belong in the same place with the regular metrics endpoint. And /db/{tenantid}/series end-point is needed in this exact format for compatibility with Influxdb compatible services. The attached pdf document includes the removal of the tenant id from the URL. 3) Docker and Kubernetes Thanks to the amazing work done by Matt Wringe, we now have a subproject in Hawkular Metrics that can be used to create components to be run on Openshift/Fabric8 (https://github.com/hawkular/hawkular-metrics/tree/master/containers). This includes: a) A customized Cassandra docker image which uses a seed provider to automatically detect other nodes running behind the same service. b) A standalone dockerized Hawkular Metrics image. This will startup a hawkular metrics instance which will automatically detect and connect to the Cassandra service. c) A kubernetes application for a single step install into OpenShift or Fabric8. The zip can even be deployed via the drag-n-drop mechanism in the Fabric8 console! This is the foundation for later integrations that would require the project package in the form of containers. Github Release: https://github.com/hawkular/hawkular-metrics/releases/tag/0.3.3 JBoss Nexus Maven artifacts: http://origin-repository.jboss.org/nexus/content/repositories/public/org/hawkular/metrics/ Jira release tracker: https://issues.jboss.org/browse/HWKMETRICS/fixforversion/12326937 Hawkular Metrics 0.3.4 and Beyond Here is a list of features and enhancements that we planned for 0.3.4 and beyond. 1) Gauge Aggregates - Long-term storage of numeric metrics at the expense of losing some fidelity. The design has been already presented and discussed during 0.3.3 release cycle. The task queue work has already been started by John. We expect at least an initial implementation to land in 0.3.4. 3) Go client - will help with integrating with third party metrics collection framework. This work is the foundation for a Heapster sink. 4) Public Java API - following the work done in 0.3.1 for the core, the goal is to separate the implementation from a public API and make that available to clients 5) Update REST testing - while the current set of tests is a good gauge for regressions, the overall coverage is still low. The plan for 0.3.2 is to increase coverage. 6) Improved docker and kubernetes support A big "Thank you" goes to John Sanda, Thomas Segismont, Mike Thompson, Matt Wringe, and Heiko Rupp for their project contributions. Special mentions go to Jeeva Kandasamy, Jirka Kremser, and Michael Burman for their project help. Thank you, Stefan Negrea Software Engineer -------------- next part -------------- A non-text attachment was scrubbed... Name: Metrics REST API - 0.3.1 vs 0.3.3.pdf Type: application/pdf Size: 68765 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150430/fb9a72c9/attachment-0001.pdf