From theute at redhat.com Tue Sep 1 03:08:13 2015 From: theute at redhat.com (Thomas Heute) Date: Tue, 1 Sep 2015 09:08:13 +0200 Subject: [Hawkular-dev] Add JDBC Driver support in Agent In-Reply-To: <55E48554.2070601@redhat.com> References: <55E48554.2070601@redhat.com> Message-ID: <55E54EDD.8060105@redhat.com> On 08/31/2015 06:48 PM, Peter Palaga wrote: > Hi *, > > there is a requirement for Agent to support adding JDBC drivers > https://issues.jboss.org/browse/HAWKULAR-517 > > The named Jira does not name any standalone/domain/local/remote related > limitations for the functionality, hence the functionality should be > there for all modes. > > As already noticed on IRC, WildFly 9's jboss-cli.sh offers the > possibility to add a module, but it works only for the local machine in > standalone mode. Remote servers or domain mode are not supported. > > So clearly, for now, we can support the standalone local mode using the > same means as jboss-cli, but how should we proceed further? So that the agent subsystem can install the driver on the WF server it is installed on ? > Should we > (a) wait for WildFly to implement the missing functionality, or Is there a JIRA ? > (b) contribute to Wildfly or Unlikely > (c) create a minimal Agent to install on remote servers to provide the > functionality missing in WildFly itself (as proposed by Heiko)? Personally I am fine if we require the "complete" subsystem to be installed so that a WF server becomes manageable (ie, no remote install). Thomas > Thanks, > > Peter > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Tue Sep 1 03:32:56 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Tue, 1 Sep 2015 09:32:56 +0200 Subject: [Hawkular-dev] Add JDBC Driver support in Agent In-Reply-To: <55E54EDD.8060105@redhat.com> References: <55E48554.2070601@redhat.com> <55E54EDD.8060105@redhat.com> Message-ID: <55E554A8.3010600@redhat.com> On 2015-09-01 09:08, Thomas Heute wrote: > > > On 08/31/2015 06:48 PM, Peter Palaga wrote: >> Hi *, >> >> there is a requirement for Agent to support adding JDBC drivers >> https://issues.jboss.org/browse/HAWKULAR-517 >> >> The named Jira does not name any standalone/domain/local/remote related >> limitations for the functionality, hence the functionality should be >> there for all modes. >> >> As already noticed on IRC, WildFly 9's jboss-cli.sh offers the >> possibility to add a module, but it works only for the local machine in >> standalone mode. Remote servers or domain mode are not supported. >> >> So clearly, for now, we can support the standalone local mode using the >> same means as jboss-cli, but how should we proceed further? > > So that the agent subsystem can install the driver on the WF server it > is installed on ? Yes, exactly. > >> Should we >> (a) wait for WildFly to implement the missing functionality, or > > Is there a JIRA ? There is this "container" Jira https://issues.jboss.org/browse/WFCORE-661 that refers to subtasks https://issues.jboss.org/browse/WFCORE-422 File upload/manipulation through management API - modules, config files https://issues.jboss.org/browse/WFCORE-428 cli - command to add/remove AS modules from non-default module root > >> (b) contribute to Wildfly or > > Unlikely > >> (c) create a minimal Agent to install on remote servers to provide the >> functionality missing in WildFly itself (as proposed by Heiko)? > > Personally I am fine if we require the "complete" subsystem to be > installed so that a WF server becomes manageable (ie, no remote install). > > Thomas > > >> Thanks, >> >> Peter >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Tue Sep 1 07:18:59 2015 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 1 Sep 2015 07:18:59 -0400 (EDT) Subject: [Hawkular-dev] Add JDBC Driver support in Agent In-Reply-To: <55E554A8.3010600@redhat.com> References: <55E48554.2070601@redhat.com> <55E54EDD.8060105@redhat.com> <55E554A8.3010600@redhat.com> Message-ID: <1695839161.3698680.1441106339212.JavaMail.zimbra@redhat.com> > >> So clearly, for now, we can support the standalone local mode using the > >> same means as jboss-cli, but how should we proceed further? > > > > So that the agent subsystem can install the driver on the WF server it > > is installed on ? > > Yes, exactly. Do we need to make that distinction? In other words, is this only going to be able to deploy the driver on the WF server the agent is installed in? The agent can manage/monitor a remote WF server (by "remote" I mean one where the agent is not co-located/installed in) - this could even be one WF server that is running on the SAME MACHINE as the agent WF server (even if its a different WF Server). So I just want to be clear the limitations this will have. Will the agent: 1) only be able to install JDBC drivers on the WF server the agent is installed in? or 2) only be able to install JDBC drivers on WF servers running on the same machine as the the agent? or 3) Any remote or local WF Server? I think you are saying this will only support 1). I just wanted to make sure that is the case. From jpkroehling at redhat.com Wed Sep 2 11:38:27 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Wed, 2 Sep 2015 17:38:27 +0200 Subject: [Hawkular-dev] Heads up: Accounts URL is changing! Message-ID: <55E717F3.6030102@redhat.com> Folks, As reported via HAWKULAR-454 , it seems that the context I use on Accounts backend is not in line with the context used by other modules. Hence, Accounts' backend URL will be changing! - If you only depend on the accounts-api module, then you might just need to update to the latest Accounts version with this change (1.0.12.Final, to be released). - If you do REST calls to /hawkular-accounts, you'll need to change it to /hawkular/accounts . - This change will happen for Accounts 1.0.12.Final . Changes in Hawkular proper (aka kettle) will be performed by me . The plan is to release 1.0.12.Final at most by the end of the next week, but hopefully on Tuesday. The only difference between 1.0.11.Final and .12.Final will be this context path change. It would be a good idea to have your component updated to .11.Final by then, to make sure any possible new issues are solely related to the context path change. - Juca. From mazz at redhat.com Wed Sep 2 13:54:06 2015 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 2 Sep 2015 13:54:06 -0400 (EDT) Subject: [Hawkular-dev] Heads up: Accounts URL is changing! In-Reply-To: <55E717F3.6030102@redhat.com> References: <55E717F3.6030102@redhat.com> Message-ID: <1478997402.4722382.1441216446980.JavaMail.zimbra@redhat.com> The agent is ready for this change. See this PR: https://github.com/hawkular/hawkular-agent/pull/41 We can merge and release when accounts 1.0.12.Final is released ----- Original Message ----- > Folks, > > As reported via HAWKULAR-454 , it seems that the context I use on > Accounts backend is not in line with the context used by other modules. > > Hence, Accounts' backend URL will be changing! > > - If you only depend on the accounts-api module, then you might just > need to update to the latest Accounts version with this change > (1.0.12.Final, to be released). > > - If you do REST calls to /hawkular-accounts, you'll need to change it > to /hawkular/accounts . > > - This change will happen for Accounts 1.0.12.Final . Changes in > Hawkular proper (aka kettle) will be performed by me . > > The plan is to release 1.0.12.Final at most by the end of the next week, > but hopefully on Tuesday. The only difference between 1.0.11.Final and > .12.Final will be this context path change. It would be a good idea to > have your component updated to .11.Final by then, to make sure any > possible new issues are solely related to the context path change. > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From tsegismo at redhat.com Wed Sep 2 14:43:52 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 2 Sep 2015 20:43:52 +0200 Subject: [Hawkular-dev] [Metrics] New commit hook In-Reply-To: <55E42A91.9010708@redhat.com> References: <55CC7243.8040506@redhat.com> <55CC73AC.6050808@redhat.com> <55E061F6.9000702@redhat.com> <1260199119.20351015.1440775697066.JavaMail.zimbra@redhat.com> <55E42A91.9010708@redhat.com> Message-ID: <55E74368.7000800@redhat.com> The PR has been merged (thanks Stefan). Please take the time to add this hook to your local metrics repo: https://raw.githubusercontent.com/hawkular/hawkular-metrics/master/api/diff-pre-commit-hook.sh Also, after installing the hook, you should rebase your pull requests on master (to avoid issues with the license checker plugin). Le 31/08/2015 12:21, Thomas Segismont a ?crit : > I've created this PR "Pre-commit hook sample for JAX-RS diffs" #331 > https://github.com/hawkular/hawkular-metrics/pull/331 > > As the name suggests, it adds a pre-commit hook sample to the "api" > directory. If we go this way, metrics devs should add this commit hook > to their local repository. > > Then whenever a PR changes the diff, it will be committed and show on > GitHub review pages. > > Compared to the Travis based solution: > - diff shows up on GitHub only if new differences were introduced > - the diff file is kept up-to-date without developer intervention > - diff shows up on GitHub as a file change, so it looks much nicer on > review pages and can be downloaded easily > > I believe it's a better solution. > > > Le 28/08/2015 17:28, Stefan Negrea a ?crit : >> >> ----- Original Message ----- >>> From: "Thomas Segismont" >>> To: hawkular-dev at lists.jboss.org >>> Sent: Friday, August 28, 2015 8:28:22 AM >>> Subject: Re: [Hawkular-dev] [Metrics] JAX-RS 1.1 implementation update >>> >>> I've been playing with GitHub API this morning, and I figured out to >>> post the diff as a comment in pull requests. >> >> Could the developer creating the PR add this diff manually? Or it has to be automatic on every PR? There are some PRs that do not touch the REST implementations. If we can selectively attach this to PRs that need it, I think we should do it even if it is verbose. >> >>> >>> @metrics-dev would you find that useful? Or too noisy? >> >> I would like to see an example first. >> >>> >>> I thought about this as I noted we don't always remind to make changes >>> on both implementations. >> >> We should require adding integration tests for any change in the REST interface. And since we started this dual implementation, we have been good at doing it. Adding this diff would be an excellent second check for the tedious configuration we currently have. >> >>> >>> Le 13/08/2015 12:38, Thomas Segismont a ?crit : >>>> Forgot this: I believe this branch should be reviewed and merged ASAP. >>>> >>>> And, in the future, REST API PR reviewers should make sure changes/fixes >>>> are applied to both implementations. >>>> >>>> Le 13/08/2015 12:32, Thomas Segismont a ?crit : >>>>> Hi, >>>>> >>>>> This morning I've fixed the remaining issues in the JAX-RS 1.1 >>>>> implementation branch. The REST API test suite is now fully passing >>>>> against both JAX-RS 1.1 and 2.0 implementations. >>>>> >>>>> I've added a 'diff.txt' file in the 'api' [1]. Reviewers should >>>>> carefully look at handler code differences. >>>>> >>>>> Regards, >>>>> Thomas >>>>> >>>>> >>>>> [1] diff -r --exclude=target metrics-api-jaxrs metrics-api-jaxrs-1.1 > >>>>> diff.txt >>>>> _______________________________________________ >>>>> hawkular-dev mailing list >>>>> hawkular-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>>>> >>>> >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>>> >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Wed Sep 2 15:21:08 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Wed, 2 Sep 2015 21:21:08 +0200 Subject: [Hawkular-dev] Add JDBC Driver support in Agent In-Reply-To: <1695839161.3698680.1441106339212.JavaMail.zimbra@redhat.com> References: <55E48554.2070601@redhat.com> <55E54EDD.8060105@redhat.com> <55E554A8.3010600@redhat.com> <1695839161.3698680.1441106339212.JavaMail.zimbra@redhat.com> Message-ID: <55E74C24.2060506@redhat.com> Hi John, inline... On 2015-09-01 13:18, John Mazzitelli wrote: >>>> So clearly, for now, we can support the standalone local mode using the >>>> same means as jboss-cli, but how should we proceed further? >>> >>> So that the agent subsystem can install the driver on the WF server it >>> is installed on ? >> >> Yes, exactly. > > Do we need to make that distinction? In other words, is this only going to be able to deploy the driver on the WF server the agent is installed in? > > The agent can manage/monitor a remote WF server (by "remote" I mean one where the agent is not co-located/installed in) - this could even be one WF server that is running on the SAME MACHINE as the agent WF server (even if its a different WF Server). > > So I just want to be clear the limitations this will have. Will the agent: > > 1) only be able to install JDBC drivers on the WF server the agent is installed in? Yes, this is the option I work on. > or > > 2) only be able to install JDBC drivers on WF servers running on the same machine as the the agent? I never considered this variant. Do we have means in the agent to know that the non-hosting AS is on the same machine as the AS hosting the agent? > or > > 3) Any remote or local WF Server? Out of the scope for now. -- P > I think you are saying this will only support 1). I just wanted to make sure that is the case. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jshaughn at redhat.com Wed Sep 2 15:40:02 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 2 Sep 2015 15:40:02 -0400 Subject: [Hawkular-dev] Extra back-end for our front-end In-Reply-To: <55D2EBBB.6020804@redhat.com> References: <1439386040.3178.3.camel@vrockai-laptop> <55CB9A14.1000308@redhat.com> <921524511.10766034.1439800290722.JavaMail.zimbra@redhat.com> <1439807709.2305.3.camel@vrockai-laptop> <58BB388C-9691-4AA8-B9DB-D30197460DBF@redhat.com> <92783100.14848639.1439833984033.JavaMail.zimbra@redhat.com> <55D2EBBB.6020804@redhat.com> Message-ID: <55E75092.6060408@redhat.com> On 8/18/2015 4:24 AM, Juraci Paix?o Kr?hling wrote: > On 08/17/2015 07:53 PM, Stefan Negrea wrote: >> 1) The curent microservices are not very granular and the number is small; there are only 3 major microservices to interact with and the functionality for each is broad. >> 2) Each microservice can quickly add supporting methods for the UI. This will polish the API of each microservices because it has to be easily consumable. >> 3) We are forced to stay into microservice mode (it's a mindset!) and we will continue to develop decoupled services. >> 4) Increased potential of each service to be consumed independently of the whole because: >> a) the API will be more polished >> b) each service is always used in the mode that will be integrated outside >> 5) There is no additional place to address service quirks; each service has to solve interface quirks right away to be consumable. I see the API Gateway component as the fastest and first place to accumulate technical debt. > +1 , specially for points 1 and 5. Actually, from what I can see, the "fastest and first place to accumulate technical debt" is already the UI. The UI code is trying to coordinate inventory data with it's logically related metric information. And also tying together alerting with both inventory and metrics. If we don't add hawkular-level code that can itself be a micro-service client it is unclear to me how far we can progress without having the UI client code collapse under its own complexity. This is a problem we need to address as it seems (to me) that too much logic is going into the UI code. Maybe I'm wrong, but from what I can infer from the responses above, the alternative to hawkular-level code (some sort of coordinating "gateway" services), is to build all of the logic into the UI, where re-use of common workflows (like maybe, "get the alerts for this resource") would be hidden from other potential clients. In other words, does anyone have an alternative to "gateway" services that isn't putting everything in the presentation layer? Or maybe this has already been solved because the thread has not been active. From theute at redhat.com Thu Sep 3 03:39:04 2015 From: theute at redhat.com (Thomas Heute) Date: Thu, 3 Sep 2015 09:39:04 +0200 Subject: [Hawkular-dev] Extra back-end for our front-end In-Reply-To: <55E75092.6060408@redhat.com> References: <1439386040.3178.3.camel@vrockai-laptop> <55CB9A14.1000308@redhat.com> <921524511.10766034.1439800290722.JavaMail.zimbra@redhat.com> <1439807709.2305.3.camel@vrockai-laptop> <58BB388C-9691-4AA8-B9DB-D30197460DBF@redhat.com> <92783100.14848639.1439833984033.JavaMail.zimbra@redhat.com> <55D2EBBB.6020804@redhat.com> <55E75092.6060408@redhat.com> Message-ID: <55E7F918.2080808@redhat.com> On 09/02/2015 09:40 PM, Jay Shaughnessy wrote: > > > On 8/18/2015 4:24 AM, Juraci Paix?o Kr?hling wrote: > > On 08/17/2015 07:53 PM, Stefan Negrea wrote: > >> 1) The curent microservices are not very granular and the number is small; there are only 3 major microservices to interact with and the functionality for each is broad. > >> 2) Each microservice can quickly add supporting methods for the UI. This will polish the API of each microservices because it has to be easily consumable. > >> 3) We are forced to stay into microservice mode (it's a mindset!) and we will continue to develop decoupled services. > >> 4) Increased potential of each service to be consumed independently of the whole because: > >> a) the API will be more polished > >> b) each service is always used in the mode that will be integrated outside > >> 5) There is no additional place to address service quirks; each service has to solve interface quirks right away to be consumable. I see the API Gateway component as the fastest and first place to accumulate technical debt. > > +1 , specially for points 1 and 5. > > Actually, from what I can see, the "fastest and first place to > accumulate technical debt" is already the UI. The UI code is trying to > coordinate inventory data with it's logically related metric > information. +1 > And also tying together alerting with both inventory and > metrics. +1 > If we don't add hawkular-level code that can itself be a > micro-service client it is unclear to me how far we can progress without > having the UI client code collapse under its own complexity. > This is a > problem we need to address as it seems (to me) that too much logic is > going into the UI code. Maybe I'm wrong, but from what I can infer from > the responses above, the alternative to hawkular-level code (some sort > of coordinating "gateway" services), is to build all of the logic into > the UI, where re-use of common workflows (like maybe, "get the alerts > for this resource") would be hidden from other potential clients. +1 > > In other words, does anyone have an alternative to "gateway" services > that isn't putting everything in the presentation layer? Or maybe this > has already been solved because the thread has not been active. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jshaughn at redhat.com Thu Sep 3 08:26:34 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 3 Sep 2015 08:26:34 -0400 Subject: [Hawkular-dev] [GSoC] API and Misc Thoughts In-Reply-To: <58757021-9E13-4D6E-BF6C-25BBEA8904C9@gmail.com> References: <58757021-9E13-4D6E-BF6C-25BBEA8904C9@gmail.com> Message-ID: <55E83C7A.1020109@redhat.com> Hi Artur, We didn't interact very much but thanks for your work and for this insightful feedback. I hope you can maintain some involvement in Hawkular and best of luck with your studies and future, Jay On 8/23/2015 5:47 PM, Artur Dryomov wrote: > Hey everybody, > > As some of you might now, this year I?ve worked (am working) on the > Hawkular Android Client project. As a result, I?ve received a good > impression about the API and Hawkular itself mostly as a regular > observer, because I wasn?t directly involved in planning or the > development process. > > I must say that Hawkular is evolving and it is of course great :-) As > a side effect some things are changing and I?ve needed to keep up > though. This is not a bad thing, just a notice. The documentation side > of the project improved drastically in some places?we have nice graphs > at this point [1] which before I had to draw myself to understand how > things actually work [2]. Also, new logo! > > From the API standpoint and observing the system at whole some things > can be improved though. There is a famous Hawkular-Persona vs. > Hawkular-Tenant header battle serving a great example :-) I tried to > post some thoughts I had in the process of working with the API. I?ll > try to open some more if anything will come up, but this will do at > this point. Some things are pure nitpicking, so feel free to close and > comment ;-) > > https://issues.jboss.org/browse/HAWKULAR-452 > https://issues.jboss.org/browse/HAWKULAR-454 > https://issues.jboss.org/browse/HAWKULAR-567 > https://issues.jboss.org/browse/HAWKULAR-568 > https://issues.jboss.org/browse/HAWKULAR-569 > https://issues.jboss.org/browse/HAWKULAR-570 > https://issues.jboss.org/browse/HAWKULAR-571 > https://issues.jboss.org/browse/HAWKULAR-572 > > Thanks everybody for your kind help! This was a great summer working > with you guys and I look forward to continue to improve the Android > client (at least). > > Thanks again! > Artur. > > [1]: > http://www.hawkular.org/docs/components/inventory/index.html#inventory-organization > [2]: https://github.com/hawkular/hawkular-android-client/wiki/Overview > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150903/c0e8350b/attachment.html From miburman at redhat.com Thu Sep 3 08:29:57 2015 From: miburman at redhat.com (Michael Burman) Date: Thu, 3 Sep 2015 08:29:57 -0400 (EDT) Subject: [Hawkular-dev] Extra back-end for our front-end In-Reply-To: <55E75092.6060408@redhat.com> References: <1439386040.3178.3.camel@vrockai-laptop> <55CB9A14.1000308@redhat.com> <921524511.10766034.1439800290722.JavaMail.zimbra@redhat.com> <1439807709.2305.3.camel@vrockai-laptop> <58BB388C-9691-4AA8-B9DB-D30197460DBF@redhat.com> <92783100.14848639.1439833984033.JavaMail.zimbra@redhat.com> <55D2EBBB.6020804@redhat.com> <55E75092.6060408@redhat.com> Message-ID: <1192840039.28563829.1441283397417.JavaMail.zimbra@redhat.com> Hi, Yes, there's this wonderful tool that solves the issue quite well. It's called GWT. - Micke ----- Original Message ----- From: "Jay Shaughnessy" To: hawkular-dev at lists.jboss.org Sent: Wednesday, September 2, 2015 10:40:02 PM Subject: Re: [Hawkular-dev] Extra back-end for our front-end In other words, does anyone have an alternative to "gateway" services that isn't putting everything in the presentation layer? Or maybe this has already been solved because the thread has not been active. From jshaughn at redhat.com Thu Sep 3 08:52:19 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 3 Sep 2015 08:52:19 -0400 Subject: [Hawkular-dev] Extra back-end for our front-end In-Reply-To: <1192840039.28563829.1441283397417.JavaMail.zimbra@redhat.com> References: <1439386040.3178.3.camel@vrockai-laptop> <55CB9A14.1000308@redhat.com> <921524511.10766034.1439800290722.JavaMail.zimbra@redhat.com> <1439807709.2305.3.camel@vrockai-laptop> <58BB388C-9691-4AA8-B9DB-D30197460DBF@redhat.com> <92783100.14848639.1439833984033.JavaMail.zimbra@redhat.com> <55D2EBBB.6020804@redhat.com> <55E75092.6060408@redhat.com> <1192840039.28563829.1441283397417.JavaMail.zimbra@redhat.com> Message-ID: <55E84283.8010205@redhat.com> Nice. Truth be told, I miss GWT. For that short period I could actually be a UI dev. I miss nested async responses. On 9/3/2015 8:29 AM, Michael Burman wrote: > Hi, > > Yes, there's this wonderful tool that solves the issue quite well. It's called GWT. > > - Micke > > ----- Original Message ----- > From: "Jay Shaughnessy" > To: hawkular-dev at lists.jboss.org > Sent: Wednesday, September 2, 2015 10:40:02 PM > Subject: Re: [Hawkular-dev] Extra back-end for our front-end > > In other words, does anyone have an alternative to "gateway" services > that isn't putting everything in the presentation layer? Or maybe this > has already been solved because the thread has not been active. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150903/8d0c9178/attachment.html From jsanda at redhat.com Thu Sep 3 09:52:03 2015 From: jsanda at redhat.com (John Sanda) Date: Thu, 3 Sep 2015 09:52:03 -0400 Subject: [Hawkular-dev] Extra back-end for our front-end In-Reply-To: <55E84283.8010205@redhat.com> References: <1439386040.3178.3.camel@vrockai-laptop> <55CB9A14.1000308@redhat.com> <921524511.10766034.1439800290722.JavaMail.zimbra@redhat.com> <1439807709.2305.3.camel@vrockai-laptop> <58BB388C-9691-4AA8-B9DB-D30197460DBF@redhat.com> <92783100.14848639.1439833984033.JavaMail.zimbra@redhat.com> <55D2EBBB.6020804@redhat.com> <55E75092.6060408@redhat.com> <1192840039.28563829.1441283397417.JavaMail.zimbra@redhat.com> <55E84283.8010205@redhat.com> Message-ID: > On Sep 3, 2015, at 8:52 AM, Jay Shaughnessy wrote: > > > Nice. > > Truth be told, I miss GWT. For that short period I could actually be a UI dev. I miss nested async responses. > Give Rx a spin. You can do async without the nesting :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150903/71ec45e2/attachment.html From ppalaga at redhat.com Thu Sep 3 15:33:58 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Thu, 3 Sep 2015 21:33:58 +0200 Subject: [Hawkular-dev] Splitting out feed-comm from bus Message-ID: <55E8A0A6.5060404@redhat.com> Hi *, feed-comm is a component hosting the server part of the WebSockets that both UI and Agent use for bidirectional communication. It is quite new and it was preliminarily placed to bus git repo [1]. However, feed-comm needs to depend on Inventory which makes up a cyclic dependency, because Inventory already depends on Bus. We came to a consensus with Mazz and Luk?? that the most natural solution would be the put feed-comm into a new git repo and giving it its own release cycle. Are there any concerns about this? We are also in the process of finding a better home for nest, which conceptually does not belong to bus. Moving to HK main or to a new repo are being considered. [1] https://github.com/hawkular/hawkular-bus/tree/0.5.0.Final/hawkular-feed-comm Thanks, Peter From jsanda at redhat.com Thu Sep 3 16:15:20 2015 From: jsanda at redhat.com (John Sanda) Date: Thu, 3 Sep 2015 16:15:20 -0400 Subject: [Hawkular-dev] [metrics] remove support for tagging data points Message-ID: Early on in the project, I added some support for tagging individual data points, primarily as a PoC. To the best of my knowledge, that functionality is not used. Tagging metrics on the other hand is used. I created https://issues.jboss.org/browse/HWKMETRICS-247 to address removing the relevant code and making the necessary schema changes. If this functionality is used or if there are objections, please add a comment on the ticket; otherwise, I would like to make these changes for the next metrics release. - John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150903/45fc6a3b/attachment-0001.html From ppalaga at redhat.com Fri Sep 4 09:06:02 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Fri, 4 Sep 2015 15:06:02 +0200 Subject: [Hawkular-dev] Splitting out feed-comm from bus In-Reply-To: <55E8A0A6.5060404@redhat.com> References: <55E8A0A6.5060404@redhat.com> Message-ID: <55E9973A.2080107@redhat.com> Hi *, inline... On 2015-09-03 21:33, Peter Palaga wrote: > Hi *, > > feed-comm is a component hosting the server part of the WebSockets that > both UI and Agent use for bidirectional communication. It is quite new > and it was preliminarily placed to bus git repo [1]. However, feed-comm > needs to depend on Inventory which makes up a cyclic dependency, because > Inventory already depends on Bus. > > We came to a consensus with Mazz and Luk?? that the most natural > solution would be the put feed-comm into a new git repo and giving it > its own release cycle. Are there any concerns about this? Good, qui tacet consentit videtur. We also decided to rename feed-com to command-gateway, because (1) it is serving not only feeds, but also UI clients and (2) the server part of WebSockets is often called WebSockets Gateway. In our case the gateway consists not only of WebSockets but also of Bus and therefore, WebSockets is not a part of the new name. > We are also in the process of finding a better home for nest, which > conceptually does not belong to bus. Moving to HK main or to a new repo > are being considered. Although HK main seems to be a good place for nest, we stopped considering moving nest out of bus, because it works like it is now. Thanks, -- P > [1] > https://github.com/hawkular/hawkular-bus/tree/0.5.0.Final/hawkular-feed-comm > > Thanks, > > Peter > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Fri Sep 4 16:21:51 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Fri, 4 Sep 2015 22:21:51 +0200 Subject: [Hawkular-dev] Maven, Continuous and Reproducible In-Reply-To: <55D9930F.60001@redhat.com> References: <55D4DAB7.2090100@redhat.com> <55D9930F.60001@redhat.com> Message-ID: <55E9FD5F.5020607@redhat.com> I released the version 0.0.1 of srcdeps-maven-plugin and I started using it in agent [1][2] to be able to reliably depend on the new and so far unreleased hawkular-command-gateway artifacts. -- P [1] https://github.com/ppalaga/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R54 [2] https://github.com/ppalaga/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R217 On 2015-08-23 11:31, Peter Palaga wrote: > Status update: > > * I got a positive feedback from Heiko, Juca and Jirka. > * I moved the repo under l2x6 GitHub org, (an incubator for my open > source projects) https://github.com/l2x6/srcdeps-maven-plugin > * I registered the project at Sonatype so that it can be released to > Maven Central https://issues.sonatype.org/browse/OSSRH-17256 > * Waiting for somebody to try the plugin independently from me, I'd > eventually release after that > > -- P > > On 2015-08-19 21:36, Peter Palaga wrote: >> Hi *, >> >> I think I found a way how to make our process more continuous, while >> staying with stock Maven and Components in separate git repositories. >> >> The core of the idea is that we actually do not need to release the >> components because they are not our deliverables (yes, except for >> Metrics). If we were able to declare the dependencies using git >> revisions and build them on the fly, we could get rid of both SNAPSHOTS >> and releases. >> >> I have written srcdeps-maven-plugin [1] today that does exactly that: >> * It collects dependencies with versions matching >> {whatever}-SRC-{git-commit-hash} >> * It checks out {git-commit-hash} of their sources to >> ~/.m2/dependency-sources >> * It changes the version in the sources to >> {whatever}-SRC-{git-commit-hash} >> * Builds the artifacts and installs them to the local repository >> * All the above happens before the dependency resolution starts >> so it is fully transparent for the rest of Maven. >> * The -SRC- dependencies are build only if they are not found in local >> repository - so they prolong the build only when upgrading. >> >> How to try it out: >> >> #Make sure that you have maven 3.2.5 or newer: >> mvn -version >> >> # build the plugin >> cd ~/git >> git checkout https://github.com/l2x6/srcdeps-maven-plugin.git >> cd srcdeps-maven-plugin >> mvn clean install >> >> # build hawkular with the plugin >> cd ../hawkular >> git remote add ppalaga https://github.com/ppalaga/hawkular.git >> git fetch ppalaga >> git checkout -b 150819-srcdeps >> mvn clean install ... >> >> [1] https://github.com/ppalaga/srcdeps-maven-plugin >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Fri Sep 4 16:22:05 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Fri, 4 Sep 2015 22:22:05 +0200 Subject: [Hawkular-dev] Splitting out feed-comm from bus In-Reply-To: <55E9973A.2080107@redhat.com> References: <55E8A0A6.5060404@redhat.com> <55E9973A.2080107@redhat.com> Message-ID: <55E9FD6D.9080004@redhat.com> The announced change happened https://github.com/hawkular/hawkular-command-gateway -- P On 2015-09-04 15:06, Peter Palaga wrote: > Hi *, inline... > > On 2015-09-03 21:33, Peter Palaga wrote: >> Hi *, >> >> feed-comm is a component hosting the server part of the WebSockets that >> both UI and Agent use for bidirectional communication. It is quite new >> and it was preliminarily placed to bus git repo [1]. However, feed-comm >> needs to depend on Inventory which makes up a cyclic dependency, because >> Inventory already depends on Bus. >> >> We came to a consensus with Mazz and Luk?? that the most natural >> solution would be the put feed-comm into a new git repo and giving it >> its own release cycle. Are there any concerns about this? > > Good, qui tacet consentit videtur. > > We also decided to rename feed-com to command-gateway, because (1) it is > serving not only feeds, but also UI clients and (2) the server part of > WebSockets is often called WebSockets Gateway. In our case the gateway > consists not only of WebSockets but also of Bus and therefore, > WebSockets is not a part of the new name. > >> We are also in the process of finding a better home for nest, which >> conceptually does not belong to bus. Moving to HK main or to a new repo >> are being considered. > > Although HK main seems to be a good place for nest, we stopped > considering moving nest out of bus, because it works like it is now. > > Thanks, > > -- P > >> [1] >> https://github.com/hawkular/hawkular-bus/tree/0.5.0.Final/hawkular-feed-comm >> >> Thanks, >> >> Peter >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Mon Sep 7 07:37:50 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Mon, 7 Sep 2015 13:37:50 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 21 Message-ID: <55ED770E.7090905@redhat.com> Hi *, hawkular-parent 21 brings the following: * Upgraded maven wildfly plugin to 1.1.0.Alpha3 to fix bug with passing system properties (thanks Gary!) * Define license header style for WSDL files * Added srcdeps-maven-plugin and its configuration. Now it is possible to depend on a git commit of a component as we already do in Agent: https://github.com/hawkular/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R54 I am about to send PRs to all components repos except for BTM. Thanks, Peter From tsegismo at redhat.com Mon Sep 7 12:37:03 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 7 Sep 2015 18:37:03 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 21 In-Reply-To: <55ED770E.7090905@redhat.com> References: <55ED770E.7090905@redhat.com> Message-ID: <55EDBD2F.1060302@redhat.com> If any project other than metric fails to build, this might related to the Wildfly plugin upgrade. In Metrics, we use port offsets to avoid conflicts between a locally running server, the JAX-RS 2.0 test server and the JAX-RS 1.1 test server. It seems that "WFMP-12 Run and start goals should attempt to pass management client overrides in the start command" is the cause of our broken build. https://issues.jboss.org/browse/WFMP-12 Le 07/09/2015 13:37, Peter Palaga a ?crit : > Hi *, > > hawkular-parent 21 brings the following: > > * Upgraded maven wildfly plugin to 1.1.0.Alpha3 to fix bug with passing > system properties (thanks Gary!) > * Define license header style for WSDL files > * Added srcdeps-maven-plugin and its configuration. Now it is possible > to depend on a git commit of a component as we already do in Agent: > https://github.com/hawkular/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R54 > > I am about to send PRs to all components repos except for BTM. > > Thanks, > > Peter > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From tsegismo at redhat.com Mon Sep 7 12:39:40 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 7 Sep 2015 18:39:40 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 21 In-Reply-To: <55EDBD2F.1060302@redhat.com> References: <55ED770E.7090905@redhat.com> <55EDBD2F.1060302@redhat.com> Message-ID: <55EDBDCC.4040704@redhat.com> Just realized that my statement could be misleading... :) The metrics master build is not broken. It's the PR introducing parent 21 which fails to be built. Le 07/09/2015 18:37, Thomas Segismont a ?crit : > If any project other than metric fails to build, this might related to > the Wildfly plugin upgrade. > > In Metrics, we use port offsets to avoid conflicts between a locally > running server, the JAX-RS 2.0 test server and the JAX-RS 1.1 test server. > > It seems that "WFMP-12 Run and start goals should attempt to pass > management client overrides in the start command" is the cause of our > broken build. > > https://issues.jboss.org/browse/WFMP-12 > > Le 07/09/2015 13:37, Peter Palaga a ?crit : >> Hi *, >> >> hawkular-parent 21 brings the following: >> >> * Upgraded maven wildfly plugin to 1.1.0.Alpha3 to fix bug with passing >> system properties (thanks Gary!) >> * Define license header style for WSDL files >> * Added srcdeps-maven-plugin and its configuration. Now it is possible >> to depend on a git commit of a component as we already do in Agent: >> https://github.com/hawkular/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R54 >> >> I am about to send PRs to all components repos except for BTM. >> >> Thanks, >> >> Peter >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From tsegismo at redhat.com Tue Sep 8 04:47:03 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 8 Sep 2015 10:47:03 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 21 In-Reply-To: <55EDBDCC.4040704@redhat.com> References: <55ED770E.7090905@redhat.com> <55EDBD2F.1060302@redhat.com> <55EDBDCC.4040704@redhat.com> Message-ID: <55EEA087.90801@redhat.com> FYI, I've verified this morning that WFMP-12 is the cause of the issue. I added a comment to the ticket and I'm waiting for James' reply. In the meantime, Metrics will stick with parent POM version 20. Le 07/09/2015 18:39, Thomas Segismont a ?crit : > Just realized that my statement could be misleading... :) > > The metrics master build is not broken. It's the PR introducing parent > 21 which fails to be built. > > Le 07/09/2015 18:37, Thomas Segismont a ?crit : >> If any project other than metric fails to build, this might related to >> the Wildfly plugin upgrade. >> >> In Metrics, we use port offsets to avoid conflicts between a locally >> running server, the JAX-RS 2.0 test server and the JAX-RS 1.1 test server. >> >> It seems that "WFMP-12 Run and start goals should attempt to pass >> management client overrides in the start command" is the cause of our >> broken build. >> >> https://issues.jboss.org/browse/WFMP-12 >> >> Le 07/09/2015 13:37, Peter Palaga a ?crit : >>> Hi *, >>> >>> hawkular-parent 21 brings the following: >>> >>> * Upgraded maven wildfly plugin to 1.1.0.Alpha3 to fix bug with passing >>> system properties (thanks Gary!) >>> * Define license header style for WSDL files >>> * Added srcdeps-maven-plugin and its configuration. Now it is possible >>> to depend on a git commit of a component as we already do in Agent: >>> https://github.com/hawkular/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R54 >>> >>> I am about to send PRs to all components repos except for BTM. >>> >>> Thanks, >>> >>> Peter >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Tue Sep 8 04:57:28 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Tue, 8 Sep 2015 10:57:28 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 21 In-Reply-To: <55EEA087.90801@redhat.com> References: <55ED770E.7090905@redhat.com> <55EDBD2F.1060302@redhat.com> <55EDBDCC.4040704@redhat.com> <55EEA087.90801@redhat.com> Message-ID: <55EEA2F8.3090902@redhat.com> Hi Thomas, thanks for pin-pointing the issue. Yes, no problem if you do not adopt parent 21 in metrics for now - there is nothing critical in there. -- P On 2015-09-08 10:47, Thomas Segismont wrote: > FYI, I've verified this morning that WFMP-12 is the cause of the issue. > > I added a comment to the ticket and I'm waiting for James' reply. > > In the meantime, Metrics will stick with parent POM version 20. > > Le 07/09/2015 18:39, Thomas Segismont a ?crit : >> Just realized that my statement could be misleading... :) >> >> The metrics master build is not broken. It's the PR introducing parent >> 21 which fails to be built. >> >> Le 07/09/2015 18:37, Thomas Segismont a ?crit : >>> If any project other than metric fails to build, this might related to >>> the Wildfly plugin upgrade. >>> >>> In Metrics, we use port offsets to avoid conflicts between a locally >>> running server, the JAX-RS 2.0 test server and the JAX-RS 1.1 test server. >>> >>> It seems that "WFMP-12 Run and start goals should attempt to pass >>> management client overrides in the start command" is the cause of our >>> broken build. >>> >>> https://issues.jboss.org/browse/WFMP-12 >>> >>> Le 07/09/2015 13:37, Peter Palaga a ?crit : >>>> Hi *, >>>> >>>> hawkular-parent 21 brings the following: >>>> >>>> * Upgraded maven wildfly plugin to 1.1.0.Alpha3 to fix bug with passing >>>> system properties (thanks Gary!) >>>> * Define license header style for WSDL files >>>> * Added srcdeps-maven-plugin and its configuration. Now it is possible >>>> to depend on a git commit of a component as we already do in Agent: >>>> https://github.com/hawkular/hawkular-agent/commit/81f66c2867bce5f3f70a04324bc2259fd5aebcdb#diff-600376dffeb79835ede4a0b285078036R54 >>>> >>>> I am about to send PRs to all components repos except for BTM. >>>> >>>> Thanks, >>>> >>>> Peter >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>>> >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jpkroehling at redhat.com Tue Sep 8 11:06:56 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Tue, 8 Sep 2015 17:06:56 +0200 Subject: [Hawkular-dev] Heads up: Accounts URL is changing! In-Reply-To: <55E717F3.6030102@redhat.com> References: <55E717F3.6030102@redhat.com> Message-ID: <55EEF990.3050205@redhat.com> Accounts 1.0.12.Final has just been released with the new path. I'll be preparing a PR for the main Hawkular repo soon. On this PR, it would be interesting to get all component updates so that we don't have a broken distribution there. So far, the following needs to be changed/updated: - hawkular-ui-services (PR opened, I'll bump bower.json on hawkular) - Inventory (at least, I've seen it might need a change based on messages I've seen in the logs) - hawkular-agent (Mazz has a PR ready to be merged) Are you aware of any other component that is affected by this change and that would need to have the version bumped in the main hawkular repo? Then let me know! Otherwise, I'll send the final PR on Thursday. Need more time? Just let me know and we'll rearrange it accordingly. - Juca. On 09/02/2015 05:38 PM, Juraci Paix?o Kr?hling wrote: > Folks, > > As reported via HAWKULAR-454 , it seems that the context I use on > Accounts backend is not in line with the context used by other modules. > > Hence, Accounts' backend URL will be changing! > > - If you only depend on the accounts-api module, then you might just > need to update to the latest Accounts version with this change > (1.0.12.Final, to be released). > > - If you do REST calls to /hawkular-accounts, you'll need to change it > to /hawkular/accounts . > > - This change will happen for Accounts 1.0.12.Final . Changes in > Hawkular proper (aka kettle) will be performed by me . > > The plan is to release 1.0.12.Final at most by the end of the next week, > but hopefully on Tuesday. The only difference between 1.0.11.Final and > .12.Final will be this context path change. It would be a good idea to > have your component updated to .11.Final by then, to make sure any > possible new issues are solely related to the context path change. > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mwringe at redhat.com Tue Sep 8 16:30:23 2015 From: mwringe at redhat.com (Matt Wringe) Date: Tue, 8 Sep 2015 16:30:23 -0400 Subject: [Hawkular-dev] Bucketed data issues when no datapoints fall in the bucket span. Message-ID: <55EF455F.9080007@redhat.com> I am running into a few issues when dealing with bucketed data when there is no datapoints which happen to fall within a particular bucket size. For instance, lets say I am getting data every 30 seconds. If my bucket spans at least 30 seconds than everything works fine and all my buckets contain non-empty data. But, if I were to 'zoom in' a bit and ask for a bucket span less than 30 seconds, then some of the buckets would return with empty data. Or, if my data does not come in perfectly for each of those 30 second fragments (say the timestamps are plus or minus a couple of seconds) then I could also end up with some buckets containing empty data as well. This causes some strange charting behaviour. If a bucket is empty, then its just not displayed in the chart. This means we can end up with non-continuous lines in the charts, which is not desired. Anyone know what should be done in this situation? Its not really that there is no data there and nothing should be graphed, its just that the bucket span is too small to show anything in there. Should charting just ignore empty buckets if they are not at the start or the end? Should hawkular metrics perform calculations to figure out what the missing bucket values should be? Any ideas here? From tsegismo at redhat.com Tue Sep 8 16:56:19 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 8 Sep 2015 22:56:19 +0200 Subject: [Hawkular-dev] Bucketed data issues when no datapoints fall in the bucket span. In-Reply-To: <55EF455F.9080007@redhat.com> References: <55EF455F.9080007@redhat.com> Message-ID: <55EF4B73.6020404@redhat.com> It's hard to determine what it means to interpolate values when you're looking at a metric at its resolution level. Put another way, what does it mean to ask for avg/min/median for every 30 seconds timespan if you have a point every 30 seconds? Le 08/09/2015 22:30, Matt Wringe a ?crit : > I am running into a few issues when dealing with bucketed data when > there is no datapoints which happen to fall within a particular bucket size. > > For instance, lets say I am getting data every 30 seconds. If my bucket > spans at least 30 seconds than everything works fine and all my buckets > contain non-empty data. > > But, if I were to 'zoom in' a bit and ask for a bucket span less than 30 > seconds, then some of the buckets would return with empty data. > > Or, if my data does not come in perfectly for each of those 30 second > fragments (say the timestamps are plus or minus a couple of seconds) > then I could also end up with some buckets containing empty data as well. > > This causes some strange charting behaviour. If a bucket is empty, then > its just not displayed in the chart. This means we can end up with > non-continuous lines in the charts, which is not desired. > > Anyone know what should be done in this situation? > > Its not really that there is no data there and nothing should be > graphed, its just that the bucket span is too small to show anything in > there. > > Should charting just ignore empty buckets if they are not at the start > or the end? > > Should hawkular metrics perform calculations to figure out what the > missing bucket values should be? > > Any ideas here? > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mwringe at redhat.com Tue Sep 8 17:06:20 2015 From: mwringe at redhat.com (Matt Wringe) Date: Tue, 8 Sep 2015 17:06:20 -0400 Subject: [Hawkular-dev] Bucketed data issues when no datapoints fall in the bucket span. In-Reply-To: <55EF4B73.6020404@redhat.com> References: <55EF455F.9080007@redhat.com> <55EF4B73.6020404@redhat.com> Message-ID: <55EF4DCC.7070901@redhat.com> On 08/09/15 04:56 PM, Thomas Segismont wrote: > It's hard to determine what it means to interpolate values when you're > looking at a metric at its resolution level. > > Put another way, what does it mean to ask for avg/min/median for every > 30 seconds timespan if you have a point every 30 seconds? Yeah, that is essentially the problem. Bucketed data is not all the useful once you start to get into a smaller number of data points per bucket. But then it comes into the situation when something is expecting bucketed data and we don't have metric points for the level it is requesting (eg charting). So how do you solve the chart "zooming in" situation? We could just return the actual data points in that situation and not bucketed data, but since its a different format and it gets tricky for clients to use it. > > Le 08/09/2015 22:30, Matt Wringe a ?crit : >> I am running into a few issues when dealing with bucketed data when >> there is no datapoints which happen to fall within a particular bucket size. >> >> For instance, lets say I am getting data every 30 seconds. If my bucket >> spans at least 30 seconds than everything works fine and all my buckets >> contain non-empty data. >> >> But, if I were to 'zoom in' a bit and ask for a bucket span less than 30 >> seconds, then some of the buckets would return with empty data. >> >> Or, if my data does not come in perfectly for each of those 30 second >> fragments (say the timestamps are plus or minus a couple of seconds) >> then I could also end up with some buckets containing empty data as well. >> >> This causes some strange charting behaviour. If a bucket is empty, then >> its just not displayed in the chart. This means we can end up with >> non-continuous lines in the charts, which is not desired. >> >> Anyone know what should be done in this situation? >> >> Its not really that there is no data there and nothing should be >> graphed, its just that the bucket span is too small to show anything in >> there. >> >> Should charting just ignore empty buckets if they are not at the start >> or the end? >> >> Should hawkular metrics perform calculations to figure out what the >> missing bucket values should be? >> >> Any ideas here? >> >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From mithomps at redhat.com Tue Sep 8 17:40:22 2015 From: mithomps at redhat.com (mike thompson) Date: Tue, 8 Sep 2015 14:40:22 -0700 Subject: [Hawkular-dev] Bucketed data issues when no datapoints fall in the bucket span. In-Reply-To: <55EF455F.9080007@redhat.com> References: <55EF455F.9080007@redhat.com> Message-ID: > On 8 Sep 2015, at 13:30, Matt Wringe wrote: > > I am running into a few issues when dealing with bucketed data when > there is no datapoints which happen to fall within a particular bucket size. > > For instance, lets say I am getting data every 30 seconds. If my bucket > spans at least 30 seconds than everything works fine and all my buckets > contain non-empty data. > > But, if I were to 'zoom in' a bit and ask for a bucket span less than 30 > seconds, then some of the buckets would return with empty data. > > Or, if my data does not come in perfectly for each of those 30 second > fragments (say the timestamps are plus or minus a couple of seconds) > then I could also end up with some buckets containing empty data as well. > > This causes some strange charting behaviour. If a bucket is empty, then > its just not displayed in the chart. This means we can end up with > non-continuous lines in the charts, which is not desired. > > Anyone know what should be done in this situation? > > Its not really that there is no data there and nothing should be > graphed, its just that the bucket span is too small to show anything in > there. > > Should charting just ignore empty buckets if they are not at the start > or the end? From the charts perspective: In general, No. We use the empty buckets to be interpreted as ?Unknown? which means either the agent is down or the resource is down (whereas ?down? implies that we know the resource is actually down because the agent monitor tells us so). > > Should hawkular metrics perform calculations to figure out what the > missing bucket values should be? No, again we rely upon the empty buckets to mean no data was collected during that bucket interval. > > Any ideas here? > Sounds like the time interval that you are charting is not granular enough for the buckets that you want to display. Is it possible to increase the collection frequency? Another option is to allow a raw mode that plots individual points and interpolates between the points (but you would give up the ability to show empty ?no data? areas, which are just as important, if not more than data points). BTW It would be useful to see the charts and json dataset to fully understand the issue. Having the json allows us to model the chart exactly as you see in the error condition so we can reproduce and play with it until we are happy with the results. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150908/afa1bfd9/attachment.html From mazz at redhat.com Wed Sep 9 08:32:35 2015 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 9 Sep 2015 08:32:35 -0400 (EDT) Subject: [Hawkular-dev] article: "My Philosophy on Alerting" In-Reply-To: References: Message-ID: <515830368.7820436.1441801955783.JavaMail.zimbra@redhat.com> For the alerting folks: https://docs.google.com/document/d/199PqyG3UsyXlwieHaqbGiWVa8eMWi8zzAn0YfcApr8Q/preview By Rob Ewaschuk, an ex-Site Reliability Engineer at Google. From jshaughn at redhat.com Thu Sep 10 10:51:05 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 10 Sep 2015 10:51:05 -0400 Subject: [Hawkular-dev] Hawkular Alerts 0.4.0.Final Released! In-Reply-To: <55897A75.7070700@redhat.com> References: <55897A75.7070700@redhat.com> Message-ID: <55F198D9.4050804@redhat.com> Hawkular Alerts 0.4.0.Final has been Released! Notable Features: * Standalone deployment o HWKALERTS-68 o http://www.hawkular.org/blog/2015/08/19/hawkular-alerts-standalone.html * Group Trigger support o HWKALERTS-12 * New File System Action Notifier o HWKALERTS-42 * Improved E-mail Notifications o HWKALERTS-85 * REST API improvements for working with Trigger Conditions o HWKALERTS-83 * Context Data support for Triggers, Conditions and [previously] Data o HWKALERTS-73, 85 Jira Summary for features above, as well as minor enhancements and fixes: * https://issues.jboss.org/issues/?jql=project%20%3D%20HWKALERTS%20AND%20fixVersion%20%3D%200.4.0.Final%20ORDER%20BY%20issuetype%20ASC%2C%20key%20ASC The next scheduled version is 0.5.0, targeted for October 7, 2015. Hawkular Alerts Team Jay Shaughnessy (jshaughn at redhat.com) Lucas Ponce (lponce at redhat.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150910/2f54dd4d/attachment-0001.html From mithomps at redhat.com Thu Sep 10 15:54:45 2015 From: mithomps at redhat.com (mike thompson) Date: Thu, 10 Sep 2015 12:54:45 -0700 Subject: [Hawkular-dev] Resource Path view in UI Message-ID: Now that we have switched to Resource Paths (canonical paths from Inventory) it has come up that displaying the resource path is not very readable and requires parsing to make sense of it. Please see the current use case as example: https://issues.jboss.org/secure/attachment/12392747/AlertCenter2.jpg https://issues.jboss.org/secure/attachment/12392582/Alert-detailsnu.jpg An example resource path: /t;28026b36-8fe4-4332-84c8-524e173a68bf/e;test/f;localhost/r;localhost~Local~~ To further break the above example into its pieces: tenantId: /t;28026b36-8fe4-4332-84c8-524e173a68bf environment: test feed: localhost resourceId: localhost The following snippet explaining the resource path was extracted from: http://www.hawkular.org/docs/components/inventory/index.html Canonical Paths A canonical path follows the contains relationships from a tenant down to the entity in question. The canonical path has a form illustrated by the following example: /t;tenant-id/e;env-id/r;resource-id The above example is a canonical path to a resource with ID resource-id which is located in environment env-id which is inside a tenant tenant-id. The type specifiers in the individual path segments can be these: t - tenant e - environment rt - resource type mt - metric type f - feed r - resource m - metric And as I understand it, the resource path also contains the parent resource prepended to this. So one can always figure out where the resource came from and can uniquely identify a resource. A resource path is just one kind of canonical path (the others are given in the above except). What fields should we parse out and display in the UI as individual fields? I?ll start this conversation by assuming that the environment and tenant are not very useful to display. Then what about feed? And since resource is what we are after that one is obvious as needed. WDYT? And then there is the question of how Ancestry path should be viewed ? but for now lets just stick to the above question of how should resources be viewed in the UI? Ancestry Path is another topic for later. Please refer to Jira: https://issues.jboss.org/browse/HAWKULAR-605 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150910/5334f64a/attachment.html From mazz at redhat.com Thu Sep 10 16:07:53 2015 From: mazz at redhat.com (John Mazzitelli) Date: Thu, 10 Sep 2015 16:07:53 -0400 (EDT) Subject: [Hawkular-dev] Resource Path view in UI In-Reply-To: References: Message-ID: <677081632.8841184.1441915673100.JavaMail.zimbra@redhat.com> Correct me if I'm wrong, but doesn't the agent store the resource "name" in the inventory's properties? I thought each resource has a set of properties associated with it and the agent will populate a "name" property with the name of the resource in there. If so, you could use that to display the name of the resource. Of course, you would need to query inventory for those properties and you'd need to do the same query for each ancestor (parent, grandparent, etc) if you want to show the names in an ancestry field. ----- Original Message ----- > Now that we have switched to Resource Paths (canonical paths from Inventory) > it has come up that displaying the resource path is not very readable and > requires parsing to make sense of it. > > Please see the current use case as example: > https://issues.jboss.org/secure/attachment/12392747/AlertCenter2.jpg > https://issues.jboss.org/secure/attachment/12392582/Alert-detailsnu.jpg > > > An example resource path: > /t;28026b36-8fe4-4332-84c8-524e173a68bf/e;test/f;localhost/r;localhost~Local~~ > > To further break the above example into its pieces: > tenantId : /t;28026b36-8fe4-4332-84c8-524e173a68bf > environment : test > feed : localhost > resourceId : localhost > > > > The following snippet explaining the resource path was extracted from: > http://www.hawkular.org/docs/components/inventory/index.html > > Canonical Paths > > A canonical path follows the contains relationships from a tenant down to the > entity in question. > > > The canonical path has a form illustrated by the following example: > /t; tenant-id /e; env-id /r; resource-id > > > The above example is a canonical path to a resource with ID resource-id which > is located in environment env-id which is inside a tenant tenant-id . > > > The type specifiers in the individual path segments can be these: > > > * > t - tenant > > * > e - environment > > * > rt - resource type > > * > mt - metric type > > * > f - feed > > * > r - resource > > * > m - metric > > And as I understand it, the resource path also contains the parent resource > prepended to this. So one can always figure out where the resource came from > and can uniquely identify a resource. > A resource path is just one kind of canonical path (the others are given in > the above except). > > > > What fields should we parse out and display in the UI as individual fields? > > I?ll start this conversation by assuming that the environment and tenant are > not very useful to display. Then what about feed? And since resource is what > we are after that one is obvious as needed. > WDYT? > > And then there is the question of how Ancestry path should be viewed ? but > for now lets just stick to the above question of how should resources be > viewed in the UI? > Ancestry Path is another topic for later. > > > Please refer to Jira: https://issues.jboss.org/browse/HAWKULAR-605 > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mithomps at redhat.com Thu Sep 10 20:06:26 2015 From: mithomps at redhat.com (mike thompson) Date: Thu, 10 Sep 2015 17:06:26 -0700 Subject: [Hawkular-dev] Keycloak Authentication token Message-ID: <1D1B5E8D-9BF9-42A8-82F4-BB5ED5D584E5@redhat.com> Juca, Is there an api to get the bearer token set by KeyCloak? This seems like something we should have easy access to now that we are using Web sockets that have their own authentication. The current persona contains: createdAt: "2015-09-10T14:51:06.173Z" id: "28026b36-8fe4-4332-84c8-524e173a68bf? name: "John Doe" updatedAt: "2015-09-10T14:51:06.173Z? Can we get authenticationToken added to this? Or is there another better way? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150910/fdf8bb67/attachment.html From jpkroehling at redhat.com Fri Sep 11 02:31:47 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 11 Sep 2015 08:31:47 +0200 Subject: [Hawkular-dev] Keycloak Authentication token In-Reply-To: <1D1B5E8D-9BF9-42A8-82F4-BB5ED5D584E5@redhat.com> References: <1D1B5E8D-9BF9-42A8-82F4-BB5ED5D584E5@redhat.com> Message-ID: <55F27553.2060004@redhat.com> Mike, What's the case you have in mind? From the UI, there's already a Keycloak object that exposes the tokens. From the backend, you should be able to get the token via a KeycloakPrincipal (a Principal available via JAAS), but you'd have to use it together with the Hawkular-Persona header. Depending on your use case, we can discuss making the token available via Accounts API. - Juca. On 09/11/2015 02:06 AM, mike thompson wrote: > Juca, > > Is there an api to get the bearer token set by KeyCloak? > > This seems like something we should have easy access to now that we are > using Web sockets that have their own authentication. > The current persona contains: > > createdAt: "2015-09-10T14:51:06.173Z" > id: "28026b36-8fe4-4332-84c8-524e173a68bf? > name: "John Doe" > updatedAt: "2015-09-10T14:51:06.173Z? > > Can we get authenticationToken added to this? > > Or is there another better way? > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From lclayton at redhat.com Mon Sep 14 15:46:28 2015 From: lclayton at redhat.com (Liz Clayton) Date: Mon, 14 Sep 2015 15:46:28 -0400 Subject: [Hawkular-dev] Resource Path view in UI In-Reply-To: <677081632.8841184.1441915673100.JavaMail.zimbra@redhat.com> References: <677081632.8841184.1441915673100.JavaMail.zimbra@redhat.com> Message-ID: Hi, On Thu, Sep 10, 2015 at 4:07 PM, John Mazzitelli wrote: > Correct me if I'm wrong, but doesn't the agent store the resource "name" > in the inventory's properties? I thought each resource has a set of > properties associated with it and the agent will populate a "name" property > with the name of the resource in there. If so, you could use that to > display the name of the resource. So we can refer to the resource by "name," in the Alert center summary view? > Of course, you would need to query inventory for those properties and > you'd need to do the same query for each ancestor (parent, grandparent, > etc) if you want to show the names in an ancestry field. > And the details view would/could show the path, or the name:value for each item in the path? Thanks, Liz > > ----- Original Message ----- > > Now that we have switched to Resource Paths (canonical paths from > Inventory) > > it has come up that displaying the resource path is not very readable and > > requires parsing to make sense of it. > > > > Please see the current use case as example: > > https://issues.jboss.org/secure/attachment/12392747/AlertCenter2.jpg > > https://issues.jboss.org/secure/attachment/12392582/Alert-detailsnu.jpg > > > > > > An example resource path: > > > /t;28026b36-8fe4-4332-84c8-524e173a68bf/e;test/f;localhost/r;localhost~Local~~ > > > > To further break the above example into its pieces: > > tenantId : /t;28026b36-8fe4-4332-84c8-524e173a68bf > > environment : test > > feed : localhost > > resourceId : localhost > > > > > > > > The following snippet explaining the resource path was extracted from: > > http://www.hawkular.org/docs/components/inventory/index.html > > > > Canonical Paths > > > > A canonical path follows the contains relationships from a tenant down > to the > > entity in question. > > > > > > The canonical path has a form illustrated by the following example: > > /t; tenant-id /e; env-id /r; resource-id > > > > > > The above example is a canonical path to a resource with ID resource-id > which > > is located in environment env-id which is inside a tenant tenant-id . > > > > > > The type specifiers in the individual path segments can be these: > > > > > > * > > t - tenant > > > > * > > e - environment > > > > * > > rt - resource type > > > > * > > mt - metric type > > > > * > > f - feed > > > > * > > r - resource > > > > * > > m - metric > > > > And as I understand it, the resource path also contains the parent > resource > > prepended to this. So one can always figure out where the resource came > from > > and can uniquely identify a resource. > > A resource path is just one kind of canonical path (the others are given > in > > the above except). > > > > > > > > What fields should we parse out and display in the UI as individual > fields? > > > > I?ll start this conversation by assuming that the environment and tenant > are > > not very useful to display. Then what about feed? And since resource is > what > > we are after that one is obvious as needed. > > WDYT? > > > > And then there is the question of how Ancestry path should be viewed ? > but > > for now lets just stick to the above question of how should resources be > > viewed in the UI? > > Ancestry Path is another topic for later. > > > > > > Please refer to Jira: https://issues.jboss.org/browse/HAWKULAR-605 > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150914/2b959deb/attachment.html From lkrejci at redhat.com Tue Sep 15 06:46:19 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 15 Sep 2015 12:46:19 +0200 Subject: [Hawkular-dev] Resource Path view in UI In-Reply-To: References: <677081632.8841184.1441915673100.JavaMail.zimbra@redhat.com> Message-ID: <2147900.MygGE5gWYg@localhost.localdomain> On Monday, September 14, 2015 15:46:28 Liz Clayton wrote: > Hi, > > On Thu, Sep 10, 2015 at 4:07 PM, John Mazzitelli wrote: > > Correct me if I'm wrong, but doesn't the agent store the resource "name" > > in the inventory's properties? I thought each resource has a set of > > properties associated with it and the agent will populate a "name" > > property > > with the name of the resource in there. If so, you could use that to > > display the name of the resource. > > So we can refer to the resource by "name," in the Alert center summary > view? > Well, yes and no. The name of a resource describes what the resource is in a human readable way but doesn't place it in a hierarchy. For example if you have a deployment "app.war", that name is not enough to distinguish between 5 such deployments, each on a different server. If more of them end up in the summary view, the user wouldn't be able to tell them apart. The resources are hierarchical and that hierarchy is expressed in the resource path. The problem is that resource path is essentially an internal representation, using slash-separated, type-annotated IDs, not names (which are editable, unlike IDs). In RHQ, this was solved by a showing an "ancestry" of a resource - the ancestry would start by the name of the resource itself and would also show the names of all the resources "above it" in the hierarchy, separated by a "<". E.g. "app.war < EAP15". On hover, this was further expanded to include the types of the individual resources in the ancestry to futher help with resolving possible name duplicates. To remain performant, this would probably need some support in the backend (the same way it had support in RHQ) that would pre-compute those "hierarchical names" so that the UI wouldn't need to query for them every time it needed it for every row in the summary. > > Of course, you would need to query inventory for those properties and > > you'd need to do the same query for each ancestor (parent, grandparent, > > etc) if you want to show the names in an ancestry field. > > And the details view would/could show the path, or the name:value for each > item in the path? I think I sort of answered this already above. > > Thanks, > Liz > > > ----- Original Message ----- > > > > > Now that we have switched to Resource Paths (canonical paths from > > > > Inventory) > > > > > it has come up that displaying the resource path is not very readable > > > and > > > requires parsing to make sense of it. > > > > > > Please see the current use case as example: > > > https://issues.jboss.org/secure/attachment/12392747/AlertCenter2.jpg > > > https://issues.jboss.org/secure/attachment/12392582/Alert-detailsnu.jpg > > > > > An example resource path: > > /t;28026b36-8fe4-4332-84c8-524e173a68bf/e;test/f;localhost/r;localhost~Loc > > al~~> > > > To further break the above example into its pieces: > > > tenantId : /t;28026b36-8fe4-4332-84c8-524e173a68bf > > > environment : test > > > feed : localhost > > > resourceId : localhost > > > > > > > > > > > > The following snippet explaining the resource path was extracted from: > > > http://www.hawkular.org/docs/components/inventory/index.html > > > > > > Canonical Paths > > > > > > A canonical path follows the contains relationships from a tenant down > > > > to the > > > > > entity in question. > > > > > > > > > The canonical path has a form illustrated by the following example: > > > /t; tenant-id /e; env-id /r; resource-id > > > > > > > > > The above example is a canonical path to a resource with ID resource-id > > > > which > > > > > is located in environment env-id which is inside a tenant tenant-id . > > > > > > The type specifiers in the individual path segments can be these: > > > * > > > > > > t - tenant > > > > > > * > > > > > > e - environment > > > > > > * > > > > > > rt - resource type > > > > > > * > > > > > > mt - metric type > > > > > > * > > > > > > f - feed > > > > > > * > > > > > > r - resource > > > > > > * > > > > > > m - metric > > > > > > And as I understand it, the resource path also contains the parent > > > > resource > > > > > prepended to this. So one can always figure out where the resource came > > > > from > > > > > and can uniquely identify a resource. > > > A resource path is just one kind of canonical path (the others are given > > > > in > > > > > the above except). > > > > > > > > > > > > What fields should we parse out and display in the UI as individual > > > > fields? > > > > > I?ll start this conversation by assuming that the environment and tenant > > > > are > > > > > not very useful to display. Then what about feed? And since resource is > > > > what > > > > > we are after that one is obvious as needed. > > > WDYT? > > > > > > And then there is the question of how Ancestry path should be viewed ? > > > > but > > > > > for now lets just stick to the above question of how should resources be > > > viewed in the UI? > > > Ancestry Path is another topic for later. > > > > > > > > > Please refer to Jira: https://issues.jboss.org/browse/HAWKULAR-605 > > > > > > _______________________________________________ > > > hawkular-dev mailing list > > > hawkular-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jshaughn at redhat.com Tue Sep 15 09:15:37 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Tue, 15 Sep 2015 09:15:37 -0400 Subject: [Hawkular-dev] Resource Path view in UI In-Reply-To: <2147900.MygGE5gWYg@localhost.localdomain> References: <677081632.8841184.1441915673100.JavaMail.zimbra@redhat.com> <2147900.MygGE5gWYg@localhost.localdomain> Message-ID: <55F819F9.6080807@redhat.com> On 9/15/2015 6:46 AM, Lukas Krejci wrote: > On Monday, September 14, 2015 15:46:28 Liz Clayton wrote: >> Hi, >> >> On Thu, Sep 10, 2015 at 4:07 PM, John Mazzitelli wrote: >>> Correct me if I'm wrong, but doesn't the agent store the resource "name" >>> in the inventory's properties? I thought each resource has a set of >>> properties associated with it and the agent will populate a "name" >>> property >>> with the name of the resource in there. If so, you could use that to >>> display the name of the resource. >> So we can refer to the resource by "name," in the Alert center summary >> view? >> > Well, yes and no. The name of a resource describes what the resource is in a > human readable way but doesn't place it in a hierarchy. For example if you > have a deployment "app.war", that name is not enough to distinguish between 5 > such deployments, each on a different server. If more of them end up in the > summary view, the user wouldn't be able to tell them apart. > > The resources are hierarchical and that hierarchy is expressed in the resource > path. The problem is that resource path is essentially an internal > representation, using slash-separated, type-annotated IDs, not names (which > are editable, unlike IDs). > > In RHQ, this was solved by a showing an "ancestry" of a resource - the > ancestry would start by the name of the resource itself and would also show > the names of all the resources "above it" in the hierarchy, separated by a > "<". E.g. "app.war < EAP15". On hover, this was further expanded to include > the types of the individual resources in the ancestry to futher help with > resolving possible name duplicates. > > To remain performant, this would probably need some support in the backend > (the same way it had support in RHQ) that would pre-compute those > "hierarchical names" so that the UI wouldn't need to query for them every time > it needed it for every row in the summary. +1, I think a pre-computed value is the way to go. For details on the RHQ approach, see here: https://docs.jboss.org/author/display/RHQ/Disambiguation+API#DisambiguationAPI-RHQ4Changes I'm not sure we'd need all of the type info, etc, but certainly the hierarchy of resource names. It depends on whether we want to offer links to the ancestry resources. >>> Of course, you would need to query inventory for those properties and >>> you'd need to do the same query for each ancestor (parent, grandparent, >>> etc) if you want to show the names in an ancestry field. >> And the details view would/could show the path, or the name:value for each >> item in the path? > I think I sort of answered this already above. > >> Thanks, >> Liz >> >>> ----- Original Message ----- >>> >>>> Now that we have switched to Resource Paths (canonical paths from >>> Inventory) >>> >>>> it has come up that displaying the resource path is not very readable >>>> and >>>> requires parsing to make sense of it. >>>> >>>> Please see the current use case as example: >>>> https://issues.jboss.org/secure/attachment/12392747/AlertCenter2.jpg >>>> https://issues.jboss.org/secure/attachment/12392582/Alert-detailsnu.jpg >>>> An example resource path: >>> /t;28026b36-8fe4-4332-84c8-524e173a68bf/e;test/f;localhost/r;localhost~Loc >>> al~~> >>>> To further break the above example into its pieces: >>>> tenantId : /t;28026b36-8fe4-4332-84c8-524e173a68bf >>>> environment : test >>>> feed : localhost >>>> resourceId : localhost >>>> >>>> >>>> >>>> The following snippet explaining the resource path was extracted from: >>>> http://www.hawkular.org/docs/components/inventory/index.html >>>> >>>> Canonical Paths >>>> >>>> A canonical path follows the contains relationships from a tenant down >>> to the >>> >>>> entity in question. >>>> >>>> >>>> The canonical path has a form illustrated by the following example: >>>> /t; tenant-id /e; env-id /r; resource-id >>>> >>>> >>>> The above example is a canonical path to a resource with ID resource-id >>> which >>> >>>> is located in environment env-id which is inside a tenant tenant-id . >>>> >>>> The type specifiers in the individual path segments can be these: >>>> * >>>> >>>> t - tenant >>>> >>>> * >>>> >>>> e - environment >>>> >>>> * >>>> >>>> rt - resource type >>>> >>>> * >>>> >>>> mt - metric type >>>> >>>> * >>>> >>>> f - feed >>>> >>>> * >>>> >>>> r - resource >>>> >>>> * >>>> >>>> m - metric >>>> >>>> And as I understand it, the resource path also contains the parent >>> resource >>> >>>> prepended to this. So one can always figure out where the resource came >>> from >>> >>>> and can uniquely identify a resource. >>>> A resource path is just one kind of canonical path (the others are given >>> in >>> >>>> the above except). >>>> >>>> >>>> >>>> What fields should we parse out and display in the UI as individual >>> fields? >>> >>>> I?ll start this conversation by assuming that the environment and tenant >>> are >>> >>>> not very useful to display. Then what about feed? And since resource is >>> what >>> >>>> we are after that one is obvious as needed. >>>> WDYT? >>>> >>>> And then there is the question of how Ancestry path should be viewed ? >>> but >>> >>>> for now lets just stick to the above question of how should resources be >>>> viewed in the UI? >>>> Ancestry Path is another topic for later. >>>> >>>> >>>> Please refer to Jira: https://issues.jboss.org/browse/HAWKULAR-605 >>>> >>>> _______________________________________________ >>>> hawkular-dev mailing list >>>> hawkular-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Tue Sep 15 11:01:56 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 15 Sep 2015 17:01:56 +0200 Subject: [Hawkular-dev] RfC: Remove / rename servers from / in inventory Message-ID: Hey, currently we can add (WF) servers to inventory easily, but we can't get rid of them anymore. Similarly it is not really clear what should happen if a serve changes its name. In WildFly 10 supposedly a new uuid kind of thing should appear, that would identify each WF uniquely and which could serve as the "primary key", so that the name is just a display attribute that can be changed at will, so that the renaming piece is sort of solved. Now to the semantics of "uninventory" (not deletion of physical servers). In RHQ we had uninventory that removed the server from inventory. After a discovery scan happened, it showed up again and could be re-inventoried. During uninventory, all collected metrics etc. and alert definitions for the resource were wiped (or prepared for wiping). This (wiping of recorded metrics) has more than one time been mentioned as undesired e.g. in case that the uninventory happened due to bad luck. Also in RHQ there were issues when e.g. the machine where a WF-server was running on got renamed. Some questions that come to mind: - Should we wipe data when a server gets uninventoried? - How can we prevent it from entering inventory directly after an uninventory again? - what states do we need (data wiping may take too long to be done synchronously)? - would users perhaps want a data dump to external storage before wiping? - How can we make sure that servers without that uuid can still be identified even after rename (of the machine)? - How can we make sure that servers without that uuid can still be identified even when they are moved to a different pod Heiko From jshaughn at redhat.com Tue Sep 15 13:52:23 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Tue, 15 Sep 2015 13:52:23 -0400 Subject: [Hawkular-dev] RfC: Remove / rename servers from / in inventory In-Reply-To: References: Message-ID: <55F85AD7.3070300@redhat.com> On 9/15/2015 11:01 AM, Heiko W.Rupp wrote: > Hey, > > currently we can add (WF) servers to inventory easily, but we can't get > rid of them anymore. Similarly it is not really clear what should happen > if a serve changes its name. > > In WildFly 10 supposedly a new uuid kind of thing should appear, that > would identify each WF uniquely and which could serve as the "primary > key", so that the name is just a display attribute that can be changed > at will, so that the renaming piece is sort of solved. > > Now to the semantics of "uninventory" (not deletion of physical > servers). > In RHQ we had uninventory that removed the server from inventory. > After a discovery scan happened, it showed up again and could be > re-inventoried. > During uninventory, all collected metrics etc. and alert definitions for > the resource > were wiped (or prepared for wiping). > This (wiping of recorded metrics) has more than one time been mentioned > as undesired e.g. in case that the uninventory happened due to bad luck. > Also in RHQ there were issues when e.g. the machine where a WF-server > was running on got renamed. Before answering the questions below. I'm not so sure uninventory should be a thing. In RHQ it meant a wipe of server-side data. In hawkular I wonder whether it just means turning off the feed of that data and letting it disappear from the recent time windows. Filtering, if necessary, should maybe be handled in the presentation layer. Having a feed be able to report the set of resources it *can* report on, and then supplying it a list of the resources it *should* report on, seems like a general mechanism we could somehow build out. Or, it could be less granular, just down to type level. The initial set of types should be small. > > Some questions that come to mind: > - Should we wipe data when a server gets uninventoried? No. I think data should just live on until some TTL perhaps kicks in. I do sometimes wonder how important really old data is. Maybe useful for trend analysis but not so much for problem resolution, which is probably more reliant on say the last 12-48 hours. > - How can we prevent it from entering inventory directly after an > uninventory again? If we are able to supply a feed with a list of types/resources to not report on then it's not a problem. > - what states do we need (data wiping may take too long to be done > synchronously)? Don't wipe data, let it be purged later and just be irrelevant based on the query windows being looked at. > - would users perhaps want a data dump to external storage before > wiping? Not a problem if we keep the data. > - How can we make sure that servers without that uuid can still be > identified even after rename (of the machine)? I think this is a problem to ignore. If the feed name/resource path changes then it's basically new resource. I don't think we need to be this complex. If we maintain our feed name I think the resource path should remain the same for the resources it is monitoring. If we don't do that already then we should look at doing that. > - How can we make sure that servers without that uuid can still be > identified even when they are moved to a different pod Maybe the same as above, I'm not sure. From hrupp at redhat.com Tue Sep 15 15:12:33 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 15 Sep 2015 21:12:33 +0200 Subject: [Hawkular-dev] Remove / rename servers from / in inventory In-Reply-To: <55F85AD7.3070300@redhat.com> References: <55F85AD7.3070300@redhat.com> Message-ID: On 15 Sep 2015, at 19:52, Jay Shaughnessy wrote: > No. I think data should just live on until some TTL perhaps kicks in. That would work for metrics (out of the box). What about other data like alert trigger definitions? Or group memberships? > I think this is a problem to ignore. If the feed name/resource path > changes then it's basically new resource. I don't think Do I understand you correctly that you say that a feed name once set will be the same no matter where the (WF-)server moves to? So that a resource under that feed would be the same no matter where the feed is located. For a WF-Server with the embedded agent, this is certainly true. I may may be wrong here: in OpenShift3 / Kubernetes (K8s), K8s may kill a container at any time and start it on a different machine. If an application consist of e.g. 3 servers, having one killed and restarted would imply to me that all the previous "settings" would still apply to the freshly started instance no matter where it is then located. In this case, we should probably record the fact of the restart, but the resource would at least logically (from my PoV) be the same one of 3 servers of that app. (of course that last paragraph does not apply to explicit manual uninventory). I also recall from RHQ that we had a situation where a user was deploying my-super-app-v1.war and later my-super-app-v2.war, where both deployments are the same app and even with a different name just two different versions of the same resource and not different resources. Also in this case, the user wants all metrics, trigger definitions etc. to survive the deployment of v2. From jshaughn at redhat.com Tue Sep 15 16:04:47 2015 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Tue, 15 Sep 2015 16:04:47 -0400 Subject: [Hawkular-dev] Remove / rename servers from / in inventory In-Reply-To: References: <55F85AD7.3070300@redhat.com> Message-ID: <55F879DF.8020903@redhat.com> On 9/15/2015 3:12 PM, Heiko W.Rupp wrote: > On 15 Sep 2015, at 19:52, Jay Shaughnessy wrote: >> No. I think data should just live on until some TTL perhaps kicks in. > That would work for metrics (out of the box). > What about other data like alert trigger definitions? > Or group memberships? Perhaps we look at a reaper sort of approach where we look for "dead" resources in inventory, based on some sort of criteria, and then perform clean-up. >> I think this is a problem to ignore. If the feed name/resource path >> changes then it's basically new resource. I don't think > Do I understand you correctly that you say that a feed name > once set will be the same no matter where the (WF-)server > moves to? So that a resource under that feed would be the > same no matter where the feed is located. > For a WF-Server with the embedded agent, this is certainly > true. > > I may may be wrong here: in OpenShift3 / Kubernetes (K8s), > K8s may kill a container at any time and start it on a different > machine. If an application consist of e.g. 3 servers, having > one killed and restarted would imply to me that all the previous > "settings" would still apply to the freshly started instance no > matter where it is then located. In this case, we should probably > record the fact of the restart, but the resource would at least > logically (from my PoV) be the same one of 3 servers of that app. I defer to others to answer this question but if feedIds don't survive a machine change I think we're in trouble. I rather thought a feed would just report it's machine in some way, so that machine metrics could be correlated to app performance. > (of course that last paragraph does not apply to explicit manual > uninventory). > > I also recall from RHQ that we had a situation where a user was > deploying my-super-app-v1.war and later my-super-app-v2.war, > where both deployments are the same app and even with a different > name just two different versions of the same resource and not > different resources. Also in this case, the user wants all metrics, > trigger definitions etc. to survive the deployment of v2. You are right about this and I hadn't forgot about it. But this feature is a b*tch. It was really hard in RHQ and may be nigh impossible in Hawkular unless we basically do the same thing and start stripping versions out of resource paths. Or maybe we're already doing that, I don't know. It depends on whether the agent is using the path as-is from wfly, which afaik is still using the artifact name in the path. From loleary at redhat.com Tue Sep 15 22:10:14 2015 From: loleary at redhat.com (Larry O'Leary) Date: Tue, 15 Sep 2015 21:10:14 -0500 Subject: [Hawkular-dev] Remove / rename servers from / in inventory In-Reply-To: <55F879DF.8020903@redhat.com> References: <55F85AD7.3070300@redhat.com> <55F879DF.8020903@redhat.com> Message-ID: On Tue, Sep 15, 2015 at 3:04 PM, Jay Shaughnessy wrote: > > > On 9/15/2015 3:12 PM, Heiko W.Rupp wrote: > > On 15 Sep 2015, at 19:52, Jay Shaughnessy wrote: > >> No. I think data should just live on until some TTL perhaps kicks in. > > That would work for metrics (out of the box). > > What about other data like alert trigger definitions? > > Or group memberships? > > Perhaps we look at a reaper sort of approach where we look for "dead" > resources in inventory, based on some sort of criteria, and then perform > clean-up. > >From my PoV the data should just be kept and the legacy concept of uninventory should just become a hybrid of disable and ignore. Collection of new data should not occur as the resource has been "de-activated". Alert trigger defs and group membership should be made invisible. All historic data is retained until its TTL expires. And all configuration defs/settings, collection schedules and the like, would just remain Because the resource is "disabled" re-discovery shouldn't be an issue. If the user wants to "re-activate" it, then everything remains intact. The downside to this would be that a user could not "reset" the resource and start from scratch. But, I am not really sure that this is a problem assuming default configuration/settings/templates can be reapplied. > > I also recall from RHQ that we had a situation where a user was > > deploying my-super-app-v1.war and later my-super-app-v2.war, > > where both deployments are the same app and even with a different > > name just two different versions of the same resource and not > > different resources. Also in this case, the user wants all metrics, > > trigger definitions etc. to survive the deployment of v2. > > You are right about this and I hadn't forgot about it. But this feature > is a b*tch. It was really hard in RHQ and may be nigh impossible in > Hawkular unless we basically do the same thing and start stripping > versions out of resource paths. Or maybe we're already doing that, I > don't know. It depends on whether the agent is using the path as-is from > wfly, which afaik is still using the artifact name in the path. > > I think it would be best just to provide a resource reconciliation function that allows a user to perform operation such as: - Resource A and Resource B are the same - Merge Resource A and Resource B as Resource A | B | C - Resource B is version 2 of Resource A Because resources would no longer be deleted/removed from inventory, it allows the merging of updated/new resources without any issue -- assuming the types are the same. So, from your original questions: Some questions that come to mind: > - Should we wipe data when a server gets uninventoried? No. - How can we prevent it from entering inventory directly after an > uninventory again? If it is simply disabled/ignored, we won't have to. Only need a way to allow the user to allow it to be "re-activated". This would be similar to what was done in RHQ with re-discovery. > - what states do we need (data wiping may take too long to be done > synchronously)? No necessary. Data should just expire on its own based on data retention settings. - would users perhaps want a data dump to external storage before > wiping? Yes. For the resource that was "de-activated" the user should still be able to get to its data/metric views and grab the data that has not yet expired. > - How can we make sure that servers without that uuid can still be > identified even after rename (of the machine)? > - How can we make sure that servers without that uuid can still be > identified even when they are moved to a different pod > As Jay indicated in his response, I am not sure this is really a problem. Wouldn't a new UUID equal a new resource? Assuming there was a reconciliation feature, the user could re-link a new UUID to an existing resource with a different UUID if they are confident that it is the same resource. It would be preferred that this could happen automatically, but ideally the user knows best and should have the ability to link/merge the resources. -- Larry O'Leary https://plus.google.com/+LarryOLeary -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150915/c28028f0/attachment.html From tsegismo at redhat.com Wed Sep 16 04:58:06 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 16 Sep 2015 10:58:06 +0200 Subject: [Hawkular-dev] [Metrics] HWKMETRICS-125 Use JBoss Logging as in all other Hawkular components Message-ID: <55F92F1E.7010808@redhat.com> Hi, I created a pull request for $subject. https://github.com/hawkular/hawkular-metrics/pull/352 I'm writing to share some implementation details. I followed some Hawkular wide conventions (which I gathered from mailing list threads, so tell me if it's wrong): - put logging stuff into a 'log' package - in the message logger interface, method names should start with the log level [1] - logger instances should be named 'log' [2] - only put messages which need an id (level INFO and above) in logger interfaces [3] I tried to draw up some Metrics specific conventions (some may not be specific, I just found no mention of them in the threads): - group all messages in one class for maintainability [4] - instead of creating a logger instance in the message logger interface, use XXXLogging helper class to get a logger instance with proper category (inspired by what Hibernate does) [5] - use existing message logger instance to log trace/debug messages instead of creating a separate logger instance [6] The logging 'projectCode' is 'HAWKMETRICS'. Does that fit everyone? Note that until we no longer need to support EAP 6.4, we must not use primitive arguments in logging interfaces. [7] Also, logback is now the logging backend for tests. It allows to set log level with a system property while still having a default value. [8][9] No Maven filtering/replace dance involved. If there's no issue with all of the above, then maybe we should not wait too long before merging, because a bunch of files are impacted. Thanks, Thomas [1] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-75a8549533d503e0f032038ff7442e06R63 [2] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-26d97c7615667d28c50de926537fd343R35 [3] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-1354be094ca909a30eedafc9fdf8fc1fR70 [4] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-75a8549533d503e0f032038ff7442e06R39 [5] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-fb0691f5bb1e1793a11ed3f20a3fb6d2R28 [6] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-04e1ad2ad2cadc473893730a61b3e271R607 [7] http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000378.html [8] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-e0d878a774dafba50428080c38a192b8R24 [9] https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-e0d878a774dafba50428080c38a192b8R33 From hrupp at redhat.com Wed Sep 16 05:29:39 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 16 Sep 2015 11:29:39 +0200 Subject: [Hawkular-dev] [Metrics] HWKMETRICS-125 Use JBoss Logging as in all other Hawkular components In-Reply-To: <55F92F1E.7010808@redhat.com> References: <55F92F1E.7010808@redhat.com> Message-ID: <2ABF0BAD-4F2E-4D96-AD57-66A2DFC38AAC@redhat.com> That looks sound to me. If that is agreed upon, your writeup could also be put into http://www.hawkular.org/docs/dev/development.html#_code_style to enhance the current short bullet point. Heiko On 16 Sep 2015, at 10:58, Thomas Segismont wrote: > Hi, > > I created a pull request for $subject. > https://github.com/hawkular/hawkular-metrics/pull/352 > > I'm writing to share some implementation details. > > I followed some Hawkular wide conventions (which I gathered from > mailing > list threads, so tell me if it's wrong): > - put logging stuff into a 'log' package > - in the message logger interface, method names should start with the > log level [1] > - logger instances should be named 'log' [2] > - only put messages which need an id (level INFO and above) in logger > interfaces [3] > > I tried to draw up some Metrics specific conventions (some may not be > specific, I just found no mention of them in the threads): > - group all messages in one class for maintainability [4] > - instead of creating a logger instance in the message logger > interface, > use XXXLogging helper class to get a logger instance with proper > category (inspired by what Hibernate does) [5] > - use existing message logger instance to log trace/debug messages > instead of creating a separate logger instance [6] > > The logging 'projectCode' is 'HAWKMETRICS'. Does that fit everyone? > > Note that until we no longer need to support EAP 6.4, we must not use > primitive arguments in logging interfaces. [7] > > Also, logback is now the logging backend for tests. It allows to set > log > level with a system property while still having a default value. > [8][9] > No Maven filtering/replace dance involved. > > If there's no issue with all of the above, then maybe we should not > wait > too long before merging, because a bunch of files are impacted. > > Thanks, > Thomas > > [1] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-75a8549533d503e0f032038ff7442e06R63 > [2] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-26d97c7615667d28c50de926537fd343R35 > [3] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-1354be094ca909a30eedafc9fdf8fc1fR70 > > [4] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-75a8549533d503e0f032038ff7442e06R39 > [5] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-fb0691f5bb1e1793a11ed3f20a3fb6d2R28 > [6] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-04e1ad2ad2cadc473893730a61b3e271R607 > > > [7] > http://lists.jboss.org/pipermail/hawkular-dev/2015-March/000378.html > > [8] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-e0d878a774dafba50428080c38a192b8R24 > [9] > https://github.com/hawkular/hawkular-metrics/pull/352/files#diff-e0d878a774dafba50428080c38a192b8R33 > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From lkrejci at redhat.com Thu Sep 17 11:14:59 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Thu, 17 Sep 2015 17:14:59 +0200 Subject: [Hawkular-dev] Remove / rename servers from / in inventory In-Reply-To: References: <55F879DF.8020903@redhat.com> Message-ID: <1504272.P0LYpROuMJ@localhost.localdomain> I agree 100% with what Jay and Larry suggest here, but I want to point out one other thing that Jay mentioned on the last water-cooler call. One of the problems with discovery and (un)inventory in RHQ was that by default we discovered and inventoried A LOT. This then required some users to go back and uninventory unnecessary stuff and ignore it in the discovery q. What Jay suggested and I thought was a brilliant idea was to "start small" and only discover a very "crude picture" of the resource tree with a hint of what could be discovered in addition (i.e. discover an EAP resource with some basic stats, no child resources like deployments, subsystems, etc with the hint of what could be discovered coming from the hierarchical resource types (granted those are not in inventory right now, but that is easily changed)). On Tuesday, September 15, 2015 21:10:14 Larry O'Leary wrote: > On Tue, Sep 15, 2015 at 3:04 PM, Jay Shaughnessy > > wrote: > > On 9/15/2015 3:12 PM, Heiko W.Rupp wrote: > > > On 15 Sep 2015, at 19:52, Jay Shaughnessy wrote: > > >> No. I think data should just live on until some TTL perhaps kicks in. > > > > > > That would work for metrics (out of the box). > > > What about other data like alert trigger definitions? > > > Or group memberships? > > > > Perhaps we look at a reaper sort of approach where we look for "dead" > > resources in inventory, based on some sort of criteria, and then perform > > clean-up. > > > >From my PoV the data should just be kept and the legacy concept of > > uninventory should just become a hybrid of disable and ignore. > > Collection of new data should not occur as the resource has been > "de-activated". > > Alert trigger defs and group membership should be made invisible. > > All historic data is retained until its TTL expires. And all configuration > defs/settings, collection schedules and the like, would just remain > > Because the resource is "disabled" re-discovery shouldn't be an issue. If > the user wants to "re-activate" it, then everything remains intact. The > downside to this would be that a user could not "reset" the resource and > start from scratch. But, I am not really sure that this is a problem > assuming default configuration/settings/templates can be reapplied. > > > > I also recall from RHQ that we had a situation where a user was > > > deploying my-super-app-v1.war and later my-super-app-v2.war, > > > where both deployments are the same app and even with a different > > > name just two different versions of the same resource and not > > > different resources. Also in this case, the user wants all metrics, > > > trigger definitions etc. to survive the deployment of v2. > > > > You are right about this and I hadn't forgot about it. But this feature > > is a b*tch. It was really hard in RHQ and may be nigh impossible in > > Hawkular unless we basically do the same thing and start stripping > > versions out of resource paths. Or maybe we're already doing that, I > > don't know. It depends on whether the agent is using the path as-is from > > wfly, which afaik is still using the artifact name in the path. > > I think it would be best just to provide a resource reconciliation function > that allows a user to perform operation such as: > > - Resource A and Resource B are the same > - Merge Resource A and Resource B as Resource A | B | C > - Resource B is version 2 of Resource A > > Because resources would no longer be deleted/removed from inventory, it > allows the merging of updated/new resources without any issue -- assuming > the types are the same. > > So, from your original questions: > > Some questions that come to mind: > > - Should we wipe data when a server gets uninventoried? > > No. > > - How can we prevent it from entering inventory directly after an > > > uninventory again? > > If it is simply disabled/ignored, we won't have to. Only need a way to > allow the user to allow it to be "re-activated". This would be similar to > what was done in RHQ with re-discovery. > > > - what states do we need (data wiping may take too long to be done > > synchronously)? > > No necessary. Data should just expire on its own based on data retention > settings. > > - would users perhaps want a data dump to external storage before > > > wiping? > > Yes. For the resource that was "de-activated" the user should still be able > to get to its data/metric views and grab the data that has not yet expired. > > > - How can we make sure that servers without that uuid can still be > > identified even after rename (of the machine)? > > - How can we make sure that servers without that uuid can still be > > identified even when they are moved to a different pod > > As Jay indicated in his response, I am not sure this is really a problem. > Wouldn't a new UUID equal a new resource? > > Assuming there was a reconciliation feature, the user could re-link a new > UUID to an existing resource with a different UUID if they are confident > that it is the same resource. It would be preferred that this could happen > automatically, but ideally the user knows best and should have the ability > to link/merge the resources. From jpkroehling at redhat.com Thu Sep 17 11:37:05 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Thu, 17 Sep 2015 17:37:05 +0200 Subject: [Hawkular-dev] Token errors with WebSockets and other backend calls Message-ID: <55FADE21.3010709@redhat.com> All, I've seen some problems lately that are hard to debug and may cause you to waste valuable time. In short, if you face "strange" issues related to authentication, specially when trying features that make use of WebSockets (ie: add deployment), make sure you are accessing the web UI via 127.0.0.1 , and not localhost . In case you are interested in the details, keep reading. Upon login, Keycloak issues a token for the client, taking the auth server host into consideration (as the "issuer authority"). If you use localhost, then that's the hostname that Keycloak will use inside the token. This value is later used to validate the incoming token. Ideally, all the hostnames would be a match, and that's usually the case if you use the "-b" switch when starting Wildfly. But if you don't specify, we fall back to 127.0.0.1 [1] , causing the "backend call" to be 127.0.0.1, while the "frontend call" came via localhost. I have a couple of ideas on how to solve this in our side, but until a fix is done, tested and merged, please use 127.0.0.1 on the UI. 1 - http://git.io/vnJfx - Juca. From mwringe at redhat.com Fri Sep 18 11:21:08 2015 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 18 Sep 2015 11:21:08 -0400 Subject: [Hawkular-dev] Metric id and url restrictions Message-ID: <55FC2BE4.8060606@redhat.com> Currently we are storing metric id as any string. There are no restrictions on how we name the metrics, and when you access a single metric via a url you just need to make sure its url encoded properly here. This works fine when dealing with a client and the Hawkular Metrics directly, but can cause problems when dealing with proxies or other intermediary servers which are not exactly url compliant. The big issue is with slashes in the metric id. Even when properly encoded this can cause problems with things like proxies not accepting them (and even certain web container don't like them). Since we need to be able to integrate over networks with these types of servers, we need to find a solution to resolve this in hawkular. Possible solutions - enforce naming of anything which can show up in a the URL to make sure it cannot include a '/' . This would include things like metric Id. This should probably be enabled by default, but we could have an option to disable this if a user needs it. - create a new endpoint which can be used to perform a POST instead of doing a GET on the metric endpoint. This would mean sending things like the metric id and query string as part of the message body in the POST instead of in the URL. Any thoughts? From mazz at redhat.com Fri Sep 18 11:32:18 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 18 Sep 2015 11:32:18 -0400 (EDT) Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <55FC2BE4.8060606@redhat.com> References: <55FC2BE4.8060606@redhat.com> Message-ID: <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> Metric IDs from our Wildfly Agent contain "/" because those are characters that are valid for Wildfly resources and we use the Wildfly resource names as part of the metric ID to uniquely identify them. So for IDs of metrics coming from things like "/deployment=the-app.war/subsystem=ejb3" will have "/" in them. The only alternative would be for us to replace "/" with a character that is valid for URLs but invalid for Wildfly resource names. Say "$deployment=the-app.war$subsystem=ejb3" or something like that. ----- Original Message ----- > Currently we are storing metric id as any string. There are no > restrictions on how we name the metrics, and when you access a single > metric via a url you just need to make sure its url encoded properly here. > > This works fine when dealing with a client and the Hawkular Metrics > directly, but can cause problems when dealing with proxies or other > intermediary servers which are not exactly url compliant. > > The big issue is with slashes in the metric id. Even when properly > encoded this can cause problems with things like proxies not accepting > them (and even certain web container don't like them). > > Since we need to be able to integrate over networks with these types of > servers, we need to find a solution to resolve this in hawkular. > > Possible solutions > > - enforce naming of anything which can show up in a the URL to make sure > it cannot include a '/' . This would include things like metric Id. This > should probably be enabled by default, but we could have an option to > disable this if a user needs it. > > - create a new endpoint which can be used to perform a POST instead of > doing a GET on the metric endpoint. This would mean sending things like > the metric id and query string as part of the message body in the POST > instead of in the URL. > > Any thoughts? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Fri Sep 18 12:27:31 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 18 Sep 2015 12:27:31 -0400 (EDT) Subject: [Hawkular-dev] how to identify UI clients and which should get responses In-Reply-To: <969079858.13799434.1442592615013.JavaMail.zimbra@redhat.com> Message-ID: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> I'm sending this here because I need the input from several people (specifically Heiko, Juca, and Peter, but others are free to chime in, too). Right now, the UI sends messages over the websocket to the hawkular server to make request for things. That UI request is immediately authenticated via hawkular-accounts (typically the UI sends the keycloak token that we authentiate). At this point, we put the message on the bus so the message can be routed to the proper kettle instance so the message can be forwarded to the feed over the feed websocket connection. Once the feed processes the request, a response may be sent back (asynchronously from the request) - if a response is sent back from the feed, it goes to the kettle instance via websockets, the kettle puts it on the bus, where the listener processing the UI client responses will take the bus message and forward it to the UI over the websocket. Phew. OK, my question is - how does the UI identify itself? We need some string identifier so the response message can be targeted to the proper UI client (today, we don't have UI identifiers, so we just send responses to EVERY UI client but only those UI clients connected to the kettle that the feed is connected to - not good in a clustered environment and not good for security purposes - some UIs will be different tenants and shouldn't see these messages). Here's some use cases to illustrate the problem: 1) I'm logged into the UI using two different browser tabs. In one, I send a "deploy an application" request. Once the response comes back - do I show the response in both tabs? What if I closed the tab that performed the deploy? What should happen? 2) I am logged in kettle #1 as "jdoe" and someone else is logged in kettle #2 as the same "jdoe". I deploy an application. Should that other person get the response that a new app was deployed, too? 3) I am logged into the UI and I deploy an application. I log out (perhaps close the browser) and log back in. Then the deployment finishes and the response comes in - should I get that response (even though its a new browser instance and a new websocket connection?) The way I am thinking about this is Google Drive. I could be logged into my Google Drive account on any number of browsers or tabs - when I make a change to a document in Drive, that change is immediately reflected in every browser/tab I have open to Google Drive, not just in the browser that made the change. Is this the kind of thing we want? Or do we only want the browser that is making the requests get the responses? One thing I am thinking is hawkular-accounts gives us some common ID (not the token) that we assign a UI client and that ID floats in all requests and responses. The bad thing about this is we'll never know the UI client ID until they send their first request. Another thing I am thinking is have it work just like feeds - a feed connects via the URL ws://host/hawkular-command-gateway/feed/. We could let each UI generate their own id and connect via ws://host/hawkular-command-gateway/ui/. The good thing here is the UI knows its ID immediately (since it generated it) and we know it from the start without even getting an initial message (and we can link it with that websocket session easily). Bad thing is only that UI will be able to get responses - it won't work the "Google Drive way" where a user logged into N browsers will get responses in all N browsers. Thoughts? From mazz at redhat.com Fri Sep 18 12:41:55 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 18 Sep 2015 12:41:55 -0400 (EDT) Subject: [Hawkular-dev] how to identify UI clients and which should get responses In-Reply-To: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> References: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> Message-ID: <439967931.13806149.1442594515598.JavaMail.zimbra@redhat.com> By the way, this is related to: https://issues.jboss.org/browse/HAWKULAR-498 ----- Original Message ----- > I'm sending this here because I need the input from several people > (specifically Heiko, Juca, and Peter, but others are free to chime in, too). > > Right now, the UI sends messages over the websocket to the hawkular server to > make request for things. That UI request is immediately authenticated via > hawkular-accounts (typically the UI sends the keycloak token that we > authentiate). At this point, we put the message on the bus so the message > can be routed to the proper kettle instance so the message can be forwarded > to the feed over the feed websocket connection. Once the feed processes the > request, a response may be sent back (asynchronously from the request) - if > a response is sent back from the feed, it goes to the kettle instance via > websockets, the kettle puts it on the bus, where the listener processing the > UI client responses will take the bus message and forward it to the UI over > the websocket. Phew. > > OK, my question is - how does the UI identify itself? We need some string > identifier so the response message can be targeted to the proper UI client > (today, we don't have UI identifiers, so we just send responses to EVERY UI > client but only those UI clients connected to the kettle that the feed is > connected to - not good in a clustered environment and not good for security > purposes - some UIs will be different tenants and shouldn't see these > messages). > > Here's some use cases to illustrate the problem: > > 1) I'm logged into the UI using two different browser tabs. In one, I send a > "deploy an application" request. Once the response comes back - do I show > the response in both tabs? What if I closed the tab that performed the > deploy? What should happen? > > 2) I am logged in kettle #1 as "jdoe" and someone else is logged in kettle #2 > as the same "jdoe". I deploy an application. Should that other person get > the response that a new app was deployed, too? > > 3) I am logged into the UI and I deploy an application. I log out (perhaps > close the browser) and log back in. Then the deployment finishes and the > response comes in - should I get that response (even though its a new > browser instance and a new websocket connection?) > > The way I am thinking about this is Google Drive. I could be logged into my > Google Drive account on any number of browsers or tabs - when I make a > change to a document in Drive, that change is immediately reflected in every > browser/tab I have open to Google Drive, not just in the browser that made > the change. Is this the kind of thing we want? Or do we only want the > browser that is making the requests get the responses? > > One thing I am thinking is hawkular-accounts gives us some common ID (not the > token) that we assign a UI client and that ID floats in all requests and > responses. The bad thing about this is we'll never know the UI client ID > until they send their first request. > > Another thing I am thinking is have it work just like feeds - a feed connects > via the URL ws://host/hawkular-command-gateway/feed/. We could let > each UI generate their own id and connect via > ws://host/hawkular-command-gateway/ui/. The good thing here is the UI > knows its ID immediately (since it generated it) and we know it from the > start without even getting an initial message (and we can link it with that > websocket session easily). Bad thing is only that UI will be able to get > responses - it won't work the "Google Drive way" where a user logged into N > browsers will get responses in all N browsers. > > Thoughts? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From lkrejci at redhat.com Fri Sep 18 15:45:05 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Fri, 18 Sep 2015 21:45:05 +0200 Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities Message-ID: <1880738.1F1UWefaR0@localhost.localdomain> Hi all, pending a peer-review is a new feature in inventory to support bulk creation of entities (in a single transaction). So far, our REST API was rather granular and concentrated on working with single entities or even their individual relationships. This was quite costly especially during create, update or delete operations, where the clients would need to do stuff one-by-one. This is being addressed in Inventory 0.4.0 (hopefully coming to Hawkular Alpha5) with a new REST endpoint called "/bulk". For 0.4.0 we will only support bulk creation with update and delete coming in the next versions. The bulk endpoint accepts a rather complex javascript object that can be used to specify what is being created and where. The best way to explain it is by an example: { "/t;tenant": { "environment": [ { "id": "env1", "properties": {"key": "value"} }, { "id": "env2", "properties": {"key": "value2"} } ], "resourceType": [ { "id": "URL" } ], "metricType": [ { "id": "ResponseTime" } ] }, "/t;tenant/rt;URL": { "data": [ { "role": "configurationSchema", "value": { "title": "URL config schema", "description": "A json schema describing configuration of an URL", "type": "string" } } ], "operationType": [ { "id": "ping" } ] }, "/t;tenant/e;env1": { "resource": [ { "id": "url1", "resourceTypePath": "/t;tenant/rt;URL" } ], "metric": [ { "id": "url1_responseTime", "metricTypePath": "/t;tenant/mt;ResponseTime" } ] }, "/t;tenant/e;env1/r;url1": { "data": [ { "role": "configuration", "value": "http://redhat.com" } ], "relationship": [ { "name": "incorporates", "otherEnd": "/t;tenant/e;env1/m;url1_responseTime", "direction": "outgoing" } ] } } The javascript object above would cause creation of 2 new environments, "env1" and "env2", under the tenant "tenant", a new resource type called "URL" and a new metric type called "ResponseTime". When the URL resource type is created, a configuration schema is created for it (and the configuration schema prescribes that the configuration of the URL is merely a string). Additionally, an operation called "ping" is defined for the resource type. After that a new resource, "url1" is added to environment "env1" with the "URL" resource type and a metric called "url1_responseTime" is also added to the environment. Finally a configuration is added to the resource and the resource is configured to incorporate the previously created metric. The structure of the javascript object is following: 1) The top-level keys are paths to the inventory entities *under* which the new entities should be added, e.g. the first key "/t;tenant" means that the value of that key describes what should be created directly under a tenant called "tenant". The second key, "/t;tenant/rt;URL", tells the REST API that the value of that key describes what should be added "under" that resource type. The structure of what kinds of entities can contain what kinds of entities can be found in the documentation [1]. Notice that the order of the keys in the object is significant. 2) The second-level keys describe what type of data to create under a given parent (notice that you can create more than 1 type of data under some entity types). The values of those keys are arrays of actual blueprint objects describing the new entities to be created (each entity type can have different format). The blueprint objects are the same as the ones passed in to the granular REST endpoints that already exist. The response always has the 201 - Created status code and is a JSON object with the types of entities created as keys and values are again objects with keys being the canonical paths to the entities created and values are HTTP status codes describing the result of the creation (201, 409, etc.). E.g. for the above example the following would be a response: { "environment": { "/t;tenant/e;env1": 201, "/t;tenant/e;env2": 201 }, "resourceType": { "/t;tenant/rt;URL": 201 } "metricType": { "/t;tenant/mt;ResponseTime": 201 }, "data": { "/t;tenant/rt;URL/d;configurationSchema": 201, "/t;tenant/e;env1/r;url1/d;configuration": 201 }, "operationType": { "/t;tenant/rt;URL/ot;ping": 201 }, "resource": { "/t;tenant/e;env1/r;url1": 201 }, "metric": { "/t;tenant/e;env1/m;url1_responseTime": 201 }, "relationship": { "": 201 } } Hope you will find this useful, Lukas [1] http://www.hawkular.org/docs/components/inventory/index.html#inventory-organization From mazz at redhat.com Fri Sep 18 15:48:04 2015 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 18 Sep 2015 15:48:04 -0400 (EDT) Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities In-Reply-To: <1880738.1F1UWefaR0@localhost.localdomain> References: <1880738.1F1UWefaR0@localhost.localdomain> Message-ID: <792995575.13850625.1442605684536.JavaMail.zimbra@redhat.com> Please tell me there are more of those Blueprint thingies that will let me build up this message using Java POJOs without having to manually build up the javascript/json myself :) ----- Original Message ----- > Hi all, > > pending a peer-review is a new feature in inventory to support bulk creation > of entities (in a single transaction). > > So far, our REST API was rather granular and concentrated on working with > single entities or even their individual relationships. This was quite costly > especially during create, update or delete operations, where the clients > would > need to do stuff one-by-one. > > This is being addressed in Inventory 0.4.0 (hopefully coming to Hawkular > Alpha5) with a new REST endpoint called "/bulk". > > For 0.4.0 we will only support bulk creation with update and delete coming in > the next versions. > > The bulk endpoint accepts a rather complex javascript object that can be used > to specify what is being created and where. The best way to explain it is by > an example: > > { > "/t;tenant": { > "environment": [ > { > "id": "env1", > "properties": {"key": "value"} > }, > { > "id": "env2", > "properties": {"key": "value2"} > } > ], > "resourceType": [ > { > "id": "URL" > } > ], > "metricType": [ > { > "id": "ResponseTime" > } > ] > }, > "/t;tenant/rt;URL": { > "data": [ > { > "role": "configurationSchema", > "value": { > "title": "URL config schema", > "description": "A json schema describing configuration of an URL", > "type": "string" > } > } > ], > "operationType": [ > { > "id": "ping" > } > ] > }, > "/t;tenant/e;env1": { > "resource": [ > { > "id": "url1", > "resourceTypePath": "/t;tenant/rt;URL" > } > ], > "metric": [ > { > "id": "url1_responseTime", > "metricTypePath": "/t;tenant/mt;ResponseTime" > } > ] > }, > "/t;tenant/e;env1/r;url1": { > "data": [ > { > "role": "configuration", > "value": "http://redhat.com" > } > ], > "relationship": [ > { > "name": "incorporates", > "otherEnd": "/t;tenant/e;env1/m;url1_responseTime", > "direction": "outgoing" > } > ] > } > } > > The javascript object above would cause creation of 2 new environments, > "env1" > and "env2", under the tenant "tenant", a new resource type called "URL" and a > new metric type called "ResponseTime". > > When the URL resource type is created, a configuration schema is created for > it (and the configuration schema prescribes that the configuration of the URL > is merely a string). Additionally, an operation called "ping" is defined for > the resource type. > > After that a new resource, "url1" is added to environment "env1" with the > "URL" resource type and a metric called "url1_responseTime" is also added to > the environment. Finally a configuration is added to the resource and the > resource is configured to incorporate the previously created metric. > > The structure of the javascript object is following: > > 1) The top-level keys are paths to the inventory entities *under* which the > new entities should be added, e.g. the first key "/t;tenant" means that the > value of that key describes what should be created directly under a tenant > called "tenant". The second key, "/t;tenant/rt;URL", tells the REST API that > the value of that key describes what should be added "under" that resource > type. > > The structure of what kinds of entities can contain what kinds of entities > can > be found in the documentation [1]. > > Notice that the order of the keys in the object is significant. > > 2) The second-level keys describe what type of data to create under a given > parent (notice that you can create more than 1 type of data under some entity > types). The values of those keys are arrays of actual blueprint objects > describing the new entities to be created (each entity type can have > different > format). The blueprint objects are the same as the ones passed in to the > granular REST endpoints that already exist. > > > The response always has the 201 - Created status code and is a JSON object > with the types of entities created as keys and values are again objects with > keys being the canonical paths to the entities created and values are HTTP > status codes describing the result of the creation (201, 409, etc.). E.g. for > the above example the following would be a response: > > { > "environment": { > "/t;tenant/e;env1": 201, > "/t;tenant/e;env2": 201 > }, > "resourceType": { > "/t;tenant/rt;URL": 201 > } > "metricType": { > "/t;tenant/mt;ResponseTime": 201 > }, > "data": { > "/t;tenant/rt;URL/d;configurationSchema": 201, > "/t;tenant/e;env1/r;url1/d;configuration": 201 > }, > "operationType": { > "/t;tenant/rt;URL/ot;ping": 201 > }, > "resource": { > "/t;tenant/e;env1/r;url1": 201 > }, > "metric": { > "/t;tenant/e;env1/m;url1_responseTime": 201 > }, > "relationship": { > "": 201 > } > } > > Hope you will find this useful, > > Lukas > > [1] > http://www.hawkular.org/docs/components/inventory/index.html#inventory-organization > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From lkrejci at redhat.com Fri Sep 18 16:02:27 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Fri, 18 Sep 2015 22:02:27 +0200 Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities In-Reply-To: <792995575.13850625.1442605684536.JavaMail.zimbra@redhat.com> References: <1880738.1F1UWefaR0@localhost.localdomain> <792995575.13850625.1442605684536.JavaMail.zimbra@redhat.com> Message-ID: <1554312.pk7dHiDdDJ@localhost.localdomain> On Friday, September 18, 2015 15:48:04 John Mazzitelli wrote: > Please tell me there are more of those Blueprint thingies that will let me > build up this message using Java POJOs without having to manually build up > the javascript/json myself :) > Yes, it is exactly this: HashMap> where Blueprint is org.hawkular.inventory.api.model.Blueprint interface. I could have added a builder for this to json-helper module but it actually didn't occur to me. It's not that bad with the maps actually - all you need to remember is to put the correct keys in the inner map and put only the corresponding Blueprint instances in the inner set. > ----- Original Message ----- > > > Hi all, > > > > pending a peer-review is a new feature in inventory to support bulk > > creation of entities (in a single transaction). > > > > So far, our REST API was rather granular and concentrated on working with > > single entities or even their individual relationships. This was quite > > costly especially during create, update or delete operations, where the > > clients would > > need to do stuff one-by-one. > > > > This is being addressed in Inventory 0.4.0 (hopefully coming to Hawkular > > Alpha5) with a new REST endpoint called "/bulk". > > > > For 0.4.0 we will only support bulk creation with update and delete coming > > in the next versions. > > > > The bulk endpoint accepts a rather complex javascript object that can be > > used to specify what is being created and where. The best way to explain > > it is by an example: > > > > { > > > > "/t;tenant": { > > > > "environment": [ > > > > { > > > > "id": "env1", > > "properties": {"key": "value"} > > > > }, > > { > > > > "id": "env2", > > "properties": {"key": "value2"} > > > > } > > > > ], > > "resourceType": [ > > > > { > > > > "id": "URL" > > > > } > > > > ], > > "metricType": [ > > > > { > > > > "id": "ResponseTime" > > > > } > > > > ] > > > > }, > > "/t;tenant/rt;URL": { > > > > "data": [ > > > > { > > > > "role": "configurationSchema", > > "value": { > > > > "title": "URL config schema", > > "description": "A json schema describing configuration of an > > URL", > > "type": "string" > > > > } > > > > } > > > > ], > > "operationType": [ > > > > { > > > > "id": "ping" > > > > } > > > > ] > > > > }, > > "/t;tenant/e;env1": { > > > > "resource": [ > > > > { > > > > "id": "url1", > > "resourceTypePath": "/t;tenant/rt;URL" > > > > } > > > > ], > > "metric": [ > > > > { > > > > "id": "url1_responseTime", > > "metricTypePath": "/t;tenant/mt;ResponseTime" > > > > } > > > > ] > > > > }, > > "/t;tenant/e;env1/r;url1": { > > > > "data": [ > > > > { > > > > "role": "configuration", > > "value": "http://redhat.com" > > > > } > > > > ], > > "relationship": [ > > > > { > > > > "name": "incorporates", > > "otherEnd": "/t;tenant/e;env1/m;url1_responseTime", > > "direction": "outgoing" > > > > } > > > > ] > > > > } > > > > } > > > > The javascript object above would cause creation of 2 new environments, > > "env1" > > and "env2", under the tenant "tenant", a new resource type called "URL" > > and a new metric type called "ResponseTime". > > > > When the URL resource type is created, a configuration schema is created > > for it (and the configuration schema prescribes that the configuration of > > the URL is merely a string). Additionally, an operation called "ping" is > > defined for the resource type. > > > > After that a new resource, "url1" is added to environment "env1" with the > > "URL" resource type and a metric called "url1_responseTime" is also added > > to the environment. Finally a configuration is added to the resource and > > the resource is configured to incorporate the previously created metric. > > > > The structure of the javascript object is following: > > > > 1) The top-level keys are paths to the inventory entities *under* which > > the > > new entities should be added, e.g. the first key "/t;tenant" means that > > the > > value of that key describes what should be created directly under a tenant > > called "tenant". The second key, "/t;tenant/rt;URL", tells the REST API > > that the value of that key describes what should be added "under" that > > resource type. > > > > The structure of what kinds of entities can contain what kinds of entities > > can > > be found in the documentation [1]. > > > > Notice that the order of the keys in the object is significant. > > > > 2) The second-level keys describe what type of data to create under a > > given > > parent (notice that you can create more than 1 type of data under some > > entity types). The values of those keys are arrays of actual blueprint > > objects describing the new entities to be created (each entity type can > > have different > > format). The blueprint objects are the same as the ones passed in to the > > granular REST endpoints that already exist. > > > > > > The response always has the 201 - Created status code and is a JSON object > > with the types of entities created as keys and values are again objects > > with keys being the canonical paths to the entities created and values > > are HTTP status codes describing the result of the creation (201, 409, > > etc.). E.g. for the above example the following would be a response: > > > > { > > > > "environment": { > > > > "/t;tenant/e;env1": 201, > > "/t;tenant/e;env2": 201 > > > > }, > > "resourceType": { > > > > "/t;tenant/rt;URL": 201 > > > > } > > "metricType": { > > > > "/t;tenant/mt;ResponseTime": 201 > > > > }, > > "data": { > > > > "/t;tenant/rt;URL/d;configurationSchema": 201, > > "/t;tenant/e;env1/r;url1/d;configuration": 201 > > > > }, > > "operationType": { > > > > "/t;tenant/rt;URL/ot;ping": 201 > > > > }, > > "resource": { > > > > "/t;tenant/e;env1/r;url1": 201 > > > > }, > > "metric": { > > > > "/t;tenant/e;env1/m;url1_responseTime": 201 > > > > }, > > "relationship": { > > > > "": 201 > > > > } > > > > } > > > > Hope you will find this useful, > > > > Lukas > > > > [1] > > http://www.hawkular.org/docs/components/inventory/index.html#inventory-org > > anization _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From mwringe at redhat.com Fri Sep 18 17:13:44 2015 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 18 Sep 2015 17:13:44 -0400 Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> Message-ID: <55FC7E88.1020102@redhat.com> On 18/09/15 11:32 AM, John Mazzitelli wrote: > Metric IDs from our Wildfly Agent contain "/" because those are characters that are valid for Wildfly resources and we use the Wildfly resource names as part of the metric ID to uniquely identify them. Yeah, and the cAdvisor metrics from Heapster also contain slashes, which is where I am seeing the problem right now with OpenShift proxies. Outside of the proxy, then everything works fine with having encoded slashes in the URL, which is why I don't see the issue with my own testing. But it looks like for the UI they may need to use the proxy. Its not just the OpenShift proxies that have this problem, its a common issue with some types proxies and servers as well. > > So for IDs of metrics coming from things like "/deployment=the-app.war/subsystem=ejb3" will have "/" in them. The only alternative would be for us to replace "/" with a character that is valid for URLs but invalid for Wildfly resource names. Say "$deployment=the-app.war$subsystem=ejb3" or something like that. I don't know if it really matter here how wildfly does things. Your client should be able to encode the metricid in one format when going into hawkular and then decode when they get things back from hawkular. I don't know what should be the right solution there: 1) we could enforce what is allowed in a metric id field. This means we break backwards compatibility with what we currently have, but it also means that we can open up the door to things like restricting some metrics to some internal only things (see https://issues.jboss.org/browse/HWKMETRICS-208). 2) we keep our current endpoint as is, but add a new endpoint which can do queries via a post (essentially this is just an escape route to get around the issue when dealing with systems like this). So to do the equivalence of: GET http://localhost:8080/hawkular/metrics/gauges/foo%2fbar/data?start=10&end=110&buckets=2 You could do also do something like: POST http://localhost:8080/hawkular/metrics/gauges/data { "metric-id": foo/bar, "start": 10, "end": 110, "buckets":2 } 3) Band-aid solution to fix the immediate problem: update the heapster sink to encode metric ids in a format which does not include the '/' anywhere in it. This doesn't really solve the problem, but it would at least get us past the problem until we come up with a better, more permanent solution. Any thoughts? > > > ----- Original Message ----- >> Currently we are storing metric id as any string. There are no >> restrictions on how we name the metrics, and when you access a single >> metric via a url you just need to make sure its url encoded properly here. >> >> This works fine when dealing with a client and the Hawkular Metrics >> directly, but can cause problems when dealing with proxies or other >> intermediary servers which are not exactly url compliant. >> >> The big issue is with slashes in the metric id. Even when properly >> encoded this can cause problems with things like proxies not accepting >> them (and even certain web container don't like them). >> >> Since we need to be able to integrate over networks with these types of >> servers, we need to find a solution to resolve this in hawkular. >> >> Possible solutions >> >> - enforce naming of anything which can show up in a the URL to make sure >> it cannot include a '/' . This would include things like metric Id. This >> should probably be enabled by default, but we could have an option to >> disable this if a user needs it. >> >> - create a new endpoint which can be used to perform a POST instead of >> doing a GET on the metric endpoint. This would mean sending things like >> the metric id and query string as part of the message body in the POST >> instead of in the URL. >> >> Any thoughts? >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From jpkroehling at redhat.com Mon Sep 21 03:57:51 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 21 Sep 2015 09:57:51 +0200 Subject: [Hawkular-dev] how to identify UI clients and which should get responses In-Reply-To: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> References: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> Message-ID: <55FFB87F.8010608@redhat.com> On 09/18/2015 06:27 PM, John Mazzitelli wrote: > 1) I'm logged into the UI using two different browser tabs. In one, I send a "deploy an application" request. Once the response comes back - do I show the response in both tabs? What if I closed the tab that performed the deploy? What should happen? I'd say that "notifications" could be shown in all screens. > 2) I am logged in kettle #1 as "jdoe" and someone else is logged in kettle #2 as the same "jdoe". I deploy an application. Should that other person get the response that a new app was deployed, too? *This* could be the start of a bigger feature. For instance, do I want to be notified if anyone in my organization has deployed something on a server where I'm a SuperUser? > The way I am thinking about this is Google Drive. I could be logged into my Google Drive account on any number of browsers or tabs - when I make a change to a document in Drive, that change is immediately reflected in every browser/tab I have open to Google Drive, not just in the browser that made the change. Is this the kind of thing we want? Or do we only want the browser that is making the requests get the responses? Not for all cases. See the "Export JDR" for a situation where you don't want this: I don't want to have 5 download prompts, one for each tab, if I clicked on "Export JDR" only on the first tab. So, I think *notifications* can be performed across tabs, but not all actions are the same. > One thing I am thinking is hawkular-accounts gives us some common ID (not the token) that we assign a UI client and that ID floats in all requests and responses. The bad thing about this is we'll never know the UI client ID until they send their first request. We have already the Persona for each request and for each socket, so, we might not need another ID. A consumer of the Accounts WebSocket API could just request the session IDs for a given persona. Accounts would need to be notified during the onClose of the socket, so that the session is removed, but I think this could work. > Another thing I am thinking is have it work just like feeds - a feed connects via the URL ws://host/hawkular-command-gateway/feed/. We could let each UI generate their own id and connect via ws://host/hawkular-command-gateway/ui/. The good thing here is the UI knows its ID immediately (since it generated it) and we know it from the start without even getting an initial message (and we can link it with that websocket session easily). Bad thing is only that UI will be able to get responses - it won't work the "Google Drive way" where a user logged into N browsers will get responses in all N browsers. > > Thoughts? I haven't checked how Google does it, but there's a technique for inter-tab communication using localStorage. So, doing this via Web Socket is not needed/required. In fact, we do something like this already with Keycloak: if you have two tabs opened, actions on one tab refreshes your token in the other tab. So, you are not "inactive" in the second tab, if you are active on the first. This means: you can still have the "notify all tabs" feature having only 1-to-1 communication between sockets/tabs. - Juca. From tsegismo at redhat.com Mon Sep 21 03:58:11 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 21 Sep 2015 09:58:11 +0200 Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <55FC7E88.1020102@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> <55FC7E88.1020102@redhat.com> Message-ID: <55FFB893.20802@redhat.com> Out of curiosity, can you explain what's going wrong? I don't understand why a properly url-encoded character make the proxy fail. Le 18/09/2015 23:13, Matt Wringe a ?crit : > On 18/09/15 11:32 AM, John Mazzitelli wrote: >> Metric IDs from our Wildfly Agent contain "/" because those are characters that are valid for Wildfly resources and we use the Wildfly resource names as part of the metric ID to uniquely identify them. > > Yeah, and the cAdvisor metrics from Heapster also contain slashes, which > is where I am seeing the problem right now with OpenShift proxies. > Outside of the proxy, then everything works fine with having encoded > slashes in the URL, which is why I don't see the issue with my own > testing. But it looks like for the UI they may need to use the proxy. > > Its not just the OpenShift proxies that have this problem, its a common > issue with some types proxies and servers as well. > >> >> So for IDs of metrics coming from things like "/deployment=the-app.war/subsystem=ejb3" will have "/" in them. The only alternative would be for us to replace "/" with a character that is valid for URLs but invalid for Wildfly resource names. Say "$deployment=the-app.war$subsystem=ejb3" or something like that. > > I don't know if it really matter here how wildfly does things. Your > client should be able to encode the metricid in one format when going > into hawkular and then decode when they get things back from hawkular. > > > I don't know what should be the right solution there: > > 1) we could enforce what is allowed in a metric id field. This means we > break backwards compatibility with what we currently have, but it also > means that we can open up the door to things like restricting some > metrics to some internal only things (see > https://issues.jboss.org/browse/HWKMETRICS-208). > > 2) we keep our current endpoint as is, but add a new endpoint which can > do queries via a post (essentially this is just an escape route to get > around the issue when dealing with systems like this). > > So to do the equivalence of: > > GET > http://localhost:8080/hawkular/metrics/gauges/foo%2fbar/data?start=10&end=110&buckets=2 > > You could do also do something like: > POST http://localhost:8080/hawkular/metrics/gauges/data > { > "metric-id": foo/bar, > "start": 10, > "end": 110, > "buckets":2 > } > > 3) Band-aid solution to fix the immediate problem: update the heapster > sink to encode metric ids in a format which does not include the '/' > anywhere in it. This doesn't really solve the problem, but it would at > least get us past the problem until we come up with a better, more > permanent solution. > > Any thoughts? > >> >> >> ----- Original Message ----- >>> Currently we are storing metric id as any string. There are no >>> restrictions on how we name the metrics, and when you access a single >>> metric via a url you just need to make sure its url encoded properly here. >>> >>> This works fine when dealing with a client and the Hawkular Metrics >>> directly, but can cause problems when dealing with proxies or other >>> intermediary servers which are not exactly url compliant. >>> >>> The big issue is with slashes in the metric id. Even when properly >>> encoded this can cause problems with things like proxies not accepting >>> them (and even certain web container don't like them). >>> >>> Since we need to be able to integrate over networks with these types of >>> servers, we need to find a solution to resolve this in hawkular. >>> >>> Possible solutions >>> >>> - enforce naming of anything which can show up in a the URL to make sure >>> it cannot include a '/' . This would include things like metric Id. This >>> should probably be enabled by default, but we could have an option to >>> disable this if a user needs it. >>> >>> - create a new endpoint which can be used to perform a POST instead of >>> doing a GET on the metric endpoint. This would mean sending things like >>> the metric id and query string as part of the message body in the POST >>> instead of in the URL. >>> >>> Any thoughts? >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From hrupp at redhat.com Mon Sep 21 06:08:25 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 21 Sep 2015 12:08:25 +0200 Subject: [Hawkular-dev] how to identify UI clients and which should get responses In-Reply-To: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> References: <1279749399.13803049.1442593651953.JavaMail.zimbra@redhat.com> Message-ID: <529F13E2-DB60-47B4-A46A-C4F77211ED80@redhat.com> Hey Mazz, [ While this was not asked, we should nevertheless keep it in mind for the future: those request / response pairs (of 'important' things) should be audit logged and also be available in some form of message viewer in case that the user wants to check stuff and/or it not logged in at time of completion ] > 1) I'm logged into the UI using two different browser tabs. In one, I > send a "deploy an application" request. Once the response comes back - > do I show the response in both tabs? What if I closed the tab that > performed the deploy? What should happen? Show in both. > 2) I am logged in kettle #1 as "jdoe" and someone else is logged in > kettle #2 as the same "jdoe". I deploy an application. Should that > other person get the response that a new app was deployed, too? Yes. I guess that in this case it is even more important. > 3) I am logged into the UI and I deploy an application. I log out > (perhaps close the browser) and log back in. Then the deployment > finishes and the response comes in - should I get that response (even > though its a new browser instance and a new websocket connection?) If you are logged in: yes. Otherwise see above. The questions start to become more interesting when multiple users of an organization (tenant) are logged in. How much do we want to show here? Spamming all org users with every message is certainly wrong. Allowing others to look them up (-> audit) may be a good idea (depending on permissions). > Another thing I am thinking is have it work just like feeds - a feed > connects via the URL ws://host/hawkular-command-gateway/feed/ ID>. We could let each UI generate their own id and connect via > ws://host/hawkular-command-gateway/ui/. The good thing here is > the UI knows its ID immediately (since it generated it) and we know it > from the start without even getting an initial message (and we can > link it with that websocket session easily). Bad thing is only that UI > will be able to get responses - it won't work the "Google Drive way" > where a user logged into N browsers will get responses in all N > browsers. Isn't a UI always tied to a user? In that case when the UI "logs into the server", the server(s) can keep a list of user->ui(s) mappings. I think it is very valid to open a/the websocket on initial login into the UI and then just use that all over the place where it is needed until the user logs out or closes the browser. From hrupp at redhat.com Mon Sep 21 06:11:25 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 21 Sep 2015 12:11:25 +0200 Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities In-Reply-To: <1554312.pk7dHiDdDJ@localhost.localdomain> References: <1880738.1F1UWefaR0@localhost.localdomain> <792995575.13850625.1442605684536.JavaMail.zimbra@redhat.com> <1554312.pk7dHiDdDJ@localhost.localdomain> Message-ID: <4703DAD0-AF3E-495A-B1E2-000AFDACED9E@redhat.com> On 18 Sep 2015, at 22:02, Lukas Krejci wrote: > Yes, it is exactly this: > > HashMap> where > Blueprint is > org.hawkular.inventory.api.model.Blueprint interface. Do we really want to tie in the whole Blueprint api into clients like the WF-agent? From hrupp at redhat.com Mon Sep 21 06:22:00 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 21 Sep 2015 12:22:00 +0200 Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities In-Reply-To: <1880738.1F1UWefaR0@localhost.localdomain> References: <1880738.1F1UWefaR0@localhost.localdomain> Message-ID: <1137B448-23AC-48BD-982A-54D55397F409@redhat.com> On 18 Sep 2015, at 21:45, Lukas Krejci wrote: > pending a peer-review is a new feature in inventory to support bulk > creation > of entities (in a single transaction). Very good idea. > The bulk endpoint accepts a rather complex javascript object that can > be used > to specify what is being created and where. The best way to explain it > is by > an example: I think it could be good to somewhat loosen that api again and have different (sub-)endpoints. Creation of resource types / environments has a different "severity" than creation of resources and relationships. From hrupp at redhat.com Mon Sep 21 06:26:40 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 21 Sep 2015 12:26:40 +0200 Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <55FC7E88.1020102@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> <55FC7E88.1020102@redhat.com> Message-ID: <240261E8-49D3-42C9-95B3-72DC2C465CA0@redhat.com> On 18 Sep 2015, at 23:13, Matt Wringe wrote: > 2) we keep our current endpoint as is, but add a new endpoint which can > do queries via a post (essentially this is just an escape route to get > around the issue when dealing with systems like this). I don't like this as it is not RESTful at all. And most possibly defeats any efforts to cache results from queries From miburman at redhat.com Mon Sep 21 07:15:00 2015 From: miburman at redhat.com (Michael Burman) Date: Mon, 21 Sep 2015 07:15:00 -0400 (EDT) Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <55FFB893.20802@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> <55FC7E88.1020102@redhat.com> <55FFB893.20802@redhat.com> Message-ID: <765231742.39478775.1442834100968.JavaMail.zimbra@redhat.com> Hi, There's a bug in the proxy, nothing more complicated. In this case the bug comes from Golang's net/url module, which is buggy (and which the authors refuse to fix) as it can't handle standard URLs. No magic here and I don't think this is a valid reason (as the proxy can be fixed by avoiding Golang's url.Parse() command) to change our implementation and APIs. - Micke ----- Original Message ----- From: "Thomas Segismont" To: hawkular-dev at lists.jboss.org Sent: Monday, September 21, 2015 10:58:11 AM Subject: Re: [Hawkular-dev] Metric id and url restrictions Out of curiosity, can you explain what's going wrong? I don't understand why a properly url-encoded character make the proxy fail. Le 18/09/2015 23:13, Matt Wringe a ?crit : > On 18/09/15 11:32 AM, John Mazzitelli wrote: >> Metric IDs from our Wildfly Agent contain "/" because those are characters that are valid for Wildfly resources and we use the Wildfly resource names as part of the metric ID to uniquely identify them. > > Yeah, and the cAdvisor metrics from Heapster also contain slashes, which > is where I am seeing the problem right now with OpenShift proxies. > Outside of the proxy, then everything works fine with having encoded > slashes in the URL, which is why I don't see the issue with my own > testing. But it looks like for the UI they may need to use the proxy. > > Its not just the OpenShift proxies that have this problem, its a common > issue with some types proxies and servers as well. > >> >> So for IDs of metrics coming from things like "/deployment=the-app.war/subsystem=ejb3" will have "/" in them. The only alternative would be for us to replace "/" with a character that is valid for URLs but invalid for Wildfly resource names. Say "$deployment=the-app.war$subsystem=ejb3" or something like that. > > I don't know if it really matter here how wildfly does things. Your > client should be able to encode the metricid in one format when going > into hawkular and then decode when they get things back from hawkular. > > > I don't know what should be the right solution there: > > 1) we could enforce what is allowed in a metric id field. This means we > break backwards compatibility with what we currently have, but it also > means that we can open up the door to things like restricting some > metrics to some internal only things (see > https://issues.jboss.org/browse/HWKMETRICS-208). > > 2) we keep our current endpoint as is, but add a new endpoint which can > do queries via a post (essentially this is just an escape route to get > around the issue when dealing with systems like this). > > So to do the equivalence of: > > GET > http://localhost:8080/hawkular/metrics/gauges/foo%2fbar/data?start=10&end=110&buckets=2 > > You could do also do something like: > POST http://localhost:8080/hawkular/metrics/gauges/data > { > "metric-id": foo/bar, > "start": 10, > "end": 110, > "buckets":2 > } > > 3) Band-aid solution to fix the immediate problem: update the heapster > sink to encode metric ids in a format which does not include the '/' > anywhere in it. This doesn't really solve the problem, but it would at > least get us past the problem until we come up with a better, more > permanent solution. > > Any thoughts? > >> >> >> ----- Original Message ----- >>> Currently we are storing metric id as any string. There are no >>> restrictions on how we name the metrics, and when you access a single >>> metric via a url you just need to make sure its url encoded properly here. >>> >>> This works fine when dealing with a client and the Hawkular Metrics >>> directly, but can cause problems when dealing with proxies or other >>> intermediary servers which are not exactly url compliant. >>> >>> The big issue is with slashes in the metric id. Even when properly >>> encoded this can cause problems with things like proxies not accepting >>> them (and even certain web container don't like them). >>> >>> Since we need to be able to integrate over networks with these types of >>> servers, we need to find a solution to resolve this in hawkular. >>> >>> Possible solutions >>> >>> - enforce naming of anything which can show up in a the URL to make sure >>> it cannot include a '/' . This would include things like metric Id. This >>> should probably be enabled by default, but we could have an option to >>> disable this if a user needs it. >>> >>> - create a new endpoint which can be used to perform a POST instead of >>> doing a GET on the metric endpoint. This would mean sending things like >>> the metric id and query string as part of the message body in the POST >>> instead of in the URL. >>> >>> Any thoughts? >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From tsegismo at redhat.com Mon Sep 21 08:14:46 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 21 Sep 2015 14:14:46 +0200 Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <765231742.39478775.1442834100968.JavaMail.zimbra@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> <55FC7E88.1020102@redhat.com> <55FFB893.20802@redhat.com> <765231742.39478775.1442834100968.JavaMail.zimbra@redhat.com> Message-ID: <55FFF4B6.3010706@redhat.com> Le 21/09/2015 13:15, Michael Burman a ?crit : > I don't think this is a valid reason (as the proxy can be fixed by avoiding Golang's url.Parse() command) to change our implementation and APIs. +1 From lkrejci at redhat.com Mon Sep 21 08:16:35 2015 From: lkrejci at redhat.com (Lukas Krejci) Date: Mon, 21 Sep 2015 14:16:35 +0200 Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities In-Reply-To: <4703DAD0-AF3E-495A-B1E2-000AFDACED9E@redhat.com> References: <1880738.1F1UWefaR0@localhost.localdomain> <1554312.pk7dHiDdDJ@localhost.localdomain> <4703DAD0-AF3E-495A-B1E2-000AFDACED9E@redhat.com> Message-ID: <1938887.W3KBgqWf8t@localhost.localdomain> org.hawkular.inventory.api.model.Blueprint is a hawkular-specific interface. It's got nothing to do with Tinkerpop Blueprints API. An inventory entity blueprint is a thing that is used to create an entity on a certain place in the inventory. Because of our initial discussions and uncertainty around the backend storage for inventory, the actual storage mechanism is "pluggable" in inventory and the Tinkerpop-based implementation doesn't "bubble up" into the core inventory API itself at all. On Monday, September 21, 2015 12:11:25 Heiko W.Rupp wrote: > On 18 Sep 2015, at 22:02, Lukas Krejci wrote: > > Yes, it is exactly this: > > > > HashMap> where > > Blueprint is > > org.hawkular.inventory.api.model.Blueprint interface. > > Do we really want to tie in the whole Blueprint api > into clients like the WF-agent? > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Mon Sep 21 11:41:42 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 21 Sep 2015 17:41:42 +0200 Subject: [Hawkular-dev] Coming to inventory 0.4.0 - bulk create of entities In-Reply-To: <1938887.W3KBgqWf8t@localhost.localdomain> References: <1880738.1F1UWefaR0@localhost.localdomain> <1554312.pk7dHiDdDJ@localhost.localdomain> <4703DAD0-AF3E-495A-B1E2-000AFDACED9E@redhat.com> <1938887.W3KBgqWf8t@localhost.localdomain> Message-ID: <8B789B22-E7C5-442E-89F7-E749A791640E@redhat.com> On 21 Sep 2015, at 14:16, Lukas Krejci wrote: > org.hawkular.inventory.api.model.Blueprint is a hawkular-specific > interface. > It's got nothing to do with Tinkerpop Blueprints API. Ah ok. Great. Thanks for clarifying. From tsegismo at redhat.com Mon Sep 21 12:04:34 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 21 Sep 2015 18:04:34 +0200 Subject: [Hawkular-dev] [Metrics] New commit hook In-Reply-To: <55E74368.7000800@redhat.com> References: <55CC7243.8040506@redhat.com> <55CC73AC.6050808@redhat.com> <55E061F6.9000702@redhat.com> <1260199119.20351015.1440775697066.JavaMail.zimbra@redhat.com> <55E42A91.9010708@redhat.com> <55E74368.7000800@redhat.com> Message-ID: <56002A92.5010107@redhat.com> Just a line to tell you the hook has been updated in master branch, in order to ignore IDE specific files. Please update your git config as needed. Le 02/09/2015 20:43, Thomas Segismont a ?crit : > The PR has been merged (thanks Stefan). > > Please take the time to add this hook to your local metrics repo: > https://raw.githubusercontent.com/hawkular/hawkular-metrics/master/api/diff-pre-commit-hook.sh > > Also, after installing the hook, you should rebase your pull requests on > master (to avoid issues with the license checker plugin). > From hrupp at redhat.com Mon Sep 21 12:31:12 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 21 Sep 2015 18:31:12 +0200 Subject: [Hawkular-dev] [Metrics] New commit hook In-Reply-To: <56002A92.5010107@redhat.com> References: <55CC7243.8040506@redhat.com> <55CC73AC.6050808@redhat.com> <55E061F6.9000702@redhat.com> <1260199119.20351015.1440775697066.JavaMail.zimbra@redhat.com> <55E42A91.9010708@redhat.com> <55E74368.7000800@redhat.com> <56002A92.5010107@redhat.com> Message-ID: On 21 Sep 2015, at 18:04, Thomas Segismont wrote: > Just a line to tell you the hook has been updated in master branch, in > order to ignore IDE specific files. Please update your git config as > needed. Could / should that use .gitignore as ignore file to prevent duplication ? From mwringe at redhat.com Mon Sep 21 14:04:52 2015 From: mwringe at redhat.com (Matt Wringe) Date: Mon, 21 Sep 2015 14:04:52 -0400 Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <55FFF4B6.3010706@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> <55FC7E88.1020102@redhat.com> <55FFB893.20802@redhat.com> <765231742.39478775.1442834100968.JavaMail.zimbra@redhat.com> <55FFF4B6.3010706@redhat.com> Message-ID: <560046C4.2010204@redhat.com> A couple of points here: 1) I agree that its not an issue with our code here. Its a problem that exists in other third party intermediary servers which are not compliant. 2) I agree that the POST hack instead of GET could be seen as being against the REST principals, but its still REST compliant (POST is still unsafe and idempotent. It could be seen still as creating a new resource which is returned but not stored on the server). It might not be the most performant thing to use since caching servers are out of the question. Its what is currently used in InfluxDB as a way to get around this issue. This is not just an issue with the Kubernetes proxy, its also an issue with EAP, other web server, other apache components (or at least some older versions), and apparently a bunch of other proxies servers out there. Its a common enough problem in this space that we have already ran into in two places when dealing with OpenShift integration work. I don't think its acceptable to say that since we are compliant, its not our fault and we wont to anything to help get anyone get around this issue. We need to work in the real world and this means having to (potentially) deal with the common problems which exist out there. For now, it looks like we are just going to disable certain checks in EAP to bypass the error there and try and patch the kubernetes proxy to get around the issue. Hopefully this will be enough for now and that we don't run into other issues in the near future. But we may want to consider making things easier when we know this is a potential problem area. - Matt On 21/09/15 08:14 AM, Thomas Segismont wrote: > Le 21/09/2015 13:15, Michael Burman a ?crit : >> I don't think this is a valid reason (as the proxy can be fixed by avoiding Golang's url.Parse() command) to change our implementation and APIs. > +1 > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From tsegismo at redhat.com Tue Sep 22 03:17:51 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 22 Sep 2015 09:17:51 +0200 Subject: [Hawkular-dev] [Metrics] New commit hook In-Reply-To: References: <55CC7243.8040506@redhat.com> <55CC73AC.6050808@redhat.com> <55E061F6.9000702@redhat.com> <1260199119.20351015.1440775697066.JavaMail.zimbra@redhat.com> <55E42A91.9010708@redhat.com> <55E74368.7000800@redhat.com> <56002A92.5010107@redhat.com> Message-ID: <5601009F.5020203@redhat.com> Le 21/09/2015 18:31, Heiko W.Rupp a ?crit : > Could / should that use .gitignore as ignore file to > prevent duplication ? Some patterns are the same, but some patterns work only with git. It's not bad if we have a separate diff excludes file I think. We should get rid of JAX-RS 1.1 implementation in a few months anyway. From fbrychta at redhat.com Tue Sep 22 04:29:18 2015 From: fbrychta at redhat.com (Filip Brychta) Date: Tue, 22 Sep 2015 04:29:18 -0400 (EDT) Subject: [Hawkular-dev] Recent changes in hawkular-metrics performance tests In-Reply-To: <654111097.19536682.1442907841430.JavaMail.zimbra@redhat.com> Message-ID: <1882080894.19545278.1442910558368.JavaMail.zimbra@redhat.com> Hello, there is one important change in the test setup. Baselines are no longer taken from last successful job but are defined statically now. It means that any change to the baselines must be done manually. This was done to avoid silent performance drops and to have every baseline change documented. See [1] for details. Another thing is reporting improvement. Simple report containing each build step and some instructions is now available on public VM (accessible outside of VPN). It should be easy to find out which build step failed and what to do to next without looking into internal jenkins job console logs. This report is accessible from each pull request (Details link next to haw-perf-stability-test check). Example: http://209.132.178.156/report268.html Filip [1] - https://mojo.redhat.com/docs/DOC-1045245 From tsegismo at redhat.com Tue Sep 22 06:03:57 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 22 Sep 2015 12:03:57 +0200 Subject: [Hawkular-dev] Embedded Cassandra default log level Message-ID: <5601278D.8080202@redhat.com> Hi everyone, There's a bunch of log messages when you start Hawkular with embedded Cassandra. IMHO, this prevents the user to focus on informative messages. I'd like to change the default log level of embedded C* to WARN. https://github.com/hawkular/hawkular/pull/487 Regards, Thomas From lponce at redhat.com Tue Sep 22 06:25:17 2015 From: lponce at redhat.com (Lucas Ponce) Date: Tue, 22 Sep 2015 06:25:17 -0400 (EDT) Subject: [Hawkular-dev] Embedded Cassandra default log level In-Reply-To: <5601278D.8080202@redhat.com> References: <5601278D.8080202@redhat.com> Message-ID: <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> +1 ----- Original Message ----- > From: "Thomas Segismont" > To: "Discussions around Hawkular development" > Sent: Tuesday, September 22, 2015 12:03:57 PM > Subject: [Hawkular-dev] Embedded Cassandra default log level > > Hi everyone, > > There's a bunch of log messages when you start Hawkular with embedded > Cassandra. IMHO, this prevents the user to focus on informative messages. > > I'd like to change the default log level of embedded C* to WARN. > > https://github.com/hawkular/hawkular/pull/487 > > Regards, > Thomas > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jpkroehling at redhat.com Tue Sep 22 06:42:49 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Tue, 22 Sep 2015 12:42:49 +0200 Subject: [Hawkular-dev] Token errors with WebSockets and other backend calls In-Reply-To: <55FADE21.3010709@redhat.com> References: <55FADE21.3010709@redhat.com> Message-ID: <560130A9.1080802@redhat.com> Team, I prepared a workaround for this and performed a release of accounts with the fix. I sent also PRs for all projects that are consuming Accounts, as per the "Component Dependencies" from our documentation[1]. Some components were using an old version of accounts, while others were two versions behind. - For agent/command-gateway, this change is required to fix the 127.0.0.1 vs. localhost issue (HAWKULAR-615). - For most of the components, this change is optional and there's no harm in updating. This is valid for components that were on 1.0.12.Final. - For components that were on very old versions (1.0.1, for instance), it might be wise to wait for the MS5 release before merging the PR. As far as I *remember*, there were no breaking changes on the API, but we have a release in a couple of days :-) 1 - www.hawkular.org/docs/dev/development.html#component-dependencies - Juca. On 09/17/2015 05:37 PM, Juraci Paix?o Kr?hling wrote: > All, > > I've seen some problems lately that are hard to debug and may cause you > to waste valuable time. > > In short, if you face "strange" issues related to authentication, > specially when trying features that make use of WebSockets (ie: add > deployment), make sure you are accessing the web UI via 127.0.0.1 , and > not localhost . > > In case you are interested in the details, keep reading. > > Upon login, Keycloak issues a token for the client, taking the auth > server host into consideration (as the "issuer authority"). If you use > localhost, then that's the hostname that Keycloak will use inside the > token. This value is later used to validate the incoming token. Ideally, > all the hostnames would be a match, and that's usually the case if you > use the "-b" switch when starting Wildfly. But if you don't specify, we > fall back to 127.0.0.1 [1] , causing the "backend call" to be 127.0.0.1, > while the "frontend call" came via localhost. > > I have a couple of ideas on how to solve this in our side, but until a > fix is done, tested and merged, please use 127.0.0.1 on the UI. > > 1 - http://git.io/vnJfx > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From hrupp at redhat.com Tue Sep 22 07:19:39 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 22 Sep 2015 13:19:39 +0200 Subject: [Hawkular-dev] Embedded Cassandra default log level In-Reply-To: <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> References: <5601278D.8080202@redhat.com> <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> Message-ID: <1422B033-8658-40EE-B732-E684743CEACB@redhat.com> On 22 Sep 2015, at 12:25, Lucas Ponce wrote: > +1 > +2 From hrupp at redhat.com Tue Sep 22 08:02:58 2015 From: hrupp at redhat.com (hrupp at redhat.com) Date: Tue, 22 Sep 2015 12:02:58 +0000 Subject: [Hawkular-dev] Termin gestrichen: Hawkular-Team Update - Di 22. Sep. 2015 3:30PM - 4PM (hrupp@redhat.com) Message-ID: <001a11c3434c64a980052054c79a@google.com> Dieser Termin wurde gestrichen und aus Ihrem Kalender entfernt. Titel: Hawkular-Team Update This is a all-Hawkular team meeting to give updates where we are and so on. This is *open to the public*. Location: on bluejeans: https://redhat.bluejeans.com/hrupp/ or alternatively teleconference Reservationless+ , passcode 204 2160 481 You can find Dial-In telephone numbers here: https://www.intercallonline.com/listNumbersByCode.action?confCode=2042160481 RedHat internal short dial numbers are 16666 and 15555 (and probably others, depends on your location) Wann: Di 22. Sep. 2015 3:30PM - 4PM Berlin Wo: pc 204 2160 481 Kalender: hrupp at redhat.com Wer * Heiko Rupp - Organisator * Peter Palaga * Simeon Pinder * Lucas Ponce * lkrejci at redhat.com * Mike Thompson * amendonc at redhat.com * Gabriel Cardoso * hawkular-dev at lists.jboss.org * Jay Shaughnessy * snegrea at redhat.com * theute at redhat.com * gbrown at redhat.com * John Mazzitelli * jcosta at redhat.com * Viliam Rockai * John Sanda * Jiri Kremser * Thomas Segismont * miburman at redhat.com Einladung von Google Kalender: https://www.google.com/calendar/ Sie erhalten diese E-Mail unter hawkular-dev at lists.jboss.org, da Sie ein Gast bei diesem Termin sind. Lehnen Sie diesen Termin ab, um keine weiteren Informationen zu diesem Termin zu erhalten. Sie k?nnen auch unter https://www.google.com/calendar/ ein Google-Konto erstellen und Ihre Benachrichtigungseinstellungen f?r Ihren gesamten Kalender steuern. Wenn Sie diese Einladung weiterleiten, kann jeder Empf?nger Ihre Antwort auf die Einladung ?ndern. Weitere Informationen finden Sie unter https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150922/de83d8ac/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 4132 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150922/de83d8ac/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 4219 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150922/de83d8ac/attachment-0003.bin From jkremser at redhat.com Tue Sep 22 13:03:47 2015 From: jkremser at redhat.com (Jiri Kremser) Date: Tue, 22 Sep 2015 13:03:47 -0400 (EDT) Subject: [Hawkular-dev] Embedded Cassandra default log level In-Reply-To: <1422B033-8658-40EE-B732-E684743CEACB@redhat.com> References: <5601278D.8080202@redhat.com> <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> <1422B033-8658-40EE-B732-E684743CEACB@redhat.com> Message-ID: <1663299797.19694570.1442941427735.JavaMail.zimbra@redhat.com> Hi Thomas, did you find the root cause of the duplicate logging messages? I've spend 2 hours looking into it and didn't find anything. Here is what I did: I compared the contents of lib directories in hawkular=commons-embedded-cassandra-ear.ear before the issue and after the issue the new version have dependencies on slf4j, logback and log4j (quite a mess) the old one doesn't have the dependency on log4j So I've added couple of in the pom in hawkular-commons/* modules to exclude the log4j jars, but it didn't help I've also tried to provide a configuration for logback (by -Dlogback.configurationFile=foo and also programmatically in EmbeddedCassandraService.java), but it wasn't taken into consideration no success :/ ..and +1 on C* lvl to WARN (I have an alias for that and I've been running the hawkular with it for couple of months) jk ----- Original Message ----- | From: "Heiko W.Rupp" | To: "Discussions around Hawkular development" | Sent: Tuesday, 22 September, 2015 1:19:39 PM | Subject: Re: [Hawkular-dev] Embedded Cassandra default log level | | On 22 Sep 2015, at 12:25, Lucas Ponce wrote: | | > +1 | > | +2 | _______________________________________________ | hawkular-dev mailing list | hawkular-dev at lists.jboss.org | https://lists.jboss.org/mailman/listinfo/hawkular-dev | From jkremser at redhat.com Tue Sep 22 13:06:16 2015 From: jkremser at redhat.com (Jiri Kremser) Date: Tue, 22 Sep 2015 13:06:16 -0400 (EDT) Subject: [Hawkular-dev] Embedded Cassandra default log level In-Reply-To: <1663299797.19694570.1442941427735.JavaMail.zimbra@redhat.com> References: <5601278D.8080202@redhat.com> <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> <1422B033-8658-40EE-B732-E684743CEACB@redhat.com> <1663299797.19694570.1442941427735.JavaMail.zimbra@redhat.com> Message-ID: <947901317.19694907.1442941576020.JavaMail.zimbra@redhat.com> oh, I forgot to mention: this is the first commit in hawkular/hawkular with the duplicate logging issue: https://github.com/hawkular/hawkular/commit/5eba4ec0149ddfe93c8476754fe8457c1e60bf9d So basically, it's caused by the C* upgrade 2.1.6 -> 2.2.0 jk ----- Original Message ----- | From: "Jiri Kremser" | To: "Discussions around Hawkular development" | Sent: Tuesday, 22 September, 2015 7:03:47 PM | Subject: Re: [Hawkular-dev] Embedded Cassandra default log level | | Hi Thomas, | did you find the root cause of the duplicate logging messages? I've spend 2 | hours looking into it and didn't find anything. | Here is what I did: | | I compared the contents of lib directories in | hawkular=commons-embedded-cassandra-ear.ear before the issue and after the | issue | the new version have dependencies on slf4j, logback and log4j (quite a mess) | the old one doesn't have the dependency on log4j | So I've added couple of in the pom in hawkular-commons/* modules | to exclude the log4j jars, but it didn't help | | I've also tried to provide a configuration for logback (by | -Dlogback.configurationFile=foo and also programmatically in | EmbeddedCassandraService.java), but it wasn't taken into consideration | | no success :/ | | ..and +1 on C* lvl to WARN (I have an alias for that and I've been running | the hawkular with it for couple of months) | jk | | ----- Original Message ----- | | From: "Heiko W.Rupp" | | To: "Discussions around Hawkular development" | | | | Sent: Tuesday, 22 September, 2015 1:19:39 PM | | Subject: Re: [Hawkular-dev] Embedded Cassandra default log level | | | | On 22 Sep 2015, at 12:25, Lucas Ponce wrote: | | | | > +1 | | > | | +2 | | _______________________________________________ | | hawkular-dev mailing list | | hawkular-dev at lists.jboss.org | | https://lists.jboss.org/mailman/listinfo/hawkular-dev | | From tsegismo at redhat.com Wed Sep 23 04:29:29 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 23 Sep 2015 10:29:29 +0200 Subject: [Hawkular-dev] Embedded Cassandra default log level In-Reply-To: <1663299797.19694570.1442941427735.JavaMail.zimbra@redhat.com> References: <5601278D.8080202@redhat.com> <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> <1422B033-8658-40EE-B732-E684743CEACB@redhat.com> <1663299797.19694570.1442941427735.JavaMail.zimbra@redhat.com> Message-ID: <560262E9.8000800@redhat.com> Hey, thanks for your time. I am still trying to figure out what's going on. As Lucas P said, there's no issue when Hawkular points to an external C* server. I tried to reproduce with a server where embedded C* is the only deployment, without success. So my current hypothesis is that something's going wrong with the inventory EAR (which has all of C* inside). Le 22/09/2015 19:03, Jiri Kremser a ?crit : > Hi Thomas, > did you find the root cause of the duplicate logging messages? I've spend 2 hours looking into it and didn't find anything. > Here is what I did: > > I compared the contents of lib directories in hawkular=commons-embedded-cassandra-ear.ear before the issue and after the issue > the new version have dependencies on slf4j, logback and log4j (quite a mess) the old one doesn't have the dependency on log4j > So I've added couple of in the pom in hawkular-commons/* modules to exclude the log4j jars, but it didn't help > > I've also tried to provide a configuration for logback (by -Dlogback.configurationFile=foo and also programmatically in EmbeddedCassandraService.java), but it wasn't taken into consideration > > no success :/ > > ..and +1 on C* lvl to WARN (I have an alias for that and I've been running the hawkular with it for couple of months) > jk > > ----- Original Message ----- > | From: "Heiko W.Rupp" > | To: "Discussions around Hawkular development" > | Sent: Tuesday, 22 September, 2015 1:19:39 PM > | Subject: Re: [Hawkular-dev] Embedded Cassandra default log level > | > | On 22 Sep 2015, at 12:25, Lucas Ponce wrote: > | > | > +1 > | > > | +2 > | _______________________________________________ > | hawkular-dev mailing list > | hawkular-dev at lists.jboss.org > | https://lists.jboss.org/mailman/listinfo/hawkular-dev > | > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From tsegismo at redhat.com Wed Sep 23 04:54:20 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 23 Sep 2015 10:54:20 +0200 Subject: [Hawkular-dev] Embedded Cassandra default log level In-Reply-To: <560262E9.8000800@redhat.com> References: <5601278D.8080202@redhat.com> <1742650755.14924452.1442917517010.JavaMail.zimbra@redhat.com> <1422B033-8658-40EE-B732-E684743CEACB@redhat.com> <1663299797.19694570.1442941427735.JavaMail.zimbra@redhat.com> <560262E9.8000800@redhat.com> Message-ID: <560268BC.1090707@redhat.com> Le 23/09/2015 10:29, Thomas Segismont a ?crit : > As Lucas P said, there's no issue when Hawkular points to an external C* > server. > > I tried to reproduce with a server where embedded C* is the only > deployment, without success. > > So my current hypothesis is that something's going wrong with the > inventory EAR (which has all of C* inside). I could reproduce with Metrics + Embedded C*, so it's not coming from the inventory EAR. From theute at redhat.com Wed Sep 23 05:44:28 2015 From: theute at redhat.com (Thomas Heute) Date: Wed, 23 Sep 2015 11:44:28 +0200 Subject: [Hawkular-dev] Metric id and url restrictions In-Reply-To: <560046C4.2010204@redhat.com> References: <55FC2BE4.8060606@redhat.com> <2100669810.13780295.1442590338705.JavaMail.zimbra@redhat.com> <55FC7E88.1020102@redhat.com> <55FFB893.20802@redhat.com> <765231742.39478775.1442834100968.JavaMail.zimbra@redhat.com> <55FFF4B6.3010706@redhat.com> <560046C4.2010204@redhat.com> Message-ID: On Mon, Sep 21, 2015 at 8:04 PM, Matt Wringe wrote: > > This is not just an issue with the Kubernetes proxy, its also an issue > with EAP, other web server, other apache components (or at least some > older versions), and apparently a bunch of other proxies servers out > there. Its a common enough problem in this space that we have already > ran into in two places when dealing with OpenShift integration work. > > I don't think its acceptable to say that since we are compliant, its not > our fault and we wont to anything to help get anyone get around this > issue. We need to work in the real world and this means having to > (potentially) deal with the common problems which exist out there. > +1 And it's not the first time I hear about this issue, didn't we have that issue in RHQ ? (and change EAP default parameters?) I would vote on forbidding / in URLs (metricsId...) if that's the issue (and likely be even more conservative)... URL encoding/decoding is a mess we cannot fix for others (proxies). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/52cb0ca6/attachment.html From mfoley at redhat.com Wed Sep 23 09:21:26 2015 From: mfoley at redhat.com (Michael Foley) Date: Wed, 23 Sep 2015 09:21:26 -0400 (EDT) Subject: [Hawkular-dev] Hawkular MS5 Test Planning Meeting Message-ID: <453992545.56129936.1443014485406.JavaMail.zimbra@redhat.com> The following is a new meeting request: Subject: Hawkular MS5 Test Planning Meeting Organizer: "Michael Foley" Location: http://bluejeans.com/mfoley51 Time: Friday, September 25, 2015, 9:00:00 AM - 9:30:00 AM GMT -05:00 US/Canada Eastern Required: jon-qa-list at redhat.com Optional: jboss-on-team at redhat.com; jon-pm-list at redhat.com; hawkular-dev at lists.jboss.org *~*~*~*~*~*~*~*~*~* Hi, Let's have a Hawkular MS5 Test Planning meeting! We will have seen the MS5 Developer Demo yesterday (September 24th). Let's review the testing plan to and define how we will test this. The link to the etherpad is here -->> http://pad.engineering.redhat.com/HawkularMS5TestPlanning And here is the pre-plan (subject to change, of course, based on what we see and learn in the developer demo) : Hawkular MS5 Test Planning Link to developer demo -->> Jira payload * Link to planned payload https://issues.jboss.org/projects/JMAN4/versions/12327712 * https://issues.jboss.org/browse/JMAN4-34 Deploy and Manage Applications to EAP7 * https://issues.jboss.org/browse/JMAN4-35 Define and Manage Datasources in EAP7 * https://issues.jboss.org/browse/JMAN4-36 Create and Manage JDBC Drivers * https://issues.jboss.org/browse/JMAN4-46 Adminster Audit Logging * https://issues.jboss.org/browse/JMAN4-24 RH Connected Customer, Case Management MS5 Sign-off Document, Michael -->> REST API Automation ...Jeeva * Run the whole test suite to look for regressions * New tests for these Jiras * https://issues.jboss.org/browse/JMAN4-34 Deploy and Manage Applications to EAP7 * https://issues.jboss.org/browse/JMAN4-35 Define and Manage Datasources in EAP7 * https://issues.jboss.org/browse/JMAN4-36 Create and Manage JDBC Drivers * test execution results exported to Polarion Manual testcases in Polarion, Sunil, Prachi, Vojtech * add these 5 requirements * https://issues.jboss.org/browse/JMAN4-34 Deploy and Manage Applications to EAP7 * https://issues.jboss.org/browse/JMAN4-35 Define and Manage Datasources in EAP7 honda civic * https://issues.jboss.org/browse/JMAN4-36 Create and Manage JDBC Drivers * https://issues.jboss.org/browse/JMAN4-46 Adminster Audit Logging * https://issues.jboss.org/browse/JMAN4-24 RH Connected Customer, Case Management * add 5 testcases * 1 documnted testcase execution UI Automation ... Sunil, Prachi, Vojtech * Run the whole test suite to look for regressions * New tests for these Jiras * https://issues.jboss.org/browse/JMAN4-24 RH Connected Customer, Case Management * https://issues.jboss.org/browse/JMAN4-46 Adminster Audit Logging * test execution results exported to Polarion CI/CD Improvements ...Viet, Matt * improve CI/CD to include remote Wildfly * Cantos board ... these specific cards Performance CI....Filip, Vojtech * integration with Perf Repo Openshift Integration Viet, Matt, Peter * http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning * Goals for MS5 * get a few instances of Openshift+Hawkular setup * bladecenter * multi-node setup in Mountain view DONE * very simple smoke test * REST API call to deployed container to see that it is emitting metrics * scalability * UI automation * pair programming with peter ruan to get some automation in place in the openshift cucu-shift CI ... it may only be a few tests -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/0bd4ff90/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 66018 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/0bd4ff90/attachment-0001.bin From tsegismo at redhat.com Wed Sep 23 10:39:06 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 23 Sep 2015 16:39:06 +0200 Subject: [Hawkular-dev] Let embedded Cassandra service manage its own logging Message-ID: <5602B98A.8020807@redhat.com> Hi, I've been looking at the duplicate logs issue in Hawkular since yesterday. Testing has shown that embedded Cassandra is the culprit. I couldn't determine what exactly is going wrong. I suspect it's a logging framework setup mess (Cassandra has a few logging framework dependencies, even JBoss logging...) I tried to make its logs configurable via Wildfly without success. So I've sent a PR to let embedded Cassandra manage its own logging: https://github.com/hawkular/hawkular-commons/pull/16 The consequences are the following: - Cassandra logs go to "${jboss.server.log.dir}/embedded-cassandra.log" - Log files follow a time base rolling policy (similar to Wildfly server log file) - You won't see Cassandra logs in Hawkular standard output anymore It will still be possible to configure Cassandra log level via the "hawkular.log.cassandra" system property (the logback configuration file uses it). I tried the changes in a Hawkular build and they solve the duplicate logs issue. If there's no objection on the principle, could someone please review the pull request? Thanks, Thomas From mfoley at redhat.com Wed Sep 23 11:20:50 2015 From: mfoley at redhat.com (Michael Foley) Date: Wed, 23 Sep 2015 11:20:50 -0400 (EDT) Subject: [Hawkular-dev] MS5 Testing Review Message-ID: <1469510242.56242692.1443021650367.JavaMail.zimbra@redhat.com> The following is a new meeting request: Subject: MS5 Testing Review Organizer: "Michael Foley" Location: http://www.bluejeans.com/mfoley51 Time: Friday, October 16, 2015, 10:00:00 AM - 10:30:00 AM GMT -05:00 US/Canada Eastern Required: jon-qa-list at redhat.com; jon-pm-list at redhat.com Optional: hawkular-dev at lists.jboss.org; jboss-on-team at redhat.com *~*~*~*~*~*~*~*~*~* Hi, It has been 3 weeks from the MS5 Developer Demo; let's have a review of the QE qualification of MS5! Proposed Agenda: * Review the MS5 Testing Plan -->> http://pad.engineering.redhat.com/HawkularMS5TestPlanning * QEs ... add links to all PRs for MS5 test automation into your daily status emails ...and also onto this etherpad for code review * QE Deliverables: * UI Automation ... owner = Sunil (working with Prachi, Vojtech, and Matt) * add tests for new Jiras * refactor existing tests, as needed * a link to a passing test run ... onto the MS5 Test Planning etherpad (or Jiras opened for any issues or regressions) * all test results into Polarion, our system of record * REST API Automation ... owner = Jeeva (working with Matt ) * add tests for new Jiras * refactor existing tests, as needed * a link to a passing test run...onto the MS5 Test Planning etherpad (or Jiras opened for any issues or regressions) * all test results into Polarion, our system of record * Performance CI ... owner - Filip (working with Vojtech) * a link to CI ...and a statement such as "No peformance regressions in MS5" ...or "MS5 has a performance regression of x% based on this PR" * results into either Perf Repo, or Polarion * Continuous Delivery owners = Viet, Matt * a list of any CI/CD improvements made * # of new instances deployed and made available to Hawkular community * Openshift Integration Point * we are meeting separately on this, but ... * we should quickly review this etherpad -->> http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning * the MS5 deliverable for this integation point is: * 1 functional testcase into Polarion * 1 simple scalability testcase into Polarion * 1 passing testcase execution run of these 2 testcases Regards, Michael Foley QE Supervisor, Middleware BU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/68d112b1/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 7899 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/68d112b1/attachment.bin From mfoley at redhat.com Wed Sep 23 12:05:50 2015 From: mfoley at redhat.com (Michael Foley) Date: Wed, 23 Sep 2015 12:05:50 -0400 (EDT) Subject: [Hawkular-dev] MS5 Testing Review Message-ID: <1472813373.56298591.1443024349908.JavaMail.zimbra@redhat.com> The following meeting has been modified: Subject: MS5 Testing Review Organizer: "Michael Foley" Location: http://www.bluejeans.com/mfoley51 Time: Friday, October 16, 2015, 10:00:00 AM - 10:30:00 AM GMT -05:00 US/Canada Eastern Required: jon-qa-list at redhat.com; jon-pm-list at redhat.com; mmahoney at redhat.com; dgeoffro at redhat.com; snegrea at redhat.com Optional: hawkular-dev at lists.jboss.org; jboss-on-team at redhat.com *~*~*~*~*~*~*~*~*~* Hi, It has been 3 weeks from the MS5 Developer Demo; let's have a review of the QE qualification of MS5! Proposed Agenda: * Review the MS5 Testing Plan -->> http://pad.engineering.redhat.com/HawkularMS5TestPlanning * QEs ... add links to all PRs for MS5 test automation into your daily status emails ...and also onto this etherpad for code review * QE Deliverables: * UI Automation ... owner = Sunil (working with Prachi, Vojtech, and Matt) * add tests for new Jiras * refactor existing tests, as needed * a link to a passing test run ... onto the MS5 Test Planning etherpad (or Jiras opened for any issues or regressions) * all test results into Polarion, our system of record * REST API Automation ... owner = Jeeva (working with Matt ) * add tests for new Jiras * refactor existing tests, as needed * a link to a passing test run...onto the MS5 Test Planning etherpad (or Jiras opened for any issues or regressions) * all test results into Polarion, our system of record * Manual Testing...owner = Sunil (working with Prachi, Vojtech, and Matt) * add each MS5 requirement into Polarion * add testcase for each requirement and additionally whatever is demo'd * documented testcase execution run, with a Jira for anything that fails * Performance CI ... owner - Filip (working with Vojtech) * a link to CI ...and a statement such as "No peformance regressions in MS5" ...or "MS5 has a performance regression of x% based on this PR" * results into either Perf Repo, or Polarion * Continuous Delivery owners = Viet, Matt * a list of any CI/CD improvements made * # of new instances deployed and made available to Hawkular community * Openshift Integration Point * we are meeting separately on this, but ... * we should quickly review this etherpad -->> http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning * the MS5 deliverable for this integation point is: * 1 functional testcase into Polarion * 1 simple scalability testcase into Polarion * 1 passing testcase execution run of these 2 testcases Regards, Michael Foley QE Supervisor, Middleware BU -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/2fad59ca/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 9071 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150923/2fad59ca/attachment-0001.bin From ppalaga at redhat.com Thu Sep 24 04:52:44 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Thu, 24 Sep 2015 10:52:44 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 24 Message-ID: <5603B9DC.7070209@redhat.com> Hi *, hawkular-parent 24 brings the following: * srcdeps-maven-plugin 0.0.4 * fixed on Windows * less console output * source dependencies built without tests * testng added to management * wildfly-maven-plugin 1.1.0.Alpha4 was still not available when I was releasing. I have sent PRs to all components repos. Thanks, Peter From tsegismo at redhat.com Thu Sep 24 04:56:48 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 24 Sep 2015 10:56:48 +0200 Subject: [Hawkular-dev] Any idea how to move component specific docs out of website repo? Message-ID: <5603BAD0.4070306@redhat.com> Hi everyone, Currently, the website offers Hawkular docs as well as components and REST API docs. The way REST API docs are updated is very nice: when Travis builds a component's master branch (typically, after a PR is merged), it generates the API doc and then pushes the result to the website repository. I'm wondering what's needed to follow a similar process for component specific docs. Any ideas? The goal would be to move component specific doc files to the component repository. Code and doc in the same place. Thanks, Thomas From mithomps at redhat.com Thu Sep 24 23:03:28 2015 From: mithomps at redhat.com (mike thompson) Date: Thu, 24 Sep 2015 20:03:28 -0700 Subject: [Hawkular-dev] Hawkular UI [Developers] Typescript update Message-ID: If you are doing Hawkular UI development please update your Typescript compiler to 1.6 (from 1.5) as we will begin using those features and you may have issues compiling on your local development machine if you are not using 1.6. Announcing Typescript 1.6: http://blogs.msdn.com/b/typescript/archive/2015/09/16/announcing-typescript-1-6.aspx Please note, if you are just compiling Hawkular via maven no changes are needed. However for UI dev work using gulp and bower incompatibilities may occur. tsc ?version to see what version of typescript and to update: sudo npm update -g typescript It should read Version 1.6.2 after the update NOTE: This is a several day warning before the actual changes will take place this weekend. So if you are running the gulp dev build and get funny errors next week update your TS version first. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150924/e13d334b/attachment.html From mazz at redhat.com Mon Sep 28 09:54:16 2015 From: mazz at redhat.com (John Mazzitelli) Date: Mon, 28 Sep 2015 09:54:16 -0400 (EDT) Subject: [Hawkular-dev] bus API changes - BasicMessage will be interface - AbstractMessage is the new superclass In-Reply-To: <461493588.18760199.1443448338829.JavaMail.zimbra@redhat.com> Message-ID: <2043507893.18764313.1443448456360.JavaMail.zimbra@redhat.com> this is a heads up: Peter has done some refactoring work on the bus API - part of it means BasicMessage is now an interface and AbstractMessage is the new "superclass" that messages extend. IIRC, both inventory and alerts use the bus API as well. this means once this gets merged: https://github.com/hawkular/hawkular-bus/pull/44/files We'll probably need to fix inventory and alerts as well. From lponce at redhat.com Tue Sep 29 03:53:06 2015 From: lponce at redhat.com (Lucas Ponce) Date: Tue, 29 Sep 2015 03:53:06 -0400 (EDT) Subject: [Hawkular-dev] bus API changes - BasicMessage will be interface - AbstractMessage is the new superclass In-Reply-To: <2043507893.18764313.1443448456360.JavaMail.zimbra@redhat.com> References: <2043507893.18764313.1443448456360.JavaMail.zimbra@redhat.com> Message-ID: <382443876.19234739.1443513186469.JavaMail.zimbra@redhat.com> This change is for Bus target version 0.6.2, right ? Is this version planned to be used in Hawkular MS6 ? Thanks, Lucas ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Monday, September 28, 2015 3:54:16 PM > Subject: [Hawkular-dev] bus API changes - BasicMessage will be interface - AbstractMessage is the new superclass > > this is a heads up: > > Peter has done some refactoring work on the bus API - part of it means > BasicMessage is now an interface and AbstractMessage is the new "superclass" > that messages extend. > > IIRC, both inventory and alerts use the bus API as well. this means once this > gets merged: > > https://github.com/hawkular/hawkular-bus/pull/44/files > > We'll probably need to fix inventory and alerts as well. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Tue Sep 29 09:08:51 2015 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 29 Sep 2015 09:08:51 -0400 (EDT) Subject: [Hawkular-dev] bus API changes - BasicMessage will be interface - AbstractMessage is the new superclass In-Reply-To: <382443876.19234739.1443513186469.JavaMail.zimbra@redhat.com> References: <2043507893.18764313.1443448456360.JavaMail.zimbra@redhat.com> <382443876.19234739.1443513186469.JavaMail.zimbra@redhat.com> Message-ID: <40320574.19669251.1443532131635.JavaMail.zimbra@redhat.com> this changes the API in a manner that will require a new minor version (0.7.0.Final) to be released. Yes, hopefully this will go in the next hawkular release, M6. ----- Original Message ----- > This change is for Bus target version 0.6.2, right ? > > Is this version planned to be used in Hawkular MS6 ? > > Thanks, > Lucas > > ----- Original Message ----- > > From: "John Mazzitelli" > > To: "Discussions around Hawkular development" > > > > Sent: Monday, September 28, 2015 3:54:16 PM > > Subject: [Hawkular-dev] bus API changes - BasicMessage will be interface - > > AbstractMessage is the new superclass > > > > this is a heads up: > > > > Peter has done some refactoring work on the bus API - part of it means > > BasicMessage is now an interface and AbstractMessage is the new > > "superclass" > > that messages extend. > > > > IIRC, both inventory and alerts use the bus API as well. this means once > > this > > gets merged: > > > > https://github.com/hawkular/hawkular-bus/pull/44/files > > > > We'll probably need to fix inventory and alerts as well. > > > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From hrupp at redhat.com Tue Sep 29 10:37:43 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 29 Sep 2015 16:37:43 +0200 Subject: [Hawkular-dev] Recording from src-deps presentation Message-ID: <2E738E58-555F-4869-B35A-C65B233C402D@redhat.com> The presentation has been recorded and should be viable here: https://bluejeans.com/s/8CPv/ Peters slides: http://ppalaga.github.io/presentations/150929-srcdeps-maven-plugin/150929-srcdeps-maven-plugin.html#title-slide Thanks again Peter! From jpkroehling at redhat.com Tue Sep 29 11:15:06 2015 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Tue, 29 Sep 2015 17:15:06 +0200 Subject: [Hawkular-dev] Switching personas is now implemented Message-ID: <560AAAFA.2010705@redhat.com> Team, The first screens for Organizations are now implemented, which means that the account switcher (top-right corner) now includes the organizations that the user might switch to. The "SwitchedPersona" event was already implemented, and I think a feature or two was already prepared for that ("Inventory" comes to my mind). For other screens, I don't think it's currently correct, so, please test your screens and see if any action is required once this event occurs. The general rule is: if you got data that is dependent on the current tenant, you might want to re-load the data. Same for permissions: if you checked the permission before showing an item on the screen, you might want to re-evaluate that for the "current tenant". As reminder: the documentation for the events produced by Accounts can be found here: http://www.hawkular.org/docs/dev/accounts.html#_frontend_events And this is an example of how to use it: http://git.io/vc3CL - Juca. From hrupp at redhat.com Tue Sep 29 11:39:05 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 29 Sep 2015 17:39:05 +0200 Subject: [Hawkular-dev] Fwd: [wildfly-dev] Powershell scripts References: Message-ID: <0F8D35CC-2B34-4199-9B1A-02BAC10BD670@redhat.com> Windows devs FYI Forwarded message: > From: Toma? Cerar > To: wildfly-dev at lists.jboss.org > Subject: [wildfly-dev] Powershell scripts > Date: Tue, 29 Sep 2015 17:29:42 +0200 > > Hey guys, > > > just heads up, I've managed to prepare[1] all powershell scripts to > replace > the aging .bat ones. > > New scripts unify most of the shared "code" that all scripts use by > including common.ps1, > and each script only has the code that differs from common behavior > they > all share. > > As result of move we now have much more reliable and faster scripts > that > share common code, > which helps with maintaining a lot. > > New scripts (domain.ps1 & standalone.ps1) now support --background > (also > controllable via .conf.ps1) > option that runs the process in background and remembers its pid and > as > such add support for > feature that until now was only in *nix version of scripts. > > I would like to propose this PR for inclusion to WildFly 10. As it > doesn't > alter any existing behavior > but just adds & improve the script experience on Windows. > > As we get to test and verify that new scripts don't break/include any > currently available option/action > I would also like to remove the batch scripts[2], so they would just > do > call out to their .ps1 counter parts. > > Once this scripts get merged, I will send PR to also add PS scripts > for > WildFly full defined scripts, > given that most of the functionally is already in "common.ps1" new > scripts > will be just simple 5 liners [3] > > > -- > tomaz > > > [1] https://github.com/wildfly/wildfly-core/pull/1118 > [2] https://github.com/wildfly/wildfly-core/pull/1126 > [3] > https://github.com/ctomc/wildfly-core/blob/powershell/core-feature-pack/src/main/resources/content/bin/add-user.ps1 > _______________________________________________ > wildfly-dev mailing list > wildfly-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/wildfly-dev From jsanda at redhat.com Tue Sep 29 22:49:41 2015 From: jsanda at redhat.com (John Sanda) Date: Tue, 29 Sep 2015 22:49:41 -0400 Subject: [Hawkular-dev] batch vs async inserts Message-ID: Many of you have probably seen warnings in the Hawkular server log like, -------------- WARN 15:55:59 Batch of prepared statements for [hawkular_metrics.data, hawkular_metrics.metrics_idx] is of size 5665, exceeding specified threshold of 5120 by 545. -------------- This warning is generated due to batch statements being larger that a threshold defined in cassandra.yaml. It defaults to 5 KB. When the batch statement is larger than that threshold, Cassandra logs the warning. Note that the threshold is based on the actual size of the payload, not the number of statements in the batch. We should stop seeing this warning in 0.7.0 release of Metrics. See HWKMETRICS-252[1] for details. The general advice in the Cassandra community is to favor async writes in parallel over batch inserts when you are trying to improve or optimize write performance. Unlogged batches across multiple partitions is almost always a bad idea. The one exception is with unlogged batches in which all of the mutations are for the same partition. In that case, Cassandra performs the writes atomically. This is how we use batch inserts in metrics. Interestingly I have seen threads on the Cassandra mailing list that still discourage the use of batch inserts even in this case. This thread[1] provides some really interesting insights and analysis on unlogged batch inserts vs async inserts. The thread references a document with some performance analysis that is worth a look. [1] https://issues.jboss.org/browse/HWKMETRICS-252 [2] http://www.mail-archive.com/user%40cassandra.apache.org/msg43976.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20150929/1f18d4f1/attachment-0001.html From hrupp at redhat.com Wed Sep 30 03:26:14 2015 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 30 Sep 2015 09:26:14 +0200 Subject: [Hawkular-dev] Good talk about measuring latency Message-ID: <0A09EB46-0D0B-497C-9D87-8552D3E41653@redhat.com> https://www.youtube.com/watch?v=lJ8ydIuPFeU If you don't have the time - most load generators suck from "coordinated omission". Gatling seems to be one of the few that work correctly https://youtu.be/lJ8ydIuPFeU?t=38m52s From miburman at redhat.com Wed Sep 30 07:24:42 2015 From: miburman at redhat.com (Michael Burman) Date: Wed, 30 Sep 2015 07:24:42 -0400 (EDT) Subject: [Hawkular-dev] batch vs async inserts In-Reply-To: References: Message-ID: <712302532.46327367.1443612282237.JavaMail.zimbra@redhat.com> Hi, I assume this might be one of the reasons for the scalability wall I've seen in my performance tests. No matter what I did (while testing the JAX-RS 2.0 / Servlet 3.1 async I/O performances) I would always hit a limit of around 3200 inserts per second. Although, this happened with a parallel requests each sending just one metric. Neither Cassandra or EAP ate all the CPU, nor was the I/O a bottleneck or out of memory. But no matter the implementation, that was the top it would do (the servlet did less CPU - but still the performance was stuck there). Or was it hitting a limitation of some Datastax-driver-netty-concurrency limit? Either way, it's clearly limiting our performance. And even if it didn't matter there, the catch is that if that was the reason, it would become very apparent with this change of using async writes as we would do more in parallel. - Micke ----- Original Message ----- From: "John Sanda" To: "Discussions around Hawkular development" Sent: Wednesday, September 30, 2015 5:49:41 AM Subject: [Hawkular-dev] batch vs async inserts Many of you have probably seen warnings in the Hawkular server log like, -------------- WARN 15:55:59 Batch of prepared statements for [hawkular_metrics.data, hawkular_metrics.metrics_idx] is of size 5665, exceeding specified threshold of 5120 by 545. -------------- This warning is generated due to batch statements being larger that a threshold defined in cassandra.yaml. It defaults to 5 KB. When the batch statement is larger than that threshold, Cassandra logs the warning. Note that the threshold is based on the actual size of the payload, not the number of statements in the batch. We should stop seeing this warning in 0.7.0 release of Metrics. See HWKMETRICS-252[1] for details. The general advice in the Cassandra community is to favor async writes in parallel over batch inserts when you are trying to improve or optimize write performance. Unlogged batches across multiple partitions is almost always a bad idea. The one exception is with unlogged batches in which all of the mutations are for the same partition. In that case, Cassandra performs the writes atomically. This is how we use batch inserts in metrics. Interestingly I have seen threads on the Cassandra mailing list that still discourage the use of batch inserts even in this case. This thread[1] provides some really interesting insights and analysis on unlogged batch inserts vs async inserts. The thread references a document with some performance analysis that is worth a look. [1] https://issues.jboss.org/browse/HWKMETRICS-252 [2] http://www.mail-archive.com/user%40cassandra.apache.org/msg43976.html _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From tsegismo at redhat.com Wed Sep 30 08:09:41 2015 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 30 Sep 2015 14:09:41 +0200 Subject: [Hawkular-dev] batch vs async inserts In-Reply-To: <712302532.46327367.1443612282237.JavaMail.zimbra@redhat.com> References: <712302532.46327367.1443612282237.JavaMail.zimbra@redhat.com> Message-ID: <560BD105.3010509@redhat.com> FYI, I just created "HWKMETRICS-291 Provide different strategies for C* statement grouping" https://issues.jboss.org/browse/HWKMETRICS-291 Le 30/09/2015 13:24, Michael Burman a ?crit : > Hi, > > I assume this might be one of the reasons for the scalability wall I've seen in my performance tests. No matter what I did (while testing the JAX-RS 2.0 / Servlet 3.1 async I/O performances) I would always hit a limit of around 3200 inserts per second. Although, this happened with a parallel requests each sending just one metric. Neither Cassandra or EAP ate all the CPU, nor was the I/O a bottleneck or out of memory. But no matter the implementation, that was the top it would do (the servlet did less CPU - but still the performance was stuck there). Or was it hitting a limitation of some Datastax-driver-netty-concurrency limit? Either way, it's clearly limiting our performance. > > And even if it didn't matter there, the catch is that if that was the reason, it would become very apparent with this change of using async writes as we would do more in parallel. > > - Micke > > ----- Original Message ----- > From: "John Sanda" > To: "Discussions around Hawkular development" > Sent: Wednesday, September 30, 2015 5:49:41 AM > Subject: [Hawkular-dev] batch vs async inserts > > Many of you have probably seen warnings in the Hawkular server log like, > > -------------- > WARN 15:55:59 Batch of prepared statements for [hawkular_metrics.data, hawkular_metrics.metrics_idx] is of size 5665, exceeding specified threshold of 5120 by 545. > -------------- > > This warning is generated due to batch statements being larger that a threshold defined in cassandra.yaml. It defaults to 5 KB. When the batch statement is larger than that threshold, Cassandra logs the warning. Note that the threshold is based on the actual size of the payload, not the number of statements in the batch. We should stop seeing this warning in 0.7.0 release of Metrics. See HWKMETRICS-252[1] for details. > > The general advice in the Cassandra community is to favor async writes in parallel over batch inserts when you are trying to improve or optimize write performance. Unlogged batches across multiple partitions is almost always a bad idea. The one exception is with unlogged batches in which all of the mutations are for the same partition. In that case, Cassandra performs the writes atomically. This is how we use batch inserts in metrics. Interestingly I have seen threads on the Cassandra mailing list that still discourage the use of batch inserts even in this case. This thread[1] provides some really interesting insights and analysis on unlogged batch inserts vs async inserts. The thread references a document with some performance analysis that is worth a look. > > [1] https://issues.jboss.org/browse/HWKMETRICS-252 > [2] http://www.mail-archive.com/user%40cassandra.apache.org/msg43976.html > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From mazz at redhat.com Wed Sep 30 16:15:05 2015 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 30 Sep 2015 16:15:05 -0400 (EDT) Subject: [Hawkular-dev] new bus and cmdgw API going in In-Reply-To: <2046116193.21063168.1443644087096.JavaMail.zimbra@redhat.com> Message-ID: <1756089154.21063469.1443644105759.JavaMail.zimbra@redhat.com> BTW end of the day today, the new bus and cmdgw API will be in master. I have a PR that fixes pinger and avail-creator so you can see what changes (these two required minimal changes): https://github.com/hawkular/hawkular/pull/506/files Inventory and Alerts will probably need similar changes From ppalaga at redhat.com Wed Sep 30 16:30:13 2015 From: ppalaga at redhat.com (Peter Palaga) Date: Wed, 30 Sep 2015 22:30:13 +0200 Subject: [Hawkular-dev] New and noteworthy in hawkular-parent 25 Message-ID: <560C4655.6040700@redhat.com> Hi *, hawkular-parent 25 brings the following: * srcdeps-maven-plugin 0.0.5 * meets the promisses falsely done for 0.0.4: * less console output * built without tests * wildfly-maven-plugin 1.1.0.Alpha4 I have sent PRs to all components repos. Thanks, Peter _______________________________________________ hawkular-dev mailing list hawkular-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev From snegrea at redhat.com Wed Sep 30 19:33:35 2015 From: snegrea at redhat.com (Stefan Negrea) Date: Wed, 30 Sep 2015 19:33:35 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics 0.7.0 - Release & Beyond In-Reply-To: <668773895.17969803.1440452194258.JavaMail.zimbra@redhat.com> Message-ID: <1815555006.36803806.1443656015921.JavaMail.zimbra@redhat.com> Hello Everybody, I am happy to announce release 0.7.0 of Hawkular Metrics. This release is anchored by major enhancements to counter metrics, updated tag support, and many performance and stability enhancements. Here is a list of major changes in this release: 1) Cassandra Upgrade * Cassandra version 2.2.x is now required for data storage 2) Updated support for counter metrics * Counter raw data and rate data now support buckets similar to the way gauge data does. (HWKMETRICS-280, HWKMETRICS-283) * The same query parameters as gauge metrics are supported. * Tagging functionality (add, delete, update) is now identical to availability and gauge metrics. 3) Revamped tag functionality * Tag support has been updated to have identical functionality across all metric types. * Tagging data points is no longer supported; however, this functionality may resurface when requirements are better understood and there is a real use case around it (HWKMETRICS-247) * Improved metric tag storage and querying; metric tags are no longer stored in the data table (HWKMETRICS-254) 4) Data storage updates * Schema changes will require rebuilding database !!IMPORTANT!! * Interval column has been removed from all tables (HWKMETRICS-3) * Stop the warning message in Cassandra log about batch statement size threshold being exceeded (HWKMETRICS-252) * System-wide data retention default setting is now configurable via system property (HWKMETRICS-251) * Data retention can be set during tenant creation (HWKMETRICS-127) 5) Influx endpoint * time_precision parameter is now supported (Hawkular Metrics does not support microseconds precision though) * integer "overflow" fixes; long integers are now used where needed * time range restrictions support values without unit (i.e. 'time > 1010101010') 6) PTrans * logback replaces log4j as logging backend 7) REST API documentation * Improved documentation (no more broken links, more details on parameters and data types) * http://www.hawkular.org/docs/rest/rest-metrics.html Github Release: https://github.com/hawkular/hawkular-metrics/releases/tag/0.7.0 JBoss Nexus Maven artifacts: http://origin-repository.jboss.org/nexus/content/repositories/public/org/hawkular/metrics/ Jira release tracker: https://issues.jboss.org/browse/HWKMETRICS/fixforversion/12327874/ Hawkular Metrics Clients * Ruby: https://github.com/hawkular/hawkular-client-ruby * Python: https://github.com/hawkular/hawkular-client-python * Go: https://github.com/hawkular/hawkular-client-go Hawkular Metrics 0.8.0 & Beyond: 1) Gauge Aggregates - The task scheduling work is now in place to enable server side aggregates at ingestion times. 2) Improved docker and kubernetes support - this is a long term goal for the project 3) Support for counters in Python and Ruby clients 4) Improved tag support (bulk tag operations and tag queries) 5) Query time metric grouping and aggregation A big "Thank you" goes to John Sanda, Thomas Segismont, Mike Thompson, Matt Wringe, Michael Burman, Libor Zoubek, Jirka Kremser, and Heiko Rupp for their project contributions. Thank you, Stefan Negrea Software Engineer