From snegrea at redhat.com Fri Apr 1 18:24:06 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Fri, 1 Apr 2016 17:24:06 -0500 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint Message-ID: Hello Everybody, I submitted today a PR for a long standing JIRA: https://issues.jboss.org/browse/HWKMETRICS-24 ; here is a related one: https://issues.jboss.org/browse/HWKMETRICS-57 , and the PR: https://github.com/hawkular/hawkular-metrics/pull/473 The JIRAs recommend splitting '*/data' endpoint in two separate endpoints, '/raw' and '/stats'. There are two issues with the current '*/data'. First, the code was really hard to maintain because it was serving two purposes. Depending on the input parameters, the endpoint would serve raw ingested data in some cases and bucketed results in some other cases. The PR still has the old code (since it got just deprecated in this release) and it's not pretty. The second problem was the actual API interface. There was no simple way to know exactly what you get back from the endpoint because sometimes it would return simple data points and sometimes bucketed data. This was based on the query parameters specified in the request. And to make things worse, some parameters could not be mixed, for example, the user could not limit or order bucketed results; but the documentation had to include all under the same endpoint. The plan is to deprecate existing '*/data' endpoints in the upcoming release (0.15.0) and remove them in the release after that (0.16.0). That gives enough time (roughly 2 months) for all existing projects to migrate to the newer API. Does anybody see any problem with this timeline? I expect the transition to be simpler because most of the consumers were using '*/data' with the intent to get bucketed results, which is now '*/stats'. So it is just a simple change in most cases. Are there any projects negatively affected by this change in the long-term? Does the change make sense? Is the API interface for retrieving data easier to understand after the change? Thank you, Stefan Negrea Software Engineer -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160401/b3827bc0/attachment.html From jsanda at redhat.com Sat Apr 2 00:50:40 2016 From: jsanda at redhat.com (John Sanda) Date: Sat, 2 Apr 2016 00:50:40 -0400 Subject: [Hawkular-dev] [Metrics] How to react on low disk? In-Reply-To: References: Message-ID: For whatever I just saw this for this for the first time earlier today. If additional capacity is needed, the primary solution should be to deploy additional Cassandra nodes in order to decrease the load per node. Running compactions manually, i.e., major compactions, should be considered a last resort. Cassandra runs compactions automatically. These are referred to as minor compactions. With a major compaction, Cassandra will merge together all SSTables. With a minor compaction and size-tiered compaction which we use for the data table, Cassandra will merge 4 tables of similar size. The settings are configurable but we can stick with the defaults for the discussion. The reason running major compactions is discouraged is because it often results in minor compactions no longer being executed due to the way size-tiered compaction works. This means that users would have to set up a job to manually run compactions. Not the end of the world, but something that definitely needs to be taken into consideration. Another thing to consider is how deletes are handled by Cassandra. Deleted data is not immediately purged from disk. There is a grace period which is configured per table. It defaults to 10 days. Deleted data is only purged after that grace period. So if we have a bunch of data files on disk that consist of only non-deleted data and/or deleted data for which the grace period has not passed, running a major compaction probably won?t reclaim much space. With HWKMETRICS-367, we have reduced the grace period from the default of 10 days to one day. We need documentation to explain all of this as well as docs to provide some sizing guidelines. I have also floated the idea of a trigger that could either reject writes or send an email notification when disk usage exceeds a threshold, which should be well before the disk is full. If the disk is close to being full and if the data set is large enough, running a compaction could fail as it requires additional disk space. > On Mar 11, 2016, at 11:55 AM, Heiko W.Rupp wrote: > > Hey, > > > so for Hawkular-metrics (and Hawkular) we store the data in a Cassandra > database that puts files on a disk, > which can get full earlier than expected (and usually on week-ends). And > when the disk is full, Metrics does not like it. > > > What can we do in this case? > > I could imagine that on the C* nodes we run a script that uses df to > figure out the available space and tries > to run some compaction if space gets tight. > > Of course that does not solve the issue per se, but should give some air > to breathe. > Right now I fear we are not able to reduce the ttl of a metric/tenant on > the fly and have metrics do the right thing - at least if I understood > yak correctly. > That script should possibly also send an email to an admin. > > In case that we run Hawkular-full, we can determine the disk space and > feed that into Hawkular for Alerts to pick it up and then have the > machinery trigger the compaction and send the email. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Mon Apr 4 03:00:00 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 04 Apr 2016 09:00:00 +0200 Subject: [Hawkular-dev] Good article about smart notifications Message-ID: <93D7E2BB-9864-49E1-AABE-31F19FEF7006@redhat.com> https://medium.com/@intercom/designing-smart-notifications-36336b9c58fb#.fqfwf5srs From hrupp at redhat.com Mon Apr 4 03:23:31 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 04 Apr 2016 09:23:31 +0200 Subject: [Hawkular-dev] GH now allows to Squash your commits on merge Message-ID: <84EE92C2-101C-4390-A19E-E756DEB76F8E@redhat.com> https://github.com/blog/2141-squash-your-commits From tsegismo at redhat.com Mon Apr 4 04:42:41 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 4 Apr 2016 10:42:41 +0200 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: References: Message-ID: Hi Stefan, I haven't looked at the PR, the changes only apply to GET requests, correct? In other words, will POST requests to '*/data' (e.g. from feeds) still work? Thanks, Thomas 2016-04-02 0:24 GMT+02:00 Stefan Negrea : > Hello Everybody, > > I submitted today a PR for a long standing JIRA: > https://issues.jboss.org/browse/HWKMETRICS-24 ; here is a related one: > https://issues.jboss.org/browse/HWKMETRICS-57 , and the PR: > https://github.com/hawkular/hawkular-metrics/pull/473 > > The JIRAs recommend splitting '*/data' endpoint in two separate endpoints, > '/raw' and '/stats'. There are two issues with the current '*/data'. First, > the code was really hard to maintain because it was serving two purposes. > Depending on the input parameters, the endpoint would serve raw ingested > data in some cases and bucketed results in some other cases. The PR still > has the old code (since it got just deprecated in this release) and it's > not pretty. The second problem was the actual API interface. There was no > simple way to know exactly what you get back from the endpoint because > sometimes it would return simple data points and sometimes bucketed data. > This was based on the query parameters specified in the request. And to > make things worse, some parameters could not be mixed, for example, the > user could not limit or order bucketed results; but the documentation had > to include all under the same endpoint. > > The plan is to deprecate existing '*/data' endpoints in the upcoming > release (0.15.0) and remove them in the release after that (0.16.0). That > gives enough time (roughly 2 months) for all existing projects to migrate > to the newer API. Does anybody see any problem with this timeline? > > I expect the transition to be simpler because most of the consumers were > using '*/data' with the intent to get bucketed results, which is now > '*/stats'. So it is just a simple change in most cases. > > Are there any projects negatively affected by this change in the > long-term? Does the change make sense? Is the API interface for retrieving > data easier to understand after the change? > > > Thank you, > Stefan Negrea > > Software Engineer > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160404/0e2ba1e2/attachment.html From hrupp at redhat.com Mon Apr 4 04:56:15 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 04 Apr 2016 10:56:15 +0200 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: References: Message-ID: On 2 Apr 2016, at 0:24, Stefan Negrea wrote: > The plan is to deprecate existing '*/data' endpoints in the upcoming > release (0.15.0) and remove them in the release after that (0.16.0). That > gives enough time (roughly 2 months) for all existing projects to migrate > to the newer API. Does anybody see any problem with this timeline? Yes. There are users out there that use the old api in production. They may not even be aware that there is a new one. If they deploy the new hawkular-metrics (or someone does for them), their apps will break and they may not even know why. REST allows for content negotiation. So we should keep the old endpoint, deprecate them (and perhaps even spit regular warnings in the logs) and on top add the new endpoints that you suggested but with a new content type like "application/json+hawkular-v2". In fact if we keen old and new, that new content type is not exactly needed if the data format exchanged does not change. Btw: I doubt that GET ../data and POST ../data is confusing anyone and in Jax-Rs you can have those on two different methods anyway. From hrupp at redhat.com Mon Apr 4 05:32:31 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 04 Apr 2016 11:32:31 +0200 Subject: [Hawkular-dev] Article about "Managing heterogeneous environments with ManageIQ" Message-ID: <7DF9A668-EAAB-4697-A652-8C68BBFB037B@redhat.com> http://lwn.net/Articles/680060/ From snegrea at redhat.com Mon Apr 4 09:17:01 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Mon, 4 Apr 2016 08:17:01 -0500 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: References: Message-ID: Hello, The POST got migrated to '*/raw' because the data format resembles exactly the GET variation of the two new endpoints. Plus from a logical perspective, we allow users to insert only raw metrics. Thank you, Stefan Thank you, Stefan Negrea Software Engineer On Mon, Apr 4, 2016 at 3:42 AM, Thomas Segismont wrote: > Hi Stefan, > > I haven't looked at the PR, the changes only apply to GET requests, > correct? In other words, will POST requests to '*/data' (e.g. from feeds) > still work? > > Thanks, > Thomas > > 2016-04-02 0:24 GMT+02:00 Stefan Negrea : > >> Hello Everybody, >> >> I submitted today a PR for a long standing JIRA: >> https://issues.jboss.org/browse/HWKMETRICS-24 ; here is a related one: >> https://issues.jboss.org/browse/HWKMETRICS-57 , and the PR: >> https://github.com/hawkular/hawkular-metrics/pull/473 >> >> The JIRAs recommend splitting '*/data' endpoint in two separate >> endpoints, '/raw' and '/stats'. There are two issues with the current >> '*/data'. First, the code was really hard to maintain because it was >> serving two purposes. Depending on the input parameters, the endpoint would >> serve raw ingested data in some cases and bucketed results in some other >> cases. The PR still has the old code (since it got just deprecated in this >> release) and it's not pretty. The second problem was the actual API >> interface. There was no simple way to know exactly what you get back from >> the endpoint because sometimes it would return simple data points and >> sometimes bucketed data. This was based on the query parameters specified >> in the request. And to make things worse, some parameters could not be >> mixed, for example, the user could not limit or order bucketed results; but >> the documentation had to include all under the same endpoint. >> >> The plan is to deprecate existing '*/data' endpoints in the upcoming >> release (0.15.0) and remove them in the release after that (0.16.0). That >> gives enough time (roughly 2 months) for all existing projects to migrate >> to the newer API. Does anybody see any problem with this timeline? >> >> I expect the transition to be simpler because most of the consumers were >> using '*/data' with the intent to get bucketed results, which is now >> '*/stats'. So it is just a simple change in most cases. >> >> Are there any projects negatively affected by this change in the >> long-term? Does the change make sense? Is the API interface for retrieving >> data easier to understand after the change? >> >> >> Thank you, >> Stefan Negrea >> >> Software Engineer >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> > > > -- > Thomas Segismont > JBoss ON Engineering Team > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160404/d36af39d/attachment-0001.html From mwringe at redhat.com Mon Apr 4 10:21:56 2016 From: mwringe at redhat.com (Matt Wringe) Date: Mon, 4 Apr 2016 10:21:56 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: References: Message-ID: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Heiko W.Rupp" > To: "Discussions around Hawkular development" > Sent: Monday, April 4, 2016 4:56:15 AM > Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint > > On 2 Apr 2016, at 0:24, Stefan Negrea wrote: > > The plan is to deprecate existing '*/data' endpoints in the upcoming > > release (0.15.0) and remove them in the release after that (0.16.0). That > > gives enough time (roughly 2 months) for all existing projects to migrate > > to the newer API. Does anybody see any problem with this timeline? > > Yes. > There are users out there that use the old api in production. > They may not even be aware that there is a new one. > > If they deploy the new hawkular-metrics (or someone does > for them), their apps will break and they may not even know > why. +1 I think we need to make sure that we don't break the existing API for current users who are already using it. There will most likely be a bunch of complaints if we update this in OpenShift and then everyone's setup stops working. > REST allows for content negotiation. So we should keep the > old endpoint, deprecate them (and perhaps even spit regular > warnings in the logs) and on top add the new endpoints > that you suggested but with a new content type > like "application/json+hawkular-v2". If we are going to introduce a new version for the REST API, it might make sense to release Hawkular Metrics with a new major version as well. But major versions can be tricky depending on how the project handles them. Some projects will have the same major versions for years, others will update the major version almost monthly (eg kernel version versus browser version) > In fact if we keen old and new, that new content type > is not exactly needed if the data format exchanged does not > change. > > Btw: I doubt that GET ../data and POST ../data is confusing anyone > and in Jax-Rs you can have those on two different methods anyway. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jsanda at redhat.com Mon Apr 4 17:03:21 2016 From: jsanda at redhat.com (John Sanda) Date: Mon, 4 Apr 2016 17:03:21 -0400 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> Message-ID: <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> Making sure we don?t break clients is the most important aspect of this; however, there are some other questions I want to throw out. Today we only support clients adding raw data points. What if in the future though we add support for complex types like histograms and allow clients to directly store them. Would clients post that data to /raw, /stats, or something else? Today we compute aggregated metrics at query time. We are going to add support for pre-computed aggregate metrics that are persisted. Do we have different endpoints for each? I haven?t thought about it too much yet, but I don?t separate endpoints is the way to go. With HWKMETRICS-373, we are introducing support for tag based bucketing as opposed to the date range bucketing already have. There are some minor differences in the data structures returned in the response, namely a map is returned instead of an array. Should we have a different endpoints for tag based buckets vs date range buckets? We currently have one increasingly complex method to handle queries for raw data, various permutations of date range buckets, and eventually tag based buckets once my changes for HWKMETRICS-373 are merged. These changes definitely will simplify things on the server side, but do they simply or otherwise improve things for clients? I am a little concerned that we might be letting our implementation bleed into the API a little too much. For example, we segment our data in our schema design by metric type. Do we necessarily need to do that everywhere in the API? > On Apr 4, 2016, at 10:21 AM, Matt Wringe wrote: > > ----- Original Message ----- >> From: "Heiko W.Rupp" >> To: "Discussions around Hawkular development" >> Sent: Monday, April 4, 2016 4:56:15 AM >> Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint >> >> On 2 Apr 2016, at 0:24, Stefan Negrea wrote: >>> The plan is to deprecate existing '*/data' endpoints in the upcoming >>> release (0.15.0) and remove them in the release after that (0.16.0). That >>> gives enough time (roughly 2 months) for all existing projects to migrate >>> to the newer API. Does anybody see any problem with this timeline? >> >> Yes. >> There are users out there that use the old api in production. >> They may not even be aware that there is a new one. >> >> If they deploy the new hawkular-metrics (or someone does >> for them), their apps will break and they may not even know >> why. > > +1 > > I think we need to make sure that we don't break the existing API for current users who are already using it. There will most likely be a bunch of complaints if we update this in OpenShift and then everyone's setup stops working. > >> REST allows for content negotiation. So we should keep the >> old endpoint, deprecate them (and perhaps even spit regular >> warnings in the logs) and on top add the new endpoints >> that you suggested but with a new content type >> like "application/json+hawkular-v2". > > If we are going to introduce a new version for the REST API, it might make sense to release Hawkular Metrics with a new major version as well. But major versions can be tricky depending on how the project handles them. Some projects will have the same major versions for years, others will update the major version almost monthly (eg kernel version versus browser version) > >> In fact if we keen old and new, that new content type >> is not exactly needed if the data format exchanged does not >> change. >> >> Btw: I doubt that GET ../data and POST ../data is confusing anyone >> and in Jax-Rs you can have those on two different methods anyway. >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160404/fdb2bf5d/attachment.html From snegrea at redhat.com Mon Apr 4 17:52:13 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Mon, 4 Apr 2016 16:52:13 -0500 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> Message-ID: Most of the discussion so far in this thread was about accommodating existing users. We could extend the support for the deprecated code for and additional release (roughly one more month). Here is my proposal 1) 0.15.0 - existing '*/data' endpoints are kept in place but deprecated 2) 0.16.0 - all '*/data' points are still kept but the GET endpoints will be modified to serve only '*/stats' variant 3) 0.17.0 - purge all '*/data' Step 2 above makes sense because the vast majority of our current consumers use GET '/data' to retrieve bucketed results. If there are no strong disagreements about the API change itself than it is just a matter of finding the right balance to deprecate and remove the existing endpoints. Any thoughts? Thank you, Stefan Negrea Software Engineer On Mon, Apr 4, 2016 at 9:21 AM, Matt Wringe wrote: > ----- Original Message ----- > > From: "Heiko W.Rupp" > > To: "Discussions around Hawkular development" < > hawkular-dev at lists.jboss.org> > > Sent: Monday, April 4, 2016 4:56:15 AM > > Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint > > > > On 2 Apr 2016, at 0:24, Stefan Negrea wrote: > > > The plan is to deprecate existing '*/data' endpoints in the upcoming > > > release (0.15.0) and remove them in the release after that (0.16.0). > That > > > gives enough time (roughly 2 months) for all existing projects to > migrate > > > to the newer API. Does anybody see any problem with this timeline? > > > > Yes. > > There are users out there that use the old api in production. > > They may not even be aware that there is a new one. > > > > If they deploy the new hawkular-metrics (or someone does > > for them), their apps will break and they may not even know > > why. > > +1 > > I think we need to make sure that we don't break the existing API for > current users who are already using it. There will most likely be a bunch > of complaints if we update this in OpenShift and then everyone's setup > stops working. > > > REST allows for content negotiation. So we should keep the > > old endpoint, deprecate them (and perhaps even spit regular > > warnings in the logs) and on top add the new endpoints > > that you suggested but with a new content type > > like "application/json+hawkular-v2". > > If we are going to introduce a new version for the REST API, it might make > sense to release Hawkular Metrics with a new major version as well. But > major versions can be tricky depending on how the project handles them. > Some projects will have the same major versions for years, others will > update the major version almost monthly (eg kernel version versus browser > version) > > > In fact if we keen old and new, that new content type > > is not exactly needed if the data format exchanged does not > > change. > > > > Btw: I doubt that GET ../data and POST ../data is confusing anyone > > and in Jax-Rs you can have those on two different methods anyway. > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160404/993e5131/attachment-0001.html From snegrea at redhat.com Mon Apr 4 18:18:25 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Mon, 4 Apr 2016 17:18:25 -0500 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> Message-ID: On Mon, Apr 4, 2016 at 4:03 PM, John Sanda wrote: > Making sure we don?t break clients is the most important aspect of this; > however, there are some other questions I want to throw out. Today we only > support clients adding raw data points. What if in the future though we add > support for complex types like histograms and allow clients to directly > store them. Would clients post that data to /raw, /stats, or something else? > > Today we compute aggregated metrics at query time. We are going to add > support for pre-computed aggregate metrics that are persisted. Do we have > different endpoints for each? I haven?t thought about it too much yet, but > I don?t separate endpoints is the way to go. > In the actual JIRA ticket there were 3 ideas floated around: separate endpoint, or /raw endpoint (because that is not bucketed data), or /stats endpoint (because is bucketed, just server side). And I think those are all the possible variations. One thing that I do not like about '/data' and pre-computed aggregates, if we do not split that method will serve 3 or 4 different formats depending on query params. That is way too complicated for both us and the users. > > With HWKMETRICS-373, we are introducing support for tag based bucketing as > opposed to the date range bucketing already have. There are some minor > differences in the data structures returned in the response, namely a map > is returned instead of an array. Should we have a different endpoints for > tag based buckets vs date range buckets? > I like the idea of just raw and stats. I associate /raw with raw data stored by Hawkular Metrics, and that includes pre-computed aggregates too, and /stats with some sort of statistical computation done do the raw data before returned to the user. Based on this, the tag querying would fall under '/stats' if data is bucketed and would fall under '/raw' if you query for just raw data that is tagged. > > We currently have one increasingly complex method to handle queries for > raw data, various permutations of date range buckets, and eventually tag > based buckets once my changes for HWKMETRICS-373 are merged. These changes > definitely will simplify things on the server side, but do they simply or > otherwise improve things for clients? I am a little concerned that we might > be letting our implementation bleed into the API a little too much. For > example, we segment our data in our schema design by metric type. Do we > necessarily need to do that everywhere in the API? > I like to think about the API in terms of code integration and not necessarily from the perspective of an actual end-user. To me the end-users will never see our interface, they just see some pretty UI. From an integration perspective serving multiple data formats over a single endpoint is not easy to accommodate on most frameworks. But we need to balance exploding the API into too many end-points. The explanation I wrote about raw and stats will help us keep the number of end-points under control. One of the reasons the JIRA ticket was opened is Angular and their design of the REST framework. Serving multiple data formats over the same endpoint is a big no-no. And there is also a special limitation that you need to decided ahead of time if and end-point serves arrays or not; errors occur if you expect array but get single object and vice-versa. With this in mind, I think we need to focus on how consumable is our interface from a machine code perspective. And here is one more perspective to think on this. The current API served us for the past year and a half. Are the changes proposed here going to go good for the next year and half? I think based on the roadmap that we published, the planned features, and the current code, these changes are a step in a good direction. > > On Apr 4, 2016, at 10:21 AM, Matt Wringe wrote: > > ----- Original Message ----- > > From: "Heiko W.Rupp" > To: "Discussions around Hawkular development" < > hawkular-dev at lists.jboss.org> > Sent: Monday, April 4, 2016 4:56:15 AM > Subject: Re: [Hawkular-dev] Hawkular Metrics - /data enpoint > > On 2 Apr 2016, at 0:24, Stefan Negrea wrote: > > The plan is to deprecate existing '*/data' endpoints in the upcoming > release (0.15.0) and remove them in the release after that (0.16.0). That > gives enough time (roughly 2 months) for all existing projects to migrate > to the newer API. Does anybody see any problem with this timeline? > > > Yes. > There are users out there that use the old api in production. > They may not even be aware that there is a new one. > > If they deploy the new hawkular-metrics (or someone does > for them), their apps will break and they may not even know > why. > > > +1 > > I think we need to make sure that we don't break the existing API for > current users who are already using it. There will most likely be a bunch > of complaints if we update this in OpenShift and then everyone's setup > stops working. > > REST allows for content negotiation. So we should keep the > old endpoint, deprecate them (and perhaps even spit regular > warnings in the logs) and on top add the new endpoints > that you suggested but with a new content type > like "application/json+hawkular-v2". > > > If we are going to introduce a new version for the REST API, it might make > sense to release Hawkular Metrics with a new major version as well. But > major versions can be tricky depending on how the project handles them. > Some projects will have the same major versions for years, others will > update the major version almost monthly (eg kernel version versus browser > version) > > In fact if we keen old and new, that new content type > is not exactly needed if the data format exchanged does not > change. > > Btw: I doubt that GET ../data and POST ../data is confusing anyone > and in Jax-Rs you can have those on two different methods anyway. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160404/9e88eef5/attachment.html From hrupp at redhat.com Tue Apr 5 08:49:57 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 05 Apr 2016 14:49:57 +0200 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> Message-ID: <56C322E0-B255-46A1-830B-E0C3E9FD8B4E@redhat.com> Stefan, > existing users. We could extend the support for the deprecated code for and > additional release (roughly one more month). I am still not happy here. We have the current api in *production* with documentation and clients that we do not know about and that we do not control. We can't just violate that contract. And it does not matter if it happens one month earlier or later. On top if we make such an incompatible change, then we need to bump the major version number. > Here is my proposal > 1) 0.15.0 - existing '*/data' endpoints are kept in place but deprecated What about: 1) we keep the existing endpoints and mark them as deprecated. 2) we create new endpoints perhaps even under a different url-root (inside /hawkular/metrics) and only add features to the new endpoints. 3) At some point in time we declare metrics-1.0 and for this one remove the old endpoints. From hrupp at redhat.com Tue Apr 5 08:59:10 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 05 Apr 2016 14:59:10 +0200 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> Message-ID: <2576AABF-449D-4D5C-813D-D00091211ACB@redhat.com> On 4 Apr 2016, at 23:03, John Sanda wrote: > based bucketing as opposed to the date range bucketing already have. > There are some minor differences in the data structures returned in > the response, namely a map is returned instead of an array. Should we > have a different endpoints for tag based buckets vs date range > buckets? If that stays in the same endpoint, then we need to return different content-types We can't have clients guessing here. > > do they simply or otherwise improve things for clients? I am a little > concerned that we might be letting our implementation bleed into the > API a little too much. For example, we segment our data in our schema > design by metric type. Do we necessarily need to do that everywhere in > the API? I agree. We can do a lot more via content-negotiation. The data structure to add a single (raw) data point or a list of data points that belong to different metric-ids is already much too complicated. There may in the future clients like on the photo of [1] or [2] out there that may want to directly send data into h-metrics. There you don't want to fiddle with object mappers to understand complex structures and all that. [1] http://pilhuhn.blogspot.de/2016/02/sending-iot-sensor-data-to-hawkular.html [2] https://www.arduino.cc/en/Main/ArduinoMKR1000 From hrupp at redhat.com Tue Apr 5 09:04:04 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 05 Apr 2016 15:04:04 +0200 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> Message-ID: <294D8FCB-16BB-4BE3-BB1B-6B0371AE52D9@redhat.com> On 5 Apr 2016, at 0:18, Stefan Negrea wrote: > On Mon, Apr 4, 2016 at 4:03 PM, John Sanda wrote: > >> Making sure we don?t break clients is the most important aspect of >> this; >> however, there are some other questions I want to throw out. Today we >> only >> support clients adding raw data points. What if in the future though >> we add >> support for complex types like histograms and allow clients to >> directly >> store them. Would clients post that data to /raw, /stats, or >> something else? >> >> Today we compute aggregated metrics at query time. We are going to >> add >> support for pre-computed aggregate metrics that are persisted. Do we >> have >> different endpoints for each? I haven?t thought about it too much >> yet, but >> I don?t separate endpoints is the way to go. >> > > In the actual JIRA ticket there were 3 ideas floated around: separate > endpoint, or /raw endpoint (because that is not bucketed data), or > /stats > endpoint (because is bucketed, just server side). And I think those > are all > the possible variations. One thing that I do not like about '/data' > and > pre-computed aggregates, if we do not split that method will serve 3 > or 4 > different formats depending on query params. That is way too > complicated > for both us and the users. > > >> >> With HWKMETRICS-373, we are introducing support for tag based >> bucketing as >> opposed to the date range bucketing already have. There are some >> minor >> differences in the data structures returned in the response, namely a >> map >> is returned instead of an array. Should we have a different endpoints >> for >> tag based buckets vs date range buckets? >> > > I like the idea of just raw and stats. I associate /raw with raw data > stored by Hawkular Metrics, and that includes pre-computed aggregates > too, > and /stats with some sort of statistical computation done do the raw > data > before returned to the user. Based on this, the tag querying would > fall > under '/stats' if data is bucketed and would fall under '/raw' if you > query > for just raw data that is tagged. John wrote about "tag based bucketing as opposed to the date range bucketing" So both is bucket data and both /stats -- but with different return types. If this is true, they need to return different content-types. This can not just be returned like so without any notice to the used and the user doing the guesswork. From jpkroehling at redhat.com Tue Apr 5 09:20:34 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Tue, 5 Apr 2016 15:20:34 +0200 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <56C322E0-B255-46A1-830B-E0C3E9FD8B4E@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> <56C322E0-B255-46A1-830B-E0C3E9FD8B4E@redhat.com> Message-ID: <5703BBA2.2070502@redhat.com> On 05.04.2016 14:49, Heiko W.Rupp wrote: > 2) we create new endpoints perhaps even under a > different url-root (inside /hawkular/metrics) and > only add features to the new endpoints. > 3) At some point in time we declare metrics-1.0 and > for this one remove the old endpoints. Time to version the endpoints? /v1/data - Juca. From hrupp at redhat.com Tue Apr 5 09:25:29 2016 From: hrupp at redhat.com (hrupp at redhat.com) Date: Tue, 05 Apr 2016 13:25:29 +0000 Subject: [Hawkular-dev] Termin gestrichen: Hawkular-Team Update - Di 5. Apr. 2016 3:30PM - 4PM (hrupp@redhat.com) Message-ID: <047d7bf0c28e692af5052fbcc78b@google.com> Dieser Termin wurde gestrichen und aus Ihrem Kalender entfernt. Titel: Hawkular-Team Update This is a all-Hawkular team meeting to give updates where we are and so on. This is *open to the public*. Location: on bluejeans: https://redhat.bluejeans.com/hrupp/ or alternatively teleconference Reservationless+ , passcode 204 2160 481 You can find Dial-In telephone numbers here: https://www.intercallonline.com/listNumbersByCode.action?confCode=2042160481 RedHat internal short dial numbers are 16666 and 15555 (and probably others, depends on your location) Wann: Di 5. Apr. 2016 3:30PM - 4PM Berlin Wo: pc 204 2160 481 Kalender: hrupp at redhat.com Wer * Heiko Rupp - Organisator * Mike Thompson * Gabriel Cardoso * gbrown at redhat.com * Jiri Kremser * Thomas Segismont * miburman at redhat.com * theute at redhat.com * Simeon Pinder * Peter Palaga * lkrejci at redhat.com * snegrea at redhat.com * John Mazzitelli * hawkular-dev at lists.jboss.org * jcosta at redhat.com * John Sanda * amendonc at redhat.com * Jay Shaughnessy * Lucas Ponce * mwringe at redhat.com Einladung von Google Kalender: https://www.google.com/calendar/ Sie erhalten diese E-Mail unter hawkular-dev at lists.jboss.org, da Sie ein Gast bei diesem Termin sind. Lehnen Sie diesen Termin ab, um keine weiteren Informationen zu diesem Termin zu erhalten. Sie k?nnen auch unter https://www.google.com/calendar/ ein Google-Konto erstellen und Ihre Benachrichtigungseinstellungen f?r Ihren gesamten Kalender steuern. Wenn Sie diese Einladung weiterleiten, kann jeder Empf?nger Ihre Antwort auf die Einladung ?ndern. Weitere Informationen finden Sie unter https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160405/abd908a5/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 4406 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160405/abd908a5/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 4502 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160405/abd908a5/attachment-0003.bin From jshaughn at redhat.com Tue Apr 5 10:19:58 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Tue, 5 Apr 2016 10:19:58 -0400 Subject: [Hawkular-dev] A better mechanism to import events from Hawkular to MiQ? Message-ID: <5703C98E.1070007@redhat.com> Jkremser, and anyone, So I've now got some code working that can move Hawkular events (events stored via H Alerts) into MiQ as MiQ Events. As a note, not every Hawkular event will be a MiQ event because MiQ requires that supported event types are predefined. This mail is more about the mechanism used to move the events... I've started with a primitive mechanism, it's a looping REST Get with an advancing time window, using the H Alerts rest api via the hawkular-ruby-client. This has a variety of potential issues, for example: * Any Hawkular events generated when MiQ isn't running will likely be missed. * If the timestamps reported on the events are behind the MiQ polling window they will be missed. (late arrival, possibly a hawkular server time a bit behind MiQ server time). * Potentially excessive polling if the number of events is not large. Certainly some of these issues could be softened with a little more provider-side smarts, like querying more into the past and protecting against duplicate event storage, etc. But I'm wondering what thoughts people may have on a better mechanism. I know other providers in MiQ use a variety of techniques to import data, from polling, to blocking HTTP requests, to queue listeners. I should mention that the general approach of an MiQ provider is to provide an "Event Catcher", which runs in a handler process for each provider instance. The catcher is basically told by MiQ to go get events and then queues them for MiQ consumption. Let me know what you think. Also, if anyone would like to see a short demo of what I have right now I'd be happy to run a short meeting. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160405/568e01b4/attachment.html From jshaughn at redhat.com Tue Apr 5 16:46:58 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Tue, 5 Apr 2016 16:46:58 -0400 Subject: [Hawkular-dev] Hawkular Alerts 1.x Roadmap Message-ID: <57042442.3020202@redhat.com> Today, Hawkular Alerts 1.0 offers a rich and comprehensive set of features for adding alerts and events to an application: * Robust REST API * Complex Alerting * Powerful Event Handling * Pluggable Action Handlers with many provided out-of-box (e-mail, SMS, etc) * Alert Lifecycle (Ack, Resolved, etc) * Flexible Deployment, either standalone or as part of Hawkular * Group trigger management * Much More... This is the proposed roadmap for upcoming Hawkular Alerts 1.x releases. The main goal of the 1.x releases will be hardening of existing features, upgrades aligned with the main Hawkular project, fixes and enhancements identified by the community. Currently planned or proposed: * Upgrade to Cassandra 3 (as part of common upgrade in all Hawkular components). o Review and optimize schema * Improvements in the Actions plugins architecture. o Add plugins properties validation. o Introduce deployed/undeployed status on actions definitions. * Extend listeners support to clustering scenarios. * Improve REST API documentation and add additional examples. There is no release date established for Hawkular Alerts 1.1, releases will be made as demanded by the Hawkular project or other community needs. Please let us know if you have questions or suggested additions. Thanks! Hawkular Alerts Team Jay Shaughnessy (jshaughn at redhat.com) Lucas Ponce (lponce at redhat.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160405/b12f355d/attachment.html From ppalaga at redhat.com Wed Apr 6 09:25:18 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Wed, 6 Apr 2016 15:25:18 +0200 Subject: [Hawkular-dev] Titan 0.5.4 with Cassandra 3.3 Message-ID: <57050E3E.3030306@redhat.com> Hi *, Short version: I want to release our own fixed Titan 0.5.4-jboss-1 that works with Cassandra 3.3 Long version: Titan is a graph database used by Hawkular Inventory. ATM, we use Titan 0.5.4 with Cassandra 2.x. As I already noted elsewhere, Titan 0.5.4 requires Guava 15 that is incompatible with Guava 18 required by the newest c* 3.x Driver. The conflict seems to be solved in the titan05 branch [1] in the thinkaurelius repo. I asked both their release engineer directly and on their mailing list [2] if they could release Titan 0.5.5, but got no reply so far. It does not look like we can hope for a release from them soon. Therefore, I propose the following: * I'll fork Titan code from their 0.5.4 tag * I'll apply the changes necessary for Titan 0.5.4 to work with Cassandra 3.3 * I'll change the groupId from com.thinkaurelius.titan to org.hawkular.titan * I'll release the result as Titan 0.5.4-jboss-1 This should be just a temporary solution till Inventory is migrated to Titan 1.0 API that should work with the newest Cassandra. Are there any concerns about this proposal? [1] https://github.com/thinkaurelius/titan/tree/titan05 [2] https://groups.google.com/forum/#!searchin/aureliusgraphs/0.5.5/aureliusgraphs/y9QxXdA7CLo/_k_-n-avDwAJ Thanks, Peter From jshaughn at redhat.com Wed Apr 6 09:33:39 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Wed, 6 Apr 2016 09:33:39 -0400 Subject: [Hawkular-dev] Hawkular Alerts 1.x Roadmap In-Reply-To: <57042442.3020202@redhat.com> References: <57042442.3020202@redhat.com> Message-ID: <57051033.4020207@redhat.com> I have no idea why the original mail was formatted so badly, trying a resend that hopefully improves things... -J Today, Hawkular Alerts 1.0 offers a rich and comprehensive set of features for adding alerts and events to an application: * Robust REST API * Complex Alerting * Powerful Event Handling * Pluggable Action Handlers with many provided out-of-box (e-mail, SMS, etc) * Alert Lifecycle (Ack, Resolved, etc) * Flexible Deployment, either standalone or as part of Hawkular * Group trigger management * Much More... This is the proposed roadmap for upcoming Hawkular Alerts 1.x releases. The main goal of the 1.x releases will be hardening of existing features, upgrades aligned with the main Hawkular project, fixes and enhancements identified by the community. Currently planned or proposed: * Upgrade to Cassandra 3 (as part of common upgrade in all Hawkular components). o Review and optimize schema * Improvements in the Actions plugins architecture. o Add plugins properties validation. o Introduce deployed/undeployed status on actions definitions. * Extend listeners support to clustering scenarios. * Improve REST API documentation and add additional examples. There is no release date established for Hawkular Alerts 1.1, releases will be made as demanded by the Hawkular project or other community needs. Please let us know if you have questions or suggested additions. To request enhancements or track work see Jira: https://issues.jboss.org/projects/HWKALERTS Thanks! Hawkular Alerts Team Jay Shaughnessy (jshaughn at redhat.com) Lucas Ponce (lponce at redhat.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160406/ea0e65b6/attachment.html From snegrea at redhat.com Wed Apr 6 12:22:25 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Wed, 6 Apr 2016 11:22:25 -0500 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <5703BBA2.2070502@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> <56C322E0-B255-46A1-830B-E0C3E9FD8B4E@redhat.com> <5703BBA2.2070502@redhat.com> Message-ID: On Tue, Apr 5, 2016 at 8:20 AM, Juraci Paix?o Kr?hling < jpkroehling at redhat.com> wrote: > On 05.04.2016 14:49, Heiko W.Rupp wrote: > > 2) we create new endpoints perhaps even under a > > different url-root (inside /hawkular/metrics) and > > only add features to the new endpoints. > > 3) At some point in time we declare metrics-1.0 and > > for this one remove the old endpoints. > > Time to version the endpoints? /v1/data > This was discussed in the past a few times and unfortunately did not get a lot of traction. One of the primary reasons is the complexity introduced by having to support multiple REST API versions against multiple C* schemas. It is a really big effort. > > - Juca. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160406/2d6dad19/attachment.html From snegrea at redhat.com Wed Apr 6 13:25:30 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Wed, 6 Apr 2016 12:25:30 -0500 Subject: [Hawkular-dev] Hawkular Metrics - /data enpoint In-Reply-To: <294D8FCB-16BB-4BE3-BB1B-6B0371AE52D9@redhat.com> References: <1153187766.52337076.1459779716684.JavaMail.zimbra@redhat.com> <15B0DC34-7864-4F1C-BA57-8DDB52A435FD@redhat.com> <294D8FCB-16BB-4BE3-BB1B-6B0371AE52D9@redhat.com> Message-ID: Hello, One point not made so far, even today developers have to code twice against GET '*/data/' because of different possible outputs. First is GET '*/data/' with some parameters to get raw data and then another instance with identical URL but different parameters to get bucketed results. The split in API will formalize the differences and will no longer burden the developer with reading the documentation to make sure the right parameters are passed. The typical CRUD REST API design principles cannot be applied in some cases to the Hawkular Metrics interface because of the project's nature. There is little CRUD in ingesting metrics once and making that data available in a few different ways (raw, bucketed, by tags, and eventually pre-computed aggregates). And that makes it somewhat challenging to design and find good articles and examples. But here a few resources that I think are good in this context: 1) https://www.thoughtworks.com/insights/blog/rest-api-design-resource-modeling - section 'An example from the public GitHub API' 2) http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api - section 'What about actions that don't fit into the world of CRUD operations?' 3) https://developer.github.com/v3/repos/statistics/ - documentation page to get statistical data for a repo, the rest of the documentation is also interesting 4) https://developers.facebook.com/docs/marketing-api/reference/ad-keyword-stats - documentation for getting ad keyword statistics I see two common themes in the links above. First, the public APIs are really complex and the number of end-points is really not the concern; more important are ease of use, single purpose, and good documentation. And second, outside of CRUD there is little material (blogs, books, articles) about what to do. With this in mind here is the plan of action based on the feedback in this thread and many discussions we had in the community and amongst Hawkular Metrics contributors. Hawkular Metrics will proceed with merging the first PR to split and restructure '/data', the old code is deprecated but intact. We will work with the community and current users to find the best path to fully remove '*/data'. We will publish more details about the deprecation and plans for removal as soon as we have more details; also except this change to be documented in detail in the release notes until fully removed. Just to be clear, this is not a single release transition to fully remove '*/data/', so far we identified a minimum of 3 releases, but it could extend beyond that based on feedback. Thank you, Stefan Negrea Software Engineer On Tue, Apr 5, 2016 at 8:04 AM, Heiko W.Rupp wrote: > On 5 Apr 2016, at 0:18, Stefan Negrea wrote: > > > On Mon, Apr 4, 2016 at 4:03 PM, John Sanda wrote: > > > >> Making sure we don?t break clients is the most important aspect of > >> this; > >> however, there are some other questions I want to throw out. Today we > >> only > >> support clients adding raw data points. What if in the future though > >> we add > >> support for complex types like histograms and allow clients to > >> directly > >> store them. Would clients post that data to /raw, /stats, or > >> something else? > >> > >> Today we compute aggregated metrics at query time. We are going to > >> add > >> support for pre-computed aggregate metrics that are persisted. Do we > >> have > >> different endpoints for each? I haven?t thought about it too much > >> yet, but > >> I don?t separate endpoints is the way to go. > >> > > > > In the actual JIRA ticket there were 3 ideas floated around: separate > > endpoint, or /raw endpoint (because that is not bucketed data), or > > /stats > > endpoint (because is bucketed, just server side). And I think those > > are all > > the possible variations. One thing that I do not like about '/data' > > and > > pre-computed aggregates, if we do not split that method will serve 3 > > or 4 > > different formats depending on query params. That is way too > > complicated > > for both us and the users. > > > > > >> > >> With HWKMETRICS-373, we are introducing support for tag based > >> bucketing as > >> opposed to the date range bucketing already have. There are some > >> minor > >> differences in the data structures returned in the response, namely a > >> map > >> is returned instead of an array. Should we have a different endpoints > >> for > >> tag based buckets vs date range buckets? > >> > > > > I like the idea of just raw and stats. I associate /raw with raw data > > stored by Hawkular Metrics, and that includes pre-computed aggregates > > too, > > and /stats with some sort of statistical computation done do the raw > > data > > before returned to the user. Based on this, the tag querying would > > fall > > under '/stats' if data is bucketed and would fall under '/raw' if you > > query > > for just raw data that is tagged. > > > John wrote about "tag based bucketing as opposed to the date range > bucketing" > > So both is bucket data and both /stats -- but with different return > types. > If this is true, they need to return different content-types. This can > not just > be returned like so without any notice to the used and the user doing > the > guesswork. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160406/9c5d68e8/attachment-0001.html From ppalaga at redhat.com Wed Apr 6 17:01:29 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Wed, 6 Apr 2016 23:01:29 +0200 Subject: [Hawkular-dev] Titan 0.5.4 with Cassandra 3.3 In-Reply-To: <57050E3E.3030306@redhat.com> References: <57050E3E.3030306@redhat.com> Message-ID: <57057929.80201@redhat.com> Here is the branch from which that I want to release: https://github.com/ppalaga/titan/commits/0.5.4-jboss -- P On 2016-04-06 15:25, Peter Palaga wrote: > Hi *, > > Short version: I want to release our own fixed Titan 0.5.4-jboss-1 that > works with Cassandra 3.3 > > Long version: > > Titan is a graph database used by Hawkular Inventory. ATM, we use Titan > 0.5.4 with Cassandra 2.x. > > As I already noted elsewhere, Titan 0.5.4 requires Guava 15 that is > incompatible with Guava 18 required by the newest c* 3.x Driver. The > conflict seems to be solved in the titan05 branch [1] in the > thinkaurelius repo. I asked both their release engineer directly and on > their mailing list [2] if they could release Titan 0.5.5, but got no > reply so far. It does not look like we can hope for a release from them > soon. > > Therefore, I propose the following: > * I'll fork Titan code from their 0.5.4 tag > * I'll apply the changes necessary for Titan 0.5.4 to work with > Cassandra 3.3 > * I'll change the groupId from com.thinkaurelius.titan to org.hawkular.titan > * I'll release the result as Titan 0.5.4-jboss-1 > > This should be just a temporary solution till Inventory is migrated to > Titan 1.0 API that should work with the newest Cassandra. > > Are there any concerns about this proposal? > > [1] https://github.com/thinkaurelius/titan/tree/titan05 > [2] > https://groups.google.com/forum/#!searchin/aureliusgraphs/0.5.5/aureliusgraphs/y9QxXdA7CLo/_k_-n-avDwAJ > > Thanks, > > Peter > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Thu Apr 7 15:02:47 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Thu, 7 Apr 2016 21:02:47 +0200 Subject: [Hawkular-dev] Titan 0.5.4 with Cassandra 3.3 In-Reply-To: <57057929.80201@redhat.com> References: <57050E3E.3030306@redhat.com> <57057929.80201@redhat.com> Message-ID: <5706AED7.7020305@redhat.com> Released http://origin-repository.jboss.org/nexus/content/repositories/public/org/hawkular/titan/titan-cassandra/ -- P On 2016-04-06 23:01, Peter Palaga wrote: > Here is the branch from which that I want to release: > > https://github.com/ppalaga/titan/commits/0.5.4-jboss > > -- P > > On 2016-04-06 15:25, Peter Palaga wrote: >> Hi *, >> >> Short version: I want to release our own fixed Titan 0.5.4-jboss-1 that >> works with Cassandra 3.3 >> >> Long version: >> >> Titan is a graph database used by Hawkular Inventory. ATM, we use Titan >> 0.5.4 with Cassandra 2.x. >> >> As I already noted elsewhere, Titan 0.5.4 requires Guava 15 that is >> incompatible with Guava 18 required by the newest c* 3.x Driver. The >> conflict seems to be solved in the titan05 branch [1] in the >> thinkaurelius repo. I asked both their release engineer directly and on >> their mailing list [2] if they could release Titan 0.5.5, but got no >> reply so far. It does not look like we can hope for a release from them >> soon. >> >> Therefore, I propose the following: >> * I'll fork Titan code from their 0.5.4 tag >> * I'll apply the changes necessary for Titan 0.5.4 to work with >> Cassandra 3.3 >> * I'll change the groupId from com.thinkaurelius.titan to org.hawkular.titan >> * I'll release the result as Titan 0.5.4-jboss-1 >> >> This should be just a temporary solution till Inventory is migrated to >> Titan 1.0 API that should work with the newest Cassandra. >> >> Are there any concerns about this proposal? >> >> [1] https://github.com/thinkaurelius/titan/tree/titan05 >> [2] >> https://groups.google.com/forum/#!searchin/aureliusgraphs/0.5.5/aureliusgraphs/y9QxXdA7CLo/_k_-n-avDwAJ >> >> Thanks, >> >> Peter >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From ppalaga at redhat.com Thu Apr 7 16:12:01 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Thu, 7 Apr 2016 22:12:01 +0200 Subject: [Hawkular-dev] Hawkular Parent 39/40 Message-ID: <5706BF11.1060208@redhat.com> Hi *, What is new in Hawkular Parent 39 [1]: * Guava 18.0 managed - it is the version provided by WF 10. Till now, we relied on Guava 16.0 that we provided ourselves in a dedicated module because of Titan 0.5.4. * Activate the test.groovy profile based on the existence of src/test/groovy rather than on a in-pom.xml property that does not work * hawkular-build-tools 16 with the Checkstyle line length rule removed Hawkular Parent 40 [2]: * Upgrade to c* 3.4 Upgrades to Hawkular Parent 40 happened in Commons and Accounts already and are underway in Metrics [3] and Inventory [4] Thanks, Peter [1] https://github.com/hawkular/hawkular-parent-pom/commits/39 [2] https://github.com/hawkular/hawkular-parent-pom/commits/40 [3] https://github.com/hawkular/hawkular-metrics/pull/478 [4] https://github.com/hawkular/hawkular-inventory/pull/238 From snegrea at redhat.com Fri Apr 8 11:56:38 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Fri, 8 Apr 2016 10:56:38 -0500 Subject: [Hawkular-dev] Hawkular Alerts 1.x Roadmap In-Reply-To: <57051033.4020207@redhat.com> References: <57042442.3020202@redhat.com> <57051033.4020207@redhat.com> Message-ID: On Wed, Apr 6, 2016 at 8:33 AM, Jay Shaughnessy wrote: > I have no idea why the original mail was formatted so badly, trying a > resend that hopefully improves things... -J > > > Today, Hawkular Alerts 1.0 offers a rich and comprehensive set of features > for adding alerts and events to an application: > > - Robust REST API > - Complex Alerting > - Powerful Event Handling > - Pluggable Action Handlers with many provided out-of-box (e-mail, > SMS, etc) > - Alert Lifecycle (Ack, Resolved, etc) > - Flexible Deployment, either standalone or as part of Hawkular > - Group trigger management > - Much More... > > This is the proposed roadmap for upcoming Hawkular Alerts 1.x releases. > The main goal of the 1.x releases will be hardening of existing features, > upgrades aligned with the main Hawkular project, fixes and enhancements > identified by the community. Currently planned or proposed: > > - Upgrade to Cassandra 3 (as part of common upgrade in all Hawkular > components). > - Review and optimize schema > > Did you consider using the same schema management library that Hawkular Metrics just introduced in 0.15.0? ( https://github.com/hawkular/hawkular-metrics/tree/master/core/schema) > > - Improvements in the Actions plugins architecture. > - Add plugins properties validation. > - Introduce deployed/undeployed status on actions definitions. > - Extend listeners support to clustering scenarios. > - Improve REST API documentation and add additional examples. > > There is no release date established for Hawkular Alerts 1.1, releases > will be made as demanded by the Hawkular project or other community needs. > Do you plan on doing incremental releases? For example, if the C* 3.x upgrade is ready, are you going to release just that or will wait until the entire list above is finished? > > Please let us know if you have questions or suggested additions. To > request enhancements or track work see Jira: > https://issues.jboss.org/projects/HWKALERTS > > Thanks! > > Hawkular Alerts Team > Jay Shaughnessy (jshaughn at redhat.com) > Lucas Ponce (lponce at redhat.com) > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160408/4790c4ab/attachment.html From lponce at redhat.com Fri Apr 8 12:03:29 2016 From: lponce at redhat.com (Lucas Ponce) Date: Fri, 8 Apr 2016 12:03:29 -0400 (EDT) Subject: [Hawkular-dev] Hawkular Alerts 1.x Roadmap In-Reply-To: References: <57042442.3020202@redhat.com> <57051033.4020207@redhat.com> Message-ID: <1393972768.20788432.1460131409170.JavaMail.zimbra@redhat.com> > > > On Wed, Apr 6, 2016 at 8:33 AM, Jay Shaughnessy < jshaughn at redhat.com > > wrote: > > > > I have no idea why the original mail was formatted so badly, trying a resend > that hopefully improves things... -J > > > Today, Hawkular Alerts 1.0 offers a rich and comprehensive set of features > for adding alerts and events to an application: > > > * Robust REST API > * Complex Alerting > * Powerful Event Handling > * Pluggable Action Handlers with many provided out-of-box (e-mail, SMS, > etc) > * Alert Lifecycle (Ack, Resolved, etc) > * Flexible Deployment, either standalone or as part of Hawkular > * Group trigger management > * Much More... > > This is the proposed roadmap for upcoming Hawkular Alerts 1.x releases. The > main goal of the 1.x releases will be hardening of existing features, > upgrades aligned with the main Hawkular project, fixes and enhancements > identified by the community. Currently planned or proposed: > > > * Upgrade to Cassandra 3 (as part of common upgrade in all Hawkular > components). > > > * Review and optimize schema > Did you consider using the same schema management library that Hawkular > Metrics just introduced in 0.15.0? ( > https://github.com/hawkular/hawkular-metrics/tree/master/core/schema ) > > +1 I like the idea to share the same schema management. Is this library in commons in a easy consumable way ? > > > > * Improvements in the Actions plugins architecture. > > * Add plugins properties validation. > * Introduce deployed/undeployed status on actions definitions. > * Extend listeners support to clustering scenarios. * Improve REST > API documentation and add additional examples. > > There is no release date established for Hawkular Alerts 1.1, releases will > be made as demanded by the Hawkular project or other community needs. > > Do you plan on doing incremental releases? For example, if the C* 3.x upgrade > is ready, are you going to release just that or will wait until the entire > list above is finished? > > We can do it. No strong dates defined, so with this feedback I think this is a valid suggestion. > > > Please let us know if you have questions or suggested additions. To request > enhancements or track work see Jira: > https://issues.jboss.org/projects/HWKALERTS > > Thanks! > > Hawkular Alerts Team > Jay Shaughnessy ( jshaughn at redhat.com ) > Lucas Ponce ( lponce at redhat.com ) > > From jsanda at redhat.com Fri Apr 8 12:11:58 2016 From: jsanda at redhat.com (John Sanda) Date: Fri, 8 Apr 2016 12:11:58 -0400 Subject: [Hawkular-dev] Hawkular Alerts 1.x Roadmap In-Reply-To: <1393972768.20788432.1460131409170.JavaMail.zimbra@redhat.com> References: <57042442.3020202@redhat.com> <57051033.4020207@redhat.com> <1393972768.20788432.1460131409170.JavaMail.zimbra@redhat.com> Message-ID: <9F9FEF62-943B-4994-8161-D89E05C435FE@redhat.com> > On Apr 8, 2016, at 12:03 PM, Lucas Ponce wrote: > >> >> >> On Wed, Apr 6, 2016 at 8:33 AM, Jay Shaughnessy < jshaughn at redhat.com > >> wrote: >> >> >> >> I have no idea why the original mail was formatted so badly, trying a resend >> that hopefully improves things... -J >> >> >> Today, Hawkular Alerts 1.0 offers a rich and comprehensive set of features >> for adding alerts and events to an application: >> >> >> * Robust REST API >> * Complex Alerting >> * Powerful Event Handling >> * Pluggable Action Handlers with many provided out-of-box (e-mail, SMS, >> etc) >> * Alert Lifecycle (Ack, Resolved, etc) >> * Flexible Deployment, either standalone or as part of Hawkular >> * Group trigger management >> * Much More... >> >> This is the proposed roadmap for upcoming Hawkular Alerts 1.x releases. The >> main goal of the 1.x releases will be hardening of existing features, >> upgrades aligned with the main Hawkular project, fixes and enhancements >> identified by the community. Currently planned or proposed: >> >> >> * Upgrade to Cassandra 3 (as part of common upgrade in all Hawkular >> components). >> >> >> * Review and optimize schema >> Did you consider using the same schema management library that Hawkular >> Metrics just introduced in 0.15.0? ( >> https://github.com/hawkular/hawkular-metrics/tree/master/core/schema ) >> >> > > +1 > > I like the idea to share the same schema management. > Is this library in commons in a easy consumable way ? The library is in its own repo[1]. [1] https://github.com/jsanda/cassalog > >> >> >> >> * Improvements in the Actions plugins architecture. >> >> * Add plugins properties validation. >> * Introduce deployed/undeployed status on actions definitions. >> * Extend listeners support to clustering scenarios. * Improve REST >> API documentation and add additional examples. >> >> There is no release date established for Hawkular Alerts 1.1, releases will >> be made as demanded by the Hawkular project or other community needs. >> >> Do you plan on doing incremental releases? For example, if the C* 3.x upgrade >> is ready, are you going to release just that or will wait until the entire >> list above is finished? >> >> > > We can do it. No strong dates defined, so with this feedback I think this is a valid suggestion. > >> >> >> Please let us know if you have questions or suggested additions. To request >> enhancements or track work see Jira: >> https://issues.jboss.org/projects/HWKALERTS >> >> Thanks! >> >> Hawkular Alerts Team >> Jay Shaughnessy ( jshaughn at redhat.com ) >> Lucas Ponce ( lponce at redhat.com ) >> >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160408/49a7883d/attachment-0001.html From mazz at redhat.com Fri Apr 8 15:38:42 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 8 Apr 2016 15:38:42 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <860028183.20822770.1460143864894.JavaMail.zimbra@redhat.com> Message-ID: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> I'm trying to figure out how the Hawkular WildFly Agent needs to be enhanced to collect metrics from Prometheus (which is where a lot of Open Shift metrics are going). Here is how I originally understood the problem (which may be completely wrong): I am looking at this: https://prometheus.io/docs/querying/api/ So if Open Shift components are storing metrics in Prometheus, the Agent would need to query the data via something like: http://localhost:9090/api/v1/query?query=my_metric_name_seconds{label_one=my_label,another_label=some_value} The agent can take the data it gets and store it in Hawkular Metrics (using a different metric name and/or labels if we want). I am hoping Matt W. can clarify and if this is completely wrong, how does he see it working? --John Mazz From mwringe at redhat.com Fri Apr 8 17:04:39 2016 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 8 Apr 2016 17:04:39 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> References: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> Message-ID: <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Friday, April 8, 2016 3:38:42 PM > Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift > > I'm trying to figure out how the Hawkular WildFly Agent needs to be enhanced > to collect metrics from Prometheus (which is where a lot of Open Shift > metrics are going). > > Here is how I originally understood the problem (which may be completely > wrong): > > I am looking at this: https://prometheus.io/docs/querying/api/ > > So if Open Shift components are storing metrics in Prometheus, the Agent > would need to query the data via something like: > > http://localhost:9090/api/v1/query?query=my_metric_name_seconds{label_one=my_label,another_label=some_value} > > The agent can take the data it gets and store it in Hawkular Metrics (using a > different metric name and/or labels if we want). > > I am hoping Matt W. can clarify and if this is completely wrong, how does he > see it working? Not quite. There are some endpoints used in OpenShift/Kubernetes that stores metrics using the Prometheus protocol. These endpoint expose metrics in which Prometheus (or something which understands the protocol) can then bring into their system. We are looking at potentially being able to read from these endpoint types, not being able to read from a Prometheus server itself. A prime example of this is the "/metrics" endpoint which is exposed on every Kubernetes node. There is also the '/stats/ endpoint on the node which Heapster uses, but it only contains a subset of what is available under '/metrics'. It would be good to be able to bring in some of these extra metrics as well. There are also custom metrics that a container can expose which also uses this protocol. It is important when we bring in extra metrics that they contain similar tags that our metrics from Heapster already contain. This is so we can perform similar queries regardless of where the original metric came from. We should also try and generate the metric ids in a similar manner as well. From mazz at redhat.com Fri Apr 8 17:22:32 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 8 Apr 2016 17:22:32 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> References: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> Message-ID: <1570028055.20835527.1460150552055.JavaMail.zimbra@redhat.com> > There are some endpoints used in OpenShift/Kubernetes that stores metrics > using the Prometheus protocol. These endpoint expose metrics in which > Prometheus (or something which understands the protocol) can then bring into > their system. We are looking at potentially being able to read from these > endpoint types, not being able to read from a Prometheus server itself. Then it sounds more like we'd want to have some kind of plugin or component inside Hawkular-Metrics itself so H-Metrics can just pull metrics in using the Prometheus protocol (in addition to getting metrics pushed into it via its REST API). This could be an optional component to Hawkular-Metrics for those that don't need or want it. But it sounds like this is just another protocol to insert data into H-Metrics. From mwringe at redhat.com Fri Apr 8 18:07:02 2016 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 8 Apr 2016 18:07:02 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <1570028055.20835527.1460150552055.JavaMail.zimbra@redhat.com> References: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> <1570028055.20835527.1460150552055.JavaMail.zimbra@redhat.com> Message-ID: <613960713.55514985.1460153222740.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Friday, April 8, 2016 5:22:32 PM > Subject: Re: [Hawkular-dev] hawkular-agent, prometheus, openshift > > > There are some endpoints used in OpenShift/Kubernetes that stores metrics > > using the Prometheus protocol. These endpoint expose metrics in which > > Prometheus (or something which understands the protocol) can then bring > > into > > their system. We are looking at potentially being able to read from these > > endpoint types, not being able to read from a Prometheus server itself. > > Then it sounds more like we'd want to have some kind of plugin or component > inside Hawkular-Metrics itself so H-Metrics can just pull metrics in using > the Prometheus protocol (in addition to getting metrics pushed into it via > its REST API). This could be an optional component to Hawkular-Metrics for > those that don't need or want it. But it sounds like this is just another > protocol to insert data into H-Metrics. I don't see this as something which should be part of Hawkular Metrics. Essentially we are going to be dealing with people running various different types of containers in their OpenShift environment. My tomcat server may be exposing metrics in the jolokia format, my golang app over there might be exposing metrics in the prometheus protocol, my other app somewhere else might be exposing their metrics in some other protocol. We need something to figure out where those endpoints are available from, gather the metadata about the containers which are exposing those endpoints, convert the metrics from one type into something Hawkular Metrics can understand, and then write those metrics (with metadata based tags) into Hawkular Metrics. I think this needs to be part of a separate component. There is going to be a lot of code here dealing specifically with the OpenShift environment. If we are reading from a jolokia endpoint and we have an agent which knows how to store jolokia style metrics in Hawkular-Metrics, then we should of course try and reuse components. From mazz at redhat.com Fri Apr 8 18:38:16 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 8 Apr 2016 18:38:16 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <613960713.55514985.1460153222740.JavaMail.zimbra@redhat.com> References: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> <1570028055.20835527.1460150552055.JavaMail.zimbra@redhat.com> <613960713.55514985.1460153222740.JavaMail.zimbra@redhat.com> Message-ID: <537694385.20855610.1460155096216.JavaMail.zimbra@redhat.com> > We need something to figure out where those endpoints are available from, > gather the metadata about the containers which are exposing those endpoints, > convert the metrics from one type into something Hawkular Metrics can > understand, and then write those metrics (with metadata based tags) into > Hawkular Metrics. > > I think this needs to be part of a separate component. There is going to be a > lot of code here dealing specifically with the OpenShift environment. > > If we are reading from a jolokia endpoint and we have an agent which knows > how to store jolokia style metrics in Hawkular-Metrics, then we should of > course try and reuse components. OK, I'll look more into the Prometheus protocol. That's the part I was missing. It sounds like essentially we are going to be pulling metrics directly from the components emitting the data (rather than pulling metrics from the Prometheus server itself which is what I originally thought). Now that I think about it, really this sounds just like JMX/Jolokia endpoints or WildFly's remote DMR interface). I just have to learn more about what these Prometheus endpoints look like and what the data looks like over that protocol. From mwringe at redhat.com Fri Apr 8 18:48:08 2016 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 8 Apr 2016 18:48:08 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <537694385.20855610.1460155096216.JavaMail.zimbra@redhat.com> References: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> <1570028055.20835527.1460150552055.JavaMail.zimbra@redhat.com> <613960713.55514985.1460153222740.JavaMail.zimbra@redhat.com> <537694385.20855610.1460155096216.JavaMail.zimbra@redhat.com> Message-ID: <727487379.55517015.1460155688859.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "John Mazzitelli" > To: "Discussions around Hawkular development" > Sent: Friday, April 8, 2016 6:38:16 PM > Subject: Re: [Hawkular-dev] hawkular-agent, prometheus, openshift > > > We need something to figure out where those endpoints are available from, > > gather the metadata about the containers which are exposing those > > endpoints, > > convert the metrics from one type into something Hawkular Metrics can > > understand, and then write those metrics (with metadata based tags) into > > Hawkular Metrics. > > > > I think this needs to be part of a separate component. There is going to be > > a > > lot of code here dealing specifically with the OpenShift environment. > > > > If we are reading from a jolokia endpoint and we have an agent which knows > > how to store jolokia style metrics in Hawkular-Metrics, then we should of > > course try and reuse components. > > OK, I'll look more into the Prometheus protocol. That's the part I was > missing. It sounds like essentially we are going to be pulling metrics > directly from the components emitting the data (rather than pulling metrics > from the Prometheus server itself which is what I originally thought). Now > that I think about it, really this sounds just like JMX/Jolokia endpoints or > WildFly's remote DMR interface). I just have to learn more about what these > Prometheus endpoints look like and what the data looks like over that > protocol. Yep, exactly. Its all about taking things from one protocol (jolokia, prometheus, etc) and being able to send the data contained within that protocol into Hawkular Metrics. From mazz at redhat.com Sun Apr 10 00:10:47 2016 From: mazz at redhat.com (John Mazzitelli) Date: Sun, 10 Apr 2016 00:10:47 -0400 (EDT) Subject: [Hawkular-dev] hawkular-agent, prometheus, openshift In-Reply-To: <727487379.55517015.1460155688859.JavaMail.zimbra@redhat.com> References: <1499396658.20823626.1460144321995.JavaMail.zimbra@redhat.com> <312655616.55509833.1460149479544.JavaMail.zimbra@redhat.com> <1570028055.20835527.1460150552055.JavaMail.zimbra@redhat.com> <613960713.55514985.1460153222740.JavaMail.zimbra@redhat.com> <537694385.20855610.1460155096216.JavaMail.zimbra@redhat.com> <727487379.55517015.1460155688859.JavaMail.zimbra@redhat.com> Message-ID: <1116644035.20940259.1460261447879.JavaMail.zimbra@redhat.com> > > OK, I'll look more into the Prometheus protocol. That's the part I was > > missing. It sounds like essentially we are going to be pulling metrics > > directly from the components emitting the data (rather than pulling metrics > > from the Prometheus server itself which is what I originally thought). Now > > that I think about it, really this sounds just like JMX/Jolokia endpoints > > or > > WildFly's remote DMR interface). I just have to learn more about what these > > Prometheus endpoints look like and what the data looks like over that > > protocol. > > Yep, exactly. Its all about taking things from one protocol (jolokia, > prometheus, etc) and being able to send the data contained within that > protocol into Hawkular Metrics. OK, this looks very easy to do. I just wrote up a quick PoC that shows we can have a Prometheus protocol endpoint scraper that can pull metrics from any prom protocol endpoint: https://github.com/hawkular/hawkular-agent/blob/mazz/hwkagent-66-prometheus/prometheus-scraper/src/main/java/org/hawkular/agent/prometheus/Main.java#L40 this doesn't do much - just pulls metrics over HTTP from a prometheus protocol endpoint URL and parses the binary data using a prometheus java client. But it shows me it's easy (given any URL to a prometheus endpoint) to pull metrics data. We'd just push it out to hawkular-metrics (it would work just like our JMX and DMR integrations). So my task for next week will be to take this and integrate it into hawkular-wildfly-agent. From mazz at redhat.com Mon Apr 11 21:00:32 2016 From: mazz at redhat.com (John Mazzitelli) Date: Mon, 11 Apr 2016 21:00:32 -0400 (EDT) Subject: [Hawkular-dev] java-based prometheus metrics scraper In-Reply-To: <845213401.310543.1460422524671.JavaMail.zimbra@redhat.com> Message-ID: <1294561257.311268.1460422832468.JavaMail.zimbra@redhat.com> FYI: I finished a Java-based Prometheus scraper - seems to be working. It is just code that scrapes any remote Prometheus endpoint given a URL and let's you easily walk the metric data for further processing (like, if you want to insert that data into Hawkular Metrics or if you want to dump the data in XML or JSON format - the CLI does this today, see below). I'll eventually blog about this, but not until I actually merge it into master. It's in my branch right now here: https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-66-prometheus/prometheus-scraper This also includes a CLI tool - build the scraper maven module and you'll get an uber-jar that let's you scrape a remote Prometheus endpoint and dump the metric data in XML, JSON, or simple form. See the README.md for more. Now I just have to figure out how to integrate this into the Hawkular WildFly Agent :-) From lkrejci at redhat.com Tue Apr 12 06:48:52 2016 From: lkrejci at redhat.com (Lukas Krejci) Date: Tue, 12 Apr 2016 12:48:52 +0200 Subject: [Hawkular-dev] java-based prometheus metrics scraper In-Reply-To: <1294561257.311268.1460422832468.JavaMail.zimbra@redhat.com> References: <1294561257.311268.1460422832468.JavaMail.zimbra@redhat.com> Message-ID: <3422442.CfM6j9WH5c@localhost.localdomain> Why should this be part of the agent? Shouldn't this be rather a standalone utility? I mean it sure can reuse large parts of the code from the agent (talking to inv and metrics, etc), but Prometheus doesn't have much to do with Wildfly, does it? I fear that we're just bloating the agent more and more until it becomes an inseparable "blob" as RHQ agent. It already does Wfly, JMX, platform and now Prometheus. On Monday, April 11, 2016 09:00:32 PM John Mazzitelli wrote: > FYI: I finished a Java-based Prometheus scraper - seems to be working. It is > just code that scrapes any remote Prometheus endpoint given a URL and let's > you easily walk the metric data for further processing (like, if you want > to insert that data into Hawkular Metrics or if you want to dump the data > in XML or JSON format - the CLI does this today, see below). I'll > eventually blog about this, but not until I actually merge it into master. > It's in my branch right now here: > > https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-66-prometheus/ > prometheus-scraper > > This also includes a CLI tool - build the scraper maven module and you'll > get an uber-jar that let's you scrape a remote Prometheus endpoint and dump > the metric data in XML, JSON, or simple form. See the README.md for more. > > Now I just have to figure out how to integrate this into the Hawkular > WildFly Agent :-) _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Lukas Krejci From hrupp at redhat.com Tue Apr 12 07:32:45 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 12 Apr 2016 13:32:45 +0200 Subject: [Hawkular-dev] java-based prometheus metrics scraper In-Reply-To: <3422442.CfM6j9WH5c@localhost.localdomain> References: <1294561257.311268.1460422832468.JavaMail.zimbra@redhat.com> <3422442.CfM6j9WH5c@localhost.localdomain> Message-ID: On 12 Apr 2016, at 12:48, Lukas Krejci wrote: > Why should this be part of the agent? Shouldn't this be rather a standalone > utility? There are two use cases with the WF-agent: - as is inside WF/EAP: here we need to aim for minimal footprint - standalone inside WF-Swarm as proxy to get metrics from 3rd parties and feeding that into Hawkular - like the existing protocol translator (pTrans) does as well (but mostly in a passive, listening mode) From mazz at redhat.com Tue Apr 12 08:16:45 2016 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 12 Apr 2016 08:16:45 -0400 (EDT) Subject: [Hawkular-dev] java-based prometheus metrics scraper In-Reply-To: <3422442.CfM6j9WH5c@localhost.localdomain> References: <1294561257.311268.1460422832468.JavaMail.zimbra@redhat.com> <3422442.CfM6j9WH5c@localhost.localdomain> Message-ID: <1722552700.452310.1460463405890.JavaMail.zimbra@redhat.com> > Why should this be part of the agent? Shouldn't this be rather a standalone utility? The code I have now IS a standalone utility. One maven dep "org.hawkular.agent:prometheus-scraper" - pull that in and anything can become a prometheus scraper and you can use the standalone CLI to produce XML or JSON metric data. It just doesn't do anything "hawkular-y" with the data it collects. You could then write a Java wrapper around it to (for example) insert the data into Hawk-Metrics. But then, we already have such a thing (the wildfly-agent) to get started with that - so we could reuse that, too (which is the current plan). This is mainly to support an OpenShift requirement. Since we'll already have the hawkular wildfly agent integrated in OpenShift (hopefully :-) the idea is to piggyback on that work. Otherwise, we would have to integrate yet another hawkular feed into OpenShift (and from what I can tell, that is NOT a trivial exercise and Matt would probably have a seizure if we asked him to). One thing I'm contemplating is to have this Prometheus support being very independent from the current way the wildfly agent does its thing. So, for example, you could inject the wildfly agent subsystem in a standalone wildfly to use as an "agent core" but only enable the prometheus stuff (you just don't define any DMR or JMX metadata in standalone.xml). Run this in, say, swarm and you've got a standalone prometheus agent - it just happens to reuse code that the wildfly agent uses (it just so happens to run in an EAP container, but it doesn't have any DMR or JMX stuff enabled). It's just a thought. Note that AFAICS, there is no real inventory associated with Prometheus data. So right now I'm thinking this Prometheus stuff is only a "metrics only" feature. There are no resource types or resources associated with Prometheus data. I'm still thinking about this, though. Prometheus adds in a different way of looking at the world compared to how we've been doing things. This is all in flux right now. I'm perfectly willing to admit that we are growing the Hawkular WildFly Agent into something it wasn't originally envisioned to be and that we should do something else. Discussions are welcome. ----- Original Message ----- > Why should this be part of the agent? Shouldn't this be rather a standalone > utility? > > I mean it sure can reuse large parts of the code from the agent (talking to > inv and metrics, etc), but Prometheus doesn't have much to do with Wildfly, > does it? > > I fear that we're just bloating the agent more and more until it becomes an > inseparable "blob" as RHQ agent. It already does Wfly, JMX, platform and now > Prometheus. > > On Monday, April 11, 2016 09:00:32 PM John Mazzitelli wrote: > > FYI: I finished a Java-based Prometheus scraper - seems to be working. It > > is > > just code that scrapes any remote Prometheus endpoint given a URL and let's > > you easily walk the metric data for further processing (like, if you want > > to insert that data into Hawkular Metrics or if you want to dump the data > > in XML or JSON format - the CLI does this today, see below). I'll > > eventually blog about this, but not until I actually merge it into master. > > It's in my branch right now here: > > > > https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-66-prometheus/ > > prometheus-scraper > > > > This also includes a CLI tool - build the scraper maven module and you'll > > get an uber-jar that let's you scrape a remote Prometheus endpoint and dump > > the metric data in XML, JSON, or simple form. See the README.md for more. > > > > Now I just have to figure out how to integrate this into the Hawkular > > WildFly Agent :-) _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -- > > Lukas Krejci > From jsanda at redhat.com Tue Apr 12 15:33:09 2016 From: jsanda at redhat.com (John Sanda) Date: Tue, 12 Apr 2016 15:33:09 -0400 Subject: [Hawkular-dev] Reducing foot print of embedded Cassandra Message-ID: <10FE1C4D-9759-468C-94C7-5AC1582EFFAF@redhat.com> Earlier Heiko asked me if there are any changes we can make to reduce the foot print of the embedded Cassandra instance we use for development and testing. There are several configuration changes that we can make which could be helpful. I did a quick check and it looks like the hawkular server is configured with a 512 MB heap. Some recommended values are based on that assumption. * disable hinted handoff There is no need to run hinted handoff with a single node. Set the hinted_handoff_enabled property in cassandra.yaml to false. * disable key and counter caches This may decrease read performance but should reduce Cassandra?s overall memory usage. Set the key_cache_size_in_mb and counter_cache_size_mb properties in cassandra.yaml to 0. * reduce the concurrent reads/writes This will reduce the number of threads that Cassandra uses. Set the concurrent_reads, concurrent_writes, and concurrent_counter_writes properties in cassandra.yaml to 2. * reduce file cache size This is off-heap memory used for pooling SSTable buffers. For a 512 MB heap it will be 128 MB. Change the file_cache_size_in_mb property to 64. * reduce memory used for memtables This will increase the number of flushes which should be fine for write-heavy workloads. We probably don?t want to make this too small for read-heavy scenarios. Change the memtable_heap_space_in_mb and memtable_offheap_space_in_mb properties to 64. * reduce memory for index summaries This defaults to a little over 25 MB. Change the index_summary_capcaity_in_mb property to 13. * reduce the native transport threads The native transport server will use a max of 128 threads by default. Change the native_transport_max_threads property to 32. * reduce thrift server threads Change the rpc_min_threads property to 8 and pc_max_threads to 512. * tune compaction Set concurrent_compactors to 1. * disable gossip No need to run gossip with a single node. Since Cassandra runs in its own deployment, the easiest way to do this is via JMX using the stopGossiping() method on StorageServiceMBean. I need to do some testing with this to see how it works. It looks like the call has to happen after some initialization has completed. * disable compression Compression is enabled by default and is configured per table. It reduces disk space but increases CPU utilization. ALTER TABLE my_table WITH compression = { ?sstable_compression?: ?? }; From mazz at redhat.com Wed Apr 13 00:03:02 2016 From: mazz at redhat.com (John Mazzitelli) Date: Wed, 13 Apr 2016 00:03:02 -0400 (EDT) Subject: [Hawkular-dev] PoC hawkular wildfly agent and prometheus working In-Reply-To: <1111498019.675356.1460518619281.JavaMail.zimbra@redhat.com> Message-ID: <1279995386.677775.1460520182401.JavaMail.zimbra@redhat.com> I have a PoC working with the agent scraping a Prometheus endpoint and storing the metric data in Hawkular-Metrics. It came together faster than I expected :) This can be used as a standalone feature - in other words, you don't have to run the agent collecting everything (DMR + JMX + Platform + Prometheus data) though you can if you want a "uber-agent" to collect from multiple sources. But you can just have a minimally configured agent collect just Prometheus data. The config in standalone.xml would look like this (this is the entire agent subsystem config): This is just a prototype. Still lots of questions to answer like: 1) What do we want the keys to look like in H-Metrics? How to translate Prometheus keys/labels to H-Metrics? What this PoC does right now is build a huge H-Metrics key that appends the feed ID, the Prometheus metric name and Prometheus labels so it looks like this "b6fb9814-e6b5-435a-b533-411d85867720_prometheus_rule_evaluation_failures_total_rule_type=alerting_" (rule_type=alerting is a label). 2) Prometheus supports summary and histograms - what do we do with those? I ignore them for now. (btw: even histograms isn't supported in Prometheus' Java model API that is released, only supported in local snapshot build form only) 3) Do we want to create labels in H-Metrics to distinguish the Prometheus endpoint data? How? 4) No security whatsoever in here - PoC assumes in-the-clear http access. We'd need to add https, basic auth, ssl/security realms and anything else needed to access any Prometheus endpoints that are secured. I'm sure Open Shift has some things we need to work around to scrape Prometheus endpoints that are running in an OS environment - this is something we need to figure out. 5) Notice there is no metadata in the above config. There is no integration with H-Inventory - no resource types, no resources, no metric types. Prometheus doesn't really have the notion of inventory - at best, each Prometheus endpoint could be a "resource". But this PoC does nothing like that - it is a metrics-only solution (which as I understand it is all we need for the Open Shift requirement anyway). And I'm sure plenty more questions. But, point is, we have a working example of how this can be done. Code is in this branch: https://github.com/hawkular/hawkular-agent/tree/mazz/hwkagent-66-dynamic-prometheus --John Mazz From jsanda at redhat.com Thu Apr 14 09:58:34 2016 From: jsanda at redhat.com (John Sanda) Date: Thu, 14 Apr 2016 09:58:34 -0400 Subject: [Hawkular-dev] Reducing foot print of embedded Cassandra In-Reply-To: <10FE1C4D-9759-468C-94C7-5AC1582EFFAF@redhat.com> References: <10FE1C4D-9759-468C-94C7-5AC1582EFFAF@redhat.com> Message-ID: <2CBC5ED3-2FE3-4299-90B4-23CBA4973EAD@redhat.com> > On Apr 12, 2016, at 3:33 PM, John Sanda wrote: > > Earlier Heiko asked me if there are any changes we can make to reduce the foot print of the embedded Cassandra instance we use for development and testing. There are several configuration changes that we can make which could be helpful. I did a quick check and it looks like the hawkular server is configured with a 512 MB heap. Some recommended values are based on that assumption. > > * disable hinted handoff > There is no need to run hinted handoff with a single node. Set the hinted_handoff_enabled property in cassandra.yaml to false. > > * disable key and counter caches > This may decrease read performance but should reduce Cassandra?s overall memory usage. Set the key_cache_size_in_mb and counter_cache_size_mb properties in cassandra.yaml to 0. > > * reduce the concurrent reads/writes > This will reduce the number of threads that Cassandra uses. Set the concurrent_reads, concurrent_writes, and concurrent_counter_writes properties in cassandra.yaml to 2. > > * reduce file cache size > This is off-heap memory used for pooling SSTable buffers. For a 512 MB heap it will be 128 MB. Change the file_cache_size_in_mb property to 64. > > * reduce memory used for memtables > This will increase the number of flushes which should be fine for write-heavy workloads. We probably don?t want to make this too small for read-heavy scenarios. Change the memtable_heap_space_in_mb and memtable_offheap_space_in_mb properties to 64. > > * reduce memory for index summaries > This defaults to a little over 25 MB. Change the index_summary_capcaity_in_mb property to 13. > > * reduce the native transport threads > The native transport server will use a max of 128 threads by default. Change the native_transport_max_threads property to 32. > > * reduce thrift server threads > Change the rpc_min_threads property to 8 and pc_max_threads to 512. > > * tune compaction > Set concurrent_compactors to 1. > > * disable gossip > No need to run gossip with a single node. Since Cassandra runs in its own deployment, the easiest way to do this is via JMX using the stopGossiping() method on StorageServiceMBean. I need to do some testing with this to see how it works. It looks like the call has to happen after some initialization has completed. > > * disable compression > Compression is enabled by default and is configured per table. It reduces disk space but increases CPU utilization. > > ALTER TABLE my_table WITH compression = { ?sstable_compression?: ?? }; > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev There is something else I want to point out that is not related specifically to the use of embedded Cassandra. Each hawkular component uses its own, separate driver instance. This is a big no, no. We should be using a shared instance per physical cluster. I think Juca may have looked into a resource adapter at one point, but I am not sure the outcome. Might be good to revisit. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160414/cb27176a/attachment.html From ppalaga at redhat.com Fri Apr 15 05:38:49 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Fri, 15 Apr 2016 11:38:49 +0200 Subject: [Hawkular-dev] Ideas about CD with Maven Message-ID: <5710B6A9.90800@redhat.com> Hi all interested in continuous delivery and integration, Contrary to srcdeps, this guy [1] does not try to avoid releases and maven repos, instead, he just makes the release process extremely effective. Some of his ideas can be used to improve the srcdeps plugin. [1] https://axelfontaine.com/blog/dead-burried.html Best, Peter From jpkroehling at redhat.com Fri Apr 15 06:13:49 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 15 Apr 2016 12:13:49 +0200 Subject: [Hawkular-dev] Ideas about CD with Maven In-Reply-To: <5710B6A9.90800@redhat.com> References: <5710B6A9.90800@redhat.com> Message-ID: <5710BEDD.7050205@redhat.com> On 15.04.2016 11:38, Peter Palaga wrote: > Contrary to srcdeps, this guy [1] does not try to avoid releases and > maven repos, instead, he just makes the release process extremely > effective. Some of his ideas can be used to improve the srcdeps plugin. I like his approach, but I believe this means that releases are to be made only on CI servers, right? So, no more manual releasing? - Juca. From hrupp at redhat.com Fri Apr 15 06:27:42 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 15 Apr 2016 12:27:42 +0200 Subject: [Hawkular-dev] Ideas about CD with Maven In-Reply-To: <5710BEDD.7050205@redhat.com> References: <5710B6A9.90800@redhat.com> <5710BEDD.7050205@redhat.com> Message-ID: On 15 Apr 2016, at 12:13, Juraci Paix?o Kr?hling wrote: > I like his approach, but I believe this means that releases are to be > made only on CI servers, right? So, no more manual releasing? Would that be an issue? I understand that this may even be an advantage, as there we can always know a known JDK and other environment? To me this sounds good at first. Where it does not go into is if the modules of the multi-module build are pulled in from different repositories like we do. -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill From jpkroehling at redhat.com Fri Apr 15 07:29:28 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 15 Apr 2016 13:29:28 +0200 Subject: [Hawkular-dev] Ideas about CD with Maven In-Reply-To: References: <5710B6A9.90800@redhat.com> <5710BEDD.7050205@redhat.com> Message-ID: <5710D098.6000403@redhat.com> On 15.04.2016 12:27, Heiko W.Rupp wrote: > On 15 Apr 2016, at 12:13, Juraci Paix?o Kr?hling wrote: > >> I like his approach, but I believe this means that releases are to be >> made only on CI servers, right? So, no more manual releasing? > > Would that be an issue? No, quite the opposite. I'm in favor of building only on stable machines, with reproducible environments. But it does break the current workflow of most our modules. I believe they are (almost) all released locally by the developers themselves. - Juca. From hrupp at redhat.com Fri Apr 15 08:59:25 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 15 Apr 2016 14:59:25 +0200 Subject: [Hawkular-dev] Ideas about CD with Maven In-Reply-To: <5710D098.6000403@redhat.com> References: <5710B6A9.90800@redhat.com> <5710BEDD.7050205@redhat.com> <5710D098.6000403@redhat.com> Message-ID: <9CF457EB-C4C9-4D93-84F1-405C51B72BDF@redhat.com> On 15 Apr 2016, at 13:29, Juraci Paix?o Kr?hling wrote: > But it does break the current workflow of most our modules. I believe > they are (almost) all released locally by the developers themselves. With the src-dep approach we had discussed, the need for those releases would have gone down anyway. I think the key point here is that we know where stuff comes from and not that there is something with a foo-1.2.3-Final stamp in some nexus repo. So just putting a tag into the git repo may be enough. -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill From jpkroehling at redhat.com Fri Apr 15 09:14:04 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 15 Apr 2016 15:14:04 +0200 Subject: [Hawkular-dev] Do not depend on Keycloak anymore In-Reply-To: <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> Message-ID: <5710E91C.7080105@redhat.com> Bringing this discussion to the list. Summary so far: === If all those requirements got dropped, specially "multiple organizations" and "multi tenancy", then there's really no point in having Accounts. Without the UI/self-registration/SSO, I doubt we need to depend on Keycloak. I added SSO to this list because if there will be no user interaction on Hawkular, there's no "single sign on" required there. It's still not clear to me, though, if we need to care about authorization. Are we getting calls only from "trusted" clients? Also, should we use the same user base as MiQ, or should we have a "system account" only? Having a system account might make more sense here, and we won't need to integrate with their user database. With this system account, we would act like a database: any user with valid credentials would be allowed to read/write data. We would trust the client if the client said "store this data for this tenant". Instead of tokens, we could have just system accounts. Revoking a token would mean just removing this system account. === > On 15 Apr 2016, at 14:38, John Mazzitelli wrote: > > What about agents/feeds? They aren't talking to CFME ; they are > talking directly to the Hawkular Server (or Hawkular-Metrics if in > METRICS-ONLY mode, such as in Open Shift). Is there not going to be > any Basic Auth headers anymore? If there will still need to be > authentication, then it must be the JAAS credentials and not the > Accounts tokens, I suppose. If we have system accounts, then agents would use a system account, like they would use a token. Admins are free to create one system account per agent (key/secret), or use one shared account to all agents (jdoe/password). For all "we" (Hawkular backend) care, it's all done by JAAS. In fact, if all goes well, there will be no need to change anything for the agents. And I'm considering that the auth is done by Basic auth. If we *need* to have other authentication mechanisms, we need to deviate from JAAS. On 15.04.2016 14:43, Heiko W.Rupp wrote: > Yes, that *may* require a change. Or not if we e.g. have > - accounts-keycloak > - accounts-jaas > where the latter does the mapping as a jaas provider/plugin. > But yes we need to secure the agent/server comm. And we > need to apply SSL for the product (isn't that part of http/2 anyway). So far, I don't see the need in having Keycloak, or Accounts. Securing the agent/server communication is not relevant to the discussion, as it's independent from the authentication mechanism being used. - Juca. From mazz at redhat.com Fri Apr 15 11:10:31 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 15 Apr 2016 11:10:31 -0400 (EDT) Subject: [Hawkular-dev] agents In-Reply-To: <36560198.1910987.1460733018474.JavaMail.zimbra@redhat.com> Message-ID: <1398780519.1910996.1460733031958.JavaMail.zimbra@redhat.com> [sending to dev list] If you wrote an agent/feed (even if its something in your own little corner of the world in your github repo), let me know where it is, what it is, what it can do now, what you hoped it could do in the future. Don't assume I know anything about it, because I probably don't. I just want to get a "lay of the land" - what is out there, what people are thinking about wrt agents, what we have today that we can build on. That kind of thing. Thanks, John Mazz From jshaughn at redhat.com Fri Apr 15 13:33:24 2016 From: jshaughn at redhat.com (jshaughn at redhat.com) Date: Fri, 15 Apr 2016 17:33:24 +0000 Subject: [Hawkular-dev] Invitation: Watercooler: MIQ provider show + tell @ Tue Apr 19, 2016 10am - 11am (Hawkular) Message-ID: <001a113a9afc762e9305308968c2@google.com> You have been invited to the following event. Title: Watercooler: MIQ provider show + tell This is an optional gathering for MIQ Hawkular provider devs to show what they have done (merged or in the pipeline) and to informally discuss what we have and where we should go. So, minimally I will give a brief overview and demo of the event stuff we have so far. Hopefully other will do a brief demo or presentation as well. All are welcome, no one is required. When: Tue Apr 19, 2016 10am - 11am Eastern Time Where: https://redhat.bluejeans.com/u/jshaughn/ Calendar: Hawkular Who: * jshaughn at redhat.com - creator * jbmc-team * hawkular-dev at lists.jboss.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=dTJjdm03cWJqdDk2cjluMmo5NmY2YTA3cGcgaGF3a3VsYXItZGV2QGxpc3RzLmpib3NzLm9yZw&tok=NjMjcmVkaGF0LmNvbV9mbWlnMm9zdTY5a21hNDdqcmRjMnZlbjRtb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29tZThhMWUxOWJhNTUwZGE2N2Y3MzlkMmVmM2NjOTgxNDgxYmY0MjljOQ&ctz=America/New_York&hl=en Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account hawkular-dev at lists.jboss.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160415/0b03f417/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 1585 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160415/0b03f417/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 1620 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160415/0b03f417/attachment-0003.bin From jpkroehling at redhat.com Mon Apr 18 10:57:08 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 18 Apr 2016 16:57:08 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: <5710E91C.7080105@redhat.com> References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> Message-ID: <5714F5C4.9090308@redhat.com> On 15.04.2016 15:14, Juraci Paix?o Kr?hling wrote: > On 15.04.2016 14:43, Heiko W.Rupp wrote: >> Yes, that *may* require a change. Or not if we e.g. have >> - accounts-keycloak >> - accounts-jaas >> where the latter does the mapping as a jaas provider/plugin. I'm still not convinced why we would need two modules. If we assume that Hawkular is similar to a database, in the sense that end users have no access to it, then there would be no need for any advanced feature from Keycloak. Plain JAAS would suffice. - Juca. From jpkroehling at redhat.com Mon Apr 18 11:14:23 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 18 Apr 2016 17:14:23 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: <5714F5C4.9090308@redhat.com> References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> Message-ID: <5714F9CF.5000303@redhat.com> On 18.04.2016 16:57, Juraci Paix?o Kr?hling wrote: > On 15.04.2016 15:14, Juraci Paix?o Kr?hling wrote: >> On 15.04.2016 14:43, Heiko W.Rupp wrote: >>> Yes, that *may* require a change. Or not if we e.g. have >>> - accounts-keycloak >>> - accounts-jaas >>> where the latter does the mapping as a jaas provider/plugin. > > I'm still not convinced why we would need two modules. If we assume that > Hawkular is similar to a database, in the sense that end users have no > access to it, then there would be no need for any advanced feature from > Keycloak. Plain JAAS would suffice. > Sent without finishing :) Another aspect that comes with the removal of the dependency on Keycloak is surrounding tenancy. We don't have the same requirements as before, and in the case described above where Hawkular could be seen as a "database", the tenancy would/should be managed on the user-facing application. This means that we'd have a breaking change for components like Inventory and Metrics, where the tenant is currently the same as the persona, which in turn is derived from the logged in user (or organization selected on the account switcher). Not having a tenancy model anymore means that all users are of the same tenant, so, components that care about tenancy should be changed. Note that there are two ways of interpreting "tenancy" here: the first is about how data is stored, and the second is how data is accessed. Previously, a tenant would write and read only its own data. Now, tenant is just another piece of the data, so, components would not actively check if the data belongs to the current user. We trust that the user-facing application is performing these checks. - Juca. From tsegismo at redhat.com Mon Apr 18 12:43:59 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 18 Apr 2016 18:43:59 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: <5714F9CF.5000303@redhat.com> References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> <5714F9CF.5000303@redhat.com> Message-ID: 2016-04-18 17:14 GMT+02:00 Juraci Paix?o Kr?hling : > On 18.04.2016 16:57, Juraci Paix?o Kr?hling wrote: > > On 15.04.2016 15:14, Juraci Paix?o Kr?hling wrote: > >> On 15.04.2016 14:43, Heiko W.Rupp wrote: > >>> Yes, that *may* require a change. Or not if we e.g. have > >>> - accounts-keycloak > >>> - accounts-jaas > >>> where the latter does the mapping as a jaas provider/plugin. > > > > I'm still not convinced why we would need two modules. If we assume that > > Hawkular is similar to a database, in the sense that end users have no > > access to it, then there would be no need for any advanced feature from > > Keycloak. Plain JAAS would suffice. > > > > Sent without finishing :) > > Another aspect that comes with the removal of the dependency on Keycloak > is surrounding tenancy. We don't have the same requirements as before, > and in the case described above where Hawkular could be seen as a > "database", the tenancy would/should be managed on the user-facing > application. > > This means that we'd have a breaking change for components like > Inventory and Metrics, where the tenant is currently the same as the > persona, which in turn is derived from the logged in user (or > organization selected on the account switcher). Not having a tenancy > model anymore means that all users are of the same tenant, so, > components that care about tenancy should be changed. > > Note that there are two ways of interpreting "tenancy" here: the first > is about how data is stored, and the second is how data is accessed. > Previously, a tenant would write and read only its own data. Now, tenant > is just another piece of the data, so, components would not actively > check if the data belongs to the current user. We trust that the > user-facing application is performing these checks. > If we don't check that the authenticated user can only access the data he is entitled to read, it's not good. It's protecting your web application with client side checks only. > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160418/bb97403d/attachment.html From jpkroehling at redhat.com Mon Apr 18 13:27:35 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 18 Apr 2016 19:27:35 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> <5714F9CF.5000303@redhat.com> Message-ID: <57151907.2080300@redhat.com> On 18.04.2016 18:43, Thomas Segismont wrote: > If we don't check that the authenticated user can only access the data > he is entitled to read, it's not good. It's protecting your web > application with client side checks only. Here's the scenario as I understand it: jdoe -> Client 1 -> admin1 -> hawkular jsmith -> Client 1 -> admin1 -> hawkular jsmith -> Client 2 -> admin2 -> hawkular On this scheme, an user "jdoe" logs in into "Client 1". This application uses "admin1" to talk to our Hawkular backend. Our backends have no idea about tenants, as it will be managed on "client" applications (ie: MiQ). I think we could/should have a way to isolate data from "admin1"/"admin2", but those users are not "tenants" in the same sense as we have today, are they? As I understand it, "jdoe" and "jsmith" might belong to the same tenant, but this information is something that is stored inside MiQ, so, not handled by our backend. Our provider (Ruby gem), however, will have access to this information. - Juca. From hrupp at redhat.com Mon Apr 18 14:06:02 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 18 Apr 2016 20:06:02 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: <5714F5C4.9090308@redhat.com> References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> Message-ID: On 18 Apr 2016, at 16:57, Juraci Paix?o Kr?hling wrote: > On 15.04.2016 15:14, Juraci Paix?o Kr?hling wrote: >> On 15.04.2016 14:43, Heiko W.Rupp wrote: >>> Yes, that *may* require a change. Or not if we e.g. have >>> - accounts-keycloak >>> - accounts-jaas >>> where the latter does the mapping as a jaas provider/plugin. > > I'm still not convinced why we would need two modules. If we assume that > Hawkular is similar to a database, in the sense that end users have no > access to it, then there would be no need for any advanced feature from > Keycloak. Plain JAAS would suffice. We have 2 different use cases: - Hawkular as we know it. Including KC, Multi-tenancy and the full enchilada - Hawkular that works like a database for users like ManageIQ, that only need this one technical user to talk to Hawkular. From hrupp at redhat.com Mon Apr 18 14:10:55 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 18 Apr 2016 20:10:55 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: <5714F9CF.5000303@redhat.com> References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> <5714F9CF.5000303@redhat.com> Message-ID: On 18 Apr 2016, at 17:14, Juraci Paix?o Kr?hling wrote: > Another aspect that comes with the removal of the dependency on > Keycloak > is surrounding tenancy. We don't have the same requirements as before, > and in the case described above where Hawkular could be seen as a > "database", the tenancy would/should be managed on the user-facing > application. Not sure I completely understand. When e.g. RHQ uses Postgres, it always logs in as "rhqadmin/rhqadmin". There is no forwarding of the logged in RHQ-user to Postgres, but Postgres is always accessed by the technical user. > This means that we'd have a breaking change for components like > Inventory and Metrics, where the tenant is currently the same as the > persona, which in turn is derived from the logged in user (or > organization selected on the account switcher). Not having a tenancy > model anymore means that all users are of the same tenant, so, > components that care about tenancy should be changed. Yes. And this is fine, as there is only one user, which then belongs to a "default tenant". -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill From hrupp at redhat.com Mon Apr 18 14:30:29 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 18 Apr 2016 20:30:29 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> <5714F9CF.5000303@redhat.com> Message-ID: On 18 Apr 2016, at 18:43, Thomas Segismont wrote: > If we don't check that the authenticated user can only access the data he > is entitled to read, it's not good. It's protecting your web application > with client side checks only. In the use case where you only have one user, this probably does not matter too much. From jpkroehling at redhat.com Mon Apr 18 16:28:57 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 18 Apr 2016 22:28:57 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> Message-ID: <57154389.60503@redhat.com> On 18.04.2016 20:06, Heiko W.Rupp wrote: > On 18 Apr 2016, at 16:57, Juraci Paix?o Kr?hling wrote: >> I'm still not convinced why we would need two modules. If we assume that >> Hawkular is similar to a database, in the sense that end users have no >> access to it, then there would be no need for any advanced feature from >> Keycloak. Plain JAAS would suffice. > > We have 2 different use cases: > > - Hawkular as we know it. Including KC, Multi-tenancy and the full enchilada I still don't get why we need this. What's the concrete use case behind this requirement? Sure, it's nice to have it all and then some, but there has to be a reason for having this feature, right? :) KC would be "free" to have, so, I don't mind it much, but multi tenancy is done by highly customized code in our side, including nested organization management and custom permission API. If we don't have any concrete requirements around this, I'd rather remove, so that we can have better, cleaner and more elegant solutions on other concrete requirements, like Pavol's Data Mining. > - Hawkular that works like a database for users like ManageIQ, that only need > this one technical user to talk to Hawkular. This one I get. - Juca. From jpkroehling at redhat.com Mon Apr 18 16:36:50 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 18 Apr 2016 22:36:50 +0200 Subject: [Hawkular-dev] Tenancy model (was Re: Do not depend on Keycloak anymore) In-Reply-To: References: <5710C5B4.8030203@redhat.com> <5710DD2F.4070309@redhat.com> <2025202470.1835416.1460723914886.JavaMail.zimbra@redhat.com> <7872C1CC-E394-4515-A39A-30EEA10CFDD6@redhat.com> <5710E91C.7080105@redhat.com> <5714F5C4.9090308@redhat.com> <5714F9CF.5000303@redhat.com> Message-ID: <57154562.20008@redhat.com> On 18.04.2016 20:10, Heiko W.Rupp wrote: > On 18 Apr 2016, at 17:14, Juraci Paix?o Kr?hling wrote: > >> Another aspect that comes with the removal of the dependency on >> Keycloak >> is surrounding tenancy. We don't have the same requirements as before, >> and in the case described above where Hawkular could be seen as a >> "database", the tenancy would/should be managed on the user-facing >> application. > > Not sure I completely understand. When e.g. RHQ > uses Postgres, it always logs in as "rhqadmin/rhqadmin". > There is no forwarding of the logged in RHQ-user > to Postgres, but Postgres is always accessed by the > technical user. I exemplified in another email. Something like this: jdoe -> Client 1 -> admin1 -> hawkular jsmith -> Client 1 -> admin1 -> hawkular jsmith -> Client 2 -> admin2 -> hawkular Previously, our model was to have "jdoe" talking directly to Hawkular, and Hawkular Accounts would determine the tenant. Our other Hawkular components would just need to @Inject a Persona and that persona would be the tenant. We have this technical user now, so, we don't necessarily have the tenant information based on the trusted user data (signed JWT from KC + data we store ourselves in Cassandra). Accounts cannot inject an appropriate Persona anymore. We would then need to trust the client (RHQ in your example) to handle tenancy information, restricting access to data from tenant A from being seen by tenant B. Or treat tenant data as just another piece of information, without actively trying to do access control on our side. - Juca. From hrupp at redhat.com Tue Apr 19 09:23:48 2016 From: hrupp at redhat.com (hrupp at redhat.com) Date: Tue, 19 Apr 2016 13:23:48 +0000 Subject: [Hawkular-dev] Termin gestrichen: Hawkular-Team Update - Di 19. Apr. 2016 3:30PM - 4PM (hrupp@redhat.com) Message-ID: <94eb2c095f6426808d0530d66312@google.com> Dieser Termin wurde gestrichen und aus Ihrem Kalender entfernt. Titel: Hawkular-Team Update This is a all-Hawkular team meeting to give updates where we are and so on. This is *open to the public*. Location: on bluejeans: https://redhat.bluejeans.com/hrupp/ or alternatively teleconference Reservationless+ , passcode 204 2160 481 You can find Dial-In telephone numbers here: https://www.intercallonline.com/listNumbersByCode.action?confCode=2042160481 RedHat internal short dial numbers are 16666 and 15555 (and probably others, depends on your location) Wann: Di 19. Apr. 2016 3:30PM - 4PM Berlin Wo: pc 204 2160 481 Kalender: hrupp at redhat.com Wer * Heiko Rupp - Organisator * Jiri Kremser * theute at redhat.com * abonas at redhat.com * jcosta at redhat.com * Lucas Ponce * Simeon Pinder * Gabriel Cardoso * snegrea at redhat.com * John Sanda * John Mazzitelli * lkrejci at redhat.com * Jay Shaughnessy * gbrown at redhat.com * mwringe at redhat.com * miburman at redhat.com * Peter Palaga * Mike Thompson * amendonc at redhat.com * Thomas Segismont * hawkular-dev at lists.jboss.org Einladung von Google Kalender: https://www.google.com/calendar/ Sie erhalten diese E-Mail unter hawkular-dev at lists.jboss.org, da Sie ein Gast bei diesem Termin sind. Lehnen Sie diesen Termin ab, um keine weiteren Informationen zu diesem Termin zu erhalten. Sie k?nnen auch unter https://www.google.com/calendar/ ein Google-Konto erstellen und Ihre Benachrichtigungseinstellungen f?r Ihren gesamten Kalender steuern. Wenn Sie diese Einladung weiterleiten, kann jeder Empf?nger Ihre Antwort auf die Einladung ?ndern. Weitere Informationen finden Sie unter https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160419/2a18f4af/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 4270 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160419/2a18f4af/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 4359 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160419/2a18f4af/attachment-0003.bin From mazz at redhat.com Tue Apr 19 13:58:48 2016 From: mazz at redhat.com (John Mazzitelli) Date: Tue, 19 Apr 2016 13:58:48 -0400 (EDT) Subject: [Hawkular-dev] agent 0.17.1.Final released - prometheus support In-Reply-To: <2102347616.3305137.1461088654457.JavaMail.zimbra@redhat.com> Message-ID: <1350450296.3305279.1461088728382.JavaMail.zimbra@redhat.com> Hawkular WildFly Agent 0.17.1.Final has been released - this is the first release with Prometheus support. The agent can now be told to scrape Prometheus endpoints for metric data (so it now supports DMR (WildFly/EAP) monitoring, Platform monitoring, Jolokia/JMX monitoring, and now Prometheus endpoint monitoring). See my blog on the details here: http://management-platform.blogspot.com/2016/04/prometheus-metric-endpoint-parser-for.html From lkrejci at redhat.com Tue Apr 19 14:51:04 2016 From: lkrejci at redhat.com (lkrejci at redhat.com) Date: Tue, 19 Apr 2016 18:51:04 +0000 Subject: [Hawkular-dev] =?iso-8859-1?q?Pozv=E1nka=3A_Discuss_Inventory_Syn?= =?iso-8859-1?q?c_-_St_2016/04/20_3=3A30PM_-_4=3A30PM_=28Hawkular?= =?iso-8859-1?q?=29?= Message-ID: <94eb2c07f7d69772eb0530daf5ae@google.com> Byli jste pozv?ni na n?sleduj?c? ud?lost. N?zev: Discuss Inventory Sync With the first version of the inventory sync (or rather "dump" as it exists at the moment) in inventory, I'd like to invite you to discuss its features and future directions so that we make it as useful as possible for the feed writers. https://bluejeans.com/8169978803 Kdy: St 2016/04/20 3:30PM - 4:30PM St?edoevropsk? ?as - Praha Kde: https://bluejeans.com/8169978803 Kalend??: Hawkular Kdo: * lkrejci at redhat.com- autor * jmazzite at redhat.com * jshaughn at redhat.com * hrupp at redhat.com * hawkular-dev at lists.jboss.org Podrobnosti o ud?losti: https://www.google.com/calendar/event?action=VIEW&eid=cTExdWdzbXR0Z3Y2M2xoMmdwbW05bzZyMGMgaGF3a3VsYXItZGV2QGxpc3RzLmpib3NzLm9yZw&tok=NjMjcmVkaGF0LmNvbV9mbWlnMm9zdTY5a21hNDdqcmRjMnZlbjRtb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29tNDk0ZDJmNjE5NjFhZWE3OWRkOThmMDExY2NkMDZkZTU1ZjQ3Zjk4OA&ctz=Europe/Prague&hl=cs Pozv?nka z Kalend??e Google: https://www.google.com/calendar/ Tento e-mail na ??et hawkular-dev at lists.jboss.org jste obdr?eli z toho d?vodu, ?e jste ??astn?kem t?to ud?losti. Chcete-li se odhl?sit z odb?ru ozn?men? o t?to ud?losti, odm?tn?te ud?lost. P??padn? si m??ete na adrese https://www.google.com/calendar/ zaregistrovat ??et Google a ??dit nastaven? ozn?men? pro cel? sv?j kalend??. P?eposl?n?m t?to pozv?nky byste mohli komukoli z p??jemc? umo?nit, aby zm?nil va?i odpov?? na toto pozv?n?. Dal?? informace najdete na https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160419/e2537869/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 1771 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160419/e2537869/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 1809 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160419/e2537869/attachment-0001.bin From tsegismo at redhat.com Wed Apr 20 04:53:18 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 20 Apr 2016 10:53:18 +0200 Subject: [Hawkular-dev] agent 0.17.1.Final released - prometheus support In-Reply-To: <1350450296.3305279.1461088728382.JavaMail.zimbra@redhat.com> References: <2102347616.3305137.1461088654457.JavaMail.zimbra@redhat.com> <1350450296.3305279.1461088728382.JavaMail.zimbra@redhat.com> Message-ID: Congrats mazz! 2016-04-19 19:58 GMT+02:00 John Mazzitelli : > Hawkular WildFly Agent 0.17.1.Final has been released - this is the first > release with Prometheus support. The agent can now be told to scrape > Prometheus endpoints for metric data (so it now supports DMR (WildFly/EAP) > monitoring, Platform monitoring, Jolokia/JMX monitoring, and now Prometheus > endpoint monitoring). > > See my blog on the details here: > > > http://management-platform.blogspot.com/2016/04/prometheus-metric-endpoint-parser-for.html > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160420/c3dc9bde/attachment.html From lkrejci at redhat.com Wed Apr 20 12:36:30 2016 From: lkrejci at redhat.com (Lukas Krejci) Date: Wed, 20 Apr 2016 18:36:30 +0200 Subject: [Hawkular-dev] =?utf-8?q?Pozv=C3=A1nka=3A_Discuss_Inventory_Sync_?= =?utf-8?q?-_St_2016/04/20_3=3A30PM_-_4=3A30PM_=28Hawkular=29?= In-Reply-To: <94eb2c07f7d69772eb0530daf5ae@google.com> References: <94eb2c07f7d69772eb0530daf5ae@google.com> Message-ID: <3992839.tk5pFGMcNB@localhost.localdomain> The discussion we had revolved mainly about what does sync (or lack thereof) mean in the bigger picture of Hawkular. First, I will summarize the discussion and the main questions we raised (please fill in the gaps if you find I forgot about something) and then, in a subsequent email, I'll try to provide my answers to those questions to start off further discussion. == Should we delete stuff from inventory when it's not present in the sync report? On one hand, one could argue that yes, we should delete whatever the feed doesn't see, because, well, it isn't there and therefore inventory should not keep it, because then it would not represent what the true state of affairs is (this brings forth the missing versioning in inventory - if we had that, we would keep the historical record of the resource being there). On the other hand, users should "see something" when a resource they expect to exist suddenly disappears. Yet another stance on this is that if stuff is kept in inventory even after it has stopped being reported by feeds, it should not remain there forever (why is the app that I undeployed a month ago still showing up in inventory?). == What does time since last sync mean? Say we have an embedded Widfly agent that we haven't heard from for a year. Does it mean that there's just nothing new coming from it or it just means that the server the agent was running in no longer runs? And more importantly what should we do about it? The idea was to somehow store the time the last sync occurred and base further decisions on that. What this timestamp would mean would depend on what we're looking at. For a feed, this would mean the last time it synced and hence was running correctly. For a resource managed by a feed, this would mean the last time the feed saw that resource (provided we don't delete the resources during sync immediately upon not seeing them). For an embedded feed, not seeing it running either means that the server it's embedded in is not running or it also can mean that the feed was just "removed" from the server it was running in. For a standalone feed managing remote servers/processes, this would mean the feed is not running, but would not tell us anything about the servers it manages. == How do we determine what's "the last time we've seen something live?" This can be based on the last sync time or the last time we've seen some metric associated with a resource or the last time we've seen a (to be implemented) availability metric associated with a resource. == Conclusion The three questions raised above are intertwined and the answers to them will influence each other. Overall these questions revolve around determining staleness/age of the inventory data and how to react on it. On Tuesday, April 19, 2016 06:51:04 PM lkrejci at redhat.com wrote: > Byli jste pozv?ni na n?sleduj?c? ud?lost. > > N?zev: Discuss Inventory Sync > With the first version of the inventory sync (or rather "dump" as it exists > at the moment) in inventory, I'd like to invite you to discuss its > features and future directions so that we make it as useful as possible for > the feed writers. > > https://bluejeans.com/8169978803 > Kdy: St 2016/04/20 3:30PM - 4:30PM St?edoevropsk? ?as - Praha > Kde: https://bluejeans.com/8169978803 > Kalend??: Hawkular > Kdo: > * lkrejci at redhat.com- autor > * jmazzite at redhat.com > * jshaughn at redhat.com > * hrupp at redhat.com > * hawkular-dev at lists.jboss.org > > Podrobnosti o ud?losti: > https://www.google.com/calendar/event?action=VIEW&eid=cTExdWdzbXR0Z3Y2M2xoMm > dwbW05bzZyMGMgaGF3a3VsYXItZGV2QGxpc3RzLmpib3NzLm9yZw&tok=NjMjcmVkaGF0LmNvbV9 > mbWlnMm9zdTY5a21hNDdqcmRjMnZlbjRtb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29tNDk0ZDJ > mNjE5NjFhZWE3OWRkOThmMDExY2NkMDZkZTU1ZjQ3Zjk4OA&ctz=Europe/Prague&hl=cs > Pozv?nka z Kalend??e Google: https://www.google.com/calendar/ > > Tento e-mail na ??et hawkular-dev at lists.jboss.org jste obdr?eli z toho > d?vodu, ?e jste ??astn?kem t?to ud?losti. > > Chcete-li se odhl?sit z odb?ru ozn?men? o t?to ud?losti, odm?tn?te ud?lost. > P??padn? si m??ete na adrese https://www.google.com/calendar/ > zaregistrovat ??et Google a ??dit nastaven? ozn?men? pro cel? sv?j > kalend??. > > P?eposl?n?m t?to pozv?nky byste mohli komukoli z p??jemc? umo?nit, aby > zm?nil va?i odpov?? na toto pozv?n?. Dal?? informace najdete na > https://support.google.com/calendar/answer/37135#forwarding -- Lukas Krejci From ploffay at redhat.com Thu Apr 21 02:30:10 2016 From: ploffay at redhat.com (Pavol Loffay) Date: Thu, 21 Apr 2016 08:30:10 +0200 Subject: [Hawkular-dev] Hawkular Data Mining 0.1.0.Final Release Message-ID: Hello Everyone, I am happy to announce first release 0.1.0.Final of Hawkular Data Mining. Release contains several time series models and utility classes for time series modelling: *Time series models* - Simple exponential smoothing - Double exponential smoothing (Holt's linear trend) - Seasonal triple exponential smoothing (Holt Winters) - Simple moving average (Weighted moving average) - AutomaticForecaster - which automatically selects the best model - All these models are estimated using maximum likelihood estimation and available for online learning Time series manipulation & Statistics - Augmented Dickey-Fuller test - Autocorrelation function (ACF) - Time series decomposition - Time series lagging - Time series differencing - Automatic period identification If you want to try it with Hawkular: https://goo.gl/R93k02 1. clone https://github.com/hawkular/hawkular/tree/datamining - hawkular/hawkular branch datamining 2. build with -Pdev && run 3. enable predictions preferably for every metric https://github.com/hawkular/hawkular-datamining and wait for some time (see below) 4. go to the Explorer tab, choose metric and increase forecasting horizon Data Mining is currently not able to query historical metric data from metrics due to authentication issues. However it still receives metrics from bus so it is important to enable predictions as soon as possible and wait for some time to collect data (or increase collection interval). This will be fixed in the next versions. Thanks to Jiri Kremser and UI team for helping with predictive charts. Useful links: - Docs - REST API Regards Pavol -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160421/a1c6805d/attachment.html From hrupp at redhat.com Thu Apr 21 03:08:18 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Thu, 21 Apr 2016 09:08:18 +0200 Subject: [Hawkular-dev] Hawkular Data Mining 0.1.0.Final Release In-Reply-To: References: Message-ID: Hey Pavol, this is great - congratulations! On 21 Apr 2016, at 8:30, Pavol Loffay wrote: > Hello Everyone, > > I am happy to announce first release 0.1.0.Final of Hawkular Data > Mining. > Release contains several time series models and utility classes for > time > series modelling: > > *Time series models* > > - Simple exponential smoothing > - Double exponential smoothing (Holt's linear trend) > - Seasonal triple exponential smoothing (Holt Winters) > - Simple moving average (Weighted moving average) > - AutomaticForecaster - which automatically selects the best model > - All these models are estimated using maximum likelihood > estimation and > available for online learning > > Time series manipulation & Statistics > > - Augmented Dickey-Fuller test > - Autocorrelation function (ACF) > - Time series decomposition > - Time series lagging > - Time series differencing > - Automatic period identification > > > If you want to try it with Hawkular: https://goo.gl/R93k02 > > 1. clone https://github.com/hawkular/hawkular/tree/datamining - > hawkular/hawkular branch datamining > 2. build with -Pdev && run > 3. enable predictions preferably for every metric > https://github.com/hawkular/hawkular-datamining and wait for some > time > (see below) > 4. go to the Explorer tab, choose metric and increase forecasting > horizon > > Data Mining is currently not able to query historical metric data from > metrics due to authentication issues. However it still receives > metrics > from bus so it is important to enable predictions as soon as possible > and > wait for some time to collect data (or increase collection interval). > This > will be fixed in the next versions. > > Thanks to Jiri Kremser and UI team for helping with predictive charts. > > Useful links: > > - Docs > > - REST API > > > Regards > Pavol > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill From tsegismo at redhat.com Thu Apr 21 04:02:33 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Thu, 21 Apr 2016 10:02:33 +0200 Subject: [Hawkular-dev] Hawkular Data Mining 0.1.0.Final Release In-Reply-To: References: Message-ID: Congrats Pavol! 2016-04-21 8:30 GMT+02:00 Pavol Loffay : > Hello Everyone, > > I am happy to announce first release 0.1.0.Final of Hawkular Data Mining. > Release contains several time series models and utility classes for time > series modelling: > > *Time series models* > > - Simple exponential smoothing > - Double exponential smoothing (Holt's linear trend) > - Seasonal triple exponential smoothing (Holt Winters) > - Simple moving average (Weighted moving average) > - AutomaticForecaster - which automatically selects the best model > - All these models are estimated using maximum likelihood estimation > and available for online learning > > Time series manipulation & Statistics > > - Augmented Dickey-Fuller test > - Autocorrelation function (ACF) > - Time series decomposition > - Time series lagging > - Time series differencing > - Automatic period identification > > > If you want to try it with Hawkular: https://goo.gl/R93k02 > > 1. clone https://github.com/hawkular/hawkular/tree/datamining - > hawkular/hawkular branch datamining > 2. build with -Pdev && run > 3. enable predictions preferably for every metric > https://github.com/hawkular/hawkular-datamining and wait for some time > (see below) > 4. go to the Explorer tab, choose metric and increase forecasting > horizon > > Data Mining is currently not able to query historical metric data from > metrics due to authentication issues. However it still receives metrics > from bus so it is important to enable predictions as soon as possible and > wait for some time to collect data (or increase collection interval). This > will be fixed in the next versions. > > Thanks to Jiri Kremser and UI team for helping with predictive charts. > > Useful links: > > - Docs > - REST API > > > Regards > Pavol > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160421/de4131b1/attachment.html From hrupp at redhat.com Thu Apr 21 05:54:25 2016 From: hrupp at redhat.com (hrupp at redhat.com) Date: Thu, 21 Apr 2016 09:54:25 +0000 Subject: [Hawkular-dev] Termin gestrichen: Hawkular-Team Update - Di 26. Apr. 2016 3:30PM - 4PM (hrupp@redhat.com) Message-ID: <047d7bacbf8c0b92a20530fbb2fe@google.com> Dieser Termin wurde gestrichen und aus Ihrem Kalender entfernt. Titel: Hawkular-Team Update This is a all-Hawkular team meeting to give updates where we are and so on. This is *open to the public*. Location: on bluejeans: https://redhat.bluejeans.com/hrupp/ or alternatively teleconference Reservationless+ , passcode 204 2160 481 You can find Dial-In telephone numbers here: https://www.intercallonline.com/listNumbersByCode.action?confCode=2042160481 RedHat internal short dial numbers are 16666 and 15555 (and probably others, depends on your location) Wann: Di 26. Apr. 2016 3:30PM - 4PM Berlin Wo: pc 204 2160 481 Kalender: hrupp at redhat.com Wer * Heiko Rupp - Organisator * Mike Thompson * Lucas Ponce * jcosta at redhat.com * amendonc at redhat.com * Jay Shaughnessy * theute at redhat.com * John Sanda * John Mazzitelli * mwringe at redhat.com * abonas at redhat.com * Peter Palaga * gbrown at redhat.com * Gabriel Cardoso * lkrejci at redhat.com * Jiri Kremser * Thomas Segismont * hawkular-dev at lists.jboss.org * snegrea at redhat.com * Simeon Pinder * miburman at redhat.com Einladung von Google Kalender: https://www.google.com/calendar/ Sie erhalten diese E-Mail unter hawkular-dev at lists.jboss.org, da Sie ein Gast bei diesem Termin sind. Lehnen Sie diesen Termin ab, um keine weiteren Informationen zu diesem Termin zu erhalten. Sie k?nnen auch unter https://www.google.com/calendar/ ein Google-Konto erstellen und Ihre Benachrichtigungseinstellungen f?r Ihren gesamten Kalender steuern. Wenn Sie diese Einladung weiterleiten, kann jeder Empf?nger Ihre Antwort auf die Einladung ?ndern. Weitere Informationen finden Sie unter https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160421/4a39b17e/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 4270 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160421/4a39b17e/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 4359 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160421/4a39b17e/attachment-0003.bin From ploffay at redhat.com Thu Apr 21 08:08:08 2016 From: ploffay at redhat.com (Pavol Loffay) Date: Thu, 21 Apr 2016 14:08:08 +0200 Subject: [Hawkular-dev] Hawkular Data Mining 0.1.0.Final Release In-Reply-To: References: Message-ID: Blog post is available here http://www.hawkular.org/blog/2016/04/21/datamining-first-release.html Pavol On Thu, Apr 21, 2016 at 10:02 AM, Thomas Segismont wrote: > Congrats Pavol! > > 2016-04-21 8:30 GMT+02:00 Pavol Loffay : > >> Hello Everyone, >> >> I am happy to announce first release 0.1.0.Final of Hawkular Data Mining. >> Release contains several time series models and utility classes for time >> series modelling: >> >> *Time series models* >> >> - Simple exponential smoothing >> - Double exponential smoothing (Holt's linear trend) >> - Seasonal triple exponential smoothing (Holt Winters) >> - Simple moving average (Weighted moving average) >> - AutomaticForecaster - which automatically selects the best model >> - All these models are estimated using maximum likelihood estimation >> and available for online learning >> >> Time series manipulation & Statistics >> >> - Augmented Dickey-Fuller test >> - Autocorrelation function (ACF) >> - Time series decomposition >> - Time series lagging >> - Time series differencing >> - Automatic period identification >> >> >> If you want to try it with Hawkular: https://goo.gl/R93k02 >> >> 1. clone https://github.com/hawkular/hawkular/tree/datamining - >> hawkular/hawkular branch datamining >> 2. build with -Pdev && run >> 3. enable predictions preferably for every metric >> https://github.com/hawkular/hawkular-datamining and wait for some >> time (see below) >> 4. go to the Explorer tab, choose metric and increase forecasting >> horizon >> >> Data Mining is currently not able to query historical metric data from >> metrics due to authentication issues. However it still receives metrics >> from bus so it is important to enable predictions as soon as possible and >> wait for some time to collect data (or increase collection interval). This >> will be fixed in the next versions. >> >> Thanks to Jiri Kremser and UI team for helping with predictive charts. >> >> Useful links: >> >> - Docs >> - REST API >> >> >> Regards >> Pavol >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> > > > -- > Thomas Segismont > JBoss ON Engineering Team > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160421/15814532/attachment.html From theute at redhat.com Fri Apr 22 03:15:32 2016 From: theute at redhat.com (Thomas Heute) Date: Fri, 22 Apr 2016 09:15:32 +0200 Subject: [Hawkular-dev] Hawkular Data Mining 0.1.0.Final Release In-Reply-To: References: Message-ID: Great stuff, thanks ! On Thu, Apr 21, 2016 at 2:08 PM, Pavol Loffay wrote: > Blog post is available here > http://www.hawkular.org/blog/2016/04/21/datamining-first-release.html > > > Pavol > > On Thu, Apr 21, 2016 at 10:02 AM, Thomas Segismont > wrote: > >> Congrats Pavol! >> >> 2016-04-21 8:30 GMT+02:00 Pavol Loffay : >> >>> Hello Everyone, >>> >>> I am happy to announce first release 0.1.0.Final of Hawkular Data >>> Mining. Release contains several time series models and utility classes for >>> time series modelling: >>> >>> *Time series models* >>> >>> - Simple exponential smoothing >>> - Double exponential smoothing (Holt's linear trend) >>> - Seasonal triple exponential smoothing (Holt Winters) >>> - Simple moving average (Weighted moving average) >>> - AutomaticForecaster - which automatically selects the best model >>> - All these models are estimated using maximum likelihood estimation >>> and available for online learning >>> >>> Time series manipulation & Statistics >>> >>> - Augmented Dickey-Fuller test >>> - Autocorrelation function (ACF) >>> - Time series decomposition >>> - Time series lagging >>> - Time series differencing >>> - Automatic period identification >>> >>> >>> If you want to try it with Hawkular: https://goo.gl/R93k02 >>> >>> 1. clone https://github.com/hawkular/hawkular/tree/datamining - >>> hawkular/hawkular branch datamining >>> 2. build with -Pdev && run >>> 3. enable predictions preferably for every metric >>> https://github.com/hawkular/hawkular-datamining and wait for some >>> time (see below) >>> 4. go to the Explorer tab, choose metric and increase forecasting >>> horizon >>> >>> Data Mining is currently not able to query historical metric data from >>> metrics due to authentication issues. However it still receives metrics >>> from bus so it is important to enable predictions as soon as possible and >>> wait for some time to collect data (or increase collection interval). This >>> will be fixed in the next versions. >>> >>> Thanks to Jiri Kremser and UI team for helping with predictive charts. >>> >>> Useful links: >>> >>> - Docs >>> >>> - REST API >>> >>> >>> Regards >>> Pavol >>> >>> _______________________________________________ >>> hawkular-dev mailing list >>> hawkular-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev >>> >>> >> >> >> -- >> Thomas Segismont >> JBoss ON Engineering Team >> >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> >> > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160422/d2786bab/attachment-0001.html From jpkroehling at redhat.com Fri Apr 22 07:47:47 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 22 Apr 2016 13:47:47 +0200 Subject: [Hawkular-dev] What are your Authentication and Authorization needs? Message-ID: <571A0F63.7010105@redhat.com> Team, Due to the changes in requirements for Hawkular, I'm collecting the needs we have around authentication and authorization. On another thread, I mentioned that I'm inclined to settle on JAAS, but before I invest more time on that, I'd like to hear from you on your needs. Ultimately, securing your component is of my interest, but I don't have a complete picture on what your needs are. The current plan is to reduce Accounts 3.x to a minimum. Ideally, we won't even need Accounts 3.x, if we find out that all we need is JAAS. As a summary, here's what we have in Accounts that might currently be in use by other modules: - WebSocket authentication of messages. Each WebSocket message coming to the server has an "authentication" property, with credentials or tokens. The idea is that long-running connections would re-authenticate from time to time, without closing/opening the connection. The obvious easy solution here is to force-close a connection on the server side, and do a reconnect on the client. Ideally, however, we would find a way to authenticate the user with JAAS API. If that's possible, Accounts can help and consumers of WebSocket auth (ie: command gateway) won't need to be changed. - Multi tenancy: so far, I have not heard any concrete use cases for multi tenancy other than "it would be nice to have". Remembering that multi tenancy here is different than storing tenant information. We could certainly keep storing tenant information, but I don't see a case where we would need to perform authorization there. - Token support: this is not, per se, supported by vanilla JAAS. On the other hand, this can be emulated by having multiple "opaque" accounts on the JAAS user storage. So, this is not a big deal, except that the token API will cease existing (ie: any REST calls to /secret-store). This should affect *only* the UI and the Android client. - Functional users instead of real users: this is the main change. Users won't be "real" users anymore (ie: jpkroehling), but technical users (ie: miq-prod-01). Effectively, this is similar to authentication done by backend services, such as Cassandra and Postgresql. For us, this would be 100% handled by JAAS, so, we don't need to worry whether users are real or functional, but we would recommend using functional users. - Authorization: we currently have an API for authorization. I don't think anyone is using that, except perhaps for Inventory. As such, I would just remove it in favor of JAAS' authorization. This effectively means that you should be able to annotate "@RolesAllowed()" into your business methods, and JAAS would ensure that the user belongs to any of the authorized roles. We would, however, need to define the roles we need. I'd suggest starting with "read-only" and "read-write" as roles. - Events from the auth server: components are currently able to register a listener on a JMS queue or CDI event, to receive Keycloak server events (user logged in, user registered, ...). This will also cease to exist. It was implemented as a requirement from Inventory, but I think it's not being used anymore. - Feature pack: components building feature packs or consuming Accounts feature pack would need to use Nest instead. I don't expect this to be a big issue, though. - User jdoe : this will still be available on dev builds, on both the "read-only" and "read-write" roles that I suggested above. I appreciate that you are all busy, so, I'll assume you are OK with the proposed changes if I don't hear from you until next Friday :) - Juca. From mwringe at redhat.com Fri Apr 22 10:23:10 2016 From: mwringe at redhat.com (Matt Wringe) Date: Fri, 22 Apr 2016 10:23:10 -0400 (EDT) Subject: [Hawkular-dev] What are your Authentication and Authorization needs? In-Reply-To: <571A0F63.7010105@redhat.com> References: <571A0F63.7010105@redhat.com> Message-ID: <1152887070.63479320.1461334990724.JavaMail.zimbra@redhat.com> We have different authentication and authorization needs with Hawkular Metrics in OpenShift. I am not sure what exactly the plan is for introducing more Hawkular components into OpenShift though, so the requirements there may not really affect what is going on with the Hawkular components aside from Metrics (which we are using a custom setup and not Accounts). ----- Original Message ----- > From: "Juraci Paix?o Kr?hling" > To: "Discussions around Hawkular development" > Sent: Friday, April 22, 2016 7:47:47 AM > Subject: [Hawkular-dev] What are your Authentication and Authorization needs? > > Team, > > Due to the changes in requirements for Hawkular, I'm collecting the > needs we have around authentication and authorization. > > On another thread, I mentioned that I'm inclined to settle on JAAS, but > before I invest more time on that, I'd like to hear from you on your > needs. Ultimately, securing your component is of my interest, but I > don't have a complete picture on what your needs are. > > The current plan is to reduce Accounts 3.x to a minimum. Ideally, we > won't even need Accounts 3.x, if we find out that all we need is JAAS. > > As a summary, here's what we have in Accounts that might currently be in > use by other modules: > > - WebSocket authentication of messages. Each WebSocket message coming to > the server has an "authentication" property, with credentials or tokens. > The idea is that long-running connections would re-authenticate from > time to time, without closing/opening the connection. The obvious easy > solution here is to force-close a connection on the server side, and do > a reconnect on the client. Ideally, however, we would find a way to > authenticate the user with JAAS API. If that's possible, Accounts can > help and consumers of WebSocket auth (ie: command gateway) won't need to > be changed. > > - Multi tenancy: so far, I have not heard any concrete use cases for > multi tenancy other than "it would be nice to have". Remembering that > multi tenancy here is different than storing tenant information. We > could certainly keep storing tenant information, but I don't see a case > where we would need to perform authorization there. > > - Token support: this is not, per se, supported by vanilla JAAS. On the > other hand, this can be emulated by having multiple "opaque" accounts on > the JAAS user storage. So, this is not a big deal, except that the token > API will cease existing (ie: any REST calls to /secret-store). This > should affect *only* the UI and the Android client. > > - Functional users instead of real users: this is the main change. Users > won't be "real" users anymore (ie: jpkroehling), but technical users > (ie: miq-prod-01). Effectively, this is similar to authentication done > by backend services, such as Cassandra and Postgresql. For us, this > would be 100% handled by JAAS, so, we don't need to worry whether users > are real or functional, but we would recommend using functional users. > > - Authorization: we currently have an API for authorization. I don't > think anyone is using that, except perhaps for Inventory. As such, I > would just remove it in favor of JAAS' authorization. This effectively > means that you should be able to annotate "@RolesAllowed()" into your > business methods, and JAAS would ensure that the user belongs to any of > the authorized roles. We would, however, need to define the roles we > need. I'd suggest starting with "read-only" and "read-write" as roles. > > - Events from the auth server: components are currently able to register > a listener on a JMS queue or CDI event, to receive Keycloak server > events (user logged in, user registered, ...). This will also cease to > exist. It was implemented as a requirement from Inventory, but I think > it's not being used anymore. > > - Feature pack: components building feature packs or consuming Accounts > feature pack would need to use Nest instead. I don't expect this to be a > big issue, though. > > - User jdoe : this will still be available on dev builds, on both the > "read-only" and "read-write" roles that I suggested above. > > I appreciate that you are all busy, so, I'll assume you are OK with the > proposed changes if I don't hear from you until next Friday :) > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From snegrea at redhat.com Fri Apr 22 12:35:19 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Fri, 22 Apr 2016 11:35:19 -0500 Subject: [Hawkular-dev] Future Packaging of Hawkular Message-ID: Hello Everybody, With a growing community and an increasingly large number of sub-projects, it is the perfect time to rethink the packaging of Hawkular. This email is a blueprint for changes to come to Hawkular as well as establish some patterns for futures decisions. *Motivation* So far we have created Hawkular "All-in-one" releases that contain everything, Hawkular server components, UI, the agent and also embedded versions of Cassandra and KeyCloak. It's certainly the case that for community we will still deliver a fully featured all-in-one package. We also need to consider separate distributions for other projects that integrate with Hawkular. The recent effort to integrate with ManageIQ comes with its own set of requirements for services. Not all of the Hawkular components are needed and required components may be configured differently. For example the UI is supplied from within ManageIQ side and Hawkular is to be run headless. Other components like Cassandra will not run embedded, so there is no need to supply it in an embedded package. KeyCloak will probably not be needed because we only have one technical user that will access Hawkular services. *Components / Services* Here a list of projects that are part of the Hawkular organization: - Metrics - scalable, asynchronous, multi tenant, long term metrics storage engine - Inventory - registry of "things" that contains info about your applications, servers, etc. and also keeps track of their relationships with each other - Alerts - alerting engine that allows trigger definitions to evaluate incoming data, generate alerts (or events), and react with flexible actions ond lifecycle management - CommandGW - messaging framework for communicating between components - Bus - messaging framework for communicating between components - BTM - provide capabilities to monitor the flow of a business transaction instance and enable performance analysis of the individual components that make up an application - Agent - used to monitor WildFly and related projects - Accounts - is the user/organization module that provides authentication, authorization and configuration for accounts - Data Mining - alert predictions based on time series data - Hawkular UI - Hawkular Charts - Angular Directives for Metrics Visualization - Embedded Cassandra - embedded Cassandra server specifically configured for project use There are also various clients like the Ruby Gem or the Android client and supporting infrastructure projects. Those are not part of this discussion. *Options for **P**ackages* The distribution packages need to follow product integrations with a special distribution for community. The community package could potentially include every single package from the Hawkular organization. However, the rest of the packages need to be created along the lines of product use and include only the minimal amount of components, services and third-party libraries to satisfy integration requirements. The distributions should be seen as funnels, the Hawkular community creates a set of sub-projects that then get combined into few distributions that in turn get consumed. Following this analogy, the number of packaged distributions should also be kept to a minimum possible. For now we settled on 3 packages detailed below, all will be available for download on JBoss.org. With the exception of the community distribution: 1) the QA team will be engaged in testing the package 2) the sub-components will strive to be aligned in terms of tech stack and maturity 3) an automated test suite is required for packages that combine more than one project *1. Hawkular - Community Distribution* Will include every single service listed above. The UI is a requirement because some services are very hard to understand or use without a friendly interface. The UI will be a community only effort at this point since there are no plans for productization. The current Hawkular repository will be adjusted to reflect this new mission; the current UI will be moved to a separate repository. This package builds on Hawkular Core Services as described next. *2. Hawkular **Core** S**ervices* Geared towards MiQ integration, it will include only components needed for the MiQ provider. This package uses the Metrics Distribution as described below. Components included: Metrics, Alerts, Inventory, CommandGW, and Bus Reasons for some exclusions: Keycloak (different security model), Embedded Cassandra (only full C* deployments will be supported), and no Hawkular UI because it will use a special purpose UI for MiQ. *3. **Metrics** D**istribution* Geared towards usage as a pure Time Series Database (TSDB) and will be exclusively made of Hawkular Metrics service. *Expansion and Contraction* Since its inception Hawkular has been growing by all metrics: projects, lines of code, binary downloads, contributors, etc. But uncontrolled increase in some metrics can be detrimental to the organization. This is especially true for the number of projects. We will always need new projects to experiment, but the reverse might be true for mature and established projects. As the packaging and requirements for each distribution mature we need to consider the reverse, merging separate sub-projects. *Packaging Format* This email did not touch on the physical packaging formats (rpm, zip, war, jar) for the 3 distributions. We will follow-up with additional details in the coming weeks. All feedback is more than welcomed so feel free to reach us (Heiko and Stefan) directly or reply to this thread. Heiko & Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160422/84a339c0/attachment-0001.html From mazz at redhat.com Fri Apr 22 12:42:32 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 22 Apr 2016 12:42:32 -0400 (EDT) Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: References: Message-ID: <45409008.4323790.1461343352252.JavaMail.zimbra@redhat.com> How do we plan on distributing the Hawkular WildFly Agent now? Before, we had a WAR deployed in the server that had a single servlet running that would take an HTTP GET or POST request and based on the request would package up the agent installer with the latest agent extension and properly config the installer so when the agent was installed it could talk to that server where it came from. It would also allow the installer to give the agent the proper token credentials so it could talk to the server. Is that WAR/servlet still going to be in the headless version? Just curious what the plans were for that. ----- Original Message ----- > Hello Everybody, > > With a growing community and an increasingly large number of sub-projects, it > is the perfect time to rethink the packaging of Hawkular. This email is a > blueprint for changes to come to Hawkular as well as establish some patterns > for futures decisions. > > > > Motivation > > So far we have created Hawkular "All - in - one" releases that contain > everything , Hawkular server components, UI, the agent and also embedded > versions of Cassandra and KeyCloak. It's certainly the case that for > community we w ill still deliver a fully featured all-in-one package . We > also need to consider separate distributions for other projects that > integrate with Hawkular. > > The recent effort to integrate with ManageIQ comes with its own set of > requirements for services . Not all of the Hawkular components are needed > and required components may be configured differently . For example the UI > is supplied from within ManageIQ side and Hawkular is to be run headless. > Other components like Cassandra will not run embedded, so there is no need > to supply it in an embedded package . KeyCloak will probably not be need ed > because we only have one technical user that will access Hawkular services. > > > > Components / Services > > Here a list of projects that are part of the Hawkular organization: > > > * Metrics - scalable, asynchronous, multi tenant, long term metrics > storage engine > > > * Inventory - registry of "things" that contains info about your > applications, servers, etc. and also keeps track of their relationships > with each other > > > * Alerts - alerting engine that allows trigger definitions to evaluate > incoming data, generate alerts (or events), and react with flexible > actions ond lifecycle management > > > * CommandGW - messaging framework for communicating between components > > > * Bus - messaging framework for communicating between components > > > * BTM - provide capabilities to monitor the flow of a business > transaction instance and enable performance analysis of the individual > components that make up an application > > > * Agent - used to monitor WildFly and related projects > > > * Accounts - is the user/organization module that provides > authentication, authorization and configuration for accounts > > > * Data Mining - alert predictions based on time series data > > > * Hawkular UI > > > * Hawkular Charts - Angular Directives for Metrics Visualization > > > * Embedded Cassandra - embedded Cassandra server specifically configured > for project use > > T here are also various clients like the Ruby Gem or the Android client and > supporting infrastructure projects. Those are not part of this discussion. > > > > Options for P ackages > > The distribution packages need to follow product integrations with a special > distribution for community. The community package could potentially include > every single package from the Hawkular organization. However, the rest of > the packages need to be created along the lines of product use and include > only the minimal amount of components, services and third-party libraries to > satisfy integration requirements. > > The distributions should be seen as funnels, the Hawkular community creates a > set of sub-projects that then get combined into few distributions that in > turn get consumed. Following this analogy, the number of packaged > distributions should also be kept to a minimum possible. For now we settled > on 3 packages detailed below, all will be available for download on > JBoss.org. > > With the exception of the community distribution: > 1) the QA team will be engaged in testing the package > 2) the sub-components wil l strive to be aligned in terms of tech stack and > maturity > 3) an automated test suite is required for packages that combine more than > one project > > 1. Hawkular - Community Distribution > > Will include every single service listed above. The UI is a requirement > because some services are very hard to understand or use without a friendly > interface. The UI will be a community only effort at this point since there > are no plans for productization. > > The current Hawkular repository will be adjusted to reflect this new mission; > the current UI will be moved to a separate repository. This package builds > on Hawkular Core S ervices as described next . > > 2. Hawkular Core S ervices > > Geared towards MiQ integration, it will include only components needed for > the MiQ provider. This package uses the Metrics Distribution as described > below. > > Components included: Metrics, Alerts, Inventory, CommandGW, and Bus > > Reasons for some exclusions: K eycloak (different security model), E mbedded > Cassandra (only full C* deployments will be supported), and no Hawkular UI > because it will use a special purpose UI for MiQ. > > 3. Metrics D istribution > > Geared towards usage as a pure Time Series D atabase (TSDB) and will be > exclusively made of Hawkular Metrics service. > > > > Expansion and Contraction > > Since its inception Hawkular has been growing by all metrics: projects, lines > of code, binary downloads, contributors, etc. But uncontrolled increase in > some metrics can be detrimental to the organization. This is especially true > for the number of projects. We will always need new projects to experiment, > but the reverse might be true for mature and established projects. As the > packaging and requirements for each distribution mature we need to consider > the reverse, merging separate sub-projects. > > > > Packaging Format > > This email did not touch on the physical packaging formats (rpm, zip, war, > jar) for the 3 distributions. We will follow-up with additional details in > the coming weeks. > > > > All feedback is more than welcomed so feel free to reach us (Heiko and > Stefan) directly or reply to this thread. > > > > Heiko & Stefan > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From snegrea at redhat.com Fri Apr 22 13:28:00 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Fri, 22 Apr 2016 12:28:00 -0500 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <45409008.4323790.1461343352252.JavaMail.zimbra@redhat.com> References: <45409008.4323790.1461343352252.JavaMail.zimbra@redhat.com> Message-ID: On Fri, Apr 22, 2016 at 11:42 AM, John Mazzitelli wrote: > How do we plan on distributing the Hawkular WildFly Agent now? Before, we > had a WAR deployed in the server that had a single servlet running that > would take an HTTP GET or POST request and based on the request would > package up the agent installer with the latest agent extension and properly > config the installer so when the agent was installed it could talk to that > server where it came from. It would also allow the installer to give the > agent the proper token credentials so it could talk to the server. > > Is that WAR/servlet still going to be in the headless version? Just > curious what the plans were for that. This detail was specifically left out because we will need to clarify the requirements. Also, the way you described, the Agent is not necessarily part of the package but a payload of the package. Users can download an Agent via the distribution but is not an active service of the distribution; for example Metrics is an active service, Agent is not. The focus of this first email was on active services. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160422/b5e15b85/attachment.html From hrupp at redhat.com Fri Apr 22 13:58:38 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 22 Apr 2016 19:58:38 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <45409008.4323790.1461343352252.JavaMail.zimbra@redhat.com> References: <45409008.4323790.1461343352252.JavaMail.zimbra@redhat.com> Message-ID: On 22 Apr 2016, at 18:42, John Mazzitelli wrote: > How do we plan on distributing the Hawkular WildFly Agent now? Before, > we had a WAR deployed in the server that had The agent is certainly a part of the logical Hawkular-Core-Services package. See also attached graphic. -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill -------------- next part -------------- A non-text attachment was scrubbed... Name: hawk-pack.png Type: image/png Size: 59203 bytes Desc: not available Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160422/636c02cc/attachment-0001.png From mazz at redhat.com Fri Apr 22 14:59:41 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 22 Apr 2016 14:59:41 -0400 (EDT) Subject: [Hawkular-dev] agent 0.17.2.Final - now supports metric sets for prometheus endpoints In-Reply-To: <1328961998.4351981.1461351573427.JavaMail.zimbra@redhat.com> Message-ID: <1069866214.4351985.1461351581098.JavaMail.zimbra@redhat.com> I released another minor release of the agent in order for a new feature to be available. You can now filter metrics to be collected from Prometheus endpoints. Before you had no choice but to collect and store every metric from every Prometheus endpoint being monitored. That is still the default behavior, but now you have the option to selectively choose what metrics you want to collect and store (ignoring all the rest). See this blog entry for more: http://www.hawkular.org/blog/2016/04/22/collecting-metrics-from-prometheus-endpoints.html From lponce at redhat.com Mon Apr 25 03:13:03 2016 From: lponce at redhat.com (Lucas Ponce) Date: Mon, 25 Apr 2016 03:13:03 -0400 (EDT) Subject: [Hawkular-dev] What are your Authentication and Authorization needs? In-Reply-To: <571A0F63.7010105@redhat.com> References: <571A0F63.7010105@redhat.com> Message-ID: <149057485.4675131.1461568383225.JavaMail.zimbra@redhat.com> Hi Juca, Thanks for bringing this. For alerting, all model is tenant-based, and I don't see that aspect is going to change (or we can change it without a major refactor). In alerting there is not fine grained authorization, it should be a major overkill. So, no new security requeriments from this component, internally we work with the tenant that is translated from hawkular accounts (or taken from a header in standalone scenarios). Lucas ----- Mensaje original ----- > De: "Juraci Paix?o Kr?hling" > Para: "Discussions around Hawkular development" > Enviados: Viernes, 22 de Abril 2016 13:47:47 > Asunto: [Hawkular-dev] What are your Authentication and Authorization needs? > > Team, > > Due to the changes in requirements for Hawkular, I'm collecting the > needs we have around authentication and authorization. > > On another thread, I mentioned that I'm inclined to settle on JAAS, but > before I invest more time on that, I'd like to hear from you on your > needs. Ultimately, securing your component is of my interest, but I > don't have a complete picture on what your needs are. > > The current plan is to reduce Accounts 3.x to a minimum. Ideally, we > won't even need Accounts 3.x, if we find out that all we need is JAAS. > > As a summary, here's what we have in Accounts that might currently be in > use by other modules: > > - WebSocket authentication of messages. Each WebSocket message coming to > the server has an "authentication" property, with credentials or tokens. > The idea is that long-running connections would re-authenticate from > time to time, without closing/opening the connection. The obvious easy > solution here is to force-close a connection on the server side, and do > a reconnect on the client. Ideally, however, we would find a way to > authenticate the user with JAAS API. If that's possible, Accounts can > help and consumers of WebSocket auth (ie: command gateway) won't need to > be changed. > > - Multi tenancy: so far, I have not heard any concrete use cases for > multi tenancy other than "it would be nice to have". Remembering that > multi tenancy here is different than storing tenant information. We > could certainly keep storing tenant information, but I don't see a case > where we would need to perform authorization there. > > - Token support: this is not, per se, supported by vanilla JAAS. On the > other hand, this can be emulated by having multiple "opaque" accounts on > the JAAS user storage. So, this is not a big deal, except that the token > API will cease existing (ie: any REST calls to /secret-store). This > should affect *only* the UI and the Android client. > > - Functional users instead of real users: this is the main change. Users > won't be "real" users anymore (ie: jpkroehling), but technical users > (ie: miq-prod-01). Effectively, this is similar to authentication done > by backend services, such as Cassandra and Postgresql. For us, this > would be 100% handled by JAAS, so, we don't need to worry whether users > are real or functional, but we would recommend using functional users. > > - Authorization: we currently have an API for authorization. I don't > think anyone is using that, except perhaps for Inventory. As such, I > would just remove it in favor of JAAS' authorization. This effectively > means that you should be able to annotate "@RolesAllowed()" into your > business methods, and JAAS would ensure that the user belongs to any of > the authorized roles. We would, however, need to define the roles we > need. I'd suggest starting with "read-only" and "read-write" as roles. > > - Events from the auth server: components are currently able to register > a listener on a JMS queue or CDI event, to receive Keycloak server > events (user logged in, user registered, ...). This will also cease to > exist. It was implemented as a requirement from Inventory, but I think > it's not being used anymore. > > - Feature pack: components building feature packs or consuming Accounts > feature pack would need to use Nest instead. I don't expect this to be a > big issue, though. > > - User jdoe : this will still be available on dev builds, on both the > "read-only" and "read-write" roles that I suggested above. > > I appreciate that you are all busy, so, I'll assume you are OK with the > proposed changes if I don't hear from you until next Friday :) > > - Juca. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jpkroehling at redhat.com Mon Apr 25 03:22:21 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 25 Apr 2016 09:22:21 +0200 Subject: [Hawkular-dev] What are your Authentication and Authorization needs? In-Reply-To: <149057485.4675131.1461568383225.JavaMail.zimbra@redhat.com> References: <571A0F63.7010105@redhat.com> <149057485.4675131.1461568383225.JavaMail.zimbra@redhat.com> Message-ID: <7bbb3a40-fd71-043f-7982-b458b501c017@redhat.com> On 25.04.2016 09:13, Lucas Ponce wrote: > For alerting, all model is tenant-based, and I don't see that aspect is going to change (or we can change it without a major refactor). There won't be any more tenancy information coming from Accounts, because there won't be any tenancy information coming to Accounts :) I'm afraid you'd have to change it. > So, no new security requeriments from this component, internally we work with the tenant that is translated from hawkular accounts (or taken from a header in standalone scenarios). Your clients (MiQ, Ruby gem, UI, ...) will have to know about tenants and send it to Alerts on the payload. Accounts won't touch it. If you (and other components) *require* tenancy information for some concrete use case, we might discuss how we could handle it in a common way to all components. Otherwise, I'd just assume that the same requirement you had for multi tenancy went away when the requirement for multi tenancy on Accounts went away. - Juca. From hrupp at redhat.com Mon Apr 25 03:25:14 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 25 Apr 2016 09:25:14 +0200 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin Message-ID: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> Hello, please welcome Anuj and Austin as this years students on Hawkular. Anuj will work on the Android Client, where he has already done a lot of work. Austin will work on the agent for Vert.x to support inventory (sync) etc and to sort of bring it on par with the WildFly agent. Austin and Anuj are 2 out of 10 students for jboss.org this year. You can read more here: http://pilhuhn.blogspot.de/2016/04/welcome-gsoc-2016-students-to-jboss.html Heiko From theute at redhat.com Mon Apr 25 03:26:30 2016 From: theute at redhat.com (Thomas Heute) Date: Mon, 25 Apr 2016 09:26:30 +0200 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> Message-ID: Welcome ! On Mon, Apr 25, 2016 at 9:25 AM, Heiko W.Rupp wrote: > Hello, > > please welcome Anuj and Austin as this years students on Hawkular. > > Anuj will work on the Android Client, where he has already done a lot of > work. > Austin will work on the agent for Vert.x to support inventory (sync) etc > and to > sort of bring it on par with the WildFly agent. > > Austin and Anuj are 2 out of 10 students for jboss.org this year. > You can read more here: > http://pilhuhn.blogspot.de/2016/04/welcome-gsoc-2016-students-to-jboss.html > > Heiko > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/903e12b2/attachment.html From lponce at redhat.com Mon Apr 25 03:31:56 2016 From: lponce at redhat.com (Lucas Ponce) Date: Mon, 25 Apr 2016 03:31:56 -0400 (EDT) Subject: [Hawkular-dev] What are your Authentication and Authorization needs? In-Reply-To: <7bbb3a40-fd71-043f-7982-b458b501c017@redhat.com> References: <571A0F63.7010105@redhat.com> <149057485.4675131.1461568383225.JavaMail.zimbra@redhat.com> <7bbb3a40-fd71-043f-7982-b458b501c017@redhat.com> Message-ID: <62209943.4681477.1461569516237.JavaMail.zimbra@redhat.com> ----- Mensaje original ----- > De: "Juraci Paix?o Kr?hling" > Para: "Lucas Ponce" , "Discussions around Hawkular development" > Enviados: Lunes, 25 de Abril 2016 9:22:21 > Asunto: Re: [Hawkular-dev] What are your Authentication and Authorization needs? > > On 25.04.2016 09:13, Lucas Ponce wrote: > > For alerting, all model is tenant-based, and I don't see that aspect is > > going to change (or we can change it without a major refactor). > > There won't be any more tenancy information coming from Accounts, > because there won't be any tenancy information coming to Accounts :) I'm > afraid you'd have to change it. > > > So, no new security requeriments from this component, internally we work > > with the tenant that is translated from hawkular accounts (or taken from a > > header in standalone scenarios). > > Your clients (MiQ, Ruby gem, UI, ...) will have to know about tenants > and send it to Alerts on the payload. Accounts won't touch it. > > If you (and other components) *require* tenancy information for some > concrete use case, we might discuss how we could handle it in a common > way to all components. Otherwise, I'd just assume that the same > requirement you had for multi tenancy went away when the requirement for > multi tenancy on Accounts went away. > > - Juca. > So, if I interpret correctly, it seems that the change will be that we should handle the tenant aspect explictly as we do for standalone scenarios. Well, that's not a big change at all, I think we covered that usecase with the Hawkular-Tenant header. From jpkroehling at redhat.com Mon Apr 25 03:32:46 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 25 Apr 2016 09:32:46 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: References: Message-ID: <8b0a2095-14ac-66f4-d9cd-289b9689b534@redhat.com> On 22.04.2016 18:35, Stefan Negrea wrote: > *Motivation* > > So far we have created Hawkular "All-in-one" releases that contain > everything,Hawkular server components, UI, the agent and also embedded > versions of Cassandra and KeyCloak. It's certainly the case that for > community we willstill deliver a fully featured all-in-one package. We > also need to consider separate distributions for other projects that > integrate with Hawkular. > > The recent effort to integrate with ManageIQ comes with its own set of > requirements for services. Not all of the Hawkular components are > neededand required components maybe configured differently.For example > the UI is supplied from within ManageIQ side and Hawkular is to be run > headless. Other components like Cassandra will not run embedded, so > there is no need to supply it in an embedded package. KeyCloak will > probably not be needed because we only have one technical user thatwill > accessHawkularservices. Let me reverse the question: what is the motivation for having the "Community Distribution"? Why would one use it instead of MiQ + Hawkular? What are the concrete use cases? What can we offer on this distribution that MiQ + Hawkular can't? Keeping a distribution like this if we have no concrete use cases or clear benefits will only cause us to spend time creating/maintaining when we could be spending time on other more relevant parts. I asked a similar question already a couple of times in another threads, but I still couldn't get a good answer. I'm starting to think that everyone can clearly see the need for this, except me... - Juca. From jpkroehling at redhat.com Mon Apr 25 03:37:48 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Mon, 25 Apr 2016 09:37:48 +0200 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> Message-ID: <2a5958de-5158-fc0b-fd3b-274d46005999@redhat.com> On 25.04.2016 09:25, Heiko W.Rupp wrote: > please welcome Anuj and Austin as this years students on Hawkular. Welcome Austin! Glad to see you made it, Anuj! - Juca. From hrupp at redhat.com Mon Apr 25 03:49:51 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 25 Apr 2016 09:49:51 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <8b0a2095-14ac-66f4-d9cd-289b9689b534@redhat.com> References: <8b0a2095-14ac-66f4-d9cd-289b9689b534@redhat.com> Message-ID: <071B7C56-868C-46A3-8DBB-6A517AD6F34C@redhat.com> On 25 Apr 2016, at 9:32, Juraci Paix?o Kr?hling wrote: > Let me reverse the question: what is the motivation for having the > "Community Distribution"? Why would one use it instead of MiQ + > Hawkular? What are the concrete use cases? What can we offer on this > distribution that MiQ + Hawkular can't? The community distribution is a clear superset of the Hawkular-core-services, containing components that are important like e.g BTM, but which are not part of the core-services. From lponce at redhat.com Mon Apr 25 03:52:34 2016 From: lponce at redhat.com (Lucas Ponce) Date: Mon, 25 Apr 2016 03:52:34 -0400 (EDT) Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> Message-ID: <141622627.4688363.1461570754192.JavaMail.zimbra@redhat.com> Welcome Anuj and Austin ! Hope they will have a good experience working with the code, and don't hesitate to ask questions or comments on #hawkular chat room (Freenode). Lucas ----- Mensaje original ----- > De: "Heiko W.Rupp" > Para: "Discussions around Hawkular development" > Enviados: Lunes, 25 de Abril 2016 9:25:14 > Asunto: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin > > Hello, > > please welcome Anuj and Austin as this years students on Hawkular. > > Anuj will work on the Android Client, where he has already done a lot of > work. > Austin will work on the agent for Vert.x to support inventory (sync) etc > and to > sort of bring it on par with the WildFly agent. > > Austin and Anuj are 2 out of 10 students for jboss.org this year. > You can read more here: > http://pilhuhn.blogspot.de/2016/04/welcome-gsoc-2016-students-to-jboss.html > > Heiko > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From tsegismo at redhat.com Mon Apr 25 04:09:20 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Mon, 25 Apr 2016 10:09:20 +0200 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: <141622627.4688363.1461570754192.JavaMail.zimbra@redhat.com> References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> <141622627.4688363.1461570754192.JavaMail.zimbra@redhat.com> Message-ID: Welcome guys! Looking forward to working with you! 2016-04-25 9:52 GMT+02:00 Lucas Ponce : > Welcome Anuj and Austin ! > > Hope they will have a good experience working with the code, and don't > hesitate to ask questions or comments on #hawkular chat room (Freenode). > > Lucas > > ----- Mensaje original ----- > > De: "Heiko W.Rupp" > > Para: "Discussions around Hawkular development" < > hawkular-dev at lists.jboss.org> > > Enviados: Lunes, 25 de Abril 2016 9:25:14 > > Asunto: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin > > > > Hello, > > > > please welcome Anuj and Austin as this years students on Hawkular. > > > > Anuj will work on the Android Client, where he has already done a lot of > > work. > > Austin will work on the agent for Vert.x to support inventory (sync) etc > > and to > > sort of bring it on par with the WildFly agent. > > > > Austin and Anuj are 2 out of 10 students for jboss.org this year. > > You can read more here: > > > http://pilhuhn.blogspot.de/2016/04/welcome-gsoc-2016-students-to-jboss.html > > > > Heiko > > _______________________________________________ > > hawkular-dev mailing list > > hawkular-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/fb06d902/attachment.html From anuj1708 at gmail.com Mon Apr 25 04:17:45 2016 From: anuj1708 at gmail.com (Anuj Garg) Date: Mon, 25 Apr 2016 13:47:45 +0530 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> <141622627.4688363.1461570754192.JavaMail.zimbra@redhat.com> Message-ID: Thanks for giving me such a nice environment and opportunity to learn and grow. Special thanks for pilhuhn, passos and juca for helping and guiding me at every step. Anuj -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/66c066fa/attachment.html From jshaughn at redhat.com Mon Apr 25 08:48:30 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Mon, 25 Apr 2016 08:48:30 -0400 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> Message-ID: <571E121E.8040004@redhat.com> Welcome to the team! Jay On 4/25/2016 3:25 AM, Heiko W.Rupp wrote: > Hello, > > please welcome Anuj and Austin as this years students on Hawkular. > > Anuj will work on the Android Client, where he has already done a lot of > work. > Austin will work on the agent for Vert.x to support inventory (sync) etc > and to > sort of bring it on par with the WildFly agent. > > Austin and Anuj are 2 out of 10 students for jboss.org this year. > You can read more here: > http://pilhuhn.blogspot.de/2016/04/welcome-gsoc-2016-students-to-jboss.html > > Heiko > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/2447e104/attachment.html From ppalaga at redhat.com Mon Apr 25 09:24:26 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Mon, 25 Apr 2016 15:24:26 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: References: Message-ID: <571E1A8A.7010407@redhat.com> Hi Stefan and Heiko, thanks for sharing your vision. A few questions/remarks inline... On 2016-04-22 18:35, Stefan Negrea wrote: > Hello Everybody, > > With a growing community and an increasingly large number of > sub-projects, it is the perfect time to rethink the packaging of > Hawkular. This email is a blueprint for changes to come to Hawkular as > well as establish some patterns for futures decisions. > > > > *Motivation* > > So far we have created Hawkular "All-in-one" releases that contain > everything,Hawkular server components, UI, the agent and also embedded > versions of Cassandra and KeyCloak. It's certainly the case that for > community we willstill deliver a fully featured all-in-one package. We > also need to consider separate distributions for other projects that > integrate with Hawkular. > > The recent effort to integrate with ManageIQ comes with its own set of > requirements for services. Not all of the Hawkular components are > neededand required components maybe configured differently.For example > the UI is supplied from within ManageIQ side and Hawkular is to be run > headless. Other components like Cassandra will not run embedded, so > there is no need to supply it in an embedded package. KeyCloak will > probably not be needed because we only have one technical user > thatwill accessHawkularservices. > > > > *Components / Services* > > Here a list of projects that are part of the Hawkular organization: > > * Metrics - scalable, asynchronous, multi tenant, long term metrics > storage engine > > * Inventory - registry of "things" that contains info about your > applications, servers, etc. and also keeps track of their > relationships with each other > > * Alerts - alerting engine that allows trigger definitions to > evaluate incoming data, generate alerts (or events), and react > with flexible actions ond lifecycle management > > * CommandGW- messaging framework for communicating between components > > * Bus - messaging framework for communicating between components > Maybe Nest should be listed here. AFAIK, Bus is always consumed together with Nest, hence the one can replace "Bus" with "Nest+Bus" in your original proposal and I'll be satisfied :) > > * BTM - provide capabilities to monitor the flow of a business > transaction instance and enable performance analysis of the > individual components that make up an application > > * Agent - used to monitor WildFly and related projects > > * Accounts - is the user/organization module that provides > authentication, authorization and configuration for accounts > > * Data Mining - alert predictions based on time series data > > * Hawkular UI > > * Hawkular Charts - Angular Directives for Metrics Visualization > > * Embedded Cassandra - embedded Cassandra server specifically > configured for project use > > > There arealsovarious clients like the Ruby Gem or the Android client > and supporting infrastructure projects. Those are not part of this > discussion. > > > > *Options for **P**ackages* > > The distribution packages need to follow product integrations with a > special distribution for community. The community package could > potentially include every single package from the Hawkular > organization. However, the rest of the packages need to be created > along the lines of product use and include only the minimal amount of > components, services and third-party libraries to satisfy integration > requirements. > > The distributions should be seen as funnels, the Hawkular community > creates a set of sub-projects that then get combined into few > distributions that in turn get consumed. Following this analogy, the > number of packaged distributions should also be kept to a minimum > possible. For now we settled on 3 packages detailed below, all will be > available for download on JBoss.org. > > With the exception of the community distribution: > 1) the QA team will be engaged in testing the package > 2) the sub-components willstrive to be aligned in terms of tech stack > and maturity > 3) an automated test suite is required for packages that combine more > than one project > > *1. Hawkular - Community Distribution* > > Will include every single service listed above. The UI is a > requirement because some services are very hard to understand or use > without a friendly interface. The UI will be a community only effort > at this point since there are no plans for productization. > > The current Hawkular repository will be adjusted to reflect this new > mission; the current UI will be moved to a separate repository. This > package builds on Hawkular CoreServices as described next. > > *2. Hawkular **Core**S**ervices* > > Geared towards MiQ integration, it will include only components needed > for the MiQ provider. This package uses the Metrics Distribution as > described below. > > Components included: Metrics, Alerts, Inventory, CommandGW, and Bus > > Reasons for some exclusions: Keycloak(different security model), > Embedded Cassandra (only full C* deployments will be supported), andno > Hawkular UI because it will use a special purpose UI for MiQ. You say say something about Keycloak, but you do not mention Accounts. I see that Accounts 2.x will not be included and I suppose that Accouns 3.x will perhaps not be needed at all (as follows from the parallel thread), right? > > *3. **Metrics**D**istribution* > > Geared towards usage as a pure Time Series Database (TSDB) and will be > exclusively made of HawkularMetrics service. > > > > *Expansion and Contraction* > > Since its inception Hawkular has been growing by all metrics: > projects, lines of code, binary downloads, contributors, etc. But > uncontrolled increase in some metrics can be detrimental to the > organization. This is especially true for the number of projects. We > will always need new projects to experiment, but the reverse might be > true for mature and established projects. As the packaging and > requirements for each distribution mature we need to consider the > reverse, merging separate sub-projects. If we decide that there will be no Accounts 3.x, then Command Gateway (being quite small and closely related to Bus) can move to Commons git repository. Command Gateway does not need to depend on Inventory since Commons 0.6.0.Final. The change was recently merged to CmdGW. Thanks, Peter > *Packaging Format* > > This email did not touch on the physical packaging formats (rpm, zip, > war, jar) for the 3 distributions. We will follow-up with additional > details in the coming weeks. > > > > All feedback is more than welcomed so feel free to reach us (Heiko and > Stefan) directly or reply to this thread. > > > > Heiko & Stefan > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/3c810665/attachment-0001.html From jshaughn at redhat.com Mon Apr 25 09:29:30 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Mon, 25 Apr 2016 09:29:30 -0400 Subject: [Hawkular-dev] What are your Authentication and Authorization needs? In-Reply-To: <62209943.4681477.1461569516237.JavaMail.zimbra@redhat.com> References: <571A0F63.7010105@redhat.com> <149057485.4675131.1461568383225.JavaMail.zimbra@redhat.com> <7bbb3a40-fd71-043f-7982-b458b501c017@redhat.com> <62209943.4681477.1461569516237.JavaMail.zimbra@redhat.com> Message-ID: <571E1BBA.2070907@redhat.com> Juca, I think Lucas is correct, Alerts has the multi-tenancy model built in and so requires a tenantId on everything. We already have a standalone distribution (for use outside of Hawkular) that drives off of the Hawkular-Tenant header, so I guess we would just continue to use that mechanism in all cases. I guess that means we may drop the h-accounts dependency but I will discuss further with Lucas and we'll continue to monitor the accounts changes. Lucas, this may further drive the need for schema refactoring because if we only receive a single tenant on everything coming from MIQ, we will get very little data distribution. On 4/25/2016 3:31 AM, Lucas Ponce wrote: > > ----- Mensaje original ----- >> De: "Juraci Paix?o Kr?hling" >> Para: "Lucas Ponce" , "Discussions around Hawkular development" >> Enviados: Lunes, 25 de Abril 2016 9:22:21 >> Asunto: Re: [Hawkular-dev] What are your Authentication and Authorization needs? >> >> On 25.04.2016 09:13, Lucas Ponce wrote: >>> For alerting, all model is tenant-based, and I don't see that aspect is >>> going to change (or we can change it without a major refactor). >> There won't be any more tenancy information coming from Accounts, >> because there won't be any tenancy information coming to Accounts :) I'm >> afraid you'd have to change it. >> >>> So, no new security requeriments from this component, internally we work >>> with the tenant that is translated from hawkular accounts (or taken from a >>> header in standalone scenarios). >> Your clients (MiQ, Ruby gem, UI, ...) will have to know about tenants >> and send it to Alerts on the payload. Accounts won't touch it. >> >> If you (and other components) *require* tenancy information for some >> concrete use case, we might discuss how we could handle it in a common >> way to all components. Otherwise, I'd just assume that the same >> requirement you had for multi tenancy went away when the requirement for >> multi tenancy on Accounts went away. >> >> - Juca. >> > So, if I interpret correctly, it seems that the change will be that we should handle the tenant aspect explictly as we do for standalone scenarios. > > Well, that's not a big change at all, I think we covered that usecase with the Hawkular-Tenant header. > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/996d07ab/attachment.html From hrupp at redhat.com Mon Apr 25 11:02:12 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Mon, 25 Apr 2016 17:02:12 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <571E1A8A.7010407@redhat.com> References: <571E1A8A.7010407@redhat.com> Message-ID: <702EBFA6-370E-400B-87C4-D3BB0151BA21@redhat.com> Hey Peter, >> * Bus - messaging framework for communicating between components >> > Maybe Nest should be listed here. AFAIK, Bus is always consumed > together with Nest, hence the one can replace "Bus" with "Nest+Bus" in > your original proposal and I'll be satisfied :) Yes sure, thanks for pointing it out. >> use a special purpose UI for MiQ. > You say say something about Keycloak, but you do not mention Accounts. > I see that Accounts 2.x will not be included and I suppose that > Accouns 3.x will perhaps not be needed at all (as follows from the > parallel thread), right? Perhaps? :-) I think you and/or Juca know better which version of Accounts will be needed if at all. For core-services we do not need the full multi-tenancy with KC solution we have and which should still be available in the community distro. Here we have a technical user that MiQ uses to connect to Hawkular-core-services(HCS). And then most probably the same user that the agents use to talk to HCS. We may decide that we add more than one user where e.g. only the super-admin has r/w rights and other users have r/o. But all those could be configured in a properties file or via the JBoss-EAP way of setting up users. For agents talking to the HCS server, we should still have a way to get tokens so that the user credentials don't need to be stored in the agent. > If we decide that there will be no Accounts 3.x, then Command Gateway > (being quite small and closely related to Bus) can move to Commons git > repository. Command Gateway does not need to depend on Inventory since > Commons 0.6.0.Final. The change was recently merged to CmdGW. Sounds good to me. From auszon3 at gmail.com Mon Apr 25 11:57:41 2016 From: auszon3 at gmail.com (Austin Kuo) Date: Mon, 25 Apr 2016 15:57:41 +0000 Subject: [Hawkular-dev] GSoC 2016 Student Introduction : Austin Message-ID: Hi all, This is Austin Kuo I'm a senior CS student in NTHU, Taiwan. I'll work on : Hawkular-agent For Vert.x. I'm so happy to be selected and looking forward to working with you guys! Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/55e8d5bd/attachment.html From mithomps at redhat.com Mon Apr 25 12:33:10 2016 From: mithomps at redhat.com (mike thompson) Date: Mon, 25 Apr 2016 09:33:10 -0700 Subject: [Hawkular-dev] Please welcome our GSoC students Anuj and Austin In-Reply-To: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> References: <2074A5DB-EE83-4417-A0D3-263DF922329A@redhat.com> Message-ID: <09FFA7B3-8593-49BE-92C4-39FEF4702FD6@redhat.com> Welcome Guys! Can?t wait to see the new ?Hotness? each of you will bring to Hawkular! ? Mike > On 25 Apr 2016, at 00:25, Heiko W.Rupp wrote: > > Hello, > > please welcome Anuj and Austin as this years students on Hawkular. > > Anuj will work on the Android Client, where he has already done a lot of > work. > Austin will work on the agent for Vert.x to support inventory (sync) etc > and to > sort of bring it on par with the WildFly agent. > > Austin and Anuj are 2 out of 10 students for jboss.org this year. > You can read more here: > http://pilhuhn.blogspot.de/2016/04/welcome-gsoc-2016-students-to-jboss.html > > Heiko > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From snegrea at redhat.com Mon Apr 25 17:07:02 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Mon, 25 Apr 2016 16:07:02 -0500 Subject: [Hawkular-dev] Hawkular.org - Top Menu Message-ID: Hello Everybody, I sent last Friday a Hawkular.org PR with a small change, the top menu now includes Metrics as a distinct entry. Here is the PR https://github.com/hawkular/hawkular.github.io/pull/163 . The rationale for this change is to help current and potential users find information about Metrics faster. John Sanda rewrote the entire user guide (very expansive now) and we also have a few more ideas to create more content. If there are no major objections, I would like to have the PR merged this week prior to the release of Metrics 0.16.0. But this also points to a different issue. I do not think the current top menu is adequate for the direction of Hawkular. Adding Metrics to the top menu is the first step, but the entire menu needs to be overhauled. Any thoughts? Thank you, Stefan Negrea Software Engineer -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160425/e0a806f5/attachment.html From theute at redhat.com Tue Apr 26 01:42:15 2016 From: theute at redhat.com (Thomas Heute) Date: Tue, 26 Apr 2016 07:42:15 +0200 Subject: [Hawkular-dev] Hawkular.org - Top Menu In-Reply-To: References: Message-ID: Agreed, several sections need to be revisited as well On Mon, Apr 25, 2016 at 11:07 PM, Stefan Negrea wrote: > Hello Everybody, > > I sent last Friday a Hawkular.org PR with a small change, the top menu now > includes Metrics as a distinct entry. Here is the PR > https://github.com/hawkular/hawkular.github.io/pull/163 . The rationale > for this change is to help current and potential users find information > about Metrics faster. John Sanda rewrote the entire user guide (very > expansive now) and we also have a few more ideas to create more content. If > there are no major objections, I would like to have the PR merged this week > prior to the release of Metrics 0.16.0. > > > But this also points to a different issue. I do not think the current top > menu is adequate for the direction of Hawkular. Adding Metrics to the top > menu is the first step, but the entire menu needs to be overhauled. Any > thoughts? > > > > Thank you, > Stefan Negrea > > Software Engineer > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160426/bb3ea798/attachment.html From hrupp at redhat.com Tue Apr 26 02:17:10 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 26 Apr 2016 08:17:10 +0200 Subject: [Hawkular-dev] Hawkular.org - Top Menu In-Reply-To: References: Message-ID: <4D33C3F6-F014-4CDA-934F-017046BC25D8@redhat.com> On 25 Apr 2016, at 23:07, Stefan Negrea wrote: > no major objections, I would like to have the PR merged this week prior to > the release of Metrics 0.16.0. +1 And I agree that hawkular.org will need a lot more work going forward. From hrupp at redhat.com Tue Apr 26 03:47:29 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Tue, 26 Apr 2016 09:47:29 +0200 Subject: [Hawkular-dev] Tuesday all-team cancelled Message-ID: <419BBE26-EDAD-4BCA-A5B6-B232B22CE398@redhat.com> Hey, I have cancelled the Tuesday all-team meeting (the 9:30 EST one) going forward. This format was less of a success lately and going forward we will schedule individual meetings for specific topics. Heiko From tsegismo at redhat.com Tue Apr 26 03:54:51 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 26 Apr 2016 09:54:51 +0200 Subject: [Hawkular-dev] GSoC 2016 Student Introduction : Austin In-Reply-To: References: Message-ID: Glad to have you on board! Guys, as you can guess, I will mentor Austin for his time with us. His main goal is to make Vert.x report resources in Inventory. This would open the door to better integration with the rest of Hawkular family (Metrics explorer to start with). In the future, we may also support Vert.x in the middleware provider for ManageIQ. 2016-04-25 17:57 GMT+02:00 Austin Kuo : > Hi all, > This is Austin Kuo > I'm a senior CS student in NTHU, Taiwan. > I'll work on : Hawkular-agent For Vert.x. > I'm so happy to be selected and looking forward to working with you guys! > > Austin > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160426/d0f327a7/attachment.html From snegrea at redhat.com Tue Apr 26 11:57:57 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Tue, 26 Apr 2016 10:57:57 -0500 Subject: [Hawkular-dev] Hawkular Java Client Message-ID: Hello Everybody, On Monday we migrated the Hawkular Java Client GitHub repository from the QE organization to the Hawkular proper organization. The existing Hawkular repository was empty so we purged it and migrated the Hawkular-QE counterpart as is. The project is currently maintained by Jeeva Kandasamy and was started by Viet Nguyen, both from the Hawkular QA team. Please send a big thank you to both of them for their amazing work. The goal of the project is not only to have a ready-to-use Java client but also showcase how various Hawkular APIs can be used. Going forward Jeeva will be an active core contributor but the project is looking for more help. If you are interested in helping please reply to this thread or contact me directly. Github URL: https://github.com/hawkular/hawkular-client-java Thank you, Stefan Negrea Software Engineer -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160426/1864b5bc/attachment.html From tsegismo at redhat.com Tue Apr 26 12:50:43 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Tue, 26 Apr 2016 18:50:43 +0200 Subject: [Hawkular-dev] Hawkular Java Client In-Reply-To: References: Message-ID: Awesome! Thank you guys! Le 26 avr. 2016 5:58 PM, "Stefan Negrea" a ?crit : > Hello Everybody, > > On Monday we migrated the Hawkular Java Client GitHub repository from the > QE organization to the Hawkular proper organization. The existing Hawkular > repository was empty so we purged it and migrated the Hawkular-QE > counterpart as is. The project is currently maintained by Jeeva Kandasamy > and was started by Viet Nguyen, both from the Hawkular QA team. Please send > a big thank you to both of them for their amazing work. > > The goal of the project is not only to have a ready-to-use Java client but > also showcase how various Hawkular APIs can be used. > > Going forward Jeeva will be an active core contributor but the project is > looking for more help. If you are interested in helping please reply to > this thread or contact me directly. > > > Github URL: > https://github.com/hawkular/hawkular-client-java > > > Thank you, > Stefan Negrea > > Software Engineer > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160426/39639e2f/attachment.html From theute at redhat.com Tue Apr 26 12:56:49 2016 From: theute at redhat.com (Thomas Heute) Date: Tue, 26 Apr 2016 18:56:49 +0200 Subject: [Hawkular-dev] Hawkular Java Client In-Reply-To: References: Message-ID: Very nice ! Thanks ! On Tue, Apr 26, 2016 at 5:57 PM, Stefan Negrea wrote: > Hello Everybody, > > On Monday we migrated the Hawkular Java Client GitHub repository from the > QE organization to the Hawkular proper organization. The existing Hawkular > repository was empty so we purged it and migrated the Hawkular-QE > counterpart as is. The project is currently maintained by Jeeva Kandasamy > and was started by Viet Nguyen, both from the Hawkular QA team. Please send > a big thank you to both of them for their amazing work. > > The goal of the project is not only to have a ready-to-use Java client but > also showcase how various Hawkular APIs can be used. > > Going forward Jeeva will be an active core contributor but the project is > looking for more help. If you are interested in helping please reply to > this thread or contact me directly. > > > Github URL: > https://github.com/hawkular/hawkular-client-java > > > Thank you, > Stefan Negrea > > Software Engineer > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160426/f264ae36/attachment.html From lkrejci at redhat.com Wed Apr 27 05:26:29 2016 From: lkrejci at redhat.com (Lukas Krejci) Date: Wed, 27 Apr 2016 11:26:29 +0200 Subject: [Hawkular-dev] Hawkular Inventory 0.15.0.Final Released Message-ID: <6118236.TtMVctzhht@localhost.localdomain> Hi all, I'm happy to announce the release of Hawkular Inventory 0.15.0.Final. This release was brewing for a long time and brings 1 important and big feature - inventory synchronization for feeds. To ensure only the minimal amount of work is done when syncing, feeds, resource and metric types as well as resources and metrics have now an associated "identity hash", which is a Merkle tree hash of the entity's ID, important data (depends on entity type) and the hashes of its children. This way one can quickly check if a feed reported any new changes since the last time - just compare its identity hash with the last value known to you. You can now describe the structure of the inventory a feed wants to report locally on the feed and report it back to inventory server - it will ensure that all creates, updates and deletes are applied so that inventory reflects what the feed "sees". Apart from this there's been a handful of bugfixes and improvements and updates to REST documentation (that don't seem to be reflected on hawkular.org yet). In the future releases, we plan to refine the inventory sync further, restructure the REST API to get rid of the potential ambiguities and many other things. Special thanks go out to Justine Tunney, Heiko Rupp, Jirka Kremser, Pavol Loffay and Peter Palaga for their contributions to the release. With kind regards, -- Lukas Krejci From theute at redhat.com Wed Apr 27 07:49:40 2016 From: theute at redhat.com (Thomas Heute) Date: Wed, 27 Apr 2016 13:49:40 +0200 Subject: [Hawkular-dev] Hawkular Inventory 0.15.0.Final Released In-Reply-To: <6118236.TtMVctzhht@localhost.localdomain> References: <6118236.TtMVctzhht@localhost.localdomain> Message-ID: Congrats ! On Wed, Apr 27, 2016 at 11:26 AM, Lukas Krejci wrote: > Hi all, > > I'm happy to announce the release of Hawkular Inventory 0.15.0.Final. > > This release was brewing for a long time and brings 1 important and big > feature - inventory synchronization for feeds. > > To ensure only the minimal amount of work is done when syncing, feeds, > resource and metric types as well as resources and metrics have now an > associated "identity hash", which is a Merkle tree hash of the entity's ID, > important data (depends on entity type) and the hashes of its children. > This > way one can quickly check if a feed reported any new changes since the last > time - just compare its identity hash with the last value known to you. > > You can now describe the structure of the inventory a feed wants to report > locally on the feed and report it back to inventory server - it will ensure > that all creates, updates and deletes are applied so that inventory > reflects > what the feed "sees". > > Apart from this there's been a handful of bugfixes and improvements and > updates to REST documentation (that don't seem to be reflected on > hawkular.org > yet). > > In the future releases, we plan to refine the inventory sync further, > restructure the REST API to get rid of the potential ambiguities and many > other things. > > Special thanks go out to Justine Tunney, Heiko Rupp, Jirka Kremser, Pavol > Loffay and Peter Palaga for their contributions to the release. > > With kind regards, > > -- > > Lukas Krejci > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160427/ec0d5a67/attachment-0001.html From hrupp at redhat.com Wed Apr 27 09:34:07 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Wed, 27 Apr 2016 15:34:07 +0200 Subject: [Hawkular-dev] Hawkular Inventory 0.15.0.Final Released In-Reply-To: <6118236.TtMVctzhht@localhost.localdomain> References: <6118236.TtMVctzhht@localhost.localdomain> Message-ID: <6BB0D04B-D08F-4A96-BC12-CD43DDDE2E09@redhat.com> Congrats, that is huge! From tsegismo at redhat.com Wed Apr 27 10:44:03 2016 From: tsegismo at redhat.com (Thomas Segismont) Date: Wed, 27 Apr 2016 16:44:03 +0200 Subject: [Hawkular-dev] Hawkular Inventory 0.15.0.Final Released In-Reply-To: <6118236.TtMVctzhht@localhost.localdomain> References: <6118236.TtMVctzhht@localhost.localdomain> Message-ID: Congrats to the team! 2016-04-27 11:26 GMT+02:00 Lukas Krejci : > Hi all, > > I'm happy to announce the release of Hawkular Inventory 0.15.0.Final. > > This release was brewing for a long time and brings 1 important and big > feature - inventory synchronization for feeds. > > To ensure only the minimal amount of work is done when syncing, feeds, > resource and metric types as well as resources and metrics have now an > associated "identity hash", which is a Merkle tree hash of the entity's ID, > important data (depends on entity type) and the hashes of its children. > This > way one can quickly check if a feed reported any new changes since the last > time - just compare its identity hash with the last value known to you. > > You can now describe the structure of the inventory a feed wants to report > locally on the feed and report it back to inventory server - it will ensure > that all creates, updates and deletes are applied so that inventory > reflects > what the feed "sees". > > Apart from this there's been a handful of bugfixes and improvements and > updates to REST documentation (that don't seem to be reflected on > hawkular.org > yet). > > In the future releases, we plan to refine the inventory sync further, > restructure the REST API to get rid of the potential ambiguities and many > other things. > > Special thanks go out to Justine Tunney, Heiko Rupp, Jirka Kremser, Pavol > Loffay and Peter Palaga for their contributions to the release. > > With kind regards, > > -- > > Lukas Krejci > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -- Thomas Segismont JBoss ON Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160427/9921bee2/attachment.html From jpkroehling at redhat.com Thu Apr 28 08:43:29 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Thu, 28 Apr 2016 14:43:29 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: References: Message-ID: <15139e2e-d4cf-5a05-681b-7f10d1353c61@redhat.com> On 22.04.2016 18:35, Stefan Negrea wrote: > *1. Hawkular - Community Distribution* > > The current Hawkular repository will be adjusted to reflect this new > mission; the current UI will be moved to a separate repository. This > package builds on Hawkular CoreServices as described next. When will this split happen? I'm almost done in removing the Keycloak dependency from Accounts, so, KC has to be added as dependency directly on this distribution. Who will be doing the packaging for this community distribution? - Juca. From gbrown at redhat.com Thu Apr 28 09:18:06 2016 From: gbrown at redhat.com (Gary Brown) Date: Thu, 28 Apr 2016 09:18:06 -0400 (EDT) Subject: [Hawkular-dev] UI build problem on windows In-Reply-To: <970050696.43602434.1461849313632.JavaMail.zimbra@redhat.com> Message-ID: <527179702.43603054.1461849486457.JavaMail.zimbra@redhat.com> Hi Someone has reported an issue building the BTM UI on windows: [INFO] > bufferutil at 1.2.1 install D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\h awkular-btm-master\hawkular-btm-master\ui\target\gulp-build\node_modules\gulp-pr otractor\node_modules\protractor\node_modules\selenium-webdriver\node_modules\ws \node_modules\bufferutil [INFO] > node-gyp rebuild [INFO] [INFO] [INFO] D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\hawkular-btm-master\hawkular -btm-master\ui\target\gulp-build\node_modules\gulp-protractor\node_modules\protr actor\node_modules\selenium-webdriver\node_modules\ws\node_modules\bufferutil>if not defined npm_config_node_gyp (node "D:\Hawkular_SetUp_New\Latest_Hawkular\gb rown\hawkular-btm-master\hawkular-btm-master\ui\target\gulp-build\node\node_modu les\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node rebuild ) [ERROR] gyp ERR! configure error [ERROR] gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable. [ERROR] gyp ERR! stack at failNoPython (D:\Hawkular_SetUp_New\Latest_Hawkula r\gbrown\hawkular-btm-master\hawkular-btm-master\ui\target\gulp-build\node\node_ modules\npm\node_modules\node-gyp\lib\configure.js:116:14) [ERROR] gyp ERR! stack at D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\hawku lar-btm-master\hawkular-btm-master\ui\target\gulp-build\node\node_modules\npm\no de_modules\node-gyp\lib\configure.js:71:11 [ERROR] gyp ERR! stack at FSReqWrap.oncomplete (fs.js:82:15) [ERROR] gyp ERR! System Windows_NT 6.1.7601 [ERROR] gyp ERR! command "D:\\Hawkular_SetUp_New\\Latest_Hawkular\\gbrown\\hawku lar-btm-master\\hawkular-btm-master\\ui\\target\\gulp-build\\node\\node.exe" "D: \\Hawkular_SetUp_New\\Latest_Hawkular\\gbrown\\hawkular-btm-master\\hawkular-btm -master\\ui\\target\\gulp-build\\node\\node_modules\\npm\\node_modules\\node-gyp \\bin\\node-gyp.js" "rebuild" [ERROR] gyp ERR! cwd D:\Hawkular_SetUp_New\Latest_Hawkular\gbrown\hawkular-btm-m aster\hawkular-btm-master\ui\target\gulp-build\node_modules\gulp-protractor\node _modules\protractor\node_modules\selenium-webdriver\node_modules\ws\node_modules \bufferutil [ERROR] gyp ERR! node -v v4.2.2 [ERROR] gyp ERR! node-gyp -v v3.0.3 [ERROR] gyp ERR! not ok Seems like a known issue (e.g. https://github.com/niallobrien/generator-gulp-foundation/issues/13) with some possible workarounds, by installing various other components, but was just wondering if anyone knows of a better/simple workaround? Not sure if we can remove the use of node-gyp as I guess it is used by the frontend-maven-plugin? Regards Gary From jshaughn at redhat.com Thu Apr 28 14:06:09 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Thu, 28 Apr 2016 14:06:09 -0400 Subject: [Hawkular-dev] communty distribution question Message-ID: <57225111.7070404@redhat.com> This relates to the "[Hawkular-dev] Future Packaging of Hawkular" thread. I started this thread to discuss a specific scenario that I don't know how to handle. In addition to the components listed in that e-mail, there are a few other things in the Hawkular repo that weren't mentioned. For example, we have pinger and we have the inventory-event bus listener. I point out those two because I think they now exist solely to support the community UI. URL monitoring and Alert Center do not carry over to MIQ. With respect to the alert center we have out-of-box trigger definitions that are currently still active and are defined specifically to support the UI Alert center in Hawkular classic. And so we have a situation where code I would otherwise remove may need to exist to support the UI in the community distribution. Do we really want to keep a community-edition UI and the server-code that *only* supports what it does? If we remove it it will severely cripple the UI. But I fear that the UI will anyway quickly deteriorate unless we make concerted efforts to maintain compatible server code. Even today I think the UI is starting to falter. I'm not sure but when I tried to use the master version today to perform a deployment, it just hung. So in line with Juca's questioning, what sort of effort do we put into the community UI? A lot, and keep it working as best as we can and ensure we mark server code that is relevant to community UI support? None, and just make the server code be the headless provider it needs to be to support MIQ, or something in the middle (which usually ends up being a waste of time). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160428/58fc6969/attachment.html From jwon at redhat.com Thu Apr 28 22:12:16 2016 From: jwon at redhat.com (Jong Seok Won) Date: Fri, 29 Apr 2016 11:12:16 +0900 Subject: [Hawkular-dev] Is there the feature to provide timegap between ripple restart of WildFly(JBoss EAP) cluster in Hawkular? Message-ID: Hi all~, Is there the feature to provide timegap between ripple restart of WildFly(JBoss EAP) cluster in Hawkular? If there is no feature, I'm curious there is a roadmap for this feature. Actually, this kind of feature is useful in a production environment. I've already checked RHQ(JON) however there is no for ripple restart. RHQ(JON) provides below in restart operation page of Group menu. 1. "Execute: in the order specified below 2. "Halt on Failure?" Thanks in advance. -- Kindly Regards, Ted Won -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160429/ebcf7870/attachment.html From ppalaga at redhat.com Fri Apr 29 03:40:31 2016 From: ppalaga at redhat.com (Peter Palaga) Date: Fri, 29 Apr 2016 09:40:31 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <15139e2e-d4cf-5a05-681b-7f10d1353c61@redhat.com> References: <15139e2e-d4cf-5a05-681b-7f10d1353c61@redhat.com> Message-ID: <57230FEF.5030206@redhat.com> Hi Juca, inline... On 2016-04-28 14:43, Juraci Paix?o Kr?hling wrote: > On 22.04.2016 18:35, Stefan Negrea wrote: >> *1. Hawkular - Community Distribution* >> >> The current Hawkular repository will be adjusted to reflect this new >> mission; the current UI will be moved to a separate repository. This >> package builds on Hawkular CoreServices as described next. > > When will this split happen? I'm almost done in removing the Keycloak > dependency from Accounts, so, KC has to be added as dependency directly > on this distribution. > > Who will be doing the packaging for this community distribution? I the current proposal (based on this thread and further discussion with Stefan) I stared to work towards is the following: (1) Hawkular Community Distribution will live in https://github.com/hawkular/hawkular (2) Hawkular Core Services will live in https://github.com/hawkular/hawkular-core-services https://issues.jboss.org/browse/HAWKULAR-1069 (3) hawkular-core-services will depend on Alerts, Command Gateway, Inventory, Metrics and Commons. (4) Alerts, Command Gateway, Inventory and Metrics will be made independent of Accounts. hawkular-core-services will thus depend neither on Accounts or Keycloak. So far, I created a Jira only for Command Gateway https://issues.jboss.org/browse/HAWKULAR-1070 (5) Hawkular Community Distribution will either depend on (a) hawkular-core-services or (b) directly on Alerts, Command Gateway, Inventory and Metrics. I tend to prefer (a) because it could save some duplicate itesting that could happen in core-services only rather than in both core-services and hawkular main. @Juca: I hope the above is in accordance with how we (Peter and Juca) spoke about the future of Accounts. Please protest loudly if I misunderstood something. https://github.com/hawkular/hawkular is thus the place where the dependency on Accounts/KC should come in. I think you (Juca) are free to hack towards that. Nevertheless, having Accounts-free Alerts, Command Gateway, Inventory and Metrics is a prerequisite for that. Thanks, Peter From jpkroehling at redhat.com Fri Apr 29 04:01:24 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 29 Apr 2016 10:01:24 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <57230FEF.5030206@redhat.com> References: <15139e2e-d4cf-5a05-681b-7f10d1353c61@redhat.com> <57230FEF.5030206@redhat.com> Message-ID: On 29.04.2016 09:40, Peter Palaga wrote: > @Juca: I hope the above is in accordance with how we (Peter and Juca) > spoke about the future of Accounts. Please protest loudly if I > misunderstood something. That sounds perfectly aligned with my understanding. > https://github.com/hawkular/hawkular is thus the place where the > dependency on Accounts/KC should come in. I think you (Juca) are free to > hack towards that. Nevertheless, having Accounts-free Alerts, Command > Gateway, Inventory and Metrics is a prerequisite for that. Alright, so, we can perhaps split the work on making the components independent of Accounts. I don't think there's much to be done by me at the moment on hawkular/hawkular, as it will be changed a *lot* on the next days. I'd rather have hawkular/hawkular-core-services ready before I start messing with hawkular/hawkular. - Juca. From hrupp at redhat.com Fri Apr 29 04:28:43 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 10:28:43 +0200 Subject: [Hawkular-dev] Future Packaging of Hawkular In-Reply-To: <57230FEF.5030206@redhat.com> References: <15139e2e-d4cf-5a05-681b-7f10d1353c61@redhat.com> <57230FEF.5030206@redhat.com> Message-ID: <9D158FD6-9665-4ED6-880E-5193C460F5EA@redhat.com> > (5) Hawkular Community Distribution will either depend on (a) > hawkular-core-services or (b) directly on Alerts, Command Gateway, Yes on a) From hrupp at redhat.com Fri Apr 29 04:31:03 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 10:31:03 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <57225111.7070404@redhat.com> References: <57225111.7070404@redhat.com> Message-ID: <7DB5D234-6E56-4C39-A63D-E9E1812A1A9D@redhat.com> On 28 Apr 2016, at 20:06, Jay Shaughnessy wrote: > This relates to the "[Hawkular-dev] Future Packaging of Hawkular" > thread. I started this thread to discuss a specific scenario that I > don't know how to handle. > > In addition to the components listed in that e-mail, there are a few > other things in the Hawkular repo that weren't mentioned. For > example, we have pinger and we have the inventory-event bus listener. > I point out those two because I think they now exist solely to support > the community UI. URL monitoring and Alert Center do not carry over > to MIQ. With respect to the alert center we have out-of-box trigger > definitions that are currently still active and are defined > specifically to support the UI Alert center in Hawkular classic. And > so we have a situation where code I would otherwise remove may need to > exist to support the UI in the community distribution. Do we really > want to keep a community-edition UI and the server-code that *only* > supports what it does? If we remove it it will severely cripple the > UI. But I fear that the UI will anyway quickly deteriorate unless we > make concerted efforts to maintain compatible server code. Even today > I think the UI is starting to falter. I'm not sure but when I tried > to use the master version today to perform a deployment, it just hung. > > So in line with Juca's questioning, what sort of effort do we put into > the community UI? A lot, and keep it working as best as we can and > ensure we mark server code that is relevant to community UI support? > None, and just make the server code be the headless provider it needs > to be to support MIQ, or something in the middle (which usually ends > up being a waste of time). > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill From gbrown at redhat.com Fri Apr 29 04:44:20 2016 From: gbrown at redhat.com (Gary Brown) Date: Fri, 29 Apr 2016 04:44:20 -0400 (EDT) Subject: [Hawkular-dev] communty distribution question In-Reply-To: <57225111.7070404@redhat.com> References: <57225111.7070404@redhat.com> Message-ID: <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> First of all, apologies if I am reiterating points already discussed, as I haven't had a chance to go through all emails in the thread as there was a lot to catch up on :) I agree that a middle solution is not really going to work, and could end up leaving community users with the impression that the project is not well supported. So the two options mentioned: 1) Fully supported community UI Issue here is that community users will be using capabilities, and a style of management UI, that may never be productised, so they may like the project but not be able to obtain a support subscription. 2) No community UI I think the problem we were looking to address here is making something easily consumable to help build the community. Now the decision has been made to unify management products, I think the community equivalent should be as close to the product as possible to better drive community participation to develop features that can move into the product. So, rather than having a single community UI/distribution, maybe we should focus on the separate deliverables we are looking to support, and see whether community support can be provided to make them easy to consume? So the community distributions could be: 1) The headless server with core services - so providing a community distribution of the server, with instructions on how to set up manageiq/provider. May not be straightforward initially, but this needs to be simplified for the benefit of both manageiq and other projects in Red Hat that are developing "providers". 2) Metrics (possibly with optional Alerts) server - provide a distribution and instructions on support with projects like grafana - so make it easy to get a complete solution up and running quickly 3) BTM - just provide a standalone community edition for now So at the moment I don't see a compelling reason for a fully integration community distribution with UI. Regards Gary ----- Original Message ----- > > This relates to the "[Hawkular-dev] Future Packaging of Hawkular" thread. I > started this thread to discuss a specific scenario that I don't know how to > handle. > > In addition to the components listed in that e-mail, there are a few other > things in the Hawkular repo that weren't mentioned. For example, we have > pinger and we have the inventory-event bus listener. I point out those two > because I think they now exist solely to support the community UI. URL > monitoring and Alert Center do not carry over to MIQ. With respect to the > alert center we have out-of-box trigger definitions that are currently still > active and are defined specifically to support the UI Alert center in > Hawkular classic. And so we have a situation where code I would otherwise > remove may need to exist to support the UI in the community distribution. Do > we really want to keep a community-edition UI and the server-code that > *only* supports what it does? If we remove it it will severely cripple the > UI. But I fear that the UI will anyway quickly deteriorate unless we make > concerted efforts to maintain compatible server code. Even today I think the > UI is starting to falter. I'm not sure but when I tried to use the master > version today to perform a deployment, it just hung. > > So in line with Juca's questioning, what sort of effort do we put into the > community UI? A lot, and keep it working as best as we can and ensure we > mark server code that is relevant to community UI support? None, and just > make the server code be the headless provider it needs to be to support MIQ, > or something in the middle (which usually ends up being a waste of time). > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jpkroehling at redhat.com Fri Apr 29 05:05:19 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 29 Apr 2016 11:05:19 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: On 29.04.2016 10:44, Gary Brown wrote: > So at the moment I don't see a compelling reason for a fully integration community distribution with UI. This is exactly what I was trying to find out when I asked the following on the "Future Packaging of Hawkular" thread: - what is the motivation for having the "Community Distribution"? I think I asked the same thing in so many different ways by now that I just ran out of words and gave up trying to get an answer... - Juca. From hrupp at redhat.com Fri Apr 29 05:41:37 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 11:41:37 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <57225111.7070404@redhat.com> References: <57225111.7070404@redhat.com> Message-ID: <382968BF-3D66-4C8B-886A-50E33FF018D0@redhat.com> On 28 Apr 2016, at 20:06, Jay Shaughnessy wrote: > falter. I'm not sure but when I tried to use the master version today > to perform a deployment, it just hung. Which incidentally is part of the UI that we will remove from the community distribution anyway. From hrupp at redhat.com Fri Apr 29 05:53:55 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 11:53:55 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: On 29 Apr 2016, at 10:44, Gary Brown wrote: > I think the problem we were looking to address here is making > something easily consumable to help build the community. Exactly. As a user I want to be able to get Hawkular going in less than 5 minutes to try it and see data on screen. > 1) The headless server with core services - so providing a community > distribution of the server, with instructions on how to set up > manageiq/provider. May not be straightforward initially, but this > needs to be simplified for the benefit of both manageiq and other > projects in Red Hat that are developing "providers". I do not believe that requiring MiQ to see some data of my standalone WildFly or IoT device(*) will improve Hawkular adoption. > So at the moment I don't see a compelling reason for a fully > integration community distribution with UI. To reiterate: we will not support the app server tab going forward, which may be have been the impression. *) where IoT devices may possibly never make it into MiQ From hrupp at redhat.com Fri Apr 29 06:00:55 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 12:00:55 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: On 29 Apr 2016, at 11:05, Juraci Paix?o Kr?hling wrote: > On 29.04.2016 10:44, Gary Brown wrote: >> So at the moment I don't see a compelling reason for a fully >> integration community distribution with UI. > > This is exactly what I was trying to find out when I asked the > following > on the "Future Packaging of Hawkular" thread: > > - what is the motivation for having the "Community Distribution"? As I said before: I believe that to be attractive for community, a product/project needs to pass the "I can get it initially going in 5 minutes" test. Which is/has been given before in all the Hawkular releases. And which will no longer be true for the hawkular-core-services, as the user needs to go through a number of additional steps to see something on screen: - install Cassandra - install some kind of UI (e.g. download + configure Grafana) - manually supply some data (curl, install wf+agent somewhere(*) I do not believe that "It was hard to create, so it should be hard to use" should be applied. Heiko *) this may still be provided inside core-services. But then as the Hawkular server is a black box, the detailed self-monitoring as we have today may change. From gbrown at redhat.com Fri Apr 29 06:11:21 2016 From: gbrown at redhat.com (Gary Brown) Date: Fri, 29 Apr 2016 06:11:21 -0400 (EDT) Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: <1182341849.43810686.1461924681734.JavaMail.zimbra@redhat.com> I agree, it must pass the 5 minute test, but think we need to address it by making the community version of MiQ+provider+hawkular headless server easier to install, otherwise what is going to encourage our community users to try it out, and contribute, to what will be the product? Regards Gary ----- Original Message ----- > On 29 Apr 2016, at 11:05, Juraci Paix?o Kr?hling wrote: > > > On 29.04.2016 10:44, Gary Brown wrote: > >> So at the moment I don't see a compelling reason for a fully > >> integration community distribution with UI. > > > > This is exactly what I was trying to find out when I asked the > > following > > on the "Future Packaging of Hawkular" thread: > > > > - what is the motivation for having the "Community Distribution"? > > As I said before: I believe that to be attractive for community, a > product/project > needs to pass the "I can get it initially going in 5 minutes" test. > Which is/has been given before in all the Hawkular releases. > And which will no longer be true for the hawkular-core-services, as the > user needs to go through a number of additional steps to see something > on screen: > - install Cassandra > - install some kind of UI (e.g. download + configure Grafana) > - manually supply some data (curl, install wf+agent somewhere(*) > > I do not believe that > > "It was hard to create, so it should be hard to use" > > should be applied. > > Heiko > > *) this may still be provided inside core-services. But then as > the Hawkular server is a black box, the detailed self-monitoring > as we have today may change. > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jpkroehling at redhat.com Fri Apr 29 07:41:19 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 29 Apr 2016 13:41:19 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: On 29.04.2016 12:00, Heiko W.Rupp wrote: > As I said before: I believe that to be attractive for community, a > product/project > needs to pass the "I can get it initially going in 5 minutes" test. From what I understand, we are maintaining a whole set of components, developing a web UI, building a separate assembler module, adding and integrating a big third-party component (Keycloak), just to make it easy for someone to use "hawkular-core-services" in a particular way. *This* is what is difficult for me to understand. I can't seem to accept that this 5 minutes test is a purpose, a "why". In the end, what people will see after this 5 minutes test is not what Hawkular is. > Which is/has been given before in all the Hawkular releases. > And which will no longer be true for the hawkular-core-services, as the > user needs to go through a number of additional steps to see something > on screen: > - install Cassandra > - install some kind of UI (e.g. download + configure Grafana) > - manually supply some data (curl, install wf+agent somewhere(*) All Hawkular releases so far had the purpose of being a monitoring platform. As a platform, our solution to the 5 minutes test was to add an embedded Cassandra and embedded Keycloak. That was our cost for the 5 minute test. Everything else was supposed to be part of the product: multi tenancy, BTM, Data Mining, charts and so on. To me, "hawkular-core-services" is essentially a data store. It is intended to be consumed by platforms "closer" to the end user, like OpenShift or MiQ. As such, I would expect as much of an UI to it as I would expect from, say, a PostgreSQL database server: I would certainly expect a way to manipulate and extract data, but I would *not* expect a full fledged user-friendly HTML5 snappy web UI. If our raison d'?tre is around core-services, then having this "bloated" distribution is doing a disfavor: it dilutes the focus we have, it prevents developers from using the time on our core business and confuses people who first got in contact with Hawkular. More important than passing the 5 minute test is to clearly answer "what is this for and what can it do for me". And *this* is what I still don't understand. > I do not believe that > > "It was hard to create, so it should be hard to use" > > should be applied. Nobody said that. - Juca. From jshaughn at redhat.com Fri Apr 29 08:10:37 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Fri, 29 Apr 2016 08:10:37 -0400 Subject: [Hawkular-dev] Is there the feature to provide timegap between ripple restart of WildFly(JBoss EAP) cluster in Hawkular? In-Reply-To: References: Message-ID: <57234F3D.6030200@redhat.com> That feature is not supported in Hawkular today. Perhaps it will be a future feature for the middleware management in ManageIQ, which the Hawkular project now supports. In RHQ you have found that for a group of servers you can execute operations in order, but as far as I know there is no interval that can be specified. That would have to be a feature request. Also, I'm fairly sure that there is no halt-on-failure option. To do what you like would probably require a custom CLI script in RHQ. I think it could be done that way. On 4/28/2016 10:12 PM, Jong Seok Won wrote: > Hi all~, > > Is there the feature to provide timegap between ripple restart of > WildFly(JBoss EAP) cluster in Hawkular? > If there is no feature, I'm curious there is a roadmap for this feature. > Actually, this kind of feature is useful in a production environment. > > I've already checked RHQ(JON) however there is no for ripple restart. > RHQ(JON) provides below in restart operation page of Group menu. > 1. "Execute: in the order specified below > 2. "Halt on Failure?" > > > Thanks in advance. > -- > > Kindly Regards, > > Ted Won > > > > > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160429/7767e689/attachment.html From jshaughn at redhat.com Fri Apr 29 08:46:59 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Fri, 29 Apr 2016 08:46:59 -0400 Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: <572357C3.8090704@redhat.com> > To reiterate: we will not support the app server tab going forward, > which may be have been the impression. Where is this initially stated, I must have missed that. Knowing what the community/demo UI feature-set would be very helpful because that will indicate which hawkular back-end code to maintain, even if it is not needed for the hawkular provider. From hrupp at redhat.com Fri Apr 29 08:49:53 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 14:49:53 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> Message-ID: <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> On 29 Apr 2016, at 13:41, Juraci Paix?o Kr?hling wrote: > In the end, what people will see after this 5 minutes test is not what > Hawkular is. So the clear suggestion and opinion here is to remove everything that is not absolutely required by the ManageIQ integration? -- Reg. Adresse: Red Hat GmbH, Technopark II, Haus C, Werner-von-Siemens-Ring 14, D-85630 Grasbrunn Handelsregister: Amtsgericht M?nchen HRB 153243 Gesch?ftsf?hrer: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill From hrupp at redhat.com Fri Apr 29 08:56:01 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 14:56:01 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> Message-ID: <2B996A81-BECD-4DD7-B479-0DDFB9904DE4@redhat.com> On 29 Apr 2016, at 14:49, Heiko W.Rupp wrote: > On 29 Apr 2016, at 13:41, Juraci Paix?o Kr?hling wrote: >> In the end, what people will see after this 5 minutes test is not what >> Hawkular is. > > So the clear suggestion and opinion here is to remove everything > that is not absolutely required by the ManageIQ integration? And going forward with this, as we only have 2 use cases going forward. - support ManageIQ - support Hawkular-Metrics Can we collapse the source into one or two source trees to save on all the integration / release / ... efforts? With the above there is no point in having any general purpose setup anymore. From jpkroehling at redhat.com Fri Apr 29 08:57:41 2016 From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=) Date: Fri, 29 Apr 2016 14:57:41 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> Message-ID: <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> On 29.04.2016 14:49, Heiko W.Rupp wrote: > On 29 Apr 2016, at 13:41, Juraci Paix?o Kr?hling wrote: >> In the end, what people will see after this 5 minutes test is not what >> Hawkular is. > > So the clear suggestion and opinion here is to remove everything > that is not absolutely required by the ManageIQ integration? Might be, I don't know. Here we cycle back to the question about the purpose. What's Hawkular's purpose? If our purpose is to "provide Middleware management to ManageIQ", then why not. There are so many interesting things to be done at Middleware management that it's a shame we'd spend so much time on other non-core stuff... - Juca. From jsanda at redhat.com Fri Apr 29 09:58:01 2016 From: jsanda at redhat.com (John Sanda) Date: Fri, 29 Apr 2016 09:58:01 -0400 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> Message-ID: <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> > On Apr 29, 2016, at 8:57 AM, Juraci Paix?o Kr?hling wrote: > > On 29.04.2016 14:49, Heiko W.Rupp wrote: >> On 29 Apr 2016, at 13:41, Juraci Paix?o Kr?hling wrote: >>> In the end, what people will see after this 5 minutes test is not what >>> Hawkular is. >> >> So the clear suggestion and opinion here is to remove everything >> that is not absolutely required by the ManageIQ integration? > > Might be, I don't know. Here we cycle back to the question about the > purpose. What's Hawkular's purpose? If our purpose is to "provide > Middleware management to ManageIQ", then why not. There are so many > interesting things to be done at Middleware management that it's a shame > we'd spend so much time on other non-core stuff? > Before making a decision one way or another, I agree with Juca that we need to answer the question, what is the purpose of Hawkular now? Do we really have legitimate reasons to invest the time, effort, and resources not needed for the ManageIQ integration? Maybe we do. I don?t know. And personally, I don?t think simply growing/supporting the community is very good motivation. From snegrea at redhat.com Fri Apr 29 09:58:42 2016 From: snegrea at redhat.com (Stefan Negrea) Date: Fri, 29 Apr 2016 08:58:42 -0500 Subject: [Hawkular-dev] Hawkular-Archive - New Org Message-ID: Hello, I would like to propose a new organization in Github to host projects that no longer receive development. "Hawkular-Archive" seems a good name for this purpose. The goal is to keep the main Hawkular org focused on important projects and at the same time preserve work that was already done but away from the main organization. >From a quick look at the current organization here are two repositories that can be moved right away: hawkular-metrics-openshift (Openshit 2.x cartridge for Hawkular Metrics 0.2.7 or earlier) and hawkular-bus (code moved to another repo). Am I am sure we can find other repositories to move. In the long run we can develop some criteria for archiving projects but for now we can just do a one-time major cleanup. Any repositories that should be archived right away? Any other suggestions for a name? Any thoughts on the idea in general? Thank you, Stefan Negrea Software Engineer -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160429/21c07b39/attachment-0001.html From hrupp at redhat.com Fri Apr 29 10:48:38 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 16:48:38 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> Message-ID: On 29 Apr 2016, at 15:58, John Sanda wrote: > the ManageIQ integration? Maybe we do. I don?t know. And personally, > I don?t think simply growing/supporting the community is very good > motivation. You may have experienced in the past, that members of the community have helped tremendously with issues around Storage Nodes. For me such feedback is a big motivation. From gbrown at redhat.com Fri Apr 29 11:02:55 2016 From: gbrown at redhat.com (Gary Brown) Date: Fri, 29 Apr 2016 11:02:55 -0400 (EDT) Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> Message-ID: <1521566022.43907670.1461942175271.JavaMail.zimbra@redhat.com> The community is very important to us - as discussed on the recent call, it is essential to get bake time in the community before project's are productised. But this is the key point - the community should be using the upstream version of what will be productised - not just the backend with a completely different front end. The (MiQ MW) UI is also an important part of the upstream project(s) that needs bake time and community user feedback - so it is essential that community users are able to easily setup and use MiQ+provider+headless server. If we are saying the only reason to have our own community distribution with hawkular specific UI is for ease of use, then it implies we don't care how complex our product is to setup. I think this is the wrong approach, we should use this difficulty in the current setup to motivate us to fix it rather than trying to avoid the issue. Regards Gary ----- Original Message ----- > On 29 Apr 2016, at 15:58, John Sanda wrote: > > the ManageIQ integration? Maybe we do. I don?t know. And personally, > > I don?t think simply growing/supporting the community is very good > > motivation. > > You may have experienced in the past, that members of the community > have helped tremendously with issues around Storage Nodes. > > For me such feedback is a big motivation. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > From jdoyle at redhat.com Fri Apr 29 11:04:42 2016 From: jdoyle at redhat.com (John Doyle) Date: Fri, 29 Apr 2016 11:04:42 -0400 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> Message-ID: On Fri, Apr 29, 2016 at 9:58 AM, John Sanda wrote: > >> On Apr 29, 2016, at 8:57 AM, Juraci Paix?o Kr?hling wrote: >> >> On 29.04.2016 14:49, Heiko W.Rupp wrote: >>> On 29 Apr 2016, at 13:41, Juraci Paix?o Kr?hling wrote: >>>> In the end, what people will see after this 5 minutes test is not what >>>> Hawkular is. >>> >>> So the clear suggestion and opinion here is to remove everything >>> that is not absolutely required by the ManageIQ integration? >> >> Might be, I don't know. Here we cycle back to the question about the >> purpose. What's Hawkular's purpose? If our purpose is to "provide >> Middleware management to ManageIQ", then why not. There are so many >> interesting things to be done at Middleware management that it's a shame >> we'd spend so much time on other non-core stuff? >> > > Before making a decision one way or another, I agree with Juca that we need to answer the question, what is the purpose of Hawkular now? Do we really have legitimate reasons to invest the time, effort, and resources not needed for the ManageIQ integration? Maybe we do. I don?t know. And personally, I don?t think simply growing/supporting the community is very good motivation. It's hard to predict what the future will hold, but already we're providing middleware metrics to openshift, and will be providing MW mgmt to CF. There is also a shift happening with some of our long time customers to stop delivering packaged applications to their customers and transition to a SaaS model based upon subscriptions. If that is a trend, then hawkular, headless or not would be a valuable piece in our portfolio. I don't believe that "all management' will have to go through CF for all cases. > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev From hrupp at redhat.com Fri Apr 29 11:17:30 2016 From: hrupp at redhat.com (Heiko W.Rupp) Date: Fri, 29 Apr 2016 17:17:30 +0200 Subject: [Hawkular-dev] communty distribution question In-Reply-To: <572357C3.8090704@redhat.com> References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <572357C3.8090704@redhat.com> Message-ID: On 29 Apr 2016, at 14:46, Jay Shaughnessy wrote: > Where is this initially stated, I must have missed that. Knowing what > the community/demo UI feature-set would be very helpful because that > will indicate which hawkular back-end code to maintain, even if it is > not needed for the hawkular provider. It may not have been written down wrt the "app server" tab, but was at least said several times or to say it more positively: to keep the explorer so that people can browse through inventory and see metrics. The alert center is a mixed thing. I believe we should keep it. But in a more generic form - i.e. throw out the specific code related to app servers to be a counterpart to the explorer So more of a (simple) inventory/metrics dash/charting ui I don't think we need to necessarily keep the Pinger around From jsanda at redhat.com Fri Apr 29 12:14:06 2016 From: jsanda at redhat.com (John Sanda) Date: Fri, 29 Apr 2016 12:14:06 -0400 Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> Message-ID: > On Apr 29, 2016, at 10:48 AM, Heiko W.Rupp wrote: > > On 29 Apr 2016, at 15:58, John Sanda wrote: >> the ManageIQ integration? Maybe we do. I don?t know. And personally, >> I don?t think simply growing/supporting the community is very good >> motivation. > > You may have experienced in the past, that members of the community > have helped tremendously with issues around Storage Nodes. > > For me such feedback is a big motivation. Yes the feedback was invaluable, but note the comparison. Storage nodes and RHQ in general had well-defined priorities around it. The things being discussed here no long have well-defined priorities. If maintaining this stuff for the sake of growing the community is reason enough, then why aren?t more of the former core RHQ devs still actively contributing to RHQ? The reason is very simple. It is no longer a priority. From jsanda at redhat.com Fri Apr 29 12:18:21 2016 From: jsanda at redhat.com (John Sanda) Date: Fri, 29 Apr 2016 12:18:21 -0400 Subject: [Hawkular-dev] communty distribution question In-Reply-To: References: <57225111.7070404@redhat.com> <699051824.43797496.1461919460609.JavaMail.zimbra@redhat.com> <006EABD9-F693-4E78-9DCF-9BC85EA43372@redhat.com> <44f2137a-a03a-ff20-2b22-f504d62914aa@redhat.com> <1E7C14C9-28B3-4233-BC24-9C4E67059DF8@redhat.com> Message-ID: <40CF4503-39B4-4BDD-8055-78D3600035DD@redhat.com> > On Apr 29, 2016, at 11:04 AM, John Doyle wrote: > > On Fri, Apr 29, 2016 at 9:58 AM, John Sanda > wrote: >> >>> On Apr 29, 2016, at 8:57 AM, Juraci Paix?o Kr?hling wrote: >>> >>> On 29.04.2016 14:49, Heiko W.Rupp wrote: >>>> On 29 Apr 2016, at 13:41, Juraci Paix?o Kr?hling wrote: >>>>> In the end, what people will see after this 5 minutes test is not what >>>>> Hawkular is. >>>> >>>> So the clear suggestion and opinion here is to remove everything >>>> that is not absolutely required by the ManageIQ integration? >>> >>> Might be, I don't know. Here we cycle back to the question about the >>> purpose. What's Hawkular's purpose? If our purpose is to "provide >>> Middleware management to ManageIQ", then why not. There are so many >>> interesting things to be done at Middleware management that it's a shame >>> we'd spend so much time on other non-core stuff? >>> >> >> Before making a decision one way or another, I agree with Juca that we need to answer the question, what is the purpose of Hawkular now? Do we really have legitimate reasons to invest the time, effort, and resources not needed for the ManageIQ integration? Maybe we do. I don?t know. And personally, I don?t think simply growing/supporting the community is very good motivation. > > It's hard to predict what the future will hold, but already we're > providing middleware metrics to openshift, and will be providing MW > mgmt to CF. There is also a shift happening with some of our long time > customers to stop delivering packaged applications to their customers > and transition to a SaaS model based upon subscriptions. If that is a > trend, then hawkular, headless or not would be a valuable piece in our > portfolio. I don't believe that "all management' will have to go > through CF for all cases. Thanks for the insight. This is the kind of info I was looking for. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160429/cb5c8ca1/attachment-0001.html From mazz at redhat.com Fri Apr 29 12:42:41 2016 From: mazz at redhat.com (John Mazzitelli) Date: Fri, 29 Apr 2016 12:42:41 -0400 (EDT) Subject: [Hawkular-dev] Hawkular-Archive - New Org In-Reply-To: References: Message-ID: <566384428.1686940.1461948161763.JavaMail.zimbra@redhat.com> > I would like to propose a new organization in Github to host projects that no > longer receive development. "Hawkular-Archive" seems a good name for this > purpose. The goal is to keep the main Hawkular org focused on important > projects and at the same time preserve work that was already done but away > from the main organization. > > From a quick look at the current organization here are two repositories that > can be moved right away: hawkular-metrics-openshift (Openshit 2.x cartridge > for Hawkular Metrics 0.2.7 or earlier) and hawkular-bus (code moved to > another repo). Am I am sure we can find other repositories to move. > > In the long run we can develop some criteria for archiving projects but for > now we can just do a one-time major cleanup. > > Any repositories that should be archived right away? Any other suggestions > for a name? Any thoughts on the idea in general? This should be archived - it is obsolete - this was what the hawkular-agent grew out of, but this wildfly-monitor is no longer needed. https://github.com/hawkular/wildfly-monitor From jshaughn at redhat.com Fri Apr 29 14:17:12 2016 From: jshaughn at redhat.com (Jay Shaughnessy) Date: Fri, 29 Apr 2016 14:17:12 -0400 Subject: [Hawkular-dev] Hawkular-Archive - New Org In-Reply-To: <566384428.1686940.1461948161763.JavaMail.zimbra@redhat.com> References: <566384428.1686940.1461948161763.JavaMail.zimbra@redhat.com> Message-ID: <5723A528.1090901@redhat.com> This seems like a good thing, a place for stuff to live, as an example or reference, or possibly to be revived later. Pinger and avail_creator may be good candidates as well. On 4/29/2016 12:42 PM, John Mazzitelli wrote: >> I would like to propose a new organization in Github to host projects that no >> longer receive development. "Hawkular-Archive" seems a good name for this >> purpose. The goal is to keep the main Hawkular org focused on important >> projects and at the same time preserve work that was already done but away >> from the main organization. >> >> From a quick look at the current organization here are two repositories that >> can be moved right away: hawkular-metrics-openshift (Openshit 2.x cartridge >> for Hawkular Metrics 0.2.7 or earlier) and hawkular-bus (code moved to >> another repo). Am I am sure we can find other repositories to move. >> >> In the long run we can develop some criteria for archiving projects but for >> now we can just do a one-time major cleanup. >> >> Any repositories that should be archived right away? Any other suggestions >> for a name? Any thoughts on the idea in general? > This should be archived - it is obsolete - this was what the hawkular-agent grew out of, but this wildfly-monitor is no longer needed. > > https://github.com/hawkular/wildfly-monitor > _______________________________________________ > hawkular-dev mailing list > hawkular-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160429/4404ee7d/attachment.html