From garethahealy at gmail.com  Sun Jan  1 18:15:43 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Sun, 1 Jan 2017 23:15:43 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
Message-ID: <CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>

Have raised:

   - https://issues.jboss.org/browse/HWKMETRICS-563

Will work on the grafana datasource support in next few days.

Cheers.

On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli <mazz at redhat.com> wrote:

> Thanks.
>
> You should submit a feature request in the H-Metrics JIRA and link that
> JIRA to a PR for the H-Metrics team to peer review:
>
> https://issues.jboss.org/projects/HWKMETRICS
>
>
> ----- Original Message -----
> > I did some hacking (day 1 of Rx, so probably not the best "solution").
> But
> > works...
> >
> >    -
> >    https://github.com/garethahealy/hawkular-metrics/commit/
> 75c616f9be71a0b85ce5dee310c4dff828bb8f38
> >
> >
> > Sample output:
> >
> >    - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ece87096c
> 89
> >
> >
> > And some test cURL commands:
> >
> >    - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad7
> 88
> >
> >
> > Cheers.
> >
> > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli <mazz at redhat.com>
> wrote:
> >
> > > This would be a feature request on Hawkular-Metrics (if they don't do
> > > something like this already - I do not know. I will defer to the
> H-Metrics
> > > folks to talk about how they do querying off of tags).
> > >
> > > ----- Original Message -----
> > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
> > > single
> > > > metric with tagged datapoints, i.e.:
> > > >
> > > >
> > > >
> > > >
> > > > https://github.com/coreos/etcd/blob/master/
> Documentation/v2/metrics.md#
> > > http-requests
> > > >
> > > >
> > > >
> > > >
> > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
> > > [counter]
> > > > datapoints for metric named
> > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_
> received_total]:
> > > [
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
> > > > ]
> > > >
> > > > But when trying to view this via the grafana datasource, only 1
> metric
> > > and
> > > > the aggregated counts are shown. What i'd like to do is something
> like
> > > the
> > > > below:
> > > >
> > > >
> > > >
> > > > {
> > > > "start": 1482999755690,
> > > > "end": 1483000020093,
> > > > "order": "ASC",
> > > > "tags": "pod_namespace:etcd-testing",
> > > > "groupDatapointsByTagKey": "method"
> > > > }
> > > >
> > > > Search via tags or name (as-is) and group the datapoints by a tag
> key,
> > > which
> > > > would give you 3 lines, instead of 1.
> > > >
> > > > Does that sound possible?
> > > >
> > > > Cheers.
> > > >
> > > > _______________________________________________
> > > > hawkular-dev mailing list
> > > > hawkular-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170101/5cf31d83/attachment.html 

From jtakvori at redhat.com  Mon Jan  2 04:42:15 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Mon, 2 Jan 2017 10:42:15 +0100
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
Message-ID: <CAJo5TF=JPkox+UniUPKHM8nX67E7BZb0QTcB05GbDM2VxwF0OA@mail.gmail.com>

Hi,

Gareth, I see your PR is currently only for counters only, for the feature
to be complete it should also be implemented on gauges, availability and
strings.
And also, you will have the same issue if you try to use the "stats"
endpoints (in Grafana it is used for single stats panels), so here also it
should be implemented...

Which leads me to questioning: basically this feature is to "split" a time
series based on data points tags. Is it really something we want to
implement on the server side?
Maybe the problem is that this series is built as an aggregation of several
different "things", and should be split into several series at insertion?

On Mon, Jan 2, 2017 at 12:15 AM, Gareth Healy <garethahealy at gmail.com>
wrote:

> Have raised:
>
>    - https://issues.jboss.org/browse/HWKMETRICS-563
>
> Will work on the grafana datasource support in next few days.
>
> Cheers.
>
> On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli <mazz at redhat.com> wrote:
>
>> Thanks.
>>
>> You should submit a feature request in the H-Metrics JIRA and link that
>> JIRA to a PR for the H-Metrics team to peer review:
>>
>> https://issues.jboss.org/projects/HWKMETRICS
>>
>>
>> ----- Original Message -----
>> > I did some hacking (day 1 of Rx, so probably not the best "solution").
>> But
>> > works...
>> >
>> >    -
>> >    https://github.com/garethahealy/hawkular-metrics/commit/75c6
>> 16f9be71a0b85ce5dee310c4dff828bb8f38
>> >
>> >
>> > Sample output:
>> >
>> >    - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ec
>> e87096c89
>> >
>> >
>> > And some test cURL commands:
>> >
>> >    - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af
>> 317fad788
>> >
>> >
>> > Cheers.
>> >
>> > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli <mazz at redhat.com>
>> wrote:
>> >
>> > > This would be a feature request on Hawkular-Metrics (if they don't do
>> > > something like this already - I do not know. I will defer to the
>> H-Metrics
>> > > folks to talk about how they do querying off of tags).
>> > >
>> > > ----- Original Message -----
>> > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
>> > > single
>> > > > metric with tagged datapoints, i.e.:
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > https://github.com/coreos/etcd/blob/master/Documentation/v2/
>> metrics.md#
>> > > http-requests
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
>> > > [counter]
>> > > > datapoints for metric named
>> > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_r
>> eceived_total]:
>> > > [
>> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
>> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
>> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
>> > > > ]
>> > > >
>> > > > But when trying to view this via the grafana datasource, only 1
>> metric
>> > > and
>> > > > the aggregated counts are shown. What i'd like to do is something
>> like
>> > > the
>> > > > below:
>> > > >
>> > > >
>> > > >
>> > > > {
>> > > > "start": 1482999755690,
>> > > > "end": 1483000020093,
>> > > > "order": "ASC",
>> > > > "tags": "pod_namespace:etcd-testing",
>> > > > "groupDatapointsByTagKey": "method"
>> > > > }
>> > > >
>> > > > Search via tags or name (as-is) and group the datapoints by a tag
>> key,
>> > > which
>> > > > would give you 3 lines, instead of 1.
>> > > >
>> > > > Does that sound possible?
>> > > >
>> > > > Cheers.
>> > > >
>> > > > _______________________________________________
>> > > > hawkular-dev mailing list
>> > > > hawkular-dev at lists.jboss.org
>> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
>> > > >
>> > > _______________________________________________
>> > > hawkular-dev mailing list
>> > > hawkular-dev at lists.jboss.org
>> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
>> > >
>> >
>>
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170102/02fb6948/attachment.html 

From tcunning at redhat.com  Tue Jan  3 09:20:04 2017
From: tcunning at redhat.com (Thomas Cunningham)
Date: Tue, 3 Jan 2017 09:20:04 -0500
Subject: [Hawkular-dev] HawkFX on Mac OS X issues?
In-Reply-To: <17CA7990-BA77-4214-9185-173A94863F4E@redhat.com>
References: <CALAW+WLi28xfZeBQyfxqeh=vi_TWbHoh=kbUVVkDf8G_HNpUWg@mail.gmail.com>
	<17CA7990-BA77-4214-9185-173A94863F4E@redhat.com>
Message-ID: <CALAW+WKnB=uV5n8DE4GRQZoW7NkF695A+1LktNKjZFhHzX4U8A@mail.gmail.com>

Heiko,

Switching the flags or running bundle exec jruby hawkfx.rb seems to get me
farther, but I see the following with either.   Is there anything special I
should do during jruby install?

lilguylaptop:hawkfx cunningt$ jruby -G -S hawkfx.rb

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
warning: `<<' after local variable or literal is interpreted as binary
operator

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
warning: even though it seems like here document

Exception running Application:

#<NameError: uninitialized constant ValueElement::Logging

Did you mean?  Logger>

org/jruby/RubyModule.java:3346:in `const_missing'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/value_elts.rb:149:in
`processValue'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/value_elts.rb:67:in
`processStartElement'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/jrubyfx-fxmlloader.rb:440:in
`processStartElement'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/jrubyfx-fxmlloader.rb:351:in
`load'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.2.0-java/lib/jrubyfx/controller.rb:129:in
`load_into'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.2.0-java/lib/jrubyfx/core_ext/stage.rb:82:in
`fxml'

hawkfx.rb:18:in `block in start'

org/jruby/RubyBasicObject.java:1667:in `instance_eval'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.2.0-java/lib/jrubyfx/module.rb:49:in
`with'

hawkfx.rb:17:in `start'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.2.0-java/lib/jrubyfx/java_fx_impl.rb:119:in
`block in launch_app_after_platform'




On Fri, Dec 23, 2016 at 3:29 AM, Heiko W.Rupp <hrupp at redhat.com> wrote:

> On 22 Dec 2016, at 21:52, Thomas Cunningham wrote:
>
> > Hi,
> >
> > I'm trying to use HawkFX on Mac OS X - I have it working great on Fedora
> > but would like to set it up on my Mac OS X box as well.    I've got no
> > experience with ruby or jruby so I may be doing something wrong here in
> > installation, because I'm seeing the following :
> >
> > lilguylaptop:hawkfx cunningt$ jruby -S -G hawkfx.rb
>
> Can you try to run -G -S hawkfx.rb (reverse the flags)?
>
> I am running on OS/X myself and it works (with the reversed
> flags).
>
> Otherwise "bundle exec jruby hawkfx.rb" should
> also work.
>
> Note that Jruby 9.1.6.0 has an issue, but you use
> 9.1.5.0 which is good.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170103/9f2e81a7/attachment-0001.html 

From hrupp at redhat.com  Tue Jan  3 09:44:47 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Tue, 03 Jan 2017 15:44:47 +0100
Subject: [Hawkular-dev] HawkFX on Mac OS X issues?
In-Reply-To: <CALAW+WKnB=uV5n8DE4GRQZoW7NkF695A+1LktNKjZFhHzX4U8A@mail.gmail.com>
References: <CALAW+WLi28xfZeBQyfxqeh=vi_TWbHoh=kbUVVkDf8G_HNpUWg@mail.gmail.com>
	<17CA7990-BA77-4214-9185-173A94863F4E@redhat.com>
	<CALAW+WKnB=uV5n8DE4GRQZoW7NkF695A+1LktNKjZFhHzX4U8A@mail.gmail.com>
Message-ID: <FA6AF04F-F8CE-4CC2-9D11-C7F874B6626B@redhat.com>

Hey Thomas,

On 3 Jan 2017, at 15:20, Thomas Cunningham wrote:

> lilguylaptop:hawkfx cunningt$ jruby -G -S hawkfx.rb
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
> warning: `<<' after local variable or literal is interpreted as binary
> operator
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
> warning: even though it seems like here document

Those two are harmless.

>
> Exception running Application:
>
> #<NameError: uninitialized constant ValueElement::Logging
>
> Did you mean?  Logger>

I haven't seen that before

> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.2.0-java/lib/jrubyfx/controller.rb:129:in
> `load_into'

I see that you are using JrubyFx 1.2, while I am still on 1.1.1
I did not even know that 1.2 is out.

I did just wipe jruby-9.1.5.0 (rvm remove jruby-9.1.5.0) and then

$ rvm install jruby-9.1.5.0
$ rvm use jruby-9.1.5.0
$ gem install bundler
$ cd hawkfx
$ bundle install

and it pulled in jrubyfx-1.1.1

and it starts like this

snert:hawkfx hrupp$ jruby -G -S hawkfx.rb
/Users/hrupp/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158: 
warning: `<<' after local variable or literal is interpreted as binary 
operator
/Users/hrupp/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158: 
warning: even though it seems like here document


From tcunning at redhat.com  Tue Jan  3 11:10:08 2017
From: tcunning at redhat.com (Thomas Cunningham)
Date: Tue, 3 Jan 2017 11:10:08 -0500
Subject: [Hawkular-dev] HawkFX on Mac OS X issues?
In-Reply-To: <FA6AF04F-F8CE-4CC2-9D11-C7F874B6626B@redhat.com>
References: <CALAW+WLi28xfZeBQyfxqeh=vi_TWbHoh=kbUVVkDf8G_HNpUWg@mail.gmail.com>
	<17CA7990-BA77-4214-9185-173A94863F4E@redhat.com>
	<CALAW+WKnB=uV5n8DE4GRQZoW7NkF695A+1LktNKjZFhHzX4U8A@mail.gmail.com>
	<FA6AF04F-F8CE-4CC2-9D11-C7F874B6626B@redhat.com>
Message-ID: <CALAW+WJ_XLxb5Rey2CqLcet5h6DSfft7i-77R2g_dCmbjxG3Sg@mail.gmail.com>

Hmm...   I tried your series of steps, and bundle install seems to be
pulling in jrubyfx-1.2 for me :

lilguylaptop:hawkfx cunningt$ rvm --version

rvm 1.28.0 (latest) by Wayne E. Seguin <wayneeseguin at gmail.com>, Michal
Papis <mpapis at gmail.com> [https://rvm.io/]

bundle install output :
https://gist.github.com/cunningt/a6c018e5e38fff61110a1e1d1a1bf6ad


However, if I remove jrubyfx-1.2, and install jrubyfx-1.1.1, and bundle
install, I'm seeing :


lilguylaptop:hawkfx cunningt$ jruby -G -S hawkfx.rb

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
warning: `<<' after local variable or literal is interpreted as binary
operator

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
warning: even though it seems like here document

Exception running Application:

#<NameError: uninitialized constant ValueElement::Logging

Did you mean?  Logger>

org/jruby/RubyModule.java:3346:in `const_missing'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/value_elts.rb:149:in
`processValue'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/fxmlloader/value_elts.rb:67:in
`processStartElement'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/jrubyfx-fxmlloader.rb:440:in
`processStartElement'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader-0.4.1-java/lib/jrubyfx-fxmlloader.rb:351:in
`load'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.1.1-java/lib/jrubyfx/controller.rb:123:in
`load_into'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.1.1-java/lib/jrubyfx/core_ext/stage.rb:82:in
`fxml'

hawkfx.rb:18:in `block in start'

org/jruby/RubyBasicObject.java:1667:in `instance_eval'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.1.1-java/lib/jrubyfx/module.rb:49:in
`with'

hawkfx.rb:17:in `start'

/Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.1.1-java/lib/jrubyfx/java_fx_impl.rb:119:in
`block in launch_app_after_platform'




On Tue, Jan 3, 2017 at 9:44 AM, Heiko W.Rupp <hrupp at redhat.com> wrote:

> Hey Thomas,
>
> On 3 Jan 2017, at 15:20, Thomas Cunningham wrote:
>
> > lilguylaptop:hawkfx cunningt$ jruby -G -S hawkfx.rb
> >
> > /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
> > warning: `<<' after local variable or literal is interpreted as binary
> > operator
> >
> > /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
> > warning: even though it seems like here document
>
> Those two are harmless.
>
> >
> > Exception running Application:
> >
> > #<NameError: uninitialized constant ValueElement::Logging
> >
> > Did you mean?  Logger>
>
> I haven't seen that before
>
> > /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.
> 2.0-java/lib/jrubyfx/controller.rb:129:in
> > `load_into'
>
> I see that you are using JrubyFx 1.2, while I am still on 1.1.1
> I did not even know that 1.2 is out.
>
> I did just wipe jruby-9.1.5.0 (rvm remove jruby-9.1.5.0) and then
>
> $ rvm install jruby-9.1.5.0
> $ rvm use jruby-9.1.5.0
> $ gem install bundler
> $ cd hawkfx
> $ bundle install
>
> and it pulled in jrubyfx-1.1.1
>
> and it starts like this
>
> snert:hawkfx hrupp$ jruby -G -S hawkfx.rb
> /Users/hrupp/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
> warning: `<<' after local variable or literal is interpreted as binary
> operator
> /Users/hrupp/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158:
> warning: even though it seems like here document
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170103/0c42443d/attachment.html 

From jsanda at redhat.com  Tue Jan  3 11:15:52 2017
From: jsanda at redhat.com (John Sanda)
Date: Tue, 3 Jan 2017 11:15:52 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
Message-ID: <AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>

Gareth, I replied in the ticket. Please take a look at https://issues.jboss.org/browse/HWKMETRICS-373 <https://issues.jboss.org/browse/HWKMETRICS-373> to see if that provides the functionality you need. 

> On Jan 1, 2017, at 6:15 PM, Gareth Healy <garethahealy at gmail.com> wrote:
> 
> Have raised:
> https://issues.jboss.org/browse/HWKMETRICS-563 <https://issues.jboss.org/browse/HWKMETRICS-563>
> Will work on the grafana datasource support in next few days.
> 
> Cheers.
> 
> On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli <mazz at redhat.com <mailto:mazz at redhat.com>> wrote:
> Thanks.
> 
> You should submit a feature request in the H-Metrics JIRA and link that JIRA to a PR for the H-Metrics team to peer review:
> 
> https://issues.jboss.org/projects/HWKMETRICS <https://issues.jboss.org/projects/HWKMETRICS>
> 
> 
> ----- Original Message -----
> > I did some hacking (day 1 of Rx, so probably not the best "solution"). But
> > works...
> >
> >    -
> >    https://github.com/garethahealy/hawkular-metrics/commit/75c616f9be71a0b85ce5dee310c4dff828bb8f38 <https://github.com/garethahealy/hawkular-metrics/commit/75c616f9be71a0b85ce5dee310c4dff828bb8f38>
> >
> >
> > Sample output:
> >
> >    - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ece87096c89 <https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ece87096c89>
> >
> >
> > And some test cURL commands:
> >
> >    - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad788 <https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad788>
> >
> >
> > Cheers.
> >
> > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli <mazz at redhat.com <mailto:mazz at redhat.com>> wrote:
> >
> > > This would be a feature request on Hawkular-Metrics (if they don't do
> > > something like this already - I do not know. I will defer to the H-Metrics
> > > folks to talk about how they do querying off of tags).
> > >
> > > ----- Original Message -----
> > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
> > > single
> > > > metric with tagged datapoints, i.e.:
> > > >
> > > >
> > > >
> > > >
> > > > https://github.com/coreos/etcd/blob/master/Documentation/v2/metrics.md# <https://github.com/coreos/etcd/blob/master/Documentation/v2/metrics.md#>
> > > http-requests
> > > >
> > > >
> > > >
> > > >
> > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
> > > [counter]
> > > > datapoints for metric named
> > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_received_total]:
> > > [
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
> > > > ]
> > > >
> > > > But when trying to view this via the grafana datasource, only 1 metric
> > > and
> > > > the aggregated counts are shown. What i'd like to do is something like
> > > the
> > > > below:
> > > >
> > > >
> > > >
> > > > {
> > > > "start": 1482999755690,
> > > > "end": 1483000020093,
> > > > "order": "ASC",
> > > > "tags": "pod_namespace:etcd-testing",
> > > > "groupDatapointsByTagKey": "method"
> > > > }
> > > >
> > > > Search via tags or name (as-is) and group the datapoints by a tag key,
> > > which
> > > > would give you 3 lines, instead of 1.
> > > >
> > > > Does that sound possible?
> > > >
> > > > Cheers.
> > > >
> > > > _______________________________________________
> > > > hawkular-dev mailing list
> > > > hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
> > > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
> > >
> >
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170103/27ec87f3/attachment-0001.html 

From tcunning at redhat.com  Tue Jan  3 13:08:26 2017
From: tcunning at redhat.com (Thomas Cunningham)
Date: Tue, 3 Jan 2017 13:08:26 -0500
Subject: [Hawkular-dev] HawkFX on Mac OS X issues?
In-Reply-To: <CALAW+WJ_XLxb5Rey2CqLcet5h6DSfft7i-77R2g_dCmbjxG3Sg@mail.gmail.com>
References: <CALAW+WLi28xfZeBQyfxqeh=vi_TWbHoh=kbUVVkDf8G_HNpUWg@mail.gmail.com>
	<17CA7990-BA77-4214-9185-173A94863F4E@redhat.com>
	<CALAW+WKnB=uV5n8DE4GRQZoW7NkF695A+1LktNKjZFhHzX4U8A@mail.gmail.com>
	<FA6AF04F-F8CE-4CC2-9D11-C7F874B6626B@redhat.com>
	<CALAW+WJ_XLxb5Rey2CqLcet5h6DSfft7i-77R2g_dCmbjxG3Sg@mail.gmail.com>
Message-ID: <CALAW+W+jDaaiH2+eGEbk9Y6ZBxxCOz92F_NUF6o1v7ia45bsrw@mail.gmail.com>

Heiko,

Thank you for all of the help on IRC - just in case anyone else runs into
this, I was using :

jdk1.8.0_31.jdk and rvm 1.28.0

I upgraded my Java to jdk1.8.0_111.jdk and downgraded rvm to 1.27.0, and
ran through the steps that Heiko outlined above (wipe and reinstall jruby),
and that fixed everything.

Thanks again for the help.






On Tue, Jan 3, 2017 at 11:10 AM, Thomas Cunningham <tcunning at redhat.com>
wrote:

> Hmm...   I tried your series of steps, and bundle install seems to be
> pulling in jrubyfx-1.2 for me :
>
> lilguylaptop:hawkfx cunningt$ rvm --version
>
> rvm 1.28.0 (latest) by Wayne E. Seguin <wayneeseguin at gmail.com>, Michal
> Papis <mpapis at gmail.com> [https://rvm.io/]
>
> bundle install output : https://gist.github.com/cunningt/
> a6c018e5e38fff61110a1e1d1a1bf6ad
>
>
> However, if I remove jrubyfx-1.2, and install jrubyfx-1.1.1, and bundle
> install, I'm seeing :
>
>
> lilguylaptop:hawkfx cunningt$ jruby -G -S hawkfx.rb
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158: warning: `<<' after
> local variable or literal is interpreted as binary operator
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/elts.rb:158: warning: even though it
> seems like here document
>
> Exception running Application:
>
> #<NameError: uninitialized constant ValueElement::Logging
>
> Did you mean?  Logger>
>
> org/jruby/RubyModule.java:3346:in `const_missing'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/value_elts.rb:149:in `processValue'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/fxmlloader/value_elts.rb:67:in
> `processStartElement'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/jrubyfx-fxmlloader.rb:440:in
> `processStartElement'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-
> fxmlloader-0.4.1-java/lib/jrubyfx-fxmlloader.rb:351:in `load'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.
> 1.1-java/lib/jrubyfx/controller.rb:123:in `load_into'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.
> 1.1-java/lib/jrubyfx/core_ext/stage.rb:82:in `fxml'
>
> hawkfx.rb:18:in `block in start'
>
> org/jruby/RubyBasicObject.java:1667:in `instance_eval'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.
> 1.1-java/lib/jrubyfx/module.rb:49:in `with'
>
> hawkfx.rb:17:in `start'
>
> /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.
> 1.1-java/lib/jrubyfx/java_fx_impl.rb:119:in `block in
> launch_app_after_platform'
>
>
>
>
> On Tue, Jan 3, 2017 at 9:44 AM, Heiko W.Rupp <hrupp at redhat.com> wrote:
>
>> Hey Thomas,
>>
>> On 3 Jan 2017, at 15:20, Thomas Cunningham wrote:
>>
>> > lilguylaptop:hawkfx cunningt$ jruby -G -S hawkfx.rb
>> >
>> > /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloa
>> der-0.4.1-java/lib/fxmlloader/elts.rb:158:
>> > warning: `<<' after local variable or literal is interpreted as binary
>> > operator
>> >
>> > /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloa
>> der-0.4.1-java/lib/fxmlloader/elts.rb:158:
>> > warning: even though it seems like here document
>>
>> Those two are harmless.
>>
>> >
>> > Exception running Application:
>> >
>> > #<NameError: uninitialized constant ValueElement::Logging
>> >
>> > Did you mean?  Logger>
>>
>> I haven't seen that before
>>
>> > /Users/cunningt/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-1.2.0-
>> java/lib/jrubyfx/controller.rb:129:in
>> > `load_into'
>>
>> I see that you are using JrubyFx 1.2, while I am still on 1.1.1
>> I did not even know that 1.2 is out.
>>
>> I did just wipe jruby-9.1.5.0 (rvm remove jruby-9.1.5.0) and then
>>
>> $ rvm install jruby-9.1.5.0
>> $ rvm use jruby-9.1.5.0
>> $ gem install bundler
>> $ cd hawkfx
>> $ bundle install
>>
>> and it pulled in jrubyfx-1.1.1
>>
>> and it starts like this
>>
>> snert:hawkfx hrupp$ jruby -G -S hawkfx.rb
>> /Users/hrupp/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader
>> -0.4.1-java/lib/fxmlloader/elts.rb:158:
>> warning: `<<' after local variable or literal is interpreted as binary
>> operator
>> /Users/hrupp/.rvm/gems/jruby-9.1.5.0/gems/jrubyfx-fxmlloader
>> -0.4.1-java/lib/fxmlloader/elts.rb:158:
>> warning: even though it seems like here document
>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170103/a266da9c/attachment.html 

From garethahealy at gmail.com  Tue Jan  3 13:20:10 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Tue, 3 Jan 2017 18:20:10 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
Message-ID: <CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>

@John S,
>From looking at the JIRA, i've tried calling /stats and /data (*deprecated*)
but don't get back data i'd want.

If i search via tags; i get all empty buckets (both metric and datapoint
tags). If i search via metricNames; i get back buckets but since its
aggregated i am back to the original problem.

Am i querying via tags incorrectly? cURL commands i've been using:

   - create data - https://gist.github.com/gareth
   ahealy/0f46aad5d2da41b82aad5af317fad788
   <https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad788>
   - get buckets - https://gist.github.com/gareth
   ahealy/3a5913b5a6956d709366ab6985fefec6
   <https://gist.github.com/garethahealy/3a5913b5a6956d709366ab6985fefec6>


@Joel,
I'd not done the other endpoints as i wanted to prove it first, and then if
accepted, complete the rest.

Your point about insertion; I'll let others comment who know the metrics
codebase better than myself.

Cheers.

On Tue, Jan 3, 2017 at 4:15 PM, John Sanda <jsanda at redhat.com> wrote:

> Gareth, I replied in the ticket. Please take a look at
> https://issues.jboss.org/browse/HWKMETRICS-373 to see if that provides
> the functionality you need.
>
> On Jan 1, 2017, at 6:15 PM, Gareth Healy <garethahealy at gmail.com> wrote:
>
> Have raised:
>
>    - https://issues.jboss.org/browse/HWKMETRICS-563
>
> Will work on the grafana datasource support in next few days.
>
> Cheers.
>
> On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli <mazz at redhat.com> wrote:
>
>> Thanks.
>>
>> You should submit a feature request in the H-Metrics JIRA and link that
>> JIRA to a PR for the H-Metrics team to peer review:
>>
>> https://issues.jboss.org/projects/HWKMETRICS
>>
>>
>> ----- Original Message -----
>> > I did some hacking (day 1 of Rx, so probably not the best "solution").
>> But
>> > works...
>> >
>> >    -
>> >    https://github.com/garethahealy/hawkular-metrics/commit/75c6
>> 16f9be71a0b85ce5dee310c4dff828bb8f38
>> >
>> >
>> > Sample output:
>> >
>> >    - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ec
>> e87096c89
>> >
>> >
>> > And some test cURL commands:
>> >
>> >    - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af
>> 317fad788
>> >
>> >
>> > Cheers.
>> >
>> > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli <mazz at redhat.com>
>> wrote:
>> >
>> > > This would be a feature request on Hawkular-Metrics (if they don't do
>> > > something like this already - I do not know. I will defer to the
>> H-Metrics
>> > > folks to talk about how they do querying off of tags).
>> > >
>> > > ----- Original Message -----
>> > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
>> > > single
>> > > > metric with tagged datapoints, i.e.:
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > https://github.com/coreos/etcd/blob/master/Documentation/v2/
>> metrics.md#
>> > > http-requests
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
>> > > [counter]
>> > > > datapoints for metric named
>> > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_r
>> eceived_total]:
>> > > [
>> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
>> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
>> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
>> > > > ]
>> > > >
>> > > > But when trying to view this via the grafana datasource, only 1
>> metric
>> > > and
>> > > > the aggregated counts are shown. What i'd like to do is something
>> like
>> > > the
>> > > > below:
>> > > >
>> > > >
>> > > >
>> > > > {
>> > > > "start": 1482999755690,
>> > > > "end": 1483000020093,
>> > > > "order": "ASC",
>> > > > "tags": "pod_namespace:etcd-testing",
>> > > > "groupDatapointsByTagKey": "method"
>> > > > }
>> > > >
>> > > > Search via tags or name (as-is) and group the datapoints by a tag
>> key,
>> > > which
>> > > > would give you 3 lines, instead of 1.
>> > > >
>> > > > Does that sound possible?
>> > > >
>> > > > Cheers.
>> > > >
>> > > > _______________________________________________
>> > > > hawkular-dev mailing list
>> > > > hawkular-dev at lists.jboss.org
>> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
>> > > >
>> > > _______________________________________________
>> > > hawkular-dev mailing list
>> > > hawkular-dev at lists.jboss.org
>> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
>> > >
>> >
>>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170103/bf054da7/attachment-0001.html 

From mazz at redhat.com  Tue Jan  3 13:38:28 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Tue, 3 Jan 2017 13:38:28 -0500 (EST)
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
Message-ID: <514543075.1039358.1483468708054.JavaMail.zimbra@redhat.com>

>From reading that JIRA, it seems like that Hawkular-Metrics feature only works if you have tags that are on individual data points (as opposed to the metric definition itself). Not sure if I am mistaken or not, but just wanted to point this out because, for the record, the Hawkular OpenShift Agent does NOT tag individual data points. It only inserts tags on the metric definition. Seems inefficient to tag individual data points with the exact same tags over and over again - hence we put the tags on metric definitions.


----- Original Message -----
> @John S,
> From looking at the JIRA, i've tried calling /stats and /data ( deprecated )
> but don't get back data i'd want.
> 
> If i search via tags; i get all empty buckets (both metric and datapoint
> tags). If i search via metricNames; i get back buckets but since its
> aggregated i am back to the original problem.
> 
> Am i querying via tags incorrectly? cURL commands i've been using:
> 
> 
>     * create data -
>     https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad788
>     * get buckets -
>     https://gist.github.com/garethahealy/3a5913b5a6956d709366ab6985fefec6
> 
> @Joel,
> I'd not done the other endpoints as i wanted to prove it first, and then if
> accepted, complete the rest.
> 
> Your point about insertion; I'll let others comment who know the metrics
> codebase better than myself.
> 
> Cheers.
> 
> On Tue, Jan 3, 2017 at 4:15 PM, John Sanda < jsanda at redhat.com > wrote:
> 
> 
> 
> Gareth, I replied in the ticket. Please take a look at
> https://issues.jboss.org/browse/HWKMETRICS-373 to see if that provides the
> functionality you need.
> 
> 
> 
> 
> On Jan 1, 2017, at 6:15 PM, Gareth Healy < garethahealy at gmail.com > wrote:
> 
> Have raised:
> 
> 
>     * https://issues.jboss.org/browse/HWKMETRICS-563
> 
> Will work on the grafana datasource support in next few days.
> 
> Cheers.
> 
> On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli < mazz at redhat.com > wrote:
> 
> 
> Thanks.
> 
> You should submit a feature request in the H-Metrics JIRA and link that JIRA
> to a PR for the H-Metrics team to peer review:
> 
> https://issues.jboss.org/projects/HWKMETRICS
> 
> 
> ----- Original Message -----
> > I did some hacking (day 1 of Rx, so probably not the best "solution"). But
> > works...
> > 
> > -
> > https://github.com/garethahealy/hawkular-metrics/commit/75c616f9be71a0b85ce5dee310c4dff828bb8f38
> > 
> > 
> > Sample output:
> > 
> > - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ece87096c89
> > 
> > 
> > And some test cURL commands:
> > 
> > - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad788
> > 
> > 
> > Cheers.
> > 
> > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli < mazz at redhat.com > wrote:
> > 
> > > This would be a feature request on Hawkular-Metrics (if they don't do
> > > something like this already - I do not know. I will defer to the
> > > H-Metrics
> > > folks to talk about how they do querying off of tags).
> > > 
> > > ----- Original Message -----
> > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
> > > single
> > > > metric with tagged datapoints, i.e.:
> > > > 
> > > > 
> > > > 
> > > > 
> > > > https://github.com/coreos/etcd/blob/master/Documentation/v2/metrics.md#
> > > http-requests
> > > > 
> > > > 
> > > > 
> > > > 
> > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
> > > [counter]
> > > > datapoints for metric named
> > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_received_total]:
> > > [
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
> > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
> > > > ]
> > > > 
> > > > But when trying to view this via the grafana datasource, only 1 metric
> > > and
> > > > the aggregated counts are shown. What i'd like to do is something like
> > > the
> > > > below:
> > > > 
> > > > 
> > > > 
> > > > {
> > > > "start": 1482999755690,
> > > > "end": 1483000020093,
> > > > "order": "ASC",
> > > > "tags": "pod_namespace:etcd-testing",
> > > > "groupDatapointsByTagKey": "method"
> > > > }
> > > > 
> > > > Search via tags or name (as-is) and group the datapoints by a tag key,
> > > which
> > > > would give you 3 lines, instead of 1.
> > > > 
> > > > Does that sound possible?
> > > > 
> > > > Cheers.
> > > > 
> > > > _______________________________________________
> > > > hawkular-dev mailing list
> > > > hawkular-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > > 
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > 
> > 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From garethahealy at gmail.com  Tue Jan  3 17:05:05 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Tue, 3 Jan 2017 22:05:05 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <514543075.1039358.1483468708054.JavaMail.zimbra@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<514543075.1039358.1483468708054.JavaMail.zimbra@redhat.com>
Message-ID: <CAHWT10YHU=MHEKKVy09O_k0KjrWDxofxpkPi2vkgjZAsuamg0Q@mail.gmail.com>

Hi John,

Am not sure what you mean by "*OpenShift Agent does NOT tag individual data
points*", since this tags datapoint's:

-
https://github.com/hawkular/hawkular-openshift-agent/blob/master/collector/impl/prometheus_metrics_collector.go#L192

On Tue, Jan 3, 2017 at 6:38 PM, John Mazzitelli <mazz at redhat.com> wrote:

> >From reading that JIRA, it seems like that Hawkular-Metrics feature only
> works if you have tags that are on individual data points (as opposed to
> the metric definition itself). Not sure if I am mistaken or not, but just
> wanted to point this out because, for the record, the Hawkular OpenShift
> Agent does NOT tag individual data points. It only inserts tags on the
> metric definition. Seems inefficient to tag individual data points with the
> exact same tags over and over again - hence we put the tags on metric
> definitions.
>
>
> ----- Original Message -----
> > @John S,
> > From looking at the JIRA, i've tried calling /stats and /data (
> deprecated )
> > but don't get back data i'd want.
> >
> > If i search via tags; i get all empty buckets (both metric and datapoint
> > tags). If i search via metricNames; i get back buckets but since its
> > aggregated i am back to the original problem.
> >
> > Am i querying via tags incorrectly? cURL commands i've been using:
> >
> >
> >     * create data -
> >     https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad7
> 88
> >     * get buckets -
> >     https://gist.github.com/garethahealy/3a5913b5a6956d709366ab6985fefe
> c6
> >
> > @Joel,
> > I'd not done the other endpoints as i wanted to prove it first, and then
> if
> > accepted, complete the rest.
> >
> > Your point about insertion; I'll let others comment who know the metrics
> > codebase better than myself.
> >
> > Cheers.
> >
> > On Tue, Jan 3, 2017 at 4:15 PM, John Sanda < jsanda at redhat.com > wrote:
> >
> >
> >
> > Gareth, I replied in the ticket. Please take a look at
> > https://issues.jboss.org/browse/HWKMETRICS-373 to see if that provides
> the
> > functionality you need.
> >
> >
> >
> >
> > On Jan 1, 2017, at 6:15 PM, Gareth Healy < garethahealy at gmail.com >
> wrote:
> >
> > Have raised:
> >
> >
> >     * https://issues.jboss.org/browse/HWKMETRICS-563
> >
> > Will work on the grafana datasource support in next few days.
> >
> > Cheers.
> >
> > On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli < mazz at redhat.com >
> wrote:
> >
> >
> > Thanks.
> >
> > You should submit a feature request in the H-Metrics JIRA and link that
> JIRA
> > to a PR for the H-Metrics team to peer review:
> >
> > https://issues.jboss.org/projects/HWKMETRICS
> >
> >
> > ----- Original Message -----
> > > I did some hacking (day 1 of Rx, so probably not the best "solution").
> But
> > > works...
> > >
> > > -
> > > https://github.com/garethahealy/hawkular-metrics/commit/
> 75c616f9be71a0b85ce5dee310c4dff828bb8f38
> > >
> > >
> > > Sample output:
> > >
> > > - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ece87096c
> 89
> > >
> > >
> > > And some test cURL commands:
> > >
> > > - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad7
> 88
> > >
> > >
> > > Cheers.
> > >
> > > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli < mazz at redhat.com >
> wrote:
> > >
> > > > This would be a feature request on Hawkular-Metrics (if they don't do
> > > > something like this already - I do not know. I will defer to the
> > > > H-Metrics
> > > > folks to talk about how they do querying off of tags).
> > > >
> > > > ----- Original Message -----
> > > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
> > > > single
> > > > > metric with tagged datapoints, i.e.:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > https://github.com/coreos/etcd/blob/master/
> Documentation/v2/metrics.md#
> > > > http-requests
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
> > > > [counter]
> > > > > datapoints for metric named
> > > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_
> received_total]:
> > > > [
> > > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
> > > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
> > > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
> > > > > ]
> > > > >
> > > > > But when trying to view this via the grafana datasource, only 1
> metric
> > > > and
> > > > > the aggregated counts are shown. What i'd like to do is something
> like
> > > > the
> > > > > below:
> > > > >
> > > > >
> > > > >
> > > > > {
> > > > > "start": 1482999755690,
> > > > > "end": 1483000020093,
> > > > > "order": "ASC",
> > > > > "tags": "pod_namespace:etcd-testing",
> > > > > "groupDatapointsByTagKey": "method"
> > > > > }
> > > > >
> > > > > Search via tags or name (as-is) and group the datapoints by a tag
> key,
> > > > which
> > > > > would give you 3 lines, instead of 1.
> > > > >
> > > > > Does that sound possible?
> > > > >
> > > > > Cheers.
> > > > >
> > > > > _______________________________________________
> > > > > hawkular-dev mailing list
> > > > > hawkular-dev at lists.jboss.org
> > > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > > >
> > > > _______________________________________________
> > > > hawkular-dev mailing list
> > > > hawkular-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > >
> > >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170103/5e91e2a8/attachment.html 

From mazz at redhat.com  Tue Jan  3 17:21:43 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Tue, 3 Jan 2017 17:21:43 -0500 (EST)
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10YHU=MHEKKVy09O_k0KjrWDxofxpkPi2vkgjZAsuamg0Q@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<514543075.1039358.1483468708054.JavaMail.zimbra@redhat.com>
	<CAHWT10YHU=MHEKKVy09O_k0KjrWDxofxpkPi2vkgjZAsuamg0Q@mail.gmail.com>
Message-ID: <1129205434.1189600.1483482103793.JavaMail.zimbra@redhat.com>

Ah, OK. That's different than what I was talking about.

The Hawkular OpenShift Agent allows you to define any number of tags to apply to a metric definition. This is the "Tags" entry in your configmap or the global configs defined in the global agent config... like this:

collector:
  tags:
    pod_name: {$POD:name}
    ...

or:

endpoints:
- type: jolokia
  tags:
    foo: bar
  ...

Those tags are defined on the metric definition alone. The code you pointed out:

> https://github.com/hawkular/hawkular-openshift-agent/blob/master/collector/impl/prometheus_metrics_collector.go#L192

if you notice, the tags added to the data points there are ONLY the actual Prometheus labels that were found on the Prometheus metric itself - the agent just copies those Prometheus labels to the h-metrics data points as tags (i.e. those tags are added to each data point). The tags I talk about above (the Hawkular tags) are not added to those data points.

But perhaps that is all you care about, which would be good :-) and you can probably use what John Sanda talked about.

----- Original Message -----
> Hi John,
> 
> Am not sure what you mean by "*OpenShift Agent does NOT tag individual data
> points*", since this tags datapoint's:
> 
> -
> https://github.com/hawkular/hawkular-openshift-agent/blob/master/collector/impl/prometheus_metrics_collector.go#L192
> 
> On Tue, Jan 3, 2017 at 6:38 PM, John Mazzitelli <mazz at redhat.com> wrote:
> 
> > >From reading that JIRA, it seems like that Hawkular-Metrics feature only
> > works if you have tags that are on individual data points (as opposed to
> > the metric definition itself). Not sure if I am mistaken or not, but just
> > wanted to point this out because, for the record, the Hawkular OpenShift
> > Agent does NOT tag individual data points. It only inserts tags on the
> > metric definition. Seems inefficient to tag individual data points with the
> > exact same tags over and over again - hence we put the tags on metric
> > definitions.
> >
> >
> > ----- Original Message -----
> > > @John S,
> > > From looking at the JIRA, i've tried calling /stats and /data (
> > deprecated )
> > > but don't get back data i'd want.
> > >
> > > If i search via tags; i get all empty buckets (both metric and datapoint
> > > tags). If i search via metricNames; i get back buckets but since its
> > > aggregated i am back to the original problem.
> > >
> > > Am i querying via tags incorrectly? cURL commands i've been using:
> > >
> > >
> > >     * create data -
> > >     https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad7
> > 88
> > >     * get buckets -
> > >     https://gist.github.com/garethahealy/3a5913b5a6956d709366ab6985fefe
> > c6
> > >
> > > @Joel,
> > > I'd not done the other endpoints as i wanted to prove it first, and then
> > if
> > > accepted, complete the rest.
> > >
> > > Your point about insertion; I'll let others comment who know the metrics
> > > codebase better than myself.
> > >
> > > Cheers.
> > >
> > > On Tue, Jan 3, 2017 at 4:15 PM, John Sanda < jsanda at redhat.com > wrote:
> > >
> > >
> > >
> > > Gareth, I replied in the ticket. Please take a look at
> > > https://issues.jboss.org/browse/HWKMETRICS-373 to see if that provides
> > the
> > > functionality you need.
> > >
> > >
> > >
> > >
> > > On Jan 1, 2017, at 6:15 PM, Gareth Healy < garethahealy at gmail.com >
> > wrote:
> > >
> > > Have raised:
> > >
> > >
> > >     * https://issues.jboss.org/browse/HWKMETRICS-563
> > >
> > > Will work on the grafana datasource support in next few days.
> > >
> > > Cheers.
> > >
> > > On Fri, Dec 30, 2016 at 6:29 PM, John Mazzitelli < mazz at redhat.com >
> > wrote:
> > >
> > >
> > > Thanks.
> > >
> > > You should submit a feature request in the H-Metrics JIRA and link that
> > JIRA
> > > to a PR for the H-Metrics team to peer review:
> > >
> > > https://issues.jboss.org/projects/HWKMETRICS
> > >
> > >
> > > ----- Original Message -----
> > > > I did some hacking (day 1 of Rx, so probably not the best "solution").
> > But
> > > > works...
> > > >
> > > > -
> > > > https://github.com/garethahealy/hawkular-metrics/commit/
> > 75c616f9be71a0b85ce5dee310c4dff828bb8f38
> > > >
> > > >
> > > > Sample output:
> > > >
> > > > - https://gist.github.com/garethahealy/00a90bbee2556b6f0a338ece87096c
> > 89
> > > >
> > > >
> > > > And some test cURL commands:
> > > >
> > > > - https://gist.github.com/garethahealy/0f46aad5d2da41b82aad5af317fad7
> > 88
> > > >
> > > >
> > > > Cheers.
> > > >
> > > > On Thu, Dec 29, 2016 at 4:33 PM, John Mazzitelli < mazz at redhat.com >
> > wrote:
> > > >
> > > > > This would be a feature request on Hawkular-Metrics (if they don't do
> > > > > something like this already - I do not know. I will defer to the
> > > > > H-Metrics
> > > > > folks to talk about how they do querying off of tags).
> > > > >
> > > > > ----- Original Message -----
> > > > > > The OpenShift Agent when monitoring a prometheus endpoint creates a
> > > > > single
> > > > > > metric with tagged datapoints, i.e.:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > https://github.com/coreos/etcd/blob/master/
> > Documentation/v2/metrics.md#
> > > > > http-requests
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > I1228 21:02:01.820530 1 metrics_storage.go:155] TRACE: Stored [3]
> > > > > [counter]
> > > > > > datapoints for metric named
> > > > > > [pod/fa32a887-cd08-11e6-ab2e-525400c583ad/custom/etcd_http_
> > received_total]:
> > > > > [
> > > > > > {2016-12-28 21:02:01.638767339 +0000 UTC 622 map[method:DELETE]}
> > > > > > {2016-12-28 21:02:01.638767339 +0000 UTC 414756 map[method:GET]}
> > > > > > {2016-12-28 21:02:01.638767339 +0000 UTC 33647 map[method:PUT]}
> > > > > > ]
> > > > > >
> > > > > > But when trying to view this via the grafana datasource, only 1
> > metric
> > > > > and
> > > > > > the aggregated counts are shown. What i'd like to do is something
> > like
> > > > > the
> > > > > > below:
> > > > > >
> > > > > >
> > > > > >
> > > > > > {
> > > > > > "start": 1482999755690,
> > > > > > "end": 1483000020093,
> > > > > > "order": "ASC",
> > > > > > "tags": "pod_namespace:etcd-testing",
> > > > > > "groupDatapointsByTagKey": "method"
> > > > > > }
> > > > > >
> > > > > > Search via tags or name (as-is) and group the datapoints by a tag
> > key,
> > > > > which
> > > > > > would give you 3 lines, instead of 1.
> > > > > >
> > > > > > Does that sound possible?
> > > > > >
> > > > > > Cheers.
> > > > > >
> > > > > > _______________________________________________
> > > > > > hawkular-dev mailing list
> > > > > > hawkular-dev at lists.jboss.org
> > > > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > > > >
> > > > > _______________________________________________
> > > > > hawkular-dev mailing list
> > > > > hawkular-dev at lists.jboss.org
> > > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > > >
> > > >
> > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> > >
> > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> 

From jtakvori at redhat.com  Wed Jan  4 03:04:48 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Wed, 4 Jan 2017 09:04:48 +0100
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
Message-ID: <CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>

On Tue, Jan 3, 2017 at 7:20 PM, Gareth Healy <garethahealy at gmail.com> wrote:

>
> If i search via tags; i get all empty buckets (both metric and datapoint
> tags). If i search via metricNames; i get back buckets but since its
> aggregated i am back to the original problem.
>
>
Here, query by tag means time series tags, not data points tags. That's
probably why you don't get anything. Maybe we should more clearly
disambiguate these two concepts as there's often some confusion.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170104/fe63192c/attachment.html 

From hrupp at redhat.com  Wed Jan  4 04:01:08 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Wed, 04 Jan 2017 10:01:08 +0100
Subject: [Hawkular-dev] HawkFX on Mac OS X issues?
In-Reply-To: <CALAW+W+jDaaiH2+eGEbk9Y6ZBxxCOz92F_NUF6o1v7ia45bsrw@mail.gmail.com>
References: <CALAW+WLi28xfZeBQyfxqeh=vi_TWbHoh=kbUVVkDf8G_HNpUWg@mail.gmail.com>
	<17CA7990-BA77-4214-9185-173A94863F4E@redhat.com>
	<CALAW+WKnB=uV5n8DE4GRQZoW7NkF695A+1LktNKjZFhHzX4U8A@mail.gmail.com>
	<FA6AF04F-F8CE-4CC2-9D11-C7F874B6626B@redhat.com>
	<CALAW+WJ_XLxb5Rey2CqLcet5h6DSfft7i-77R2g_dCmbjxG3Sg@mail.gmail.com>
	<CALAW+W+jDaaiH2+eGEbk9Y6ZBxxCOz92F_NUF6o1v7ia45bsrw@mail.gmail.com>
Message-ID: <0FCC4269-CB6C-4FA0-9E0B-774A8DC2CA27@redhat.com>

Thomas,

On 3 Jan 2017, at 19:08, Thomas Cunningham wrote:

> I upgraded my Java to jdk1.8.0_111.jdk and downgraded rvm to 1.27.0, and
> ran through the steps that Heiko outlined above (wipe and reinstall jruby),
> and that fixed everything.

I am glad this works now.

  Heiko

From mazz at redhat.com  Wed Jan  4 08:06:39 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 4 Jan 2017 08:06:39 -0500 (EST)
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
Message-ID: <799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>

> Here, query by tag means time series tags, not data points tags. That's
> probably why you don't get anything. Maybe we should more clearly
> disambiguate these two concepts as there's often some confusion.

:-) And that, I think, is what I meant by tags on "metric definitions" - I think the H-Metrics team calls them "timeseries tags".

From mazz at redhat.com  Wed Jan  4 09:01:28 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 4 Jan 2017 09:01:28 -0500 (EST)
Subject: [Hawkular-dev] update your pom license headers before release
In-Reply-To: <191185263.1460025.1483538359837.JavaMail.zimbra@redhat.com>
Message-ID: <610521818.1460866.1483538488627.JavaMail.zimbra@redhat.com>

Just ran into this yesterday, so here is your annual reminder.

If you are responsible for releasing maven artifacts, you should update your pom.xml files right now to up the copyright date to 2017 in the license headers. Otherwise, the release plugin stuff will fail in the middle and you'll have to back out what you did up until that point, and then update your poms, and then try to release again.

Avoid that pain now :) and update your pom license headers now.

From snegrea at redhat.com  Thu Jan  5 16:26:25 2017
From: snegrea at redhat.com (Stefan Negrea)
Date: Thu, 5 Jan 2017 15:26:25 -0600
Subject: [Hawkular-dev] Hawkular Metrics 0.23.0 - Release
Message-ID: <CADt-fV8mWMBdZYLPps9+M_W2E1TON-FQYmrEAj+1Q+-VKFKhFA@mail.gmail.com>

Hello,

I am happy to announce release 0.23.0 of Hawkular Metrics. This release is
anchored by performance and stability improvements.

Here is a list of major changes:

   - *Performance*
      - Prevent BusyPoolException under heavy load due no available
      connection and queue reaching max size of 256 (HWKMETRICS-542
      <https://issues.jboss.org/browse/HWKMETRICS-542>)
      - Gatling load tests have a new option (loops) to specify the number
      of requests per client (HWKMETRICS-559
      <https://issues.jboss.org/browse/HWKMETRICS-559>)
   - *Deployment*
      - Resolved an issue with resource-env-ref in component war (
      HWKMETRICS-541 <https://issues.jboss.org/browse/HWKMETRICS-541>)
      - Updated packaging to support deployments on WildFly 10.1.0 (
      HWKMETRICS-558 <https://issues.jboss.org/browse/HWKMETRICS-558>)
   - *REST API*
      - Updated CORS validation to be applied prior to processing the
      request; this solves an issue where some content is still returned even
      thought a bad request status is returned (HWKMETRICS-554
      <https://issues.jboss.org/browse/HWKMETRICS-554>)
   - *Internal Monitoring*
      - Hostname is now part of the metric id when creating and storing
      internal metrics (HWKMETRICS-555
      <https://issues.jboss.org/browse/HWKMETRICS-555>)
   - *Hawkular Alerting - Updates*
      - Added support for newer condition types to the email plugin (
      HWKALERTS-208 <https://issues.jboss.org/browse/HWKALERTS-208>)
      - Allow ExternalCondition to be fired on Event submission; external
      conditions can now be matched via Event and Data submissions (
      HWKALERTS-207 <https://issues.jboss.org/browse/HWKALERTS-207>)
      - Added new NelsonCondition for native Nelson Rule detection; a brand
      new condition type to perform automatic Nelson Rule detection of
      misbehaving metrics. (HWKALERTS-209
      <https://issues.jboss.org/browse/HWKALERTS-209>)

*Hawkular Alerting - included*

   - Version 1.5.0
   <https://issues.jboss.org/projects/HWKALERTS/versions/12332918>
   - Project details and repository: Github
   <https://github.com/hawkular/hawkular-alerts>
   - Documentation: REST API
   <http://www.hawkular.org/docs/rest/rest-alerts.html>, Examples
   <https://github.com/hawkular/hawkular-alerts/tree/master/examples>,
Developer
   Guide
   <http://www.hawkular.org/community/docs/developer-guide/alerts.html>

*Hawkular Metrics Clients*

   - Python: https://github.com/hawkular/hawkular-client-python
   - Go: https://github.com/hawkular/hawkular-client-go
   - Ruby: https://github.com/hawkular/hawkular-client-ruby
   - Java: https://github.com/hawkular/hawkular-client-java



*Release Links*

Github Release:
https://github.com/hawkular/hawkular-metrics/releases/tag/0.23.0

JBoss Nexus Maven artifacts:
http://origin-repository.jboss.org/nexus/content/repositorie
s/public/org/hawkular/metrics/

Jira release tracker:
https://issues.jboss.org/projects/HWKMETRICS/versions/12332805

A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, Joel
Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their project
contributions.


Thank you,
Stefan Negrea
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170105/830e0e30/attachment.html 

From theute at redhat.com  Fri Jan  6 07:59:19 2017
From: theute at redhat.com (Thomas Heute)
Date: Fri, 6 Jan 2017 13:59:19 +0100
Subject: [Hawkular-dev] Hawkular Metrics 0.23.0 - Release
In-Reply-To: <CADt-fV8mWMBdZYLPps9+M_W2E1TON-FQYmrEAj+1Q+-VKFKhFA@mail.gmail.com>
References: <CADt-fV8mWMBdZYLPps9+M_W2E1TON-FQYmrEAj+1Q+-VKFKhFA@mail.gmail.com>
Message-ID: <CA+aN=MriH+WcDeFLRe=uUcmjPTvHTAs95ZNEdeabCZGfJxM_3A@mail.gmail.com>

Congrats !

On Thu, Jan 5, 2017 at 10:26 PM, Stefan Negrea <snegrea at redhat.com> wrote:

> Hello,
>
> I am happy to announce release 0.23.0 of Hawkular Metrics. This release is
> anchored by performance and stability improvements.
>
> Here is a list of major changes:
>
>    - *Performance*
>       - Prevent BusyPoolException under heavy load due no available
>       connection and queue reaching max size of 256 (HWKMETRICS-542
>       <https://issues.jboss.org/browse/HWKMETRICS-542>)
>       - Gatling load tests have a new option (loops) to specify the
>       number of requests per client (HWKMETRICS-559
>       <https://issues.jboss.org/browse/HWKMETRICS-559>)
>    - *Deployment*
>       - Resolved an issue with resource-env-ref in component war (
>       HWKMETRICS-541 <https://issues.jboss.org/browse/HWKMETRICS-541>)
>       - Updated packaging to support deployments on WildFly 10.1.0 (
>       HWKMETRICS-558 <https://issues.jboss.org/browse/HWKMETRICS-558>)
>    - *REST API*
>       - Updated CORS validation to be applied prior to processing the
>       request; this solves an issue where some content is still returned even
>       thought a bad request status is returned (HWKMETRICS-554
>       <https://issues.jboss.org/browse/HWKMETRICS-554>)
>    - *Internal Monitoring*
>       - Hostname is now part of the metric id when creating and storing
>       internal metrics (HWKMETRICS-555
>       <https://issues.jboss.org/browse/HWKMETRICS-555>)
>    - *Hawkular Alerting - Updates*
>       - Added support for newer condition types to the email plugin (
>       HWKALERTS-208 <https://issues.jboss.org/browse/HWKALERTS-208>)
>       - Allow ExternalCondition to be fired on Event submission; external
>       conditions can now be matched via Event and Data submissions (
>       HWKALERTS-207 <https://issues.jboss.org/browse/HWKALERTS-207>)
>       - Added new NelsonCondition for native Nelson Rule detection; a
>       brand new condition type to perform automatic Nelson Rule detection of
>       misbehaving metrics. (HWKALERTS-209
>       <https://issues.jboss.org/browse/HWKALERTS-209>)
>
> *Hawkular Alerting - included*
>
>    - Version 1.5.0
>    <https://issues.jboss.org/projects/HWKALERTS/versions/12332918>
>    - Project details and repository: Github
>    <https://github.com/hawkular/hawkular-alerts>
>    - Documentation: REST API
>    <http://www.hawkular.org/docs/rest/rest-alerts.html>, Examples
>    <https://github.com/hawkular/hawkular-alerts/tree/master/examples>, Developer
>    Guide
>    <http://www.hawkular.org/community/docs/developer-guide/alerts.html>
>
> *Hawkular Metrics Clients*
>
>    - Python: https://github.com/hawkular/hawkular-client-python
>    - Go: https://github.com/hawkular/hawkular-client-go
>    - Ruby: https://github.com/hawkular/hawkular-client-ruby
>    - Java: https://github.com/hawkular/hawkular-client-java
>
>
>
> *Release Links*
>
> Github Release: https://github.com/hawkular/hawkular-metrics/
> releases/tag/0.23.0
>
> JBoss Nexus Maven artifacts:
> http://origin-repository.jboss.org/nexus/content/repositorie
> s/public/org/hawkular/metrics/
>
> Jira release tracker:
> https://issues.jboss.org/projects/HWKMETRICS/versions/12332805
>
> A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, Joel
> Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their project
> contributions.
>
>
> Thank you,
> Stefan Negrea
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170106/d3e89c63/attachment-0001.html 

From tsegismo at redhat.com  Fri Jan  6 11:39:31 2017
From: tsegismo at redhat.com (Thomas Segismont)
Date: Fri, 6 Jan 2017 17:39:31 +0100
Subject: [Hawkular-dev] Hawkular Metrics 0.23.0 - Release
In-Reply-To: <CADt-fV8mWMBdZYLPps9+M_W2E1TON-FQYmrEAj+1Q+-VKFKhFA@mail.gmail.com>
References: <CADt-fV8mWMBdZYLPps9+M_W2E1TON-FQYmrEAj+1Q+-VKFKhFA@mail.gmail.com>
Message-ID: <CAHnC0xixZqV0A_1xeneU7yRKTgdAqAxVgecTFkOU9LLFarmdPw@mail.gmail.com>

Congrats!

i think there is a mistake in the announcement: Gatling load tests already
had a loops param. The new param is duration IIUC

2017-01-05 22:26 GMT+01:00 Stefan Negrea <snegrea at redhat.com>:

> Hello,
>
> I am happy to announce release 0.23.0 of Hawkular Metrics. This release is
> anchored by performance and stability improvements.
>
> Here is a list of major changes:
>
>    - *Performance*
>       - Prevent BusyPoolException under heavy load due no available
>       connection and queue reaching max size of 256 (HWKMETRICS-542
>       <https://issues.jboss.org/browse/HWKMETRICS-542>)
>       - Gatling load tests have a new option (loops) to specify the
>       number of requests per client (HWKMETRICS-559
>       <https://issues.jboss.org/browse/HWKMETRICS-559>)
>    - *Deployment*
>       - Resolved an issue with resource-env-ref in component war (
>       HWKMETRICS-541 <https://issues.jboss.org/browse/HWKMETRICS-541>)
>       - Updated packaging to support deployments on WildFly 10.1.0 (
>       HWKMETRICS-558 <https://issues.jboss.org/browse/HWKMETRICS-558>)
>    - *REST API*
>       - Updated CORS validation to be applied prior to processing the
>       request; this solves an issue where some content is still returned even
>       thought a bad request status is returned (HWKMETRICS-554
>       <https://issues.jboss.org/browse/HWKMETRICS-554>)
>    - *Internal Monitoring*
>       - Hostname is now part of the metric id when creating and storing
>       internal metrics (HWKMETRICS-555
>       <https://issues.jboss.org/browse/HWKMETRICS-555>)
>    - *Hawkular Alerting - Updates*
>       - Added support for newer condition types to the email plugin (
>       HWKALERTS-208 <https://issues.jboss.org/browse/HWKALERTS-208>)
>       - Allow ExternalCondition to be fired on Event submission; external
>       conditions can now be matched via Event and Data submissions (
>       HWKALERTS-207 <https://issues.jboss.org/browse/HWKALERTS-207>)
>       - Added new NelsonCondition for native Nelson Rule detection; a
>       brand new condition type to perform automatic Nelson Rule detection of
>       misbehaving metrics. (HWKALERTS-209
>       <https://issues.jboss.org/browse/HWKALERTS-209>)
>
> *Hawkular Alerting - included*
>
>    - Version 1.5.0
>    <https://issues.jboss.org/projects/HWKALERTS/versions/12332918>
>    - Project details and repository: Github
>    <https://github.com/hawkular/hawkular-alerts>
>    - Documentation: REST API
>    <http://www.hawkular.org/docs/rest/rest-alerts.html>, Examples
>    <https://github.com/hawkular/hawkular-alerts/tree/master/examples>, Developer
>    Guide
>    <http://www.hawkular.org/community/docs/developer-guide/alerts.html>
>
> *Hawkular Metrics Clients*
>
>    - Python: https://github.com/hawkular/hawkular-client-python
>    - Go: https://github.com/hawkular/hawkular-client-go
>    - Ruby: https://github.com/hawkular/hawkular-client-ruby
>    - Java: https://github.com/hawkular/hawkular-client-java
>
>
>
> *Release Links*
>
> Github Release: https://github.com/hawkular/hawkular-metrics/
> releases/tag/0.23.0
>
> JBoss Nexus Maven artifacts:
> http://origin-repository.jboss.org/nexus/content/repositorie
> s/public/org/hawkular/metrics/
>
> Jira release tracker:
> https://issues.jboss.org/projects/HWKMETRICS/versions/12332805
>
> A big "Thank you" goes to John Sanda, Matt Wringe, Michael Burman, Joel
> Takvorian, Jay Shaughnessy, Lucas Ponce, and Heiko Rupp for their project
> contributions.
>
>
> Thank you,
> Stefan Negrea
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170106/ac041642/attachment.html 

From mazz at redhat.com  Fri Jan  6 20:44:10 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Fri, 6 Jan 2017 20:44:10 -0500 (EST)
Subject: [Hawkular-dev] some new hawkular openshift agent stuff
In-Reply-To: <1372949925.3660393.1483753025341.JavaMail.zimbra@redhat.com>
Message-ID: <1555198452.3660625.1483753450272.JavaMail.zimbra@redhat.com>

FYI: some new things went into HOSA (that's the Hawkular OpenShift Agent for the uninitiated).

1. The agent now emits its own metrics and can monitor itself. Right now it just emits some basic "go" metrics like memory usage, CPU usage, etc along with one agent-specific one - a counter that counts the number of data points it has collected in its lifetime. We'll add more metrics as we figure out the things people want to see, but we have the infrastructure in place now.

2. The agent is deployed as a daemonset. This means as new nodes are brought online, an agent will go along with it (or so I'm told :)

3. The agent has changed the way it discovers what to monitor - it no longer looks at annotations on pods to determine where the configmaps are for those pods. Instead, it looks up volume declarations to see if there is an agent configmap defined. This was done to be ready for the future when new security constraints will be introduced in OpenShift which would have broken our annotation approach. This approach using volumes should not hit that issue.

NOTE: If you are building the latest agent from master, we added some dependencies so you have to update your dependencies via Glide by using the "make update-deps" target prior to building from source.

From gbrown at redhat.com  Sat Jan  7 07:36:02 2017
From: gbrown at redhat.com (Gary Brown)
Date: Sat, 7 Jan 2017 07:36:02 -0500 (EST)
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <458762184.11456833.1483790677344.JavaMail.zimbra@redhat.com>
Message-ID: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>

Hi

Wanted to discuss a proposal for recording some metric data captured from Hawkular APM in Hawkular Metrics.

For those not familiar with Hawkular APM, it captures the end to end trace instance (think of it as a distributed call stack), for each invocation of an application. This trace can include information about the communications between services, but can also include details about internal components used within the services (e.g. EJBs, database calls, etc).

First point is that if we were to record duration metrics for each 'span' captured (i.e. scope within which a task is performed), for each invocation of an application, then it would result in a large amount of data that may be of no interest. So we need to find a way for end users/developers to express which key points within an application they do want recorded as metrics.

The proposal is to allow the application/services to define a tag/property on the spans of interest, e.g. 'kpi', that would indicate to the server that the duration value for the span should be stored in H-Metrics. The value for the tag should define the name/description of the KPI.

If considered a suitable solution, then we can also propose it as a standard tag in the OpenTracing standard.

There are a couple of metrics that we could record by default, first being the trace instance completion time, and the second possibly being the individual service response times (although this could potentially also be governed by the 'kpi' tag).

Thoughts?

Regards
Gary



From jsanda at redhat.com  Sat Jan  7 11:09:56 2017
From: jsanda at redhat.com (John Sanda)
Date: Sat, 7 Jan 2017 11:09:56 -0500
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
Message-ID: <BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>

I want to make sure I understand the difference between trace and span. A trace captures the call between two services whether those are REST endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within that A calls C which in turn calls D, etc. and then finally B is called. Would a span measure the duration of the call between C and D?

Are all spans captured/recorded by default?

Would this be optional functionality? I am wondering because it obviously requires having Hawkular Metrics running which also means Cassandra.

Under what tenant would you store these metrics?

> On Jan 7, 2017, at 7:36 AM, Gary Brown <gbrown at redhat.com> wrote:
> 
> Hi
> 
> Wanted to discuss a proposal for recording some metric data captured from Hawkular APM in Hawkular Metrics.
> 
> For those not familiar with Hawkular APM, it captures the end to end trace instance (think of it as a distributed call stack), for each invocation of an application. This trace can include information about the communications between services, but can also include details about internal components used within the services (e.g. EJBs, database calls, etc).
> 
> First point is that if we were to record duration metrics for each 'span' captured (i.e. scope within which a task is performed), for each invocation of an application, then it would result in a large amount of data that may be of no interest. So we need to find a way for end users/developers to express which key points within an application they do want recorded as metrics.
> 
> The proposal is to allow the application/services to define a tag/property on the spans of interest, e.g. 'kpi', that would indicate to the server that the duration value for the span should be stored in H-Metrics. The value for the tag should define the name/description of the KPI.
> 
> If considered a suitable solution, then we can also propose it as a standard tag in the OpenTracing standard.
> 
> There are a couple of metrics that we could record by default, first being the trace instance completion time, and the second possibly being the individual service response times (although this could potentially also be governed by the 'kpi' tag).
> 
> Thoughts?
> 
> Regards
> Gary
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev



From ploffay at redhat.com  Mon Jan  9 03:03:47 2017
From: ploffay at redhat.com (Pavol Loffay)
Date: Mon, 9 Jan 2017 09:03:47 +0100
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
Message-ID: <CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>

Hello,

let me start with terminology explanation:
Span is one unit of work. It can be a method/REST endpoint invocation or
just any unit of work.
List of spans form a trace which is currently tree like structure(every
span has its parent, except root span).
Span captures its start and end, therefore it is possible to calculate a
duration.

By default all spans reported to the server are persisted in elasticsearch.

I think there are two different things in this email:

1. OpenTracing standard "kpi" tag. Tags are key-value pairs, so I guess
value would be a metric name (e.g. "user-registration": duration of user
registration which consist of multiple service calls).  I think that any
duration is considered as KPI. Typically when users want to show durations
for a particular service they can use filtering by tags(custom/standard).
Is there really need for KPI tag?

2. integration with h-metrics. What are the benefits for users? Integration
with grafana? Cannot be grafana directly connected to elastic APM is using?
All durations (span, trace) are stored in our backend and published to JMS
so an integration with other systems should be possible.


Regards,


On Sat, Jan 7, 2017 at 5:09 PM, John Sanda <jsanda at redhat.com> wrote:

> I want to make sure I understand the difference between trace and span. A
> trace captures the call between two services whether those are REST
> endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within that
> A calls C which in turn calls D, etc. and then finally B is called. Would a
> span measure the duration of the call between C and D?
>
> Are all spans captured/recorded by default?
>
> Would this be optional functionality? I am wondering because it obviously
> requires having Hawkular Metrics running which also means Cassandra.
>
> Under what tenant would you store these metrics?
>
> > On Jan 7, 2017, at 7:36 AM, Gary Brown <gbrown at redhat.com> wrote:
> >
> > Hi
> >
> > Wanted to discuss a proposal for recording some metric data captured
> from Hawkular APM in Hawkular Metrics.
> >
> > For those not familiar with Hawkular APM, it captures the end to end
> trace instance (think of it as a distributed call stack), for each
> invocation of an application. This trace can include information about the
> communications between services, but can also include details about
> internal components used within the services (e.g. EJBs, database calls,
> etc).
> >
> > First point is that if we were to record duration metrics for each
> 'span' captured (i.e. scope within which a task is performed), for each
> invocation of an application, then it would result in a large amount of
> data that may be of no interest. So we need to find a way for end
> users/developers to express which key points within an application they do
> want recorded as metrics.
> >
> > The proposal is to allow the application/services to define a
> tag/property on the spans of interest, e.g. 'kpi', that would indicate to
> the server that the duration value for the span should be stored in
> H-Metrics. The value for the tag should define the name/description of the
> KPI.
> >
> > If considered a suitable solution, then we can also propose it as a
> standard tag in the OpenTracing standard.
> >
> > There are a couple of metrics that we could record by default, first
> being the trace instance completion time, and the second possibly being the
> individual service response times (although this could potentially also be
> governed by the 'kpi' tag).
> >
> > Thoughts?
> >
> > Regards
> > Gary
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>



-- 
Pavol Loffay
Cell: +421948286055
Red Hat
Purky?ova 111 TPB-B
612 45 Brno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170109/bcd79588/attachment-0001.html 

From hrupp at redhat.com  Mon Jan  9 03:40:29 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Mon, 09 Jan 2017 09:40:29 +0100
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
Message-ID: <AF4D534F-A56F-4483-9BDE-2A9E2B6FE636@redhat.com>

Hi,

On 7 Jan 2017, at 13:36, Gary Brown wrote:

> Wanted to discuss a proposal for recording some metric data captured 
> from Hawkular APM in Hawkular Metrics.

What exactly is the goal here?

We were talking in the past about potentially storing the APM
data inside of Cassandra (plain or via H-Metrics), but what
you wrote does not look like this is it.

From gbrown at redhat.com  Mon Jan  9 04:05:37 2017
From: gbrown at redhat.com (Gary Brown)
Date: Mon, 9 Jan 2017 04:05:37 -0500 (EST)
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
Message-ID: <1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>



----- Original Message -----
> Hello,
> 
> let me start with terminology explanation:
> Span is one unit of work. It can be a method/REST endpoint invocation or just
> any unit of work.
> List of spans form a trace which is currently tree like structure(every span
> has its parent, except root span).
> Span captures its start and end, therefore it is possible to calculate a
> duration.
> 
> By default all spans reported to the server are persisted in elasticsearch.
> 
> I think there are two different things in this email:
> 
> 1. OpenTracing standard "kpi" tag. Tags are key-value pairs, so I guess value
> would be a metric name (e.g. "user-registration": duration of user
> registration which consist of multiple service calls). I think that any
> duration is considered as KPI. Typically when users want to show durations
> for a particular service they can use filtering by tags(custom/standard). Is
> there really need for KPI tag?

This is about long term metric storage and analysis, rather than short term user based filtering. So if we want to be able to identify the key metrics that are of long term interest, it would need to either be done through some configuration on the server, or as suggested in this proposal by allowing the application to tag the relevant spans.

> 
> 2. integration with h-metrics. What are the benefits for users? Integration
> with grafana? Cannot be grafana directly connected to elastic APM is using?
> All durations (span, trace) are stored in our backend and published to JMS
> so an integration with other systems should be possible.

Long term storage of specific metrics. APM is collecting a large amount of information, so it is likely that it will only be retained for a limited time frame.

Regards
Gary

> 
> 
> Regards,
> 
> 
> On Sat, Jan 7, 2017 at 5:09 PM, John Sanda < jsanda at redhat.com > wrote:
> 
> 
> I want to make sure I understand the difference between trace and span. A
> trace captures the call between two services whether those are REST
> endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within that
> A calls C which in turn calls D, etc. and then finally B is called. Would a
> span measure the duration of the call between C and D?
> 
> Are all spans captured/recorded by default?
> 
> Would this be optional functionality? I am wondering because it obviously
> requires having Hawkular Metrics running which also means Cassandra.
> 
> Under what tenant would you store these metrics?
> 
> > On Jan 7, 2017, at 7:36 AM, Gary Brown < gbrown at redhat.com > wrote:
> > 
> > Hi
> > 
> > Wanted to discuss a proposal for recording some metric data captured from
> > Hawkular APM in Hawkular Metrics.
> > 
> > For those not familiar with Hawkular APM, it captures the end to end trace
> > instance (think of it as a distributed call stack), for each invocation of
> > an application. This trace can include information about the
> > communications between services, but can also include details about
> > internal components used within the services (e.g. EJBs, database calls,
> > etc).
> > 
> > First point is that if we were to record duration metrics for each 'span'
> > captured (i.e. scope within which a task is performed), for each
> > invocation of an application, then it would result in a large amount of
> > data that may be of no interest. So we need to find a way for end
> > users/developers to express which key points within an application they do
> > want recorded as metrics.
> > 
> > The proposal is to allow the application/services to define a tag/property
> > on the spans of interest, e.g. 'kpi', that would indicate to the server
> > that the duration value for the span should be stored in H-Metrics. The
> > value for the tag should define the name/description of the KPI.
> > 
> > If considered a suitable solution, then we can also propose it as a
> > standard tag in the OpenTracing standard.
> > 
> > There are a couple of metrics that we could record by default, first being
> > the trace instance completion time, and the second possibly being the
> > individual service response times (although this could potentially also be
> > governed by the 'kpi' tag).
> > 
> > Thoughts?
> > 
> > Regards
> > Gary
> > 
> > 
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> 
> --
> Pavol Loffay
> Cell: +421948286055
> Red Hat
> Purky?ova 111 TPB-B
> 612 45 Brno
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 


From gbrown at redhat.com  Mon Jan  9 04:08:20 2017
From: gbrown at redhat.com (Gary Brown)
Date: Mon, 9 Jan 2017 04:08:20 -0500 (EST)
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <AF4D534F-A56F-4483-9BDE-2A9E2B6FE636@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<AF4D534F-A56F-4483-9BDE-2A9E2B6FE636@redhat.com>
Message-ID: <1957958768.11585062.1483952900650.JavaMail.zimbra@redhat.com>



----- Original Message -----
> Hi,
> 
> On 7 Jan 2017, at 13:36, Gary Brown wrote:
> 
> > Wanted to discuss a proposal for recording some metric data captured
> > from Hawkular APM in Hawkular Metrics.
> 
> What exactly is the goal here?
> 
> We were talking in the past about potentially storing the APM
> data inside of Cassandra (plain or via H-Metrics), but what
> you wrote does not look like this is it.

The aim is still to try to use Cassandra as the persistence mechanism, but you are correct, this is not related to that.

I think the quantity of information being collected by APM is useful over a certain timeframe, where users want to investigate current application performance and take relevant actions. But only a subset of the information will be of interest for longer term analyais - and this is where H-Metrics could be useful, with its ability to aggregate and compression metrics over a long period of time.

Regards
Gary


> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From ploffay at redhat.com  Mon Jan  9 04:42:37 2017
From: ploffay at redhat.com (Pavol Loffay)
Date: Mon, 9 Jan 2017 10:42:37 +0100
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
	<1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
Message-ID: <CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>

On Mon, Jan 9, 2017 at 10:05 AM, Gary Brown <gbrown at redhat.com> wrote:

>
>
> ----- Original Message -----
> > Hello,
> >
> > let me start with terminology explanation:
> > Span is one unit of work. It can be a method/REST endpoint invocation or
> just
> > any unit of work.
> > List of spans form a trace which is currently tree like structure(every
> span
> > has its parent, except root span).
> > Span captures its start and end, therefore it is possible to calculate a
> > duration.
> >
> > By default all spans reported to the server are persisted in
> elasticsearch.
> >
> > I think there are two different things in this email:
> >
> > 1. OpenTracing standard "kpi" tag. Tags are key-value pairs, so I guess
> value
> > would be a metric name (e.g. "user-registration": duration of user
> > registration which consist of multiple service calls). I think that any
> > duration is considered as KPI. Typically when users want to show
> durations
> > for a particular service they can use filtering by
> tags(custom/standard). Is
> > there really need for KPI tag?
>
> This is about long term metric storage and analysis, rather than short
> term user based filtering. So if we want to be able to identify the key
> metrics that are of long term interest, it would need to either be done
> through some configuration on the server, or as suggested in this proposal
> by allowing the application to tag the relevant spans.
>
>
Long term storage wasn't mentioned in the original email. I ask for a
better explanation "KPI" tag. If there is a really need for it.


> >
> > 2. integration with h-metrics. What are the benefits for users?
> Integration
> > with grafana? Cannot be grafana directly connected to elastic APM is
> using?
> > All durations (span, trace) are stored in our backend and published to
> JMS
> > so an integration with other systems should be possible.
>
> Long term storage of specific metrics. APM is collecting a large amount of
> information, so it is likely that it will only be retained for a limited
> time frame.
>


I think that tracing tool is mostly used for a real time analysis and
debugging or tool for solving performance problems and testing
"canary"/"blue-green" deployments, but long term storage can useful or have
an access to aggregated results from older periods of time. Statistics
(average/median) of service response times. We should probably look at
zipkin ML and look if somebody was looking for this feature.

It would actually make sense to provide some kind of reports as APM does
higher level aggregations. Question is if there is an interest of this
feature. I'm not very familiar with grafana but from what I read it can
connect to elasticsearch so maybe it can be used with APM to show dasboards
and reports.


> Regards
> Gary
>
> >
> >
> > Regards,
> >
> >
> > On Sat, Jan 7, 2017 at 5:09 PM, John Sanda < jsanda at redhat.com > wrote:
> >
> >
> > I want to make sure I understand the difference between trace and span. A
> > trace captures the call between two services whether those are REST
> > endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within
> that
> > A calls C which in turn calls D, etc. and then finally B is called.
> Would a
> > span measure the duration of the call between C and D?
> >
> > Are all spans captured/recorded by default?
> >
> > Would this be optional functionality? I am wondering because it obviously
> > requires having Hawkular Metrics running which also means Cassandra.
> >
> > Under what tenant would you store these metrics?
> >
> > > On Jan 7, 2017, at 7:36 AM, Gary Brown < gbrown at redhat.com > wrote:
> > >
> > > Hi
> > >
> > > Wanted to discuss a proposal for recording some metric data captured
> from
> > > Hawkular APM in Hawkular Metrics.
> > >
> > > For those not familiar with Hawkular APM, it captures the end to end
> trace
> > > instance (think of it as a distributed call stack), for each
> invocation of
> > > an application. This trace can include information about the
> > > communications between services, but can also include details about
> > > internal components used within the services (e.g. EJBs, database
> calls,
> > > etc).
> > >
> > > First point is that if we were to record duration metrics for each
> 'span'
> > > captured (i.e. scope within which a task is performed), for each
> > > invocation of an application, then it would result in a large amount of
> > > data that may be of no interest. So we need to find a way for end
> > > users/developers to express which key points within an application
> they do
> > > want recorded as metrics.
> > >
> > > The proposal is to allow the application/services to define a
> tag/property
> > > on the spans of interest, e.g. 'kpi', that would indicate to the server
> > > that the duration value for the span should be stored in H-Metrics. The
> > > value for the tag should define the name/description of the KPI.
> > >
> > > If considered a suitable solution, then we can also propose it as a
> > > standard tag in the OpenTracing standard.
> > >
> > > There are a couple of metrics that we could record by default, first
> being
> > > the trace instance completion time, and the second possibly being the
> > > individual service response times (although this could potentially
> also be
> > > governed by the 'kpi' tag).
> > >
> > > Thoughts?
> > >
> > > Regards
> > > Gary
> > >
> > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> >
> > --
> > Pavol Loffay
> > Cell: +421948286055
> > Red Hat
> > Purky?ova 111 TPB-B
> > 612 45 Brno
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>



-- 
Pavol Loffay
Cell: +421948286055
Red Hat
Purky?ova 111 TPB-B
612 45 Brno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170109/461c5db4/attachment-0001.html 

From gbrown at redhat.com  Mon Jan  9 05:10:18 2017
From: gbrown at redhat.com (Gary Brown)
Date: Mon, 9 Jan 2017 05:10:18 -0500 (EST)
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
	<1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
	<CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>
Message-ID: <2018592486.11597381.1483956618881.JavaMail.zimbra@redhat.com>

Apologies I didn't set the context better :)

There are a couple of reasons to investigate integration with H-Metrics.

1) Long term storage of metrics, as mentioned - primarily because we are currently collecting all information for use in short term analysis. If some of that information is of long term interest to an end user, then having the ability to identify the relevant information is necessary. Two options are (a) mark the data at the point it is generated (i.e. tag it), or (b) define server side configuration to extract the relevant data. Option (a) just seems the most straightforward.

2) Integration with H-Alerts for free. So although we can (and have) integrated directly, it may be better from an integrated architecture perspective to feed the metrics through H-Metrics.

Regards
Gary

----- Original Message -----
> 
> 
> On Mon, Jan 9, 2017 at 10:05 AM, Gary Brown < gbrown at redhat.com > wrote:
> 
> 
> 
> 
> ----- Original Message -----
> > Hello,
> > 
> > let me start with terminology explanation:
> > Span is one unit of work. It can be a method/REST endpoint invocation or
> > just
> > any unit of work.
> > List of spans form a trace which is currently tree like structure(every
> > span
> > has its parent, except root span).
> > Span captures its start and end, therefore it is possible to calculate a
> > duration.
> > 
> > By default all spans reported to the server are persisted in elasticsearch.
> > 
> > I think there are two different things in this email:
> > 
> > 1. OpenTracing standard "kpi" tag. Tags are key-value pairs, so I guess
> > value
> > would be a metric name (e.g. "user-registration": duration of user
> > registration which consist of multiple service calls). I think that any
> > duration is considered as KPI. Typically when users want to show durations
> > for a particular service they can use filtering by tags(custom/standard).
> > Is
> > there really need for KPI tag?
> 
> This is about long term metric storage and analysis, rather than short term
> user based filtering. So if we want to be able to identify the key metrics
> that are of long term interest, it would need to either be done through some
> configuration on the server, or as suggested in this proposal by allowing
> the application to tag the relevant spans.
> 
> 
> Long term storage wasn't mentioned in the original email. I ask for a better
> explanation "KPI" tag. If there is a really need for it.
> 
> 
> > 
> > 2. integration with h-metrics. What are the benefits for users? Integration
> > with grafana? Cannot be grafana directly connected to elastic APM is using?
> > All durations (span, trace) are stored in our backend and published to JMS
> > so an integration with other systems should be possible.
> 
> Long term storage of specific metrics. APM is collecting a large amount of
> information, so it is likely that it will only be retained for a limited
> time frame.
> 
> 
> I think that tracing tool is mostly used for a real time analysis and
> debugging or tool for solving performance problems and testing
> "canary"/"blue-green" deployments, but long term storage can useful or have
> an access to aggregated results from older periods of time. Statistics
> (average/median) of service response times. We should probably look at
> zipkin ML and look if somebody was looking for this feature.
> 
> It would actually make sense to provide some kind of reports as APM does
> higher level aggregations. Question is if there is an interest of this
> feature. I'm not very familiar with grafana but from what I read it can
> connect to elasticsearch so maybe it can be used with APM to show dasboards
> and reports.
> 
> 
> 
> 
> Regards
> Gary
> 
> > 
> > 
> > Regards,
> > 
> > 
> > On Sat, Jan 7, 2017 at 5:09 PM, John Sanda < jsanda at redhat.com > wrote:
> > 
> > 
> > I want to make sure I understand the difference between trace and span. A
> > trace captures the call between two services whether those are REST
> > endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within that
> > A calls C which in turn calls D, etc. and then finally B is called. Would a
> > span measure the duration of the call between C and D?
> > 
> > Are all spans captured/recorded by default?
> > 
> > Would this be optional functionality? I am wondering because it obviously
> > requires having Hawkular Metrics running which also means Cassandra.
> > 
> > Under what tenant would you store these metrics?
> > 
> > > On Jan 7, 2017, at 7:36 AM, Gary Brown < gbrown at redhat.com > wrote:
> > > 
> > > Hi
> > > 
> > > Wanted to discuss a proposal for recording some metric data captured from
> > > Hawkular APM in Hawkular Metrics.
> > > 
> > > For those not familiar with Hawkular APM, it captures the end to end
> > > trace
> > > instance (think of it as a distributed call stack), for each invocation
> > > of
> > > an application. This trace can include information about the
> > > communications between services, but can also include details about
> > > internal components used within the services (e.g. EJBs, database calls,
> > > etc).
> > > 
> > > First point is that if we were to record duration metrics for each 'span'
> > > captured (i.e. scope within which a task is performed), for each
> > > invocation of an application, then it would result in a large amount of
> > > data that may be of no interest. So we need to find a way for end
> > > users/developers to express which key points within an application they
> > > do
> > > want recorded as metrics.
> > > 
> > > The proposal is to allow the application/services to define a
> > > tag/property
> > > on the spans of interest, e.g. 'kpi', that would indicate to the server
> > > that the duration value for the span should be stored in H-Metrics. The
> > > value for the tag should define the name/description of the KPI.
> > > 
> > > If considered a suitable solution, then we can also propose it as a
> > > standard tag in the OpenTracing standard.
> > > 
> > > There are a couple of metrics that we could record by default, first
> > > being
> > > the trace instance completion time, and the second possibly being the
> > > individual service response times (although this could potentially also
> > > be
> > > governed by the 'kpi' tag).
> > > 
> > > Thoughts?
> > > 
> > > Regards
> > > Gary
> > > 
> > > 
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > 
> > 
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > 
> > 
> > 
> > --
> > Pavol Loffay
> > Cell: +421948286055
> > Red Hat
> > Purky?ova 111 TPB-B
> > 612 45 Brno
> > 
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> 
> --
> Pavol Loffay
> Cell: +421948286055
> Red Hat
> Purky?ova 111 TPB-B
> 612 45 Brno
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 


From jshaughn at redhat.com  Mon Jan  9 11:02:41 2017
From: jshaughn at redhat.com (Jay Shaughnessy)
Date: Mon, 9 Jan 2017 11:02:41 -0500
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <2018592486.11597381.1483956618881.JavaMail.zimbra@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
	<1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
	<CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>
	<2018592486.11597381.1483956618881.JavaMail.zimbra@redhat.com>
Message-ID: <e77f5167-8741-0f3f-4bec-d8861a0c7594@redhat.com>


For those of us not overly familiar with APM let me see if I can 
summarize what I've read so far and see if I'm close...

Trace:  Made up of a set of spans, a trace represents a high level unit 
of work, probably a complete piece of business logic, or service 
endpoint.  And whose duration is likely more reflective of 
user-experience, like a response-time.  As such it would seem like if a 
trace duration is within expectation then there isn't too much to worry 
about and the spans making up that trace are likely not overly relevant, 
unless you want to look at response-time tuning and improvement.   But, 
a "kpi" (Key Performance Indicator, right?) tag on a trace could in 
general be useful, as you may want to be able to establish a baseline 
and perform alerting on durations that reflect user experience.

If the above is fairly on target it would seem like storing trace 
duration metrics would be a good idea but storing span duration metrics 
would be a waste, until such time that the trace is not well behaved.  
It said in one of the replies that all spans were stored in ES, I assume 
this is just for a brief time to support some ad-hoc analysis in the 
short term.

As for long term storage I'd agree that it should be user-directed, tags 
on the APM side to decide what data to store seems like a good idea, 
especially at the trace level. And if the trace behaves badly (perhaps 
based on an alert) then spans for that trace perhaps should get tagged 
(at least until the issue is resolved).

I don't really know why the solution requires ES but likely it was just 
available  and easy to use.  If we can start to incorporate Hmetrics 
that seems good, a complete migration to Hmetrics seems better ;-)   As 
Gary mentioned, certainly there is a benefit of having Halerting built 
in for use with metrics being stored.   From an alerting perspective 
we're also working with Gary/APM to perform alerting on APM events, 
which could also potentially trigger some automated tagging.



On 1/9/2017 5:10 AM, Gary Brown wrote:
> Apologies I didn't set the context better :)
>
> There are a couple of reasons to investigate integration with H-Metrics.
>
> 1) Long term storage of metrics, as mentioned - primarily because we are currently collecting all information for use in short term analysis. If some of that information is of long term interest to an end user, then having the ability to identify the relevant information is necessary. Two options are (a) mark the data at the point it is generated (i.e. tag it), or (b) define server side configuration to extract the relevant data. Option (a) just seems the most straightforward.
>
> 2) Integration with H-Alerts for free. So although we can (and have) integrated directly, it may be better from an integrated architecture perspective to feed the metrics through H-Metrics.
>
> Regards
> Gary
>
> ----- Original Message -----
>>
>> On Mon, Jan 9, 2017 at 10:05 AM, Gary Brown < gbrown at redhat.com > wrote:
>>
>>
>>
>>
>> ----- Original Message -----
>>> Hello,
>>>
>>> let me start with terminology explanation:
>>> Span is one unit of work. It can be a method/REST endpoint invocation or
>>> just
>>> any unit of work.
>>> List of spans form a trace which is currently tree like structure(every
>>> span
>>> has its parent, except root span).
>>> Span captures its start and end, therefore it is possible to calculate a
>>> duration.
>>>
>>> By default all spans reported to the server are persisted in elasticsearch.
>>>
>>> I think there are two different things in this email:
>>>
>>> 1. OpenTracing standard "kpi" tag. Tags are key-value pairs, so I guess
>>> value
>>> would be a metric name (e.g. "user-registration": duration of user
>>> registration which consist of multiple service calls). I think that any
>>> duration is considered as KPI. Typically when users want to show durations
>>> for a particular service they can use filtering by tags(custom/standard).
>>> Is
>>> there really need for KPI tag?
>> This is about long term metric storage and analysis, rather than short term
>> user based filtering. So if we want to be able to identify the key metrics
>> that are of long term interest, it would need to either be done through some
>> configuration on the server, or as suggested in this proposal by allowing
>> the application to tag the relevant spans.
>>
>>
>> Long term storage wasn't mentioned in the original email. I ask for a better
>> explanation "KPI" tag. If there is a really need for it.
>>
>>
>>> 2. integration with h-metrics. What are the benefits for users? Integration
>>> with grafana? Cannot be grafana directly connected to elastic APM is using?
>>> All durations (span, trace) are stored in our backend and published to JMS
>>> so an integration with other systems should be possible.
>> Long term storage of specific metrics. APM is collecting a large amount of
>> information, so it is likely that it will only be retained for a limited
>> time frame.
>>
>>
>> I think that tracing tool is mostly used for a real time analysis and
>> debugging or tool for solving performance problems and testing
>> "canary"/"blue-green" deployments, but long term storage can useful or have
>> an access to aggregated results from older periods of time. Statistics
>> (average/median) of service response times. We should probably look at
>> zipkin ML and look if somebody was looking for this feature.
>>
>> It would actually make sense to provide some kind of reports as APM does
>> higher level aggregations. Question is if there is an interest of this
>> feature. I'm not very familiar with grafana but from what I read it can
>> connect to elasticsearch so maybe it can be used with APM to show dasboards
>> and reports.
>>
>>
>>
>>
>> Regards
>> Gary
>>
>>>
>>> Regards,
>>>
>>>
>>> On Sat, Jan 7, 2017 at 5:09 PM, John Sanda < jsanda at redhat.com > wrote:
>>>
>>>
>>> I want to make sure I understand the difference between trace and span. A
>>> trace captures the call between two services whether those are REST
>>> endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within that
>>> A calls C which in turn calls D, etc. and then finally B is called. Would a
>>> span measure the duration of the call between C and D?
>>>
>>> Are all spans captured/recorded by default?
>>>
>>> Would this be optional functionality? I am wondering because it obviously
>>> requires having Hawkular Metrics running which also means Cassandra.
>>>
>>> Under what tenant would you store these metrics?
>>>
>>>> On Jan 7, 2017, at 7:36 AM, Gary Brown < gbrown at redhat.com > wrote:
>>>>
>>>> Hi
>>>>
>>>> Wanted to discuss a proposal for recording some metric data captured from
>>>> Hawkular APM in Hawkular Metrics.
>>>>
>>>> For those not familiar with Hawkular APM, it captures the end to end
>>>> trace
>>>> instance (think of it as a distributed call stack), for each invocation
>>>> of
>>>> an application. This trace can include information about the
>>>> communications between services, but can also include details about
>>>> internal components used within the services (e.g. EJBs, database calls,
>>>> etc).
>>>>
>>>> First point is that if we were to record duration metrics for each 'span'
>>>> captured (i.e. scope within which a task is performed), for each
>>>> invocation of an application, then it would result in a large amount of
>>>> data that may be of no interest. So we need to find a way for end
>>>> users/developers to express which key points within an application they
>>>> do
>>>> want recorded as metrics.
>>>>
>>>> The proposal is to allow the application/services to define a
>>>> tag/property
>>>> on the spans of interest, e.g. 'kpi', that would indicate to the server
>>>> that the duration value for the span should be stored in H-Metrics. The
>>>> value for the tag should define the name/description of the KPI.
>>>>
>>>> If considered a suitable solution, then we can also propose it as a
>>>> standard tag in the OpenTracing standard.
>>>>
>>>> There are a couple of metrics that we could record by default, first
>>>> being
>>>> the trace instance completion time, and the second possibly being the
>>>> individual service response times (although this could potentially also
>>>> be
>>>> governed by the 'kpi' tag).
>>>>
>>>> Thoughts?
>>>>
>>>> Regards
>>>> Gary
>>>>
>>>>
>>>> _______________________________________________
>>>> hawkular-dev mailing list
>>>> hawkular-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>
>>>
>>>
>>> --
>>> Pavol Loffay
>>> Cell: +421948286055
>>> Red Hat
>>> Purky?ova 111 TPB-B
>>> 612 45 Brno
>>>
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>>
>>
>> --
>> Pavol Loffay
>> Cell: +421948286055
>> Red Hat
>> Purky?ova 111 TPB-B
>> 612 45 Brno
>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170109/a43353b1/attachment-0001.html 

From gbrown at redhat.com  Mon Jan  9 12:16:42 2017
From: gbrown at redhat.com (Gary Brown)
Date: Mon, 9 Jan 2017 12:16:42 -0500 (EST)
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <e77f5167-8741-0f3f-4bec-d8861a0c7594@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
	<1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
	<CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>
	<2018592486.11597381.1483956618881.JavaMail.zimbra@redhat.com>
	<e77f5167-8741-0f3f-4bec-d8861a0c7594@redhat.com>
Message-ID: <559164915.11771505.1483982202160.JavaMail.zimbra@redhat.com>

Hi Jay

Thanks - yes I think the complete set of span durations will only be required while performance analysis is relevant.

Tagging specific points in the business process may be more about business KPIs, that may be required for much long term analysis - possibly related to duration but also cardinality. May need to understand specific use cases better to determine the best way to deal with this.

Anyone got some good use cases, i.e. examples of useful business KPIs?

Regards
Gary

----- Original Message -----
> 
> For those of us not overly familiar with APM let me see if I can summarize
> what I've read so far and see if I'm close...
> 
> Trace: Made up of a set of spans, a trace represents a high level unit of
> work, probably a complete piece of business logic, or service endpoint. And
> whose duration is likely more reflective of user-experience, like a
> response-time. As such it would seem like if a trace duration is within
> expectation then there isn't too much to worry about and the spans making up
> that trace are likely not overly relevant, unless you want to look at
> response-time tuning and improvement. But, a "kpi" ( Key Performance
> Indicator , right?) tag on a trace could in general be useful, as you may
> want to be able to establish a baseline and perform alerting on durations
> that reflect user experience.
> 
> If the above is fairly on target it would seem like storing trace duration
> metrics would be a good idea but storing span duration metrics would be a
> waste, until such time that the trace is not well behaved. It said in one of
> the replies that all spans were stored in ES, I assume this is just for a
> brief time to support some ad-hoc analysis in the short term.
> 
> As for long term storage I'd agree that it should be user-directed, tags on
> the APM side to decide what data to store seems like a good idea, especially
> at the trace level. And if the trace behaves badly (perhaps based on an
> alert) then spans for that trace perhaps should get tagged (at least until
> the issue is resolved).
> 
> I don't really know why the solution requires ES but likely it was just
> available and easy to use. If we can start to incorporate Hmetrics that
> seems good, a complete migration to Hmetrics seems better ;-) As Gary
> mentioned, certainly there is a benefit of having Halerting built in for use
> with metrics being stored. From an alerting perspective we're also working
> with Gary/APM to perform alerting on APM events, which could also
> potentially trigger some automated tagging.
> 
> 
> 
> On 1/9/2017 5:10 AM, Gary Brown wrote:
> 
> 
> 
> Apologies I didn't set the context better :)
> 
> There are a couple of reasons to investigate integration with H-Metrics.
> 
> 1) Long term storage of metrics, as mentioned - primarily because we are
> currently collecting all information for use in short term analysis. If some
> of that information is of long term interest to an end user, then having the
> ability to identify the relevant information is necessary. Two options are
> (a) mark the data at the point it is generated (i.e. tag it), or (b) define
> server side configuration to extract the relevant data. Option (a) just
> seems the most straightforward.
> 
> 2) Integration with H-Alerts for free. So although we can (and have)
> integrated directly, it may be better from an integrated architecture
> perspective to feed the metrics through H-Metrics.
> 
> Regards
> Gary
> 
> ----- Original Message -----
> 
> 
> 
> On Mon, Jan 9, 2017 at 10:05 AM, Gary Brown < gbrown at redhat.com > wrote:
> 
> 
> 
> 
> ----- Original Message -----
> 
> 
> 
> Hello,
> 
> let me start with terminology explanation:
> Span is one unit of work. It can be a method/REST endpoint invocation or
> just
> any unit of work.
> List of spans form a trace which is currently tree like structure(every
> span
> has its parent, except root span).
> Span captures its start and end, therefore it is possible to calculate a
> duration.
> 
> By default all spans reported to the server are persisted in elasticsearch.
> 
> I think there are two different things in this email:
> 
> 1. OpenTracing standard "kpi" tag. Tags are key-value pairs, so I guess
> value
> would be a metric name (e.g. "user-registration": duration of user
> registration which consist of multiple service calls). I think that any
> duration is considered as KPI. Typically when users want to show durations
> for a particular service they can use filtering by tags(custom/standard).
> Is
> there really need for KPI tag?
> This is about long term metric storage and analysis, rather than short term
> user based filtering. So if we want to be able to identify the key metrics
> that are of long term interest, it would need to either be done through some
> configuration on the server, or as suggested in this proposal by allowing
> the application to tag the relevant spans.
> 
> 
> Long term storage wasn't mentioned in the original email. I ask for a better
> explanation "KPI" tag. If there is a really need for it.
> 
> 
> 
> 2. integration with h-metrics. What are the benefits for users? Integration
> with grafana? Cannot be grafana directly connected to elastic APM is using?
> All durations (span, trace) are stored in our backend and published to JMS
> so an integration with other systems should be possible.
> Long term storage of specific metrics. APM is collecting a large amount of
> information, so it is likely that it will only be retained for a limited
> time frame.
> 
> 
> I think that tracing tool is mostly used for a real time analysis and
> debugging or tool for solving performance problems and testing
> "canary"/"blue-green" deployments, but long term storage can useful or have
> an access to aggregated results from older periods of time. Statistics
> (average/median) of service response times. We should probably look at
> zipkin ML and look if somebody was looking for this feature.
> 
> It would actually make sense to provide some kind of reports as APM does
> higher level aggregations. Question is if there is an interest of this
> feature. I'm not very familiar with grafana but from what I read it can
> connect to elasticsearch so maybe it can be used with APM to show dasboards
> and reports.
> 
> 
> 
> 
> Regards
> Gary
> 
> 
> 
> Regards,
> 
> 
> On Sat, Jan 7, 2017 at 5:09 PM, John Sanda < jsanda at redhat.com > wrote:
> 
> 
> I want to make sure I understand the difference between trace and span. A
> trace captures the call between two services whether those are REST
> endpoints, EJBs, etc. Let?s say we want a trace of A calling B. Within that
> A calls C which in turn calls D, etc. and then finally B is called. Would a
> span measure the duration of the call between C and D?
> 
> Are all spans captured/recorded by default?
> 
> Would this be optional functionality? I am wondering because it obviously
> requires having Hawkular Metrics running which also means Cassandra.
> 
> Under what tenant would you store these metrics?
> 
> 
> 
> On Jan 7, 2017, at 7:36 AM, Gary Brown < gbrown at redhat.com > wrote:
> 
> Hi
> 
> Wanted to discuss a proposal for recording some metric data captured from
> Hawkular APM in Hawkular Metrics.
> 
> For those not familiar with Hawkular APM, it captures the end to end
> trace
> instance (think of it as a distributed call stack), for each invocation
> of
> an application. This trace can include information about the
> communications between services, but can also include details about
> internal components used within the services (e.g. EJBs, database calls,
> etc).
> 
> First point is that if we were to record duration metrics for each 'span'
> captured (i.e. scope within which a task is performed), for each
> invocation of an application, then it would result in a large amount of
> data that may be of no interest. So we need to find a way for end
> users/developers to express which key points within an application they
> do
> want recorded as metrics.
> 
> The proposal is to allow the application/services to define a
> tag/property
> on the spans of interest, e.g. 'kpi', that would indicate to the server
> that the duration value for the span should be stored in H-Metrics. The
> value for the tag should define the name/description of the KPI.
> 
> If considered a suitable solution, then we can also propose it as a
> standard tag in the OpenTracing standard.
> 
> There are a couple of metrics that we could record by default, first
> being
> the trace instance completion time, and the second possibly being the
> individual service response times (although this could potentially also
> be
> governed by the 'kpi' tag).
> 
> Thoughts?
> 
> Regards
> Gary
> 
> 
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev --
> Pavol Loffay
> Cell: +421948286055
> Red Hat
> Purky?ova 111 TPB-B
> 612 45 Brno
> 
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev --
> Pavol Loffay
> Cell: +421948286055
> Red Hat
> Purky?ova 111 TPB-B
> 612 45 Brno
> 
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 


From mazz at redhat.com  Mon Jan  9 17:48:55 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Mon, 9 Jan 2017 17:48:55 -0500 (EST)
Subject: [Hawkular-dev] hawkular openshift agent docker images are on hub
In-Reply-To: <10909362.198939.1484001640346.JavaMail.zimbra@redhat.com>
Message-ID: <1113895806.204419.1484002135173.JavaMail.zimbra@redhat.com>

The Hawkular OpenShift Agent project is now automatically publishing the latest bits to docker hub via Travis.

The docker images will be tagged with "latest" as you see here:

* The agent itself: https://hub.docker.com/r/hawkular/hawkular-openshift-agent/tags/
* An example Prometheus endpoint: https://hub.docker.com/r/hawkular/hawkular-openshift-agent-example-prometheus-python/tags/
* An example Jolokia endpoint: https://hub.docker.com/r/hawkular/hawkular-openshift-agent-example-jolokia-wildfly/tags/

Assuming you already have the hawkular-openshift-agent repo git cloned locally...

If you already have an OpenShift environment running, log into it via "oc login" and:

1. Go to the main src/github.com/hawkular/hawkular-openshift-agent directory and run "make openshift-deploy" to deploy the agent
2. Under that main directory, go to "hack/prometheus-example" and run "make openshift-deploy" to deploy the example Prometheus endpoint
3. Under that main directory, go to "hack/jolokia-example" and run "make openshift-deploy" to deploy the example Jolokia endpoint

If you try it out, let me know if any problems occur and if there's anything that can be done to make it easier.

From jsanda at redhat.com  Mon Jan  9 22:10:14 2017
From: jsanda at redhat.com (John Sanda)
Date: Mon, 9 Jan 2017 22:10:14 -0500
Subject: [Hawkular-dev] [hawkular-apm] tracing async operations
Message-ID: <EDC8F98A-A77A-4141-9337-3E187772DB7E@redhat.com>

I am reading through some of the Hawkular APM code and have been looking at how trace fragments get created and written on the client side. One of the classes I am reviewing is FragmentBuilder.java. Its javadocs state that the sequence of events within a thread of execution should be in sequence. It made me wonder whether it is possible to trace async operations. I found https://issues.jboss.org/browse/HWKAPM-77 <https://issues.jboss.org/browse/HWKAPM-77> which apparently improves support for async execution.

Gary, can you or anyone else familiar how things work, give a brief explanation of how things work async flows? Most everything in hawkular-metrics is async and reactive. That makes debugging difficult at times where you have to rely primarily on logging. I am wondering if Hawkular APM would be a good tool for these types of situations.

- John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170109/c1a8ab8d/attachment.html 

From gbrown at redhat.com  Tue Jan 10 04:29:26 2017
From: gbrown at redhat.com (Gary Brown)
Date: Tue, 10 Jan 2017 04:29:26 -0500 (EST)
Subject: [Hawkular-dev] [hawkular-apm] tracing async operations
In-Reply-To: <EDC8F98A-A77A-4141-9337-3E187772DB7E@redhat.com>
References: <EDC8F98A-A77A-4141-9337-3E187772DB7E@redhat.com>
Message-ID: <596306368.11937187.1484040566902.JavaMail.zimbra@redhat.com>

Hi John

If I put things in OpenTracing terms, each unit of work we want to measure is a Span, where work performed in the scope of one Span, is created as a child of that Span - so building up a tree structure. So this is how units of work, whether sync or async, get represented.

So the issue is, how we identify the start and end of a Span, when they are not necessarily occurring within the same thread. And secondly, when a child activity is spawned, that is not in the same thread as its parent, how that child Span becomes associated with its parent.

The piece of code you reference is part of the original java agent, and it relies on ThreadLocal as a way to track activity within threads - but to support async behaviour, the agent also enables the trace sessions/context to be propagated across threads, using context relevant ids.

If explicitly instrumenting code with the OpenTracing API, async behaviour can be monitored by simply passing around the Span objects.

Let me know if you need further information in any particular area.

Regards
Gary

----- Original Message -----
> I am reading through some of the Hawkular APM code and have been looking at
> how trace fragments get created and written on the client side. One of the
> classes I am reviewing is FragmentBuilder.java. Its javadocs state that the
> sequence of events within a thread of execution should be in sequence. It
> made me wonder whether it is possible to trace async operations. I found
> https://issues.jboss.org/browse/HWKAPM-77
> <https://issues.jboss.org/browse/HWKAPM-77> which apparently improves
> support for async execution.
> 
> Gary, can you or anyone else familiar how things work, give a brief
> explanation of how things work async flows? Most everything in
> hawkular-metrics is async and reactive. That makes debugging difficult at
> times where you have to rely primarily on logging. I am wondering if
> Hawkular APM would be a good tool for these types of situations.
> 
> - John

From gbrown at redhat.com  Tue Jan 10 04:45:44 2017
From: gbrown at redhat.com (Gary Brown)
Date: Tue, 10 Jan 2017 04:45:44 -0500 (EST)
Subject: [Hawkular-dev] [hawkular-apm] tracing async operations
In-Reply-To: <596306368.11937187.1484040566902.JavaMail.zimbra@redhat.com>
References: <EDC8F98A-A77A-4141-9337-3E187772DB7E@redhat.com>
	<596306368.11937187.1484040566902.JavaMail.zimbra@redhat.com>
Message-ID: <2120595004.11940622.1484041544356.JavaMail.zimbra@redhat.com>

For a concrete example, we have a vertx-opentracing application:

https://github.com/hawkular/hawkular-apm/blob/master/examples/vertx-opentracing/order-manager/src/main/java/org/hawkular/apm/examples/vertx/opentracing/ordermanager/PlaceOrderHandler.java#L52

This shows where the top level span is started within the order manager service. This span is propagated through various handlers until it is finally finished here:

https://github.com/hawkular/hawkular-apm/blob/master/examples/vertx-opentracing/order-manager/src/main/java/org/hawkular/apm/examples/vertx/opentracing/ordermanager/PlaceOrderHandler.java#L126

Regards
Gary

----- Original Message -----
> Hi John
> 
> If I put things in OpenTracing terms, each unit of work we want to measure is
> a Span, where work performed in the scope of one Span, is created as a child
> of that Span - so building up a tree structure. So this is how units of
> work, whether sync or async, get represented.
> 
> So the issue is, how we identify the start and end of a Span, when they are
> not necessarily occurring within the same thread. And secondly, when a child
> activity is spawned, that is not in the same thread as its parent, how that
> child Span becomes associated with its parent.
> 
> The piece of code you reference is part of the original java agent, and it
> relies on ThreadLocal as a way to track activity within threads - but to
> support async behaviour, the agent also enables the trace sessions/context
> to be propagated across threads, using context relevant ids.
> 
> If explicitly instrumenting code with the OpenTracing API, async behaviour
> can be monitored by simply passing around the Span objects.
> 
> Let me know if you need further information in any particular area.
> 
> Regards
> Gary
> 
> ----- Original Message -----
> > I am reading through some of the Hawkular APM code and have been looking at
> > how trace fragments get created and written on the client side. One of the
> > classes I am reviewing is FragmentBuilder.java. Its javadocs state that the
> > sequence of events within a thread of execution should be in sequence. It
> > made me wonder whether it is possible to trace async operations. I found
> > https://issues.jboss.org/browse/HWKAPM-77
> > <https://issues.jboss.org/browse/HWKAPM-77> which apparently improves
> > support for async execution.
> > 
> > Gary, can you or anyone else familiar how things work, give a brief
> > explanation of how things work async flows? Most everything in
> > hawkular-metrics is async and reactive. That makes debugging difficult at
> > times where you have to rely primarily on logging. I am wondering if
> > Hawkular APM would be a good tool for these types of situations.
> > 
> > - John
> 

From jpkroehling at redhat.com  Tue Jan 10 06:51:57 2017
From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=)
Date: Tue, 10 Jan 2017 12:51:57 +0100
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <559164915.11771505.1483982202160.JavaMail.zimbra@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<BE9BE7D2-BE3D-4008-A66E-DE4BC6DCFA0A@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
	<1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
	<CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>
	<2018592486.11597381.1483956618881.JavaMail.zimbra@redhat.com>
	<e77f5167-8741-0f3f-4bec-d8861a0c7594@redhat.com>
	<559164915.11771505.1483982202160.JavaMail.zimbra@redhat.com>
Message-ID: <90ba7001-6867-00cc-6089-dab9384a4fbe@redhat.com>

On 01/09/2017 06:16 PM, Gary Brown wrote:
> Anyone got some good use cases, i.e. examples of useful business KPIs?

Not exactly "business KPI", but on the realm of "X per second", there 
are cases that we could start with that would provide immediate business 
insight/alerting to an application:

- requests in geo: so that it can be detected when a specific location 
(using a geo IP database) is having troubles. For instance, if I get a 
sudden drop of 20% in the number of requests coming from Europe, I'd 
like to get an alert. "How many users are in Europe" is an answer that 
Google Analytics gives, but might be interesting to plot on APM as well;

- mobile users: if the number of requests coming from mobile users 
(detected by user agent string) drops 20%, there might be a problem with 
the mobile version of my application;

- endpoint requests: I think we have almost all bits for this one 
already: if I get a 20% drop in the number of requests to a particular 
endpoint, I'd like to get notified. For modern backend applications, one 
endpoint is usually one business transaction, so, it's easy for an end 
user to add alerting to business transactions by watching those endpoints;

- requests: the simplest of all, just show how many requests per second 
the application is currently serving. If I get a 20% drop, there's 
something wrong "somewhere", which is already an information "good 
enough" to start with :)

- Juca.

From gbrown at redhat.com  Tue Jan 10 07:03:14 2017
From: gbrown at redhat.com (Gary Brown)
Date: Tue, 10 Jan 2017 07:03:14 -0500 (EST)
Subject: [Hawkular-dev] Integrating APM data into H-Metrics
In-Reply-To: <90ba7001-6867-00cc-6089-dab9384a4fbe@redhat.com>
References: <878344870.11457244.1483792562128.JavaMail.zimbra@redhat.com>
	<CAKLecW-vUcufK5fCEDotGeitFf2L2h+59iSULet7xtAFkh=7jQ@mail.gmail.com>
	<1521320479.11584547.1483952737544.JavaMail.zimbra@redhat.com>
	<CAKLecW-18JSxPKeo7t2d6mFHS28=7j2cG5=cxM+fwTdHZfPimQ@mail.gmail.com>
	<2018592486.11597381.1483956618881.JavaMail.zimbra@redhat.com>
	<e77f5167-8741-0f3f-4bec-d8861a0c7594@redhat.com>
	<559164915.11771505.1483982202160.JavaMail.zimbra@redhat.com>
	<90ba7001-6867-00cc-6089-dab9384a4fbe@redhat.com>
Message-ID: <1550232656.11976586.1484049794719.JavaMail.zimbra@redhat.com>

Thanks for the suggestions.

Wikipedia also provides some examples, such as customer acquisition/attrition, which equally would relate to the number of specific business transactions (representing adding and removing customers).

So it appears most KPIs would involve post processing of the trace data to derive the interesting metrics. So depends whether that should reside in APM, or whether we should just record basic standard metrics (e.g. related to num/duration etc of named transactions) and let other rules aggregate/derive/analyse these metrics?

Regards
Gary

----- Original Message -----
> On 01/09/2017 06:16 PM, Gary Brown wrote:
> > Anyone got some good use cases, i.e. examples of useful business KPIs?
> 
> Not exactly "business KPI", but on the realm of "X per second", there
> are cases that we could start with that would provide immediate business
> insight/alerting to an application:
> 
> - requests in geo: so that it can be detected when a specific location
> (using a geo IP database) is having troubles. For instance, if I get a
> sudden drop of 20% in the number of requests coming from Europe, I'd
> like to get an alert. "How many users are in Europe" is an answer that
> Google Analytics gives, but might be interesting to plot on APM as well;
> 
> - mobile users: if the number of requests coming from mobile users
> (detected by user agent string) drops 20%, there might be a problem with
> the mobile version of my application;
> 
> - endpoint requests: I think we have almost all bits for this one
> already: if I get a 20% drop in the number of requests to a particular
> endpoint, I'd like to get notified. For modern backend applications, one
> endpoint is usually one business transaction, so, it's easy for an end
> user to add alerting to business transactions by watching those endpoints;
> 
> - requests: the simplest of all, just show how many requests per second
> the application is currently serving. If I get a 20% drop, there's
> something wrong "somewhere", which is already an information "good
> enough" to start with :)
> 
> - Juca.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From garethahealy at gmail.com  Tue Jan 10 08:26:06 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Tue, 10 Jan 2017 13:26:06 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
Message-ID: <CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>

To carry on the discussion around whether the agent should create single
metrics or hawkular should group the datapoints raised by Joel.

I've done a bit more digging into different prometheus endpoints and heres
an example from some work being done by Kuberenetes:

https://github.com/kubernetes/kube-state-metrics
# HELP kube_node_info Information about a cluster node.
# TYPE kube_node_info gauge
kube_node_info{container_runtime_version="docker://1.
10.3",kernel_version="3.10.0-327.28.3.el7.x86_64",kubelet_
version="v1.3.0+52492b4",kubeproxy_version="v1.3.0+
52492b4",node="origin",os_image="CentOS Linux 7 (Core)"} 1

# HELP kube_pod_status_scheduled Describes the status of the scheduling
process for the pod.
# TYPE kube_pod_status_scheduled gauge
kube_pod_status_scheduled{condition="true",namespace="
cockpit",pod="openshift-cockpit-1-0jub2"} 1

# HELP kube_pod_container_requested_cpu_cores The number of requested cpu
cores by a container.
# TYPE kube_pod_container_requested_cpu_cores gauge
kube_pod_container_requested_cpu_cores{container="registry"
,namespace="default",node="origin",pod="docker-registry-1-i37wn"} 0.1

If the agent was to create single metrics, the idea in my mind would be as
follows:

   - Add a new boolean flag to create unique metrics based on labels
   (default=false, to keep functionality as-is)
   - Allow use of "id" field to contain value replacements based on label
   value (probably need to be sanitised / escaped??)

i.e.:

metrics:
> - name: kube_pod_container_requested_cpu_cores
>   id: kube_pod_container_requested_cpu_cores_${container}_${node}
>   unique_metric_per_label: true


Which would create a gauge called:

kube_pod_container_requested_cpu_cores_registry_origin


Does that sound feasible? and thoughts?

On Wed, Jan 4, 2017 at 1:06 PM, John Mazzitelli <mazz at redhat.com> wrote:

> > Here, query by tag means time series tags, not data points tags. That's
> > probably why you don't get anything. Maybe we should more clearly
> > disambiguate these two concepts as there's often some confusion.
>
> :-) And that, I think, is what I meant by tags on "metric definitions" - I
> think the H-Metrics team calls them "timeseries tags".
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170110/2d275686/attachment.html 

From ploffay at redhat.com  Tue Jan 10 15:25:17 2017
From: ploffay at redhat.com (Pavol Loffay)
Date: Tue, 10 Jan 2017 21:25:17 +0100
Subject: [Hawkular-dev] [hawkular-apm] tracing async operations
In-Reply-To: <2120595004.11940622.1484041544356.JavaMail.zimbra@redhat.com>
References: <EDC8F98A-A77A-4141-9337-3E187772DB7E@redhat.com>
	<596306368.11937187.1484040566902.JavaMail.zimbra@redhat.com>
	<2120595004.11940622.1484041544356.JavaMail.zimbra@redhat.com>
Message-ID: <CAKLecW-jK=sf3ik9wZLAgp3udY09E58SiZJz1MckRiosLsE_aQ@mail.gmail.com>

Hello John,

this might also help:
https://github.com/opentracing/specification/blob/master/specification.md#the-opentracing-data-model
and also following subsection
https://github.com/opentracing/specification/blob/master/specification.md#references-between-spans
.


Pavol

On Tue, Jan 10, 2017 at 10:45 AM, Gary Brown <gbrown at redhat.com> wrote:

> For a concrete example, we have a vertx-opentracing application:
>
> https://github.com/hawkular/hawkular-apm/blob/master/
> examples/vertx-opentracing/order-manager/src/main/java/
> org/hawkular/apm/examples/vertx/opentracing/ordermanager/
> PlaceOrderHandler.java#L52
>
> This shows where the top level span is started within the order manager
> service. This span is propagated through various handlers until it is
> finally finished here:
>
> https://github.com/hawkular/hawkular-apm/blob/master/
> examples/vertx-opentracing/order-manager/src/main/java/
> org/hawkular/apm/examples/vertx/opentracing/ordermanager/
> PlaceOrderHandler.java#L126
>
> Regards
> Gary
>
> ----- Original Message -----
> > Hi John
> >
> > If I put things in OpenTracing terms, each unit of work we want to
> measure is
> > a Span, where work performed in the scope of one Span, is created as a
> child
> > of that Span - so building up a tree structure. So this is how units of
> > work, whether sync or async, get represented.
> >
> > So the issue is, how we identify the start and end of a Span, when they
> are
> > not necessarily occurring within the same thread. And secondly, when a
> child
> > activity is spawned, that is not in the same thread as its parent, how
> that
> > child Span becomes associated with its parent.
> >
> > The piece of code you reference is part of the original java agent, and
> it
> > relies on ThreadLocal as a way to track activity within threads - but to
> > support async behaviour, the agent also enables the trace
> sessions/context
> > to be propagated across threads, using context relevant ids.
> >
> > If explicitly instrumenting code with the OpenTracing API, async
> behaviour
> > can be monitored by simply passing around the Span objects.
> >
> > Let me know if you need further information in any particular area.
> >
> > Regards
> > Gary
> >
> > ----- Original Message -----
> > > I am reading through some of the Hawkular APM code and have been
> looking at
> > > how trace fragments get created and written on the client side. One of
> the
> > > classes I am reviewing is FragmentBuilder.java. Its javadocs state
> that the
> > > sequence of events within a thread of execution should be in sequence.
> It
> > > made me wonder whether it is possible to trace async operations. I
> found
> > > https://issues.jboss.org/browse/HWKAPM-77
> > > <https://issues.jboss.org/browse/HWKAPM-77> which apparently improves
> > > support for async execution.
> > >
> > > Gary, can you or anyone else familiar how things work, give a brief
> > > explanation of how things work async flows? Most everything in
> > > hawkular-metrics is async and reactive. That makes debugging difficult
> at
> > > times where you have to rely primarily on logging. I am wondering if
> > > Hawkular APM would be a good tool for these types of situations.
> > >
> > > - John
> >
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>



-- 
Pavol Loffay
Cell: +421948286055
Red Hat
Purky?ova 111 TPB-B
612 45 Brno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170110/032dcff0/attachment.html 

From mazz at redhat.com  Tue Jan 10 19:41:05 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Tue, 10 Jan 2017 19:41:05 -0500 (EST)
Subject: [Hawkular-dev] hosa using secrets for endpoint credentials
In-Reply-To: <1126565447.1043306.1484094469054.JavaMail.zimbra@redhat.com>
Message-ID: <579292406.1045046.1484095265278.JavaMail.zimbra@redhat.com>

Right now, the Hawkular OpenShift Agent (HOSA) can pass HTTP authentication headers to endpoints it is monitoring, but you have to declare the credentials in the pod's configmap's endpoints section:

endpoints:
- type: jolokia
  credentials:
    username: myuser
    password: mypass

We would like to figure a better way. One way Heiko mentioned was to see if we can use OpenShift's secrets right here in the credentials section.

So I created a PoC to see if and how it can work. I have it working here in my own branch:

https://github.com/jmazzitelli/hawkular-openshift-agent/tree/use-secrets

After building and deploying the agent in my OpenShift environment, I then created a secret (via the OS console) in my project where my jolokia pod lives - the secret is called "foo2" which has two keys defined: "password" and "username". I then tell the agent about this by passing in credentials as I describe above, but I prefix the values with "secret:" to tell the agent to expect the actual values to be found in the secret. The full syntax of the credentials values are "secret:<secret name>/<secret key>". So for example, I can have this in my configmap:

endpoints:
- type: jolokia
  credentials:
    username: secret:foo2/username
    password: secret:foo2/password

It can optionally use bearer tokens:

endpoints:
- type: jolokia
  credentials:
    token: secret:foo2/password

There is one problem with this. I need to add a cluster role to the agent to read secrets (I need verb "get" on resource "secrets" - for testing, I am using the "system:node" role since that is one of the few that has that permission - we'd really want a cluster role that only has "get"/"secrets" - we don't need all the perms that "system:node" provides - we'd have to create our role if need be).

But is this good? I do not know of any other way for the agent to be able to read secrets. Is it OK to require the agent to have "get" "secrets" permission? Is there another way to access secrets?

From theute at redhat.com  Wed Jan 11 03:39:05 2017
From: theute at redhat.com (Thomas Heute)
Date: Wed, 11 Jan 2017 09:39:05 +0100
Subject: [Hawkular-dev] Hawkular in OpenShift
Message-ID: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>

When I use:
oc cluster up --metrics=true

Hawkular metrics fails until I do:
oc adm policy add-role-to-user view
system:serviceaccount:openshift-infra:hawkular -n openshift-infra

Should that last command be part of the --metrics=true magic ?

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170111/4d66daf0/attachment.html 

From jpkroehling at redhat.com  Wed Jan 11 05:15:29 2017
From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=)
Date: Wed, 11 Jan 2017 11:15:29 +0100
Subject: [Hawkular-dev] hosa using secrets for endpoint credentials
In-Reply-To: <579292406.1045046.1484095265278.JavaMail.zimbra@redhat.com>
References: <579292406.1045046.1484095265278.JavaMail.zimbra@redhat.com>
Message-ID: <1aedb1f0-fedf-3981-3c6c-7613329202ff@redhat.com>

On 01/11/2017 01:41 AM, John Mazzitelli wrote:
> There is one problem with this. I need to add a cluster role to the agent to read secrets (I need verb "get" on resource "secrets" - for testing, I am using the "system:node" role since that is one of the few that has that permission - we'd really want a cluster role that only has "get"/"secrets" - we don't need all the perms that "system:node" provides - we'd have to create our role if need be).

We also use secrets on Hawkular APM, and I don't remember having to 
create specific cluster roles. I believe that, as long as the secret 
lives in the same "application" as the consumer of the secret, there's 
no need for an extra role.

I'm not sure the way we are doing can be also done for the HOSA, but 
here's an overview on how we use secrets for APM:

We create the secret, and the values here should be base64 encoded:
https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L6-L13

We specify that we want this secret to be a volume:
https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L76-L79

And we mount this volume into the container:
https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L51-L54

We have a standalone-wrapper.sh, which then tries to find the admin's 
username and password the server should create. One of the possibilities 
is reading the client secret values:
https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/hawkular-apm-server/standalone-wrapper.sh#L16-L26

- Juca.

From mazz at redhat.com  Wed Jan 11 08:52:59 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 11 Jan 2017 08:52:59 -0500 (EST)
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
Message-ID: <834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>

> When I use:
> oc cluster up --metrics=true
> 
> Hawkular metrics fails until I do:
> oc adm policy add-role-to-user view
> system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> 
> Should that last command be part of the --metrics=true magic ?


I do not have to do that. But then, I'm running from a local master build so I have the latest-n-greatest - maybe you have an older version?

I know when I do "cluster up" I usually have to wait a bit for metrics to fully start and only then do I see metrics in the UI (sometimes this takes a few minutes on a slow machine). And before its ready, I get a warning message at the top of the UI and it only goes away until metrics is ready AND I browse to the metrics status page.

I do, however, give my "admin" user full cluster-admin rights just so I can see everything when I log into the UI:

   $ oc adm policy add-cluster-role-to-user cluster-admin admin

For the record, I have a script I use to run "cluster up" - I don't just run it "oc cluster up --metrics". There are other things that have to be done to make sure it all works (at least that's been my experience - mainly, I have to turn off firewalld) - this is what I use:

https://github.com/hawkular/hawkular-openshift-agent/blob/master/hack/cluster-openshift.sh

This also has a convenient "status" option so I can get a few status things and it has "down" which also does some other things to clean up in addition to 
"oc cluster down"

From mazz at redhat.com  Wed Jan 11 08:54:49 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 11 Jan 2017 08:54:49 -0500 (EST)
Subject: [Hawkular-dev] hosa using secrets for endpoint credentials
In-Reply-To: <1aedb1f0-fedf-3981-3c6c-7613329202ff@redhat.com>
References: <579292406.1045046.1484095265278.JavaMail.zimbra@redhat.com>
	<1aedb1f0-fedf-3981-3c6c-7613329202ff@redhat.com>
Message-ID: <634517437.1279491.1484142889770.JavaMail.zimbra@redhat.com>

> We also use secrets on Hawkular APM, and I don't remember having to
> create specific cluster roles. I believe that, as long as the secret
> lives in the same "application" as the consumer of the secret, there's
> no need for an extra role.

That's exactly the issue. The agent is in charge of monitoring the node - so it can cross all the projects - a pod to be monitored may not even be in the same project (aka namespace) as the agent.

----- Original Message -----
> On 01/11/2017 01:41 AM, John Mazzitelli wrote:
> > There is one problem with this. I need to add a cluster role to the agent
> > to read secrets (I need verb "get" on resource "secrets" - for testing, I
> > am using the "system:node" role since that is one of the few that has that
> > permission - we'd really want a cluster role that only has "get"/"secrets"
> > - we don't need all the perms that "system:node" provides - we'd have to
> > create our role if need be).
> 
> We also use secrets on Hawkular APM, and I don't remember having to
> create specific cluster roles. I believe that, as long as the secret
> lives in the same "application" as the consumer of the secret, there's
> no need for an extra role.
> 
> I'm not sure the way we are doing can be also done for the HOSA, but
> here's an overview on how we use secrets for APM:
> 
> We create the secret, and the values here should be base64 encoded:
> https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L6-L13
> 
> We specify that we want this secret to be a volume:
> https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L76-L79
> 
> And we mount this volume into the container:
> https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L51-L54
> 
> We have a standalone-wrapper.sh, which then tries to find the admin's
> username and password the server should create. One of the possibilities
> is reading the client secret values:
> https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/hawkular-apm-server/standalone-wrapper.sh#L16-L26
> 
> - Juca.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From mwringe at redhat.com  Wed Jan 11 09:20:08 2017
From: mwringe at redhat.com (Matt Wringe)
Date: Wed, 11 Jan 2017 09:20:08 -0500 (EST)
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
	<834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
Message-ID: <1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>

----- Original Message -----
> From: "John Mazzitelli" <mazz at redhat.com>
> To: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
> Sent: Wednesday, 11 January, 2017 8:52:59 AM
> Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> 
> > When I use:
> > oc cluster up --metrics=true
> > 
> > Hawkular metrics fails until I do:
> > oc adm policy add-role-to-user view
> > system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> > 
> > Should that last command be part of the --metrics=true magic ?
> 
> 
> I do not have to do that. But then, I'm running from a local master build so
> I have the latest-n-greatest - maybe you have an older version?

I suspect you are using an older version of 'oc'. Can you let us know which version you are using? And can you also let us know what version the Hawkular Metrics docker image is?

If its a released version of 'oc' it should only ever use a versioned Hawkular Metrics which matches the version of OpenShift.


> 
> I know when I do "cluster up" I usually have to wait a bit for metrics to
> fully start and only then do I see metrics in the UI (sometimes this takes a
> few minutes on a slow machine). And before its ready, I get a warning
> message at the top of the UI and it only goes away until metrics is ready
> AND I browse to the metrics status page.
> 
> I do, however, give my "admin" user full cluster-admin rights just so I can
> see everything when I log into the UI:
> 
>    $ oc adm policy add-cluster-role-to-user cluster-admin admin
> 
> For the record, I have a script I use to run "cluster up" - I don't just run
> it "oc cluster up --metrics". There are other things that have to be done to
> make sure it all works (at least that's been my experience - mainly, I have
> to turn off firewalld) - this is what I use:
> 
> https://github.com/hawkular/hawkular-openshift-agent/blob/master/hack/cluster-openshift.sh
> 
> This also has a convenient "status" option so I can get a few status things
> and it has "down" which also does some other things to clean up in addition
> to
> "oc cluster down"
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From mwringe at redhat.com  Wed Jan 11 09:23:49 2017
From: mwringe at redhat.com (Matt Wringe)
Date: Wed, 11 Jan 2017 09:23:49 -0500 (EST)
Subject: [Hawkular-dev] hosa using secrets for endpoint credentials
In-Reply-To: <634517437.1279491.1484142889770.JavaMail.zimbra@redhat.com>
References: <579292406.1045046.1484095265278.JavaMail.zimbra@redhat.com>
	<1aedb1f0-fedf-3981-3c6c-7613329202ff@redhat.com>
	<634517437.1279491.1484142889770.JavaMail.zimbra@redhat.com>
Message-ID: <966544909.15079302.1484144629989.JavaMail.zimbra@redhat.com>

----- Original Message -----
> From: "John Mazzitelli" <mazz at redhat.com>
> To: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
> Sent: Wednesday, 11 January, 2017 8:54:49 AM
> Subject: Re: [Hawkular-dev] hosa using secrets for endpoint credentials
> 
> > We also use secrets on Hawkular APM, and I don't remember having to
> > create specific cluster roles. I believe that, as long as the secret
> > lives in the same "application" as the consumer of the secret, there's
> > no need for an extra role.
> 
> That's exactly the issue. The agent is in charge of monitoring the node - so
> it can cross all the projects - a pod to be monitored may not even be in the
> same project (aka namespace) as the agent.

If we can't get secrets with something like the cluster-reader permission, I don't know if we should be using it. Reading secrets starts to become tricky and it might not be something we can get approval for.

> ----- Original Message -----
> > On 01/11/2017 01:41 AM, John Mazzitelli wrote:
> > > There is one problem with this. I need to add a cluster role to the agent
> > > to read secrets (I need verb "get" on resource "secrets" - for testing, I
> > > am using the "system:node" role since that is one of the few that has
> > > that
> > > permission - we'd really want a cluster role that only has
> > > "get"/"secrets"
> > > - we don't need all the perms that "system:node" provides - we'd have to
> > > create our role if need be).
> > 
> > We also use secrets on Hawkular APM, and I don't remember having to
> > create specific cluster roles. I believe that, as long as the secret
> > lives in the same "application" as the consumer of the secret, there's
> > no need for an extra role.
> > 
> > I'm not sure the way we are doing can be also done for the HOSA, but
> > here's an overview on how we use secrets for APM:
> > 
> > We create the secret, and the values here should be base64 encoded:
> > https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L6-L13
> > 
> > We specify that we want this secret to be a volume:
> > https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L76-L79
> > 
> > And we mount this volume into the container:
> > https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/openshift-templates/hawkular-apm-server-deployment.yml#L51-L54
> > 
> > We have a standalone-wrapper.sh, which then tries to find the admin's
> > username and password the server should create. One of the possibilities
> > is reading the client secret values:
> > https://github.com/jboss-dockerfiles/hawkular-apm/blob/master/hawkular-apm-server/standalone-wrapper.sh#L16-L26
> > 
> > - Juca.
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From theute at redhat.com  Wed Jan 11 09:39:48 2017
From: theute at redhat.com (Thomas Heute)
Date: Wed, 11 Jan 2017 15:39:48 +0100
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
	<834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
	<1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>
Message-ID: <CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>

[theute at localhost vertx-opentracing]$ oc version
oc v1.3.1
kubernetes v1.3.0+52492b4
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://10.202.9.50:8443
openshift v1.5.0-alpha.1+9e682de-55
kubernetes v1.4.0+776c994


and Hawkular Metrics 0.21.5.Final

Thomas

On Wed, Jan 11, 2017 at 3:20 PM, Matt Wringe <mwringe at redhat.com> wrote:

> ----- Original Message -----
> > From: "John Mazzitelli" <mazz at redhat.com>
> > To: "Discussions around Hawkular development" <
> hawkular-dev at lists.jboss.org>
> > Sent: Wednesday, 11 January, 2017 8:52:59 AM
> > Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> >
> > > When I use:
> > > oc cluster up --metrics=true
> > >
> > > Hawkular metrics fails until I do:
> > > oc adm policy add-role-to-user view
> > > system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> > >
> > > Should that last command be part of the --metrics=true magic ?
> >
> >
> > I do not have to do that. But then, I'm running from a local master
> build so
> > I have the latest-n-greatest - maybe you have an older version?
>
> I suspect you are using an older version of 'oc'. Can you let us know
> which version you are using? And can you also let us know what version the
> Hawkular Metrics docker image is?
>
> If its a released version of 'oc' it should only ever use a versioned
> Hawkular Metrics which matches the version of OpenShift.
>
>
> >
> > I know when I do "cluster up" I usually have to wait a bit for metrics to
> > fully start and only then do I see metrics in the UI (sometimes this
> takes a
> > few minutes on a slow machine). And before its ready, I get a warning
> > message at the top of the UI and it only goes away until metrics is ready
> > AND I browse to the metrics status page.
> >
> > I do, however, give my "admin" user full cluster-admin rights just so I
> can
> > see everything when I log into the UI:
> >
> >    $ oc adm policy add-cluster-role-to-user cluster-admin admin
> >
> > For the record, I have a script I use to run "cluster up" - I don't just
> run
> > it "oc cluster up --metrics". There are other things that have to be
> done to
> > make sure it all works (at least that's been my experience - mainly, I
> have
> > to turn off firewalld) - this is what I use:
> >
> > https://github.com/hawkular/hawkular-openshift-agent/blob/
> master/hack/cluster-openshift.sh
> >
> > This also has a convenient "status" option so I can get a few status
> things
> > and it has "down" which also does some other things to clean up in
> addition
> > to
> > "oc cluster down"
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170111/6f36b730/attachment-0001.html 

From mazz at redhat.com  Wed Jan 11 09:47:37 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 11 Jan 2017 09:47:37 -0500 (EST)
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
	<834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
	<1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>
	<CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>
Message-ID: <1402349726.1329150.1484146057495.JavaMail.zimbra@redhat.com>

I suspect it is because your oc is outdated.

Here's mine:

oc v1.5.0-alpha.0+3b2bbe5
kubernetes v1.4.0+776c994
features: Basic-Auth

Server https://192.168.1.15:8443
openshift v1.5.0-alpha.0+3b2bbe5
kubernetes v1.4.0+776c994

H-Metrics: 0.21.5.Final


----- Original Message -----
> [theute at localhost vertx-opentracing]$ oc version
> oc v1.3.1
> kubernetes v1.3.0+52492b4
> features: Basic-Auth GSSAPI Kerberos SPNEGO
> 
> Server https://10.202.9.50:8443
> openshift v1.5.0-alpha.1+9e682de-55
> kubernetes v1.4.0+776c994
> 
> 
> and Hawkular Metrics 0.21.5.Final
> 
> Thomas
> 
> On Wed, Jan 11, 2017 at 3:20 PM, Matt Wringe <mwringe at redhat.com> wrote:
> 
> > ----- Original Message -----
> > > From: "John Mazzitelli" <mazz at redhat.com>
> > > To: "Discussions around Hawkular development" <
> > hawkular-dev at lists.jboss.org>
> > > Sent: Wednesday, 11 January, 2017 8:52:59 AM
> > > Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> > >
> > > > When I use:
> > > > oc cluster up --metrics=true
> > > >
> > > > Hawkular metrics fails until I do:
> > > > oc adm policy add-role-to-user view
> > > > system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> > > >
> > > > Should that last command be part of the --metrics=true magic ?
> > >
> > >
> > > I do not have to do that. But then, I'm running from a local master
> > build so
> > > I have the latest-n-greatest - maybe you have an older version?
> >
> > I suspect you are using an older version of 'oc'. Can you let us know
> > which version you are using? And can you also let us know what version the
> > Hawkular Metrics docker image is?
> >
> > If its a released version of 'oc' it should only ever use a versioned
> > Hawkular Metrics which matches the version of OpenShift.
> >
> >
> > >
> > > I know when I do "cluster up" I usually have to wait a bit for metrics to
> > > fully start and only then do I see metrics in the UI (sometimes this
> > takes a
> > > few minutes on a slow machine). And before its ready, I get a warning
> > > message at the top of the UI and it only goes away until metrics is ready
> > > AND I browse to the metrics status page.
> > >
> > > I do, however, give my "admin" user full cluster-admin rights just so I
> > can
> > > see everything when I log into the UI:
> > >
> > >    $ oc adm policy add-cluster-role-to-user cluster-admin admin
> > >
> > > For the record, I have a script I use to run "cluster up" - I don't just
> > run
> > > it "oc cluster up --metrics". There are other things that have to be
> > done to
> > > make sure it all works (at least that's been my experience - mainly, I
> > have
> > > to turn off firewalld) - this is what I use:
> > >
> > > https://github.com/hawkular/hawkular-openshift-agent/blob/
> > master/hack/cluster-openshift.sh
> > >
> > > This also has a convenient "status" option so I can get a few status
> > things
> > > and it has "down" which also does some other things to clean up in
> > addition
> > > to
> > > "oc cluster down"
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> >
> 

From theute at redhat.com  Wed Jan 11 09:54:27 2017
From: theute at redhat.com (Thomas Heute)
Date: Wed, 11 Jan 2017 15:54:27 +0100
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <1402349726.1329150.1484146057495.JavaMail.zimbra@redhat.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
	<834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
	<1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>
	<CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>
	<1402349726.1329150.1484146057495.JavaMail.zimbra@redhat.com>
Message-ID: <CA+aN=MpdYY9+U22S1WLEC-2LErBx3VaJHig0nVuq56Pf_VTB_g@mail.gmail.com>

Ok, it worked well beside that :)

I guess I was too lazy and took whatever came with F24.

Glad to know this is not needed with most recent versions

On Wed, Jan 11, 2017 at 3:47 PM, John Mazzitelli <mazz at redhat.com> wrote:

> I suspect it is because your oc is outdated.
>
> Here's mine:
>
> oc v1.5.0-alpha.0+3b2bbe5
> kubernetes v1.4.0+776c994
> features: Basic-Auth
>
> Server https://192.168.1.15:8443
> openshift v1.5.0-alpha.0+3b2bbe5
> kubernetes v1.4.0+776c994
>
> H-Metrics: 0.21.5.Final
>
>
> ----- Original Message -----
> > [theute at localhost vertx-opentracing]$ oc version
> > oc v1.3.1
> > kubernetes v1.3.0+52492b4
> > features: Basic-Auth GSSAPI Kerberos SPNEGO
> >
> > Server https://10.202.9.50:8443
> > openshift v1.5.0-alpha.1+9e682de-55
> > kubernetes v1.4.0+776c994
> >
> >
> > and Hawkular Metrics 0.21.5.Final
> >
> > Thomas
> >
> > On Wed, Jan 11, 2017 at 3:20 PM, Matt Wringe <mwringe at redhat.com> wrote:
> >
> > > ----- Original Message -----
> > > > From: "John Mazzitelli" <mazz at redhat.com>
> > > > To: "Discussions around Hawkular development" <
> > > hawkular-dev at lists.jboss.org>
> > > > Sent: Wednesday, 11 January, 2017 8:52:59 AM
> > > > Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> > > >
> > > > > When I use:
> > > > > oc cluster up --metrics=true
> > > > >
> > > > > Hawkular metrics fails until I do:
> > > > > oc adm policy add-role-to-user view
> > > > > system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> > > > >
> > > > > Should that last command be part of the --metrics=true magic ?
> > > >
> > > >
> > > > I do not have to do that. But then, I'm running from a local master
> > > build so
> > > > I have the latest-n-greatest - maybe you have an older version?
> > >
> > > I suspect you are using an older version of 'oc'. Can you let us know
> > > which version you are using? And can you also let us know what version
> the
> > > Hawkular Metrics docker image is?
> > >
> > > If its a released version of 'oc' it should only ever use a versioned
> > > Hawkular Metrics which matches the version of OpenShift.
> > >
> > >
> > > >
> > > > I know when I do "cluster up" I usually have to wait a bit for
> metrics to
> > > > fully start and only then do I see metrics in the UI (sometimes this
> > > takes a
> > > > few minutes on a slow machine). And before its ready, I get a warning
> > > > message at the top of the UI and it only goes away until metrics is
> ready
> > > > AND I browse to the metrics status page.
> > > >
> > > > I do, however, give my "admin" user full cluster-admin rights just
> so I
> > > can
> > > > see everything when I log into the UI:
> > > >
> > > >    $ oc adm policy add-cluster-role-to-user cluster-admin admin
> > > >
> > > > For the record, I have a script I use to run "cluster up" - I don't
> just
> > > run
> > > > it "oc cluster up --metrics". There are other things that have to be
> > > done to
> > > > make sure it all works (at least that's been my experience - mainly,
> I
> > > have
> > > > to turn off firewalld) - this is what I use:
> > > >
> > > > https://github.com/hawkular/hawkular-openshift-agent/blob/
> > > master/hack/cluster-openshift.sh
> > > >
> > > > This also has a convenient "status" option so I can get a few status
> > > things
> > > > and it has "down" which also does some other things to clean up in
> > > addition
> > > > to
> > > > "oc cluster down"
> > > > _______________________________________________
> > > > hawkular-dev mailing list
> > > > hawkular-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > >
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> > >
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170111/219dddcb/attachment.html 

From mwringe at redhat.com  Wed Jan 11 11:27:33 2017
From: mwringe at redhat.com (Matt Wringe)
Date: Wed, 11 Jan 2017 11:27:33 -0500 (EST)
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
	<834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
	<1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>
	<CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>
Message-ID: <1684768987.15136155.1484152053085.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Thomas Heute" <theute at redhat.com>
> To: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
> Sent: Wednesday, 11 January, 2017 9:39:48 AM
> Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> 
> [theute at localhost vertx-opentracing]$ oc version
> oc v1.3.1
> kubernetes v1.3.0+52492b4
> features: Basic-Auth GSSAPI Kerberos SPNEGO
> 
> Server https://10.202.9.50:8443
> openshift v1.5.0-alpha.1+9e682de-55
> kubernetes v1.4.0+776c994
> 
> 
> and Hawkular Metrics 0.21.5.Final

The version of the Hawkular Metrics docker image, not the version of Hawkular Metrics.

It should be shown in the console under the overview page.

> Thomas
> 
> On Wed, Jan 11, 2017 at 3:20 PM, Matt Wringe < mwringe at redhat.com > wrote:
> 
> 
> ----- Original Message -----
> > From: "John Mazzitelli" < mazz at redhat.com >
> > To: "Discussions around Hawkular development" <
> > hawkular-dev at lists.jboss.org >
> > Sent: Wednesday, 11 January, 2017 8:52:59 AM
> > Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> > 
> > > When I use:
> > > oc cluster up --metrics=true
> > > 
> > > Hawkular metrics fails until I do:
> > > oc adm policy add-role-to-user view
> > > system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> > > 
> > > Should that last command be part of the --metrics=true magic ?
> > 
> > 
> > I do not have to do that. But then, I'm running from a local master build
> > so
> > I have the latest-n-greatest - maybe you have an older version?
> 
> I suspect you are using an older version of 'oc'. Can you let us know which
> version you are using? And can you also let us know what version the
> Hawkular Metrics docker image is?
> 
> If its a released version of 'oc' it should only ever use a versioned
> Hawkular Metrics which matches the version of OpenShift.
> 
> 
> > 
> > I know when I do "cluster up" I usually have to wait a bit for metrics to
> > fully start and only then do I see metrics in the UI (sometimes this takes
> > a
> > few minutes on a slow machine). And before its ready, I get a warning
> > message at the top of the UI and it only goes away until metrics is ready
> > AND I browse to the metrics status page.
> > 
> > I do, however, give my "admin" user full cluster-admin rights just so I can
> > see everything when I log into the UI:
> > 
> > $ oc adm policy add-cluster-role-to-user cluster-admin admin
> > 
> > For the record, I have a script I use to run "cluster up" - I don't just
> > run
> > it "oc cluster up --metrics". There are other things that have to be done
> > to
> > make sure it all works (at least that's been my experience - mainly, I have
> > to turn off firewalld) - this is what I use:
> > 
> > https://github.com/hawkular/hawkular-openshift-agent/blob/master/hack/cluster-openshift.sh
> > 
> > This also has a convenient "status" option so I can get a few status things
> > and it has "down" which also does some other things to clean up in addition
> > to
> > "oc cluster down"
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From garethahealy at gmail.com  Wed Jan 11 15:15:13 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Wed, 11 Jan 2017 20:15:13 +0000
Subject: [Hawkular-dev] some new hawkular openshift agent stuff
In-Reply-To: <1555198452.3660625.1483753450272.JavaMail.zimbra@redhat.com>
References: <1372949925.3660393.1483753025341.JavaMail.zimbra@redhat.com>
	<1555198452.3660625.1483753450272.JavaMail.zimbra@redhat.com>
Message-ID: <CAHWT10agJjKUDReJkO_-5CrnwXRb-gg2_hi_eeUJCTCvsG1oxw@mail.gmail.com>

Hi John,

Just tried deploying latest agent but get an error. Am running the below
steps to deploy the agent:

oc adm policy add-cluster-role-to-user cluster-reader
system:serviceaccount:openshift-infra:hawkular-agent
oc create -f https://raw.githubusercontent.com/hawkular/hawkular-openshift-agent/master/deploy/openshift/hawkular-openshift-agent-configmap.yaml
-n openshift-infra
oc process -f https://raw.githubusercontent.com/hawkular/hawkular-openshift-agent/master/deploy/openshift/hawkular-openshift-agent.yaml
IMAGE_VERSION=latest | oc create -n openshift-infra -f -


But get this error when the agent starts:

E0111 18:34:11.353256       1 node_event_consumer.go:72] Error obtaining
information about the agent pod
[openshift-infra/hawkular-openshift-agent-h1ynb]. err=User
"system:serviceaccount:openshift-infra:hawkular-openshift-agent" cannot get
pods in project "openshift-infra"

I've also got a custom pod that has a configmap mounted, but the agent
never gets past this error message so doesn't collect any metrics.

Output of oc describe on my custom pod shows the volume mount:

Volume Mounts:
      /var/run/configmap/hawkular-agent from hawkular-openshift-agent (rw)

Any ideas?

On Sat, Jan 7, 2017 at 1:44 AM, John Mazzitelli <mazz at redhat.com> wrote:

> FYI: some new things went into HOSA (that's the Hawkular OpenShift Agent
> for the uninitiated).
>
> 1. The agent now emits its own metrics and can monitor itself. Right now
> it just emits some basic "go" metrics like memory usage, CPU usage, etc
> along with one agent-specific one - a counter that counts the number of
> data points it has collected in its lifetime. We'll add more metrics as we
> figure out the things people want to see, but we have the infrastructure in
> place now.
>
> 2. The agent is deployed as a daemonset. This means as new nodes are
> brought online, an agent will go along with it (or so I'm told :)
>
> 3. The agent has changed the way it discovers what to monitor - it no
> longer looks at annotations on pods to determine where the configmaps are
> for those pods. Instead, it looks up volume declarations to see if there is
> an agent configmap defined. This was done to be ready for the future when
> new security constraints will be introduced in OpenShift which would have
> broken our annotation approach. This approach using volumes should not hit
> that issue.
>
> NOTE: If you are building the latest agent from master, we added some
> dependencies so you have to update your dependencies via Glide by using the
> "make update-deps" target prior to building from source.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170111/ddd2c085/attachment-0001.html 

From mazz at redhat.com  Wed Jan 11 15:21:55 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 11 Jan 2017 15:21:55 -0500 (EST)
Subject: [Hawkular-dev] some new hawkular openshift agent stuff
In-Reply-To: <CAHWT10agJjKUDReJkO_-5CrnwXRb-gg2_hi_eeUJCTCvsG1oxw@mail.gmail.com>
References: <1372949925.3660393.1483753025341.JavaMail.zimbra@redhat.com>
	<1555198452.3660625.1483753450272.JavaMail.zimbra@redhat.com>
	<CAHWT10agJjKUDReJkO_-5CrnwXRb-gg2_hi_eeUJCTCvsG1oxw@mail.gmail.com>
Message-ID: <337392204.1556224.1484166115042.JavaMail.zimbra@redhat.com>

Gareth,

I've seen this before and when I did it is because the agent doesn't have the cluster-reader role.

So something happened with the command:

oc adm policy add-cluster-role-to-user cluster-reader system:serviceaccount:openshift-infra:hawkular-agent

I can tell you just recently I changed names to ensure all names in HOSA are consistent (before, you would see some things named "hawkular-agent" and other times you'd see "hawkular-openshift-agent" and other times you'd see things named "openshift-agent" - I made all names consistent... "hawkular-openshift-agent").

One of the changes is the name of the service account - it is now "hawkular-openshift-agent" not "hawkular-agent"

So if you are running newer code, check that. Your command above should be:

oc adm policy add-cluster-role-to-user cluster-reader system:serviceaccount:openshift-infra:hawkular-openshift-agent

Without that role, the agent will not work at all.

You can see the Makefile we use to deploy in our dev environments - see the "openshift-deploy" target here:

https://github.com/hawkular/hawkular-openshift-agent/blob/master/Makefile#L42-L46

----- Original Message -----
> Hi John,
> 
> Just tried deploying latest agent but get an error. Am running the below
> steps to deploy the agent:
> 
> oc adm policy add-cluster-role-to-user cluster-reader
> system:serviceaccount:openshift-infra:hawkular-agent
> oc create -f
> https://raw.githubusercontent.com/hawkular/hawkular-openshift-agent/master/deploy/openshift/hawkular-openshift-agent-configmap.yaml
> -n openshift-infra
> oc process -f
> https://raw.githubusercontent.com/hawkular/hawkular-openshift-agent/master/deploy/openshift/hawkular-openshift-agent.yaml
> IMAGE_VERSION=latest | oc create -n openshift-infra -f -
> 
> 
> But get this error when the agent starts:
> 
> E0111 18:34:11.353256       1 node_event_consumer.go:72] Error obtaining
> information about the agent pod
> [openshift-infra/hawkular-openshift-agent-h1ynb]. err=User
> "system:serviceaccount:openshift-infra:hawkular-openshift-agent" cannot get
> pods in project "openshift-infra"
> 
> I've also got a custom pod that has a configmap mounted, but the agent
> never gets past this error message so doesn't collect any metrics.
> 
> Output of oc describe on my custom pod shows the volume mount:
> 
> Volume Mounts:
>       /var/run/configmap/hawkular-agent from hawkular-openshift-agent (rw)
> 
> Any ideas?
> 
> On Sat, Jan 7, 2017 at 1:44 AM, John Mazzitelli <mazz at redhat.com> wrote:
> 
> > FYI: some new things went into HOSA (that's the Hawkular OpenShift Agent
> > for the uninitiated).
> >
> > 1. The agent now emits its own metrics and can monitor itself. Right now
> > it just emits some basic "go" metrics like memory usage, CPU usage, etc
> > along with one agent-specific one - a counter that counts the number of
> > data points it has collected in its lifetime. We'll add more metrics as we
> > figure out the things people want to see, but we have the infrastructure in
> > place now.
> >
> > 2. The agent is deployed as a daemonset. This means as new nodes are
> > brought online, an agent will go along with it (or so I'm told :)
> >
> > 3. The agent has changed the way it discovers what to monitor - it no
> > longer looks at annotations on pods to determine where the configmaps are
> > for those pods. Instead, it looks up volume declarations to see if there is
> > an agent configmap defined. This was done to be ready for the future when
> > new security constraints will be introduced in OpenShift which would have
> > broken our annotation approach. This approach using volumes should not hit
> > that issue.
> >
> > NOTE: If you are building the latest agent from master, we added some
> > dependencies so you have to update your dependencies via Glide by using the
> > "make update-deps" target prior to building from source.
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> 

From garethahealy at gmail.com  Wed Jan 11 15:28:03 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Wed, 11 Jan 2017 20:28:03 +0000
Subject: [Hawkular-dev] some new hawkular openshift agent stuff
In-Reply-To: <337392204.1556224.1484166115042.JavaMail.zimbra@redhat.com>
References: <1372949925.3660393.1483753025341.JavaMail.zimbra@redhat.com>
	<1555198452.3660625.1483753450272.JavaMail.zimbra@redhat.com>
	<CAHWT10agJjKUDReJkO_-5CrnwXRb-gg2_hi_eeUJCTCvsG1oxw@mail.gmail.com>
	<337392204.1556224.1484166115042.JavaMail.zimbra@redhat.com>
Message-ID: <CAHWT10bmDwP1svOOZMmJYzgY6cuOd4z1NYOJZ0-EQrYVqaD3vA@mail.gmail.com>

Hi John,

Thanks, it was the service account rename. Its working now.

Cheers.

On Wed, Jan 11, 2017 at 8:21 PM, John Mazzitelli <mazz at redhat.com> wrote:

> Gareth,
>
> I've seen this before and when I did it is because the agent doesn't have
> the cluster-reader role.
>
> So something happened with the command:
>
> oc adm policy add-cluster-role-to-user cluster-reader
> system:serviceaccount:openshift-infra:hawkular-agent
>
> I can tell you just recently I changed names to ensure all names in HOSA
> are consistent (before, you would see some things named "hawkular-agent"
> and other times you'd see "hawkular-openshift-agent" and other times you'd
> see things named "openshift-agent" - I made all names consistent...
> "hawkular-openshift-agent").
>
> One of the changes is the name of the service account - it is now
> "hawkular-openshift-agent" not "hawkular-agent"
>
> So if you are running newer code, check that. Your command above should be:
>
> oc adm policy add-cluster-role-to-user cluster-reader
> system:serviceaccount:openshift-infra:hawkular-openshift-agent
>
> Without that role, the agent will not work at all.
>
> You can see the Makefile we use to deploy in our dev environments - see
> the "openshift-deploy" target here:
>
> https://github.com/hawkular/hawkular-openshift-agent/blob/
> master/Makefile#L42-L46
>
> ----- Original Message -----
> > Hi John,
> >
> > Just tried deploying latest agent but get an error. Am running the below
> > steps to deploy the agent:
> >
> > oc adm policy add-cluster-role-to-user cluster-reader
> > system:serviceaccount:openshift-infra:hawkular-agent
> > oc create -f
> > https://raw.githubusercontent.com/hawkular/hawkular-
> openshift-agent/master/deploy/openshift/hawkular-openshift-
> agent-configmap.yaml
> > -n openshift-infra
> > oc process -f
> > https://raw.githubusercontent.com/hawkular/hawkular-
> openshift-agent/master/deploy/openshift/hawkular-openshift-agent.yaml
> > IMAGE_VERSION=latest | oc create -n openshift-infra -f -
> >
> >
> > But get this error when the agent starts:
> >
> > E0111 18:34:11.353256       1 node_event_consumer.go:72] Error obtaining
> > information about the agent pod
> > [openshift-infra/hawkular-openshift-agent-h1ynb]. err=User
> > "system:serviceaccount:openshift-infra:hawkular-openshift-agent" cannot
> get
> > pods in project "openshift-infra"
> >
> > I've also got a custom pod that has a configmap mounted, but the agent
> > never gets past this error message so doesn't collect any metrics.
> >
> > Output of oc describe on my custom pod shows the volume mount:
> >
> > Volume Mounts:
> >       /var/run/configmap/hawkular-agent from hawkular-openshift-agent
> (rw)
> >
> > Any ideas?
> >
> > On Sat, Jan 7, 2017 at 1:44 AM, John Mazzitelli <mazz at redhat.com> wrote:
> >
> > > FYI: some new things went into HOSA (that's the Hawkular OpenShift
> Agent
> > > for the uninitiated).
> > >
> > > 1. The agent now emits its own metrics and can monitor itself. Right
> now
> > > it just emits some basic "go" metrics like memory usage, CPU usage, etc
> > > along with one agent-specific one - a counter that counts the number of
> > > data points it has collected in its lifetime. We'll add more metrics
> as we
> > > figure out the things people want to see, but we have the
> infrastructure in
> > > place now.
> > >
> > > 2. The agent is deployed as a daemonset. This means as new nodes are
> > > brought online, an agent will go along with it (or so I'm told :)
> > >
> > > 3. The agent has changed the way it discovers what to monitor - it no
> > > longer looks at annotations on pods to determine where the configmaps
> are
> > > for those pods. Instead, it looks up volume declarations to see if
> there is
> > > an agent configmap defined. This was done to be ready for the future
> when
> > > new security constraints will be introduced in OpenShift which would
> have
> > > broken our annotation approach. This approach using volumes should not
> hit
> > > that issue.
> > >
> > > NOTE: If you are building the latest agent from master, we added some
> > > dependencies so you have to update your dependencies via Glide by
> using the
> > > "make update-deps" target prior to building from source.
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170111/8d68a6bf/attachment.html 

From mazz at redhat.com  Wed Jan 11 17:16:19 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 11 Jan 2017 17:16:19 -0500 (EST)
Subject: [Hawkular-dev] hosa and its own role
In-Reply-To: <594559581.1574370.1484172284647.JavaMail.zimbra@redhat.com>
Message-ID: <1281418260.1575656.1484172979492.JavaMail.zimbra@redhat.com>

Playing around with OpenShift roles, I found the agent doesn't need the vast majority of permissions the cluster-reader role provides.

So, rather than assign the agent to the cluster-reader role, I instead create a single role for the agent to be given where that role provides only the permissions the agent actually needs to do its job and no others:

https://github.com/hawkular/hawkular-openshift-agent/pull/87/files#diff-e7dc415f5f89921c318c41da6b565347

So far, this looks to be working. Heiko, feel free to try this out. Its part of that use-secrets PR/branch.

From snegrea at redhat.com  Thu Jan 12 15:45:28 2017
From: snegrea at redhat.com (snegrea at redhat.com)
Date: Thu, 12 Jan 2017 20:45:28 +0000
Subject: [Hawkular-dev] Invitation: New Tag Query Language - Hawkular
 Metrics @ Tue Jan 17, 2017 10am - 11am (CST) (hawkular-dev@lists.jboss.org)
Message-ID: <94eb2c0761ce2abefa0545ebcc66@google.com>

You have been invited to the following event.

Title: New Tag Query Language - Hawkular Metrics
The goal is to review and get feedback on the latest attempt to overhaul  
the tag query capabilities for Hawkular Metrics.

PR: https://github.com/hawkular/hawkular-metrics/pull/725

JIRA: https://issues.jboss.org/browse/HWKMETRICS-523
When: Tue Jan 17, 2017 10am ? 11am Central Time
Where: http://bluejeans.com/3980552127
Calendar: hawkular-dev at lists.jboss.org
Who:
     * snegrea at redhat.com - creator
     * jsanda at redhat.com
     * jtakvori at redhat.com
     * miburman at redhat.com
     * mwringe at redhat.com
     * jshaughn at redhat.com
     * hawkular-dev at lists.jboss.org

Event details:  
https://www.google.com/calendar/event?action=VIEW&eid=NHE3NG5paGc3NGRhbjgyamVjbjkwM2JpZzggaGF3a3VsYXItZGV2QGxpc3RzLmpib3NzLm9yZw&tok=NjMjcmVkaGF0LmNvbV9qMWVudWRzbGtnNTlscjJxcGhxODQ0a20wMEBncm91cC5jYWxlbmRhci5nb29nbGUuY29tOWNkYTk1M2RkODBhM2U0MDhjMzU5YTE0MGIwMjc1MTdlMjg1NzBjMw&ctz=America/Chicago&hl=en

Invitation from Google Calendar: https://www.google.com/calendar/

You are receiving this courtesy email at the account  
hawkular-dev at lists.jboss.org because you are an attendee of this event.

To stop receiving future updates for this event, decline this event.  
Alternatively you can sign up for a Google account at  
https://www.google.com/calendar/ and control your notification settings for  
your entire calendar.

Forwarding this invitation could allow any recipient to modify your RSVP  
response. Learn more at  
https://support.google.com/calendar/answer/37135#forwarding
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170112/442ef1e6/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/calendar
Size: 2186 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170112/442ef1e6/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: invite.ics
Type: application/ics
Size: 2229 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170112/442ef1e6/attachment-0003.bin 

From theute at redhat.com  Mon Jan 16 07:42:56 2017
From: theute at redhat.com (Thomas Heute)
Date: Mon, 16 Jan 2017 13:42:56 +0100
Subject: [Hawkular-dev] Hawkular in OpenShift
In-Reply-To: <1684768987.15136155.1484152053085.JavaMail.zimbra@redhat.com>
References: <CA+aN=MrpvC2aLSNQ9zBBG-xYhVnDq04kkfHvdhfBBEP-QdPALg@mail.gmail.com>
	<834692627.1279073.1484142779343.JavaMail.zimbra@redhat.com>
	<1069011043.15076062.1484144408221.JavaMail.zimbra@redhat.com>
	<CA+aN=MrmpYScOGmy41shZ5V+StpBofXiVMZYqsQGLySCHvN8ZA@mail.gmail.com>
	<1684768987.15136155.1484152053085.JavaMail.zimbra@redhat.com>
Message-ID: <CA+aN=MqB1D+k4QBS5_B9J9QLrJrchin_t1JF-AUisdQH2y1u3w@mail.gmail.com>

With newer versions of oc it worked well without it !

Thanks,
Thomas

On Wed, Jan 11, 2017 at 5:27 PM, Matt Wringe <mwringe at redhat.com> wrote:

>
>
> ----- Original Message -----
> > From: "Thomas Heute" <theute at redhat.com>
> > To: "Discussions around Hawkular development" <
> hawkular-dev at lists.jboss.org>
> > Sent: Wednesday, 11 January, 2017 9:39:48 AM
> > Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> >
> > [theute at localhost vertx-opentracing]$ oc version
> > oc v1.3.1
> > kubernetes v1.3.0+52492b4
> > features: Basic-Auth GSSAPI Kerberos SPNEGO
> >
> > Server https://10.202.9.50:8443
> > openshift v1.5.0-alpha.1+9e682de-55
> > kubernetes v1.4.0+776c994
> >
> >
> > and Hawkular Metrics 0.21.5.Final
>
> The version of the Hawkular Metrics docker image, not the version of
> Hawkular Metrics.
>
> It should be shown in the console under the overview page.
>
> > Thomas
> >
> > On Wed, Jan 11, 2017 at 3:20 PM, Matt Wringe < mwringe at redhat.com >
> wrote:
> >
> >
> > ----- Original Message -----
> > > From: "John Mazzitelli" < mazz at redhat.com >
> > > To: "Discussions around Hawkular development" <
> > > hawkular-dev at lists.jboss.org >
> > > Sent: Wednesday, 11 January, 2017 8:52:59 AM
> > > Subject: Re: [Hawkular-dev] Hawkular in OpenShift
> > >
> > > > When I use:
> > > > oc cluster up --metrics=true
> > > >
> > > > Hawkular metrics fails until I do:
> > > > oc adm policy add-role-to-user view
> > > > system:serviceaccount:openshift-infra:hawkular -n openshift-infra
> > > >
> > > > Should that last command be part of the --metrics=true magic ?
> > >
> > >
> > > I do not have to do that. But then, I'm running from a local master
> build
> > > so
> > > I have the latest-n-greatest - maybe you have an older version?
> >
> > I suspect you are using an older version of 'oc'. Can you let us know
> which
> > version you are using? And can you also let us know what version the
> > Hawkular Metrics docker image is?
> >
> > If its a released version of 'oc' it should only ever use a versioned
> > Hawkular Metrics which matches the version of OpenShift.
> >
> >
> > >
> > > I know when I do "cluster up" I usually have to wait a bit for metrics
> to
> > > fully start and only then do I see metrics in the UI (sometimes this
> takes
> > > a
> > > few minutes on a slow machine). And before its ready, I get a warning
> > > message at the top of the UI and it only goes away until metrics is
> ready
> > > AND I browse to the metrics status page.
> > >
> > > I do, however, give my "admin" user full cluster-admin rights just so
> I can
> > > see everything when I log into the UI:
> > >
> > > $ oc adm policy add-cluster-role-to-user cluster-admin admin
> > >
> > > For the record, I have a script I use to run "cluster up" - I don't
> just
> > > run
> > > it "oc cluster up --metrics". There are other things that have to be
> done
> > > to
> > > make sure it all works (at least that's been my experience - mainly, I
> have
> > > to turn off firewalld) - this is what I use:
> > >
> > > https://github.com/hawkular/hawkular-openshift-agent/blob/
> master/hack/cluster-openshift.sh
> > >
> > > This also has a convenient "status" option so I can get a few status
> things
> > > and it has "down" which also does some other things to clean up in
> addition
> > > to
> > > "oc cluster down"
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170116/25047c23/attachment.html 

From mazz at redhat.com  Mon Jan 16 08:29:54 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Mon, 16 Jan 2017 08:29:54 -0500 (EST)
Subject: [Hawkular-dev] what to name metrics in HOSA?
In-Reply-To: <1942709215.3405331.1484572767409.JavaMail.zimbra@redhat.com>
Message-ID: <617617090.3429978.1484573394769.JavaMail.zimbra@redhat.com>

HOSA has its own Prometheus endpoint that emits its own metrics. Right now, we just have one custom agent metric but we plan to add more. Before I get too far, I'm trying to figure out a good prefix to use for metric names.

I was looking over the Prometheus naming conventions for metric names here: https://prometheus.io/docs/practices/naming/

In addition, I found additional naming conventions listed in the Prom Go client comments here: https://github.com/prometheus/client_golang/blob/master/prometheus/metric.go#L63-L67

Right now the one custom agent metric is called:

hawkular_openshift_agent_metric_data_points_collected_total

I think it's too long :) And the "subsystem" is two words (openshift_agent) when the Go comment says (and all other prometheus metrics I've seen) use one word with no underscore.

I think starting it with "hawkular_" is good because looking at the metric you immediately know it is from a Hawkular component. But I don't know what the subsystem should be.

I was thinking:

hawkular_openshiftagent_<metric-name>

That is a one-word subsystem "openshiftagent" but its still too long IMO. Maybe:

hawkular_agent_<metric-name>

But then, if our other agents emit their own metrics in the future, this will be confusing (think vert.x agent, fuse agent, whatever).

How about use the HOSA abbreviation?

hawkular_hosa_<metric-name>

That seems smaller and is more specific to the OpenShift Agent. But will "HOSA" make sense to people?

Thoughts? Suggestions?


From mwringe at redhat.com  Mon Jan 16 14:26:01 2017
From: mwringe at redhat.com (Matt Wringe)
Date: Mon, 16 Jan 2017 14:26:01 -0500 (EST)
Subject: [Hawkular-dev] what to name metrics in HOSA?
In-Reply-To: <617617090.3429978.1484573394769.JavaMail.zimbra@redhat.com>
References: <617617090.3429978.1484573394769.JavaMail.zimbra@redhat.com>
Message-ID: <1808928026.17192527.1484594761020.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "John Mazzitelli" <mazz at redhat.com>
> To: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
> Sent: Monday, 16 January, 2017 8:29:54 AM
> Subject: [Hawkular-dev] what to name metrics in HOSA?
> 
> HOSA has its own Prometheus endpoint that emits its own metrics. Right now,
> we just have one custom agent metric but we plan to add more. Before I get
> too far, I'm trying to figure out a good prefix to use for metric names.
> 
> I was looking over the Prometheus naming conventions for metric names here:
> https://prometheus.io/docs/practices/naming/
> 
> In addition, I found additional naming conventions listed in the Prom Go
> client comments here:
> https://github.com/prometheus/client_golang/blob/master/prometheus/metric.go#L63-L67
> 
> Right now the one custom agent metric is called:
> 
> hawkular_openshift_agent_metric_data_points_collected_total
> 
> I think it's too long :) And the "subsystem" is two words (openshift_agent)
> when the Go comment says (and all other prometheus metrics I've seen) use
> one word with no underscore.
> 
> I think starting it with "hawkular_" is good because looking at the metric
> you immediately know it is from a Hawkular component. But I don't know what
> the subsystem should be.
> 
> I was thinking:
> 
> hawkular_openshiftagent_<metric-name>
> 
> That is a one-word subsystem "openshiftagent" but its still too long IMO.
> Maybe:
> 
> hawkular_agent_<metric-name>
> 
> But then, if our other agents emit their own metrics in the future, this will
> be confusing (think vert.x agent, fuse agent, whatever).
> 
> How about use the HOSA abbreviation?
> 
> hawkular_hosa_<metric-name>
> 
> That seems smaller and is more specific to the OpenShift Agent. But will
> "HOSA" make sense to people?
> 
> Thoughts? Suggestions?

I don't really understand. For the agent collecting its own metrics, it shouldn't need any prefix to be applied at all. Or is this for something external to be collecting this metric?

From mazz at redhat.com  Mon Jan 16 14:54:02 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Mon, 16 Jan 2017 14:54:02 -0500 (EST)
Subject: [Hawkular-dev] what to name metrics in HOSA?
In-Reply-To: <1808928026.17192527.1484594761020.JavaMail.zimbra@redhat.com>
References: <617617090.3429978.1484573394769.JavaMail.zimbra@redhat.com>
	<1808928026.17192527.1484594761020.JavaMail.zimbra@redhat.com>
Message-ID: <1280011109.524787.1484596442296.JavaMail.zimbra@redhat.com>

> I don't really understand. For the agent collecting its own metrics, it
> shouldn't need any prefix to be applied at all. Or is this for something
> external to be collecting this metric?

This is simply "What should we call our own metrics that we emit over the Prometheus protocol such that it conforms to Prometheus conventions"?

The agent can call it anything it wants (as long as it follows the syntax rules, i.e. can have only alphanumeric characters and underscores).

But I'm just trying to be a "good public citizen" and want to name these following the same patterns, norms, and conventions that all other Prometheus endpoints out there follow, but I also want them to be easily understandable and not have extremely long names.

Note that the agent emits more than just the one I documented in my last post - because we are using Prometheus's Go client, we get for free a bunch of metrics the Go client, itself, emits - basically metrics operating system and Go platform metrics. So, for example, the agent really emits these, too:

https://github.com/hawkular/hawkular-openshift-agent/blob/master/deploy/openshift/hawkular-openshift-agent-configmap.yaml#L42-L63

You'll see metrics like:

go_memstats_alloc_bytes
go_goroutines
process_open_fds

Some of these don't actually even follow that "namespace_subsystem_name_units" convention either, so perhaps I shouldn't worry too much about having the hawkular metrics conform to that convention either??

----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "John Mazzitelli" <mazz at redhat.com>
> > To: "Discussions around Hawkular development"
> > <hawkular-dev at lists.jboss.org>
> > Sent: Monday, 16 January, 2017 8:29:54 AM
> > Subject: [Hawkular-dev] what to name metrics in HOSA?
> > 
> > HOSA has its own Prometheus endpoint that emits its own metrics. Right now,
> > we just have one custom agent metric but we plan to add more. Before I get
> > too far, I'm trying to figure out a good prefix to use for metric names.
> > 
> > I was looking over the Prometheus naming conventions for metric names here:
> > https://prometheus.io/docs/practices/naming/
> > 
> > In addition, I found additional naming conventions listed in the Prom Go
> > client comments here:
> > https://github.com/prometheus/client_golang/blob/master/prometheus/metric.go#L63-L67
> > 
> > Right now the one custom agent metric is called:
> > 
> > hawkular_openshift_agent_metric_data_points_collected_total
> > 
> > I think it's too long :) And the "subsystem" is two words (openshift_agent)
> > when the Go comment says (and all other prometheus metrics I've seen) use
> > one word with no underscore.
> > 
> > I think starting it with "hawkular_" is good because looking at the metric
> > you immediately know it is from a Hawkular component. But I don't know what
> > the subsystem should be.
> > 
> > I was thinking:
> > 
> > hawkular_openshiftagent_<metric-name>
> > 
> > That is a one-word subsystem "openshiftagent" but its still too long IMO.
> > Maybe:
> > 
> > hawkular_agent_<metric-name>
> > 
> > But then, if our other agents emit their own metrics in the future, this
> > will
> > be confusing (think vert.x agent, fuse agent, whatever).
> > 
> > How about use the HOSA abbreviation?
> > 
> > hawkular_hosa_<metric-name>
> > 
> > That seems smaller and is more specific to the OpenShift Agent. But will
> > "HOSA" make sense to people?
> > 
> > Thoughts? Suggestions?
> 
> I don't really understand. For the agent collecting its own metrics, it
> shouldn't need any prefix to be applied at all. Or is this for something
> external to be collecting this metric?
> 

From theute at redhat.com  Tue Jan 17 03:41:29 2017
From: theute at redhat.com (Thomas Heute)
Date: Tue, 17 Jan 2017 09:41:29 +0100
Subject: [Hawkular-dev] what to name metrics in HOSA?
In-Reply-To: <1280011109.524787.1484596442296.JavaMail.zimbra@redhat.com>
References: <617617090.3429978.1484573394769.JavaMail.zimbra@redhat.com>
	<1808928026.17192527.1484594761020.JavaMail.zimbra@redhat.com>
	<1280011109.524787.1484596442296.JavaMail.zimbra@redhat.com>
Message-ID: <CA+aN=MqxFBsXncPb3W8-DTLezdun1wL4QyujA_cqvcci+gnh2g@mail.gmail.com>

We are talking about the metrics name when stored in Hawkular Metrics,
correct ?

With multiple agents,I guess we would need some kind of isolation with a
prefix or tags or tenant or else ? What options do we have ?

How do we avoid clashes between 2 HOSA ?

Thomas

On Mon, Jan 16, 2017 at 8:54 PM, John Mazzitelli <mazz at redhat.com> wrote:

> > I don't really understand. For the agent collecting its own metrics, it
> > shouldn't need any prefix to be applied at all. Or is this for something
> > external to be collecting this metric?
>
> This is simply "What should we call our own metrics that we emit over the
> Prometheus protocol such that it conforms to Prometheus conventions"?
>
> The agent can call it anything it wants (as long as it follows the syntax
> rules, i.e. can have only alphanumeric characters and underscores).
>
> But I'm just trying to be a "good public citizen" and want to name these
> following the same patterns, norms, and conventions that all other
> Prometheus endpoints out there follow, but I also want them to be easily
> understandable and not have extremely long names.
>
> Note that the agent emits more than just the one I documented in my last
> post - because we are using Prometheus's Go client, we get for free a bunch
> of metrics the Go client, itself, emits - basically metrics operating
> system and Go platform metrics. So, for example, the agent really emits
> these, too:
>
> https://github.com/hawkular/hawkular-openshift-agent/blob/
> master/deploy/openshift/hawkular-openshift-agent-configmap.yaml#L42-L63
>
> You'll see metrics like:
>
> go_memstats_alloc_bytes
> go_goroutines
> process_open_fds
>
> Some of these don't actually even follow that "namespace_subsystem_name_units"
> convention either, so perhaps I shouldn't worry too much about having the
> hawkular metrics conform to that convention either??
>
> ----- Original Message -----
> >
> >
> > ----- Original Message -----
> > > From: "John Mazzitelli" <mazz at redhat.com>
> > > To: "Discussions around Hawkular development"
> > > <hawkular-dev at lists.jboss.org>
> > > Sent: Monday, 16 January, 2017 8:29:54 AM
> > > Subject: [Hawkular-dev] what to name metrics in HOSA?
> > >
> > > HOSA has its own Prometheus endpoint that emits its own metrics. Right
> now,
> > > we just have one custom agent metric but we plan to add more. Before I
> get
> > > too far, I'm trying to figure out a good prefix to use for metric
> names.
> > >
> > > I was looking over the Prometheus naming conventions for metric names
> here:
> > > https://prometheus.io/docs/practices/naming/
> > >
> > > In addition, I found additional naming conventions listed in the Prom
> Go
> > > client comments here:
> > > https://github.com/prometheus/client_golang/blob/master/
> prometheus/metric.go#L63-L67
> > >
> > > Right now the one custom agent metric is called:
> > >
> > > hawkular_openshift_agent_metric_data_points_collected_total
> > >
> > > I think it's too long :) And the "subsystem" is two words
> (openshift_agent)
> > > when the Go comment says (and all other prometheus metrics I've seen)
> use
> > > one word with no underscore.
> > >
> > > I think starting it with "hawkular_" is good because looking at the
> metric
> > > you immediately know it is from a Hawkular component. But I don't know
> what
> > > the subsystem should be.
> > >
> > > I was thinking:
> > >
> > > hawkular_openshiftagent_<metric-name>
> > >
> > > That is a one-word subsystem "openshiftagent" but its still too long
> IMO.
> > > Maybe:
> > >
> > > hawkular_agent_<metric-name>
> > >
> > > But then, if our other agents emit their own metrics in the future,
> this
> > > will
> > > be confusing (think vert.x agent, fuse agent, whatever).
> > >
> > > How about use the HOSA abbreviation?
> > >
> > > hawkular_hosa_<metric-name>
> > >
> > > That seems smaller and is more specific to the OpenShift Agent. But
> will
> > > "HOSA" make sense to people?
> > >
> > > Thoughts? Suggestions?
> >
> > I don't really understand. For the agent collecting its own metrics, it
> > shouldn't need any prefix to be applied at all. Or is this for something
> > external to be collecting this metric?
> >
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170117/2f5b1fdf/attachment-0001.html 

From mazz at redhat.com  Tue Jan 17 07:14:52 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Tue, 17 Jan 2017 07:14:52 -0500 (EST)
Subject: [Hawkular-dev] what to name metrics in HOSA?
In-Reply-To: <CA+aN=MqxFBsXncPb3W8-DTLezdun1wL4QyujA_cqvcci+gnh2g@mail.gmail.com>
References: <617617090.3429978.1484573394769.JavaMail.zimbra@redhat.com>
	<1808928026.17192527.1484594761020.JavaMail.zimbra@redhat.com>
	<1280011109.524787.1484596442296.JavaMail.zimbra@redhat.com>
	<CA+aN=MqxFBsXncPb3W8-DTLezdun1wL4QyujA_cqvcci+gnh2g@mail.gmail.com>
Message-ID: <284440147.890047.1484655292536.JavaMail.zimbra@redhat.com>

> We are talking about the metrics name when stored in Hawkular Metrics,
> correct ?

Yes, these will end up as part of the H-Metric names (HOSA does allow the user to define their own IDs if they don't like the metric names coming from the resource, but the name of the metric is the default and for Prometheus endpoints people will probably use them and not define their own IDs).

> With multiple agents,I guess we would need some kind of isolation with a
> prefix or tags or tenant or else ? What options do we have ?

We do use tenants - by default we set a metric's tenant to the same as the name of the OpenShift project where that metric came from. You can see that here:

https://github.com/hawkular/hawkular-openshift-agent/blob/master/deploy/openshift/hawkular-openshift-agent-configmap.yaml#L12

And we already support an ID prefix. As you say, this is to isolate metric names in order to avoid name clashes across pods. You can see we set this in the default agent config to:

   metric_id_prefix: pod/${POD:uid}/custom/

See: https://github.com/hawkular/hawkular-openshift-agent/blob/master/deploy/openshift/hawkular-openshift-agent-configmap.yaml#L16

So, for example, if the agent is running in a pod that has a UID of "deadbeef123456" then the metric will be stored in H-Metrics as:

   pod/deadbeef123456/custom/hawkular_openshift_agent_metric_data_points_collected_total

It's tenant will be that of the project (aka namespace) of where the agent is running (which will be "openshift-infra" today).

> 
> How do we avoid clashes between 2 HOSA ?

By using this metric ID prefix. As I understand it, pod UIDs are unique across nodes so the use of ${POD:uid} avoids clashes.

From jtakvori at redhat.com  Wed Jan 18 03:54:37 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Wed, 18 Jan 2017 09:54:37 +0100
Subject: [Hawkular-dev] Fwd: [hawkular-dev] Grafana querying usability
In-Reply-To: <CAJo5TFnmH3mhCm7cQBbpD5Xs3nTmCku4KB064fNpBqy9actFgQ@mail.gmail.com>
References: <CAJo5TFnmH3mhCm7cQBbpD5Xs3nTmCku4KB064fNpBqy9actFgQ@mail.gmail.com>
Message-ID: <CAJo5TFk4ityAK7JVf5eGGTQ352JORU6YNFy5tB8M35rPxhrQAA@mail.gmail.com>

Hi everybody,

I would like to get some opinions on the hawkular-grafana-datasource
querying usability, especially if you had the opportunity to create
dashboards recently and had to juggle with querying by metric name and by
tag.

Currently the panel displays different elements depending on you're
querying by metric name (see picture:
https://raw.githubusercontent.com/hawkular/hawkular-grafana-datasource/master/docs/images/search-by-name.png
)

Querying by name is quite straightforward, but it can be cumbersome when
there's a lot of available metrics and you have to scroll among suggestions
to find the one you want.
?

Or by tag  (see picture:
https://raw.githubusercontent.com/hawkular/hawkular-grafana-datasource/master/docs/images/search-by-tag.png
 )

?
The "query by tag" interface is not very intuitive IMO (you define a list
of key-value pairs), moreover to this date there is no auto-completion on
tag name.

There's been some features in metrics recently that, I think, can enable a
better UI on Grafana side. First, there is Micke's feature "Allow fetching
of available tagNames" [1] that enables suggestions (auto-completion) on
tag name. And most importantly there's the new "tag query language" that
could (should) have its place in Grafana.


So I would have several suggestions for improving the UI and queries.

*1*: First of all I think we can remove the "Search by name" / "Search by
tag" selector, and allow searching by name AND by tag at the same time:
providing tags would refine the available metrics in the metrics text field
suggestions (auto-completion). If this text field is left blank, all
available metrics are displayed.

Then, there's several scenarios to take advantage of the new hawkular
metrics features:

*2a*: keep current key/value pairs system, but improve with adding
suggestion on tag names.

*2b*: replace key-value pairs with the new tags query, via a simple text
field:


We may or may not include syntax validation here. We must provide some
documentation on that.

*2c*: replace key-value pairs with the new tags query, with a dedicated
"builder" UI:

?Each of these boxes in tags query would have a contextual auto-completion
feature:
- suggestions on 1: list of tag keys
- suggestions on 2: list of operators (=, !=, IN, NOT IN)
- suggestions on 3: list of tag values for the given key (with some slights
differences on brackets if it's the first element or not; closing bracket
as a choice if it's not the first element)
- suggestions on 4: operators AND / OR
?

The 2b option is obviously simpler, very fast to implement. It has the
downside of loosing all auto-completion capabilities, even compared to the
current version.

2c looks nicer and more intuitive in its usage, people won't have to read
the doc to use it. However there's several downsides:
  - Need to implement the logic => need for time for development, adds
complexity to our grafana plugin.
  - Introduces a dependency to the language on server side. When the
language evolves, we'll have to maintain this as well.


Ideas & thoughts? I think it's preferable to proceed in several steps. In a
first time I could implement *1* and *2a*, and later (and maybe giving some
time to the language to be "stabilized") going with *2c*.

[1] https://issues.jboss.org/browse/HWKMETRICS-532

Thanks
Joel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/f3c010bb/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search.png
Type: image/png
Size: 38020 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/f3c010bb/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search-with-builder.png
Type: image/png
Size: 39806 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/f3c010bb/attachment-0003.png 

From tsegismo at redhat.com  Wed Jan 18 04:25:19 2017
From: tsegismo at redhat.com (Thomas Segismont)
Date: Wed, 18 Jan 2017 10:25:19 +0100
Subject: [Hawkular-dev] Fwd: [hawkular-dev] Grafana querying usability
In-Reply-To: <CAJo5TFk4ityAK7JVf5eGGTQ352JORU6YNFy5tB8M35rPxhrQAA@mail.gmail.com>
References: <CAJo5TFnmH3mhCm7cQBbpD5Xs3nTmCku4KB064fNpBqy9actFgQ@mail.gmail.com>
	<CAJo5TFk4ityAK7JVf5eGGTQ352JORU6YNFy5tB8M35rPxhrQAA@mail.gmail.com>
Message-ID: <CAHnC0xj2W_F=v9auaxY2irCDzWZqz8TheFxernJMvy7_dFK_MA@mail.gmail.com>

1/ sounds best to me, as it would give users the power of a query language.
But auto-completion could be quite a challenge there, couldn't it?

2017-01-18 9:54 GMT+01:00 Joel Takvorian <jtakvori at redhat.com>:

> Hi everybody,
>
> I would like to get some opinions on the hawkular-grafana-datasource
> querying usability, especially if you had the opportunity to create
> dashboards recently and had to juggle with querying by metric name and by
> tag.
>
> Currently the panel displays different elements depending on you're
> querying by metric name (see picture: https://raw.githubusercontent.
> com/hawkular/hawkular-grafana-datasource/master/docs/images/
> search-by-name.png )
>
> Querying by name is quite straightforward, but it can be cumbersome when
> there's a lot of available metrics and you have to scroll among suggestions
> to find the one you want.
> ?
>
> Or by tag  (see picture: https://raw.githubusercontent.
> com/hawkular/hawkular-grafana-datasource/master/docs/images/
> search-by-tag.png )
>
> ?
> The "query by tag" interface is not very intuitive IMO (you define a list
> of key-value pairs), moreover to this date there is no auto-completion on
> tag name.
>
> There's been some features in metrics recently that, I think, can enable a
> better UI on Grafana side. First, there is Micke's feature "Allow fetching
> of available tagNames" [1] that enables suggestions (auto-completion) on
> tag name. And most importantly there's the new "tag query language" that
> could (should) have its place in Grafana.
>
>
> So I would have several suggestions for improving the UI and queries.
>
> *1*: First of all I think we can remove the "Search by name" / "Search by
> tag" selector, and allow searching by name AND by tag at the same time:
> providing tags would refine the available metrics in the metrics text field
> suggestions (auto-completion). If this text field is left blank, all
> available metrics are displayed.
>
> Then, there's several scenarios to take advantage of the new hawkular
> metrics features:
>
> *2a*: keep current key/value pairs system, but improve with adding
> suggestion on tag names.
>
> *2b*: replace key-value pairs with the new tags query, via a simple text
> field:
>
>
> We may or may not include syntax validation here. We must provide some
> documentation on that.
>
> *2c*: replace key-value pairs with the new tags query, with a dedicated
> "builder" UI:
>
> ?Each of these boxes in tags query would have a contextual auto-completion
> feature:
> - suggestions on 1: list of tag keys
> - suggestions on 2: list of operators (=, !=, IN, NOT IN)
> - suggestions on 3: list of tag values for the given key (with some
> slights differences on brackets if it's the first element or not; closing
> bracket as a choice if it's not the first element)
> - suggestions on 4: operators AND / OR
> ?
>
> The 2b option is obviously simpler, very fast to implement. It has the
> downside of loosing all auto-completion capabilities, even compared to the
> current version.
>
> 2c looks nicer and more intuitive in its usage, people won't have to read
> the doc to use it. However there's several downsides:
>   - Need to implement the logic => need for time for development, adds
> complexity to our grafana plugin.
>   - Introduces a dependency to the language on server side. When the
> language evolves, we'll have to maintain this as well.
>
>
> Ideas & thoughts? I think it's preferable to proceed in several steps. In
> a first time I could implement *1* and *2a*, and later (and maybe giving
> some time to the language to be "stabilized") going with *2c*.
>
> [1] https://issues.jboss.org/browse/HWKMETRICS-532
>
> Thanks
> Joel
>
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/eb8da52e/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search-with-builder.png
Type: image/png
Size: 39806 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/eb8da52e/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search.png
Type: image/png
Size: 38020 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/eb8da52e/attachment-0003.png 

From jtakvori at redhat.com  Wed Jan 18 04:29:36 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Wed, 18 Jan 2017 10:29:36 +0100
Subject: [Hawkular-dev] Fwd: [hawkular-dev] Grafana querying usability
In-Reply-To: <CAHnC0xj2W_F=v9auaxY2irCDzWZqz8TheFxernJMvy7_dFK_MA@mail.gmail.com>
References: <CAJo5TFnmH3mhCm7cQBbpD5Xs3nTmCku4KB064fNpBqy9actFgQ@mail.gmail.com>
	<CAJo5TFk4ityAK7JVf5eGGTQ352JORU6YNFy5tB8M35rPxhrQAA@mail.gmail.com>
	<CAHnC0xj2W_F=v9auaxY2irCDzWZqz8TheFxernJMvy7_dFK_MA@mail.gmail.com>
Message-ID: <CAJo5TFmjtGNaWfEA5NWzD3GrNg4qPTZWzzS0hYRq-TPHOiQ85Q@mail.gmail.com>

Thomas, I think you mean the "2b" proposal? (with a simple text field to
type custom query in). I can't see how to provide auto-completion with
that. I think if we go this way, we loose auto-completion.

On Wed, Jan 18, 2017 at 10:25 AM, Thomas Segismont <tsegismo at redhat.com>
wrote:

> 1/ sounds best to me, as it would give users the power of a query
> language. But auto-completion could be quite a challenge there, couldn't it?
>
> 2017-01-18 9:54 GMT+01:00 Joel Takvorian <jtakvori at redhat.com>:
>
>> Hi everybody,
>>
>> I would like to get some opinions on the hawkular-grafana-datasource
>> querying usability, especially if you had the opportunity to create
>> dashboards recently and had to juggle with querying by metric name and by
>> tag.
>>
>> Currently the panel displays different elements depending on you're
>> querying by metric name (see picture: https://raw.githubusercontent.
>> com/hawkular/hawkular-grafana-datasource/master/docs/images/
>> search-by-name.png )
>>
>> Querying by name is quite straightforward, but it can be cumbersome when
>> there's a lot of available metrics and you have to scroll among suggestions
>> to find the one you want.
>> ?
>>
>> Or by tag  (see picture: https://raw.githubusercontent.
>> com/hawkular/hawkular-grafana-datasource/master/docs/images/
>> search-by-tag.png )
>>
>> ?
>> The "query by tag" interface is not very intuitive IMO (you define a list
>> of key-value pairs), moreover to this date there is no auto-completion on
>> tag name.
>>
>> There's been some features in metrics recently that, I think, can enable
>> a better UI on Grafana side. First, there is Micke's feature "Allow
>> fetching of available tagNames" [1] that enables suggestions
>> (auto-completion) on tag name. And most importantly there's the new "tag
>> query language" that could (should) have its place in Grafana.
>>
>>
>> So I would have several suggestions for improving the UI and queries.
>>
>> *1*: First of all I think we can remove the "Search by name" / "Search
>> by tag" selector, and allow searching by name AND by tag at the same time:
>> providing tags would refine the available metrics in the metrics text field
>> suggestions (auto-completion). If this text field is left blank, all
>> available metrics are displayed.
>>
>> Then, there's several scenarios to take advantage of the new hawkular
>> metrics features:
>>
>> *2a*: keep current key/value pairs system, but improve with adding
>> suggestion on tag names.
>>
>> *2b*: replace key-value pairs with the new tags query, via a simple text
>> field:
>>
>>
>> We may or may not include syntax validation here. We must provide some
>> documentation on that.
>>
>> *2c*: replace key-value pairs with the new tags query, with a dedicated
>> "builder" UI:
>>
>> ?Each of these boxes in tags query would have a contextual
>> auto-completion feature:
>> - suggestions on 1: list of tag keys
>> - suggestions on 2: list of operators (=, !=, IN, NOT IN)
>> - suggestions on 3: list of tag values for the given key (with some
>> slights differences on brackets if it's the first element or not; closing
>> bracket as a choice if it's not the first element)
>> - suggestions on 4: operators AND / OR
>> ?
>>
>> The 2b option is obviously simpler, very fast to implement. It has the
>> downside of loosing all auto-completion capabilities, even compared to the
>> current version.
>>
>> 2c looks nicer and more intuitive in its usage, people won't have to read
>> the doc to use it. However there's several downsides:
>>   - Need to implement the logic => need for time for development, adds
>> complexity to our grafana plugin.
>>   - Introduces a dependency to the language on server side. When the
>> language evolves, we'll have to maintain this as well.
>>
>>
>> Ideas & thoughts? I think it's preferable to proceed in several steps. In
>> a first time I could implement *1* and *2a*, and later (and maybe giving
>> some time to the language to be "stabilized") going with *2c*.
>>
>> [1] https://issues.jboss.org/browse/HWKMETRICS-532
>>
>> Thanks
>> Joel
>>
>>
>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/abb6fa95/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search-with-builder.png
Type: image/png
Size: 39806 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/abb6fa95/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search.png
Type: image/png
Size: 38020 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/abb6fa95/attachment-0003.png 

From tsegismo at redhat.com  Wed Jan 18 04:53:11 2017
From: tsegismo at redhat.com (Thomas Segismont)
Date: Wed, 18 Jan 2017 10:53:11 +0100
Subject: [Hawkular-dev] Fwd: [hawkular-dev] Grafana querying usability
In-Reply-To: <CAJo5TFmjtGNaWfEA5NWzD3GrNg4qPTZWzzS0hYRq-TPHOiQ85Q@mail.gmail.com>
References: <CAJo5TFnmH3mhCm7cQBbpD5Xs3nTmCku4KB064fNpBqy9actFgQ@mail.gmail.com>
	<CAJo5TFk4ityAK7JVf5eGGTQ352JORU6YNFy5tB8M35rPxhrQAA@mail.gmail.com>
	<CAHnC0xj2W_F=v9auaxY2irCDzWZqz8TheFxernJMvy7_dFK_MA@mail.gmail.com>
	<CAJo5TFmjtGNaWfEA5NWzD3GrNg4qPTZWzzS0hYRq-TPHOiQ85Q@mail.gmail.com>
Message-ID: <CAHnC0xipP_zaoXgXZoV-MRezX0q60-sYvgFsBgd+ko-Ockc_EQ@mail.gmail.com>

No, I really meant 1/. With a single text field mixing tag filters and
names. Or did I misunderstood the idea?

It might be worth taking a look at InfluxDB's datasource code. IIRC, they
let you switch between guided (with many specific text fields) and
freestyle (query language) mode.

2017-01-18 10:29 GMT+01:00 Joel Takvorian <jtakvori at redhat.com>:

> Thomas, I think you mean the "2b" proposal? (with a simple text field to
> type custom query in). I can't see how to provide auto-completion with
> that. I think if we go this way, we loose auto-completion.
>
> On Wed, Jan 18, 2017 at 10:25 AM, Thomas Segismont <tsegismo at redhat.com>
> wrote:
>
>> 1/ sounds best to me, as it would give users the power of a query
>> language. But auto-completion could be quite a challenge there, couldn't it?
>>
>> 2017-01-18 9:54 GMT+01:00 Joel Takvorian <jtakvori at redhat.com>:
>>
>>> Hi everybody,
>>>
>>> I would like to get some opinions on the hawkular-grafana-datasource
>>> querying usability, especially if you had the opportunity to create
>>> dashboards recently and had to juggle with querying by metric name and by
>>> tag.
>>>
>>> Currently the panel displays different elements depending on you're
>>> querying by metric name (see picture: https://raw.githubusercontent.
>>> com/hawkular/hawkular-grafana-datasource/master/docs/images/
>>> search-by-name.png )
>>>
>>> Querying by name is quite straightforward, but it can be cumbersome when
>>> there's a lot of available metrics and you have to scroll among suggestions
>>> to find the one you want.
>>> ?
>>>
>>> Or by tag  (see picture: https://raw.githubusercontent.
>>> com/hawkular/hawkular-grafana-datasource/master/docs/images/
>>> search-by-tag.png )
>>>
>>> ?
>>> The "query by tag" interface is not very intuitive IMO (you define a
>>> list of key-value pairs), moreover to this date there is no auto-completion
>>> on tag name.
>>>
>>> There's been some features in metrics recently that, I think, can enable
>>> a better UI on Grafana side. First, there is Micke's feature "Allow
>>> fetching of available tagNames" [1] that enables suggestions
>>> (auto-completion) on tag name. And most importantly there's the new "tag
>>> query language" that could (should) have its place in Grafana.
>>>
>>>
>>> So I would have several suggestions for improving the UI and queries.
>>>
>>> *1*: First of all I think we can remove the "Search by name" / "Search
>>> by tag" selector, and allow searching by name AND by tag at the same time:
>>> providing tags would refine the available metrics in the metrics text field
>>> suggestions (auto-completion). If this text field is left blank, all
>>> available metrics are displayed.
>>>
>>> Then, there's several scenarios to take advantage of the new hawkular
>>> metrics features:
>>>
>>> *2a*: keep current key/value pairs system, but improve with adding
>>> suggestion on tag names.
>>>
>>> *2b*: replace key-value pairs with the new tags query, via a simple
>>> text field:
>>>
>>>
>>> We may or may not include syntax validation here. We must provide some
>>> documentation on that.
>>>
>>> *2c*: replace key-value pairs with the new tags query, with a dedicated
>>> "builder" UI:
>>>
>>> ?Each of these boxes in tags query would have a contextual
>>> auto-completion feature:
>>> - suggestions on 1: list of tag keys
>>> - suggestions on 2: list of operators (=, !=, IN, NOT IN)
>>> - suggestions on 3: list of tag values for the given key (with some
>>> slights differences on brackets if it's the first element or not; closing
>>> bracket as a choice if it's not the first element)
>>> - suggestions on 4: operators AND / OR
>>> ?
>>>
>>> The 2b option is obviously simpler, very fast to implement. It has the
>>> downside of loosing all auto-completion capabilities, even compared to the
>>> current version.
>>>
>>> 2c looks nicer and more intuitive in its usage, people won't have to
>>> read the doc to use it. However there's several downsides:
>>>   - Need to implement the logic => need for time for development, adds
>>> complexity to our grafana plugin.
>>>   - Introduces a dependency to the language on server side. When the
>>> language evolves, we'll have to maintain this as well.
>>>
>>>
>>> Ideas & thoughts? I think it's preferable to proceed in several steps.
>>> In a first time I could implement *1* and *2a*, and later (and maybe
>>> giving some time to the language to be "stabilized") going with *2c*.
>>>
>>> [1] https://issues.jboss.org/browse/HWKMETRICS-532
>>>
>>> Thanks
>>> Joel
>>>
>>>
>>>
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>
>>>
>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/8acf951c/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search-with-builder.png
Type: image/png
Size: 39806 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/8acf951c/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search.png
Type: image/png
Size: 38020 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/8acf951c/attachment-0003.png 

From jtakvori at redhat.com  Wed Jan 18 05:07:39 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Wed, 18 Jan 2017 11:07:39 +0100
Subject: [Hawkular-dev] Fwd: [hawkular-dev] Grafana querying usability
In-Reply-To: <CAHnC0xipP_zaoXgXZoV-MRezX0q60-sYvgFsBgd+ko-Ockc_EQ@mail.gmail.com>
References: <CAJo5TFnmH3mhCm7cQBbpD5Xs3nTmCku4KB064fNpBqy9actFgQ@mail.gmail.com>
	<CAJo5TFk4ityAK7JVf5eGGTQ352JORU6YNFy5tB8M35rPxhrQAA@mail.gmail.com>
	<CAHnC0xj2W_F=v9auaxY2irCDzWZqz8TheFxernJMvy7_dFK_MA@mail.gmail.com>
	<CAJo5TFmjtGNaWfEA5NWzD3GrNg4qPTZWzzS0hYRq-TPHOiQ85Q@mail.gmail.com>
	<CAHnC0xipP_zaoXgXZoV-MRezX0q60-sYvgFsBgd+ko-Ockc_EQ@mail.gmail.com>
Message-ID: <CAJo5TFkhj36f3SBmDHGbF47wY+t6oxUJ2mtJqaDMd=dJ5ir2eQ@mail.gmail.com>

Ok, my idea was rather to show the metric text field and tags fields side
by side. If tags can be used to refine the list of suggested metrics (and I
think it would really be an important feature to avoid navigating in a flat
list of hundreds (thousands?) of metrics) - then we must keep them
separated.

Heiko also suggested to allow both freestyle and guided mode for tag
queries. Just checked the influx ds, I agree we could do something similar.
The query builder can be inspired from influx' one.

On Wed, Jan 18, 2017 at 10:53 AM, Thomas Segismont <tsegismo at redhat.com>
wrote:

> No, I really meant 1/. With a single text field mixing tag filters and
> names. Or did I misunderstood the idea?
>
> It might be worth taking a look at InfluxDB's datasource code. IIRC, they
> let you switch between guided (with many specific text fields) and
> freestyle (query language) mode.
>
> 2017-01-18 10:29 GMT+01:00 Joel Takvorian <jtakvori at redhat.com>:
>
>> Thomas, I think you mean the "2b" proposal? (with a simple text field to
>> type custom query in). I can't see how to provide auto-completion with
>> that. I think if we go this way, we loose auto-completion.
>>
>> On Wed, Jan 18, 2017 at 10:25 AM, Thomas Segismont <tsegismo at redhat.com>
>> wrote:
>>
>>> 1/ sounds best to me, as it would give users the power of a query
>>> language. But auto-completion could be quite a challenge there, couldn't it?
>>>
>>> 2017-01-18 9:54 GMT+01:00 Joel Takvorian <jtakvori at redhat.com>:
>>>
>>>> Hi everybody,
>>>>
>>>> I would like to get some opinions on the hawkular-grafana-datasource
>>>> querying usability, especially if you had the opportunity to create
>>>> dashboards recently and had to juggle with querying by metric name and by
>>>> tag.
>>>>
>>>> Currently the panel displays different elements depending on you're
>>>> querying by metric name (see picture: https://raw.githubusercontent.
>>>> com/hawkular/hawkular-grafana-datasource/master/docs/images/
>>>> search-by-name.png )
>>>>
>>>> Querying by name is quite straightforward, but it can be cumbersome
>>>> when there's a lot of available metrics and you have to scroll among
>>>> suggestions to find the one you want.
>>>> ?
>>>>
>>>> Or by tag  (see picture: https://raw.githubusercontent.
>>>> com/hawkular/hawkular-grafana-datasource/master/docs/images/
>>>> search-by-tag.png )
>>>>
>>>> ?
>>>> The "query by tag" interface is not very intuitive IMO (you define a
>>>> list of key-value pairs), moreover to this date there is no auto-completion
>>>> on tag name.
>>>>
>>>> There's been some features in metrics recently that, I think, can
>>>> enable a better UI on Grafana side. First, there is Micke's feature "Allow
>>>> fetching of available tagNames" [1] that enables suggestions
>>>> (auto-completion) on tag name. And most importantly there's the new "tag
>>>> query language" that could (should) have its place in Grafana.
>>>>
>>>>
>>>> So I would have several suggestions for improving the UI and queries.
>>>>
>>>> *1*: First of all I think we can remove the "Search by name" / "Search
>>>> by tag" selector, and allow searching by name AND by tag at the same time:
>>>> providing tags would refine the available metrics in the metrics text field
>>>> suggestions (auto-completion). If this text field is left blank, all
>>>> available metrics are displayed.
>>>>
>>>> Then, there's several scenarios to take advantage of the new hawkular
>>>> metrics features:
>>>>
>>>> *2a*: keep current key/value pairs system, but improve with adding
>>>> suggestion on tag names.
>>>>
>>>> *2b*: replace key-value pairs with the new tags query, via a simple
>>>> text field:
>>>>
>>>>
>>>> We may or may not include syntax validation here. We must provide some
>>>> documentation on that.
>>>>
>>>> *2c*: replace key-value pairs with the new tags query, with a
>>>> dedicated "builder" UI:
>>>>
>>>> ?Each of these boxes in tags query would have a contextual
>>>> auto-completion feature:
>>>> - suggestions on 1: list of tag keys
>>>> - suggestions on 2: list of operators (=, !=, IN, NOT IN)
>>>> - suggestions on 3: list of tag values for the given key (with some
>>>> slights differences on brackets if it's the first element or not; closing
>>>> bracket as a choice if it's not the first element)
>>>> - suggestions on 4: operators AND / OR
>>>> ?
>>>>
>>>> The 2b option is obviously simpler, very fast to implement. It has the
>>>> downside of loosing all auto-completion capabilities, even compared to the
>>>> current version.
>>>>
>>>> 2c looks nicer and more intuitive in its usage, people won't have to
>>>> read the doc to use it. However there's several downsides:
>>>>   - Need to implement the logic => need for time for development, adds
>>>> complexity to our grafana plugin.
>>>>   - Introduces a dependency to the language on server side. When the
>>>> language evolves, we'll have to maintain this as well.
>>>>
>>>>
>>>> Ideas & thoughts? I think it's preferable to proceed in several steps.
>>>> In a first time I could implement *1* and *2a*, and later (and maybe
>>>> giving some time to the language to be "stabilized") going with *2c*.
>>>>
>>>> [1] https://issues.jboss.org/browse/HWKMETRICS-532
>>>>
>>>> Thanks
>>>> Joel
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> hawkular-dev mailing list
>>>> hawkular-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>
>>>
>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/462eef89/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search-with-builder.png
Type: image/png
Size: 39806 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/462eef89/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-grafana-search.png
Type: image/png
Size: 38020 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170118/462eef89/attachment-0003.png 

From jmazzite at redhat.com  Thu Jan 19 11:58:14 2017
From: jmazzite at redhat.com (John Mazzitelli)
Date: Thu, 19 Jan 2017 11:58:14 -0500
Subject: [Hawkular-dev] Hawkular OpenShift Agent 1.0.0.Final released
Message-ID: <1484845094.6232.18.camel@redhat.com>

The Hawkular OpenShift Agent 1.0.0.Final has been released.

It is available on DockerHub here:?https://hub.docker.com/r/hawkular/ha
wkular-openshift-agent/tags/

If you want to test it with some test endpoints, you can use our
example images that provide you with Prometheus and Jolokia endpoints:

https://hub.docker.com/r/hawkular/hawkular-openshift-agent-example-prom
etheus-python/tags/

https://hub.docker.com/r/hawkular/hawkular-openshift-agent-example-jolo
kia-wildfly/tags/

The documentation is going to be revamped, but for now, you can go
here:

https://github.com/hawkular/hawkular-openshift-agent/blob/master/README
.adoc

and you can go here for how to deploy the examples:

https://github.com/hawkular/hawkular-openshift-agent/blob/master/exampl
es/README.adoc


From garethahealy at gmail.com  Thu Jan 19 12:47:10 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Thu, 19 Jan 2017 17:47:10 +0000
Subject: [Hawkular-dev] Hawkular OpenShift Agent 1.0.0.Final released
In-Reply-To: <1484845094.6232.18.camel@redhat.com>
References: <1484845094.6232.18.camel@redhat.com>
Message-ID: <CAHWT10Y5fVGzZV6h5BvqYC19cm0_fz6VjZJmJsDVYZ4-wiz8Qg@mail.gmail.com>

Cool. Congrats John.

Cheers.

On Thu, Jan 19, 2017 at 4:58 PM, John Mazzitelli <jmazzite at redhat.com>
wrote:

> The Hawkular OpenShift Agent 1.0.0.Final has been released.
>
> It is available on DockerHub here: https://hub.docker.com/r/hawkular/ha
> wkular-openshift-agent/tags/
>
> If you want to test it with some test endpoints, you can use our
> example images that provide you with Prometheus and Jolokia endpoints:
>
> https://hub.docker.com/r/hawkular/hawkular-openshift-agent-example-prom
> etheus-python/tags/
>
> https://hub.docker.com/r/hawkular/hawkular-openshift-agent-example-jolo
> kia-wildfly/tags/
>
> The documentation is going to be revamped, but for now, you can go
> here:
>
> https://github.com/hawkular/hawkular-openshift-agent/blob/master/README
> .adoc
>
> and you can go here for how to deploy the examples:
>
> https://github.com/hawkular/hawkular-openshift-agent/blob/master/exampl
> es/README.adoc
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170119/0bd8048d/attachment.html 

From garethahealy at gmail.com  Thu Jan 19 12:50:32 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Thu, 19 Jan 2017 17:50:32 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
Message-ID: <CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>

Any thoughts on the below? i.e.: should the agent create individual metrics
or should the endpoints be able to group.

Without this, it makes prometheus gauge metrics pretty useless.

Cheers.

On Tue, Jan 10, 2017 at 1:26 PM, Gareth Healy <garethahealy at gmail.com>
wrote:

> To carry on the discussion around whether the agent should create single
> metrics or hawkular should group the datapoints raised by Joel.
>
> I've done a bit more digging into different prometheus endpoints and heres
> an example from some work being done by Kuberenetes:
>
> https://github.com/kubernetes/kube-state-metrics
> # HELP kube_node_info Information about a cluster node.
> # TYPE kube_node_info gauge
> kube_node_info{container_runtime_version="docker://1.10.3",
> kernel_version="3.10.0-327.28.3.el7.x86_64",kubelet_version=
> "v1.3.0+52492b4",kubeproxy_version="v1.3.0+52492b4",node="origin",os_image="CentOS
> Linux 7 (Core)"} 1
>
> # HELP kube_pod_status_scheduled Describes the status of the scheduling
> process for the pod.
> # TYPE kube_pod_status_scheduled gauge
> kube_pod_status_scheduled{condition="true",namespace="cockpi
> t",pod="openshift-cockpit-1-0jub2"} 1
>
> # HELP kube_pod_container_requested_cpu_cores The number of requested cpu
> cores by a container.
> # TYPE kube_pod_container_requested_cpu_cores gauge
> kube_pod_container_requested_cpu_cores{container="registry",
> namespace="default",node="origin",pod="docker-registry-1-i37wn"} 0.1
>
> If the agent was to create single metrics, the idea in my mind would be as
> follows:
>
>    - Add a new boolean flag to create unique metrics based on labels
>    (default=false, to keep functionality as-is)
>    - Allow use of "id" field to contain value replacements based on label
>    value (probably need to be sanitised / escaped??)
>
> i.e.:
>
> metrics:
>> - name: kube_pod_container_requested_cpu_cores
>>   id: kube_pod_container_requested_cpu_cores_${container}_${node}
>>   unique_metric_per_label: true
>
>
> Which would create a gauge called:
>
> kube_pod_container_requested_cpu_cores_registry_origin
>
>
> Does that sound feasible? and thoughts?
>
> On Wed, Jan 4, 2017 at 1:06 PM, John Mazzitelli <mazz at redhat.com> wrote:
>
>> > Here, query by tag means time series tags, not data points tags. That's
>> > probably why you don't get anything. Maybe we should more clearly
>> > disambiguate these two concepts as there's often some confusion.
>>
>> :-) And that, I think, is what I meant by tags on "metric definitions" -
>> I think the H-Metrics team calls them "timeseries tags".
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170119/b38fce03/attachment.html 

From jmazzite at redhat.com  Thu Jan 19 14:44:13 2017
From: jmazzite at redhat.com (John Mazzitelli)
Date: Thu, 19 Jan 2017 14:44:13 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
Message-ID: <1484855053.6232.25.camel@redhat.com>

So the problem (if I understand correctly) is that if a Prometheus
metric has labels, we need to create individual metrics because
otherwise there is going to be one hawkular metric definition but it
can't represent all the data because of the different labels on the
Prometheus metric (which really represent multiple metrics).

Won't this be a problem with counter metrics too? It all seems to stem
from the fact of Prometheus labels.

You should document this in a github issue with all the details so we
can track it there.

At a quick glance without thinking about it much, I think your proposal
makes sense:

metrics:
- name: kube_pod_container_requested_cpu_cores
? id: kube_pod_container_requested_cpu_cores_${container}_${node}
? unique_metric_per_label: true

I'd like to avoid having to specifying "unique_metric_per_label" at all
and just have it inferred. Perhaps something like this: "if ID has at
least one "${x}" it means we need to create unique metric definitions
per label"

On Thu, 2017-01-19 at 17:50 +0000, Gareth Healy wrote:
> Any thoughts on the below? i.e.: should the agent create individual
> metrics or should the endpoints be able to group.
> 
> Without this, it makes prometheus gauge metrics pretty useless.
> 
> Cheers.
> 
> On Tue, Jan 10, 2017 at 1:26 PM, Gareth Healy <garethahealy at gmail.com
> > wrote:
> > To carry on the discussion around whether the agent should create
> > single metrics or hawkular should group the datapoints raised by
> > Joel.
> > 
> > I've done a bit more digging into different prometheus endpoints
> > and heres an example from some work being done by Kuberenetes:
> > 
> > https://github.com/kubernetes/kube-state-metrics
> > # HELP kube_node_info Information about a cluster node.
> > # TYPE kube_node_info gauge
> > kube_node_info{container_runtime_version="docker://1.10.3",kernel_v
> > ersion="3.10.0-
> > 327.28.3.el7.x86_64",kubelet_version="v1.3.0+52492b4",kubeproxy_ver
> > sion="v1.3.0+52492b4",node="origin",os_image="CentOS Linux 7
> > (Core)"} 1
> > 
> > # HELP kube_pod_status_scheduled Describes the status of the
> > scheduling process for the pod.
> > # TYPE kube_pod_status_scheduled gauge
> > kube_pod_status_scheduled{condition="true",namespace="cockpit",pod=
> > "openshift-cockpit-1-0jub2"} 1
> > 
> > # HELP kube_pod_container_requested_cpu_cores The number of
> > requested cpu cores by a container.
> > # TYPE kube_pod_container_requested_cpu_cores gauge
> > kube_pod_container_requested_cpu_cores{container="registry",namespa
> > ce="default",node="origin",pod="docker-registry-1-i37wn"} 0.1
> > 
> > If the agent was to create single metrics, the idea in my mind
> > would be as follows:
> > Add a new boolean flag to create unique metrics based on labels
> > (default=false, to keep functionality as-is)
> > Allow use of "id" field to contain value replacements based on
> > label value (probably need to be sanitised / escaped??)
> > i.e.:
> > 
> > > metrics:?
> > > - name: kube_pod_container_requested_cpu_cores?
> > > ? id:
> > > kube_pod_container_requested_cpu_cores_${container}_${node}?
> > > ? unique_metric_per_label: true
> > Which would create a gauge called:
> > 
> > > kube_pod_container_requested_cpu_cores_registry_origin
> > Does that sound feasible? and thoughts?
> > 
> > On Wed, Jan 4, 2017 at 1:06 PM, John Mazzitelli <mazz at redhat.com>
> > wrote:
> > > > Here, query by tag means time series tags, not data points
> > > tags. That's
> > > > probably why you don't get anything. Maybe we should more
> > > clearly
> > > > disambiguate these two concepts as there's often some
> > > confusion.
> > > 
> > > :-) And that, I think, is what I meant by tags on "metric
> > > definitions" - I think the H-Metrics team calls them "timeseries
> > > tags".
> > > _______________________________________________
> > > hawkular-dev mailing list
> > > hawkular-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > 
> > 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

From jmazzite at redhat.com  Thu Jan 19 14:58:17 2017
From: jmazzite at redhat.com (John Mazzitelli)
Date: Thu, 19 Jan 2017 14:58:17 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <1484855053.6232.25.camel@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
Message-ID: <1484855897.6232.28.camel@redhat.com>

Note that when HOSA stores the data point in H-Metrics, it does add the
prometheus labels to that datapoint as H-Metric tags. So H-Metrics does
have each datapoint properly tagged.

I haven't followed H-Metrics close enough to know if they plan on
addressing how to query that kind of tagged data to make it useful.

On Thu, 2017-01-19 at 14:44 -0500, John Mazzitelli wrote:
> So the problem (if I understand correctly) is that if a Prometheus
> metric has labels, we need to create individual metrics because
> otherwise there is going to be one hawkular metric definition but it
> can't represent all the data because of the different labels on the
> Prometheus metric (which really represent multiple metrics).
> 
> Won't this be a problem with counter metrics too? It all seems to
> stem
> from the fact of Prometheus labels.
> 
> You should document this in a github issue with all the details so we
> can track it there.
> 
> At a quick glance without thinking about it much, I think your
> proposal
> makes sense:
> 
> metrics:
> - name: kube_pod_container_requested_cpu_cores
> ? id: kube_pod_container_requested_cpu_cores_${container}_${node}
> ? unique_metric_per_label: true
> 
> I'd like to avoid having to specifying "unique_metric_per_label" at
> all
> and just have it inferred. Perhaps something like this: "if ID has at
> least one "${x}" it means we need to create unique metric definitions
> per label"
> 
> On Thu, 2017-01-19 at 17:50 +0000, Gareth Healy wrote:
> > 
> > Any thoughts on the below? i.e.: should the agent create individual
> > metrics or should the endpoints be able to group.
> > 
> > Without this, it makes prometheus gauge metrics pretty useless.
> > 
> > Cheers.
> > 
> > On Tue, Jan 10, 2017 at 1:26 PM, Gareth Healy <garethahealy at gmail.c
> > om
> > > 
> > > wrote:
> > > To carry on the discussion around whether the agent should create
> > > single metrics or hawkular should group the datapoints raised by
> > > Joel.
> > > 
> > > I've done a bit more digging into different prometheus endpoints
> > > and heres an example from some work being done by Kuberenetes:
> > > 
> > > https://github.com/kubernetes/kube-state-metrics
> > > # HELP kube_node_info Information about a cluster node.
> > > # TYPE kube_node_info gauge
> > > kube_node_info{container_runtime_version="docker://1.10.3",kernel
> > > _v
> > > ersion="3.10.0-
> > > 327.28.3.el7.x86_64",kubelet_version="v1.3.0+52492b4",kubeproxy_v
> > > er
> > > sion="v1.3.0+52492b4",node="origin",os_image="CentOS Linux 7
> > > (Core)"} 1
> > > 
> > > # HELP kube_pod_status_scheduled Describes the status of the
> > > scheduling process for the pod.
> > > # TYPE kube_pod_status_scheduled gauge
> > > kube_pod_status_scheduled{condition="true",namespace="cockpit",po
> > > d=
> > > "openshift-cockpit-1-0jub2"} 1
> > > 
> > > # HELP kube_pod_container_requested_cpu_cores The number of
> > > requested cpu cores by a container.
> > > # TYPE kube_pod_container_requested_cpu_cores gauge
> > > kube_pod_container_requested_cpu_cores{container="registry",names
> > > pa
> > > ce="default",node="origin",pod="docker-registry-1-i37wn"} 0.1
> > > 
> > > If the agent was to create single metrics, the idea in my mind
> > > would be as follows:
> > > Add a new boolean flag to create unique metrics based on labels
> > > (default=false, to keep functionality as-is)
> > > Allow use of "id" field to contain value replacements based on
> > > label value (probably need to be sanitised / escaped??)
> > > i.e.:
> > > 
> > > > 
> > > > metrics:?
> > > > - name: kube_pod_container_requested_cpu_cores?
> > > > ? id:
> > > > kube_pod_container_requested_cpu_cores_${container}_${node}?
> > > > ? unique_metric_per_label: true
> > > Which would create a gauge called:
> > > 
> > > > 
> > > > kube_pod_container_requested_cpu_cores_registry_origin
> > > Does that sound feasible? and thoughts?
> > > 
> > > On Wed, Jan 4, 2017 at 1:06 PM, John Mazzitelli <mazz at redhat.com>
> > > wrote:
> > > > 
> > > > > 
> > > > > Here, query by tag means time series tags, not data points
> > > > tags. That's
> > > > > 
> > > > > probably why you don't get anything. Maybe we should more
> > > > clearly
> > > > > 
> > > > > disambiguate these two concepts as there's often some
> > > > confusion.
> > > > 
> > > > :-) And that, I think, is what I meant by tags on "metric
> > > > definitions" - I think the H-Metrics team calls them
> > > > "timeseries
> > > > tags".
> > > > _______________________________________________
> > > > hawkular-dev mailing list
> > > > hawkular-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > > 
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

From jsanda at redhat.com  Thu Jan 19 15:21:41 2017
From: jsanda at redhat.com (John Sanda)
Date: Thu, 19 Jan 2017 15:21:41 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <1484855897.6232.28.camel@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
Message-ID: <CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>

I think I already asked this before, but does the tag based bucketing introduced in https://issues.jboss.org/browse/HWKMETRICS-373 <https://issues.jboss.org/browse/HWKMETRICS-373> help here?

> On Jan 19, 2017, at 2:58 PM, John Mazzitelli <jmazzite at redhat.com> wrote:
> 
> Note that when HOSA stores the data point in H-Metrics, it does add the
> prometheus labels to that datapoint as H-Metric tags. So H-Metrics does
> have each datapoint properly tagged.
> 
> I haven't followed H-Metrics close enough to know if they plan on
> addressing how to query that kind of tagged data to make it useful.
> 
> On Thu, 2017-01-19 at 14:44 -0500, John Mazzitelli wrote:
>> So the problem (if I understand correctly) is that if a Prometheus
>> metric has labels, we need to create individual metrics because
>> otherwise there is going to be one hawkular metric definition but it
>> can't represent all the data because of the different labels on the
>> Prometheus metric (which really represent multiple metrics).
>> 
>> Won't this be a problem with counter metrics too? It all seems to
>> stem
>> from the fact of Prometheus labels.
>> 
>> You should document this in a github issue with all the details so we
>> can track it there.
>> 
>> At a quick glance without thinking about it much, I think your
>> proposal
>> makes sense:
>> 
>> metrics:
>> - name: kube_pod_container_requested_cpu_cores
>>   id: kube_pod_container_requested_cpu_cores_${container}_${node}
>>   unique_metric_per_label: true
>> 
>> I'd like to avoid having to specifying "unique_metric_per_label" at
>> all
>> and just have it inferred. Perhaps something like this: "if ID has at
>> least one "${x}" it means we need to create unique metric definitions
>> per label"
>> 
>> On Thu, 2017-01-19 at 17:50 +0000, Gareth Healy wrote:
>>> 
>>> Any thoughts on the below? i.e.: should the agent create individual
>>> metrics or should the endpoints be able to group.
>>> 
>>> Without this, it makes prometheus gauge metrics pretty useless.
>>> 
>>> Cheers.
>>> 
>>> On Tue, Jan 10, 2017 at 1:26 PM, Gareth Healy <garethahealy at gmail.c
>>> om
>>>> 
>>>> wrote:
>>>> To carry on the discussion around whether the agent should create
>>>> single metrics or hawkular should group the datapoints raised by
>>>> Joel.
>>>> 
>>>> I've done a bit more digging into different prometheus endpoints
>>>> and heres an example from some work being done by Kuberenetes:
>>>> 
>>>> https://github.com/kubernetes/kube-state-metrics
>>>> # HELP kube_node_info Information about a cluster node.
>>>> # TYPE kube_node_info gauge
>>>> kube_node_info{container_runtime_version="docker://1.10.3",kernel
>>>> _v
>>>> ersion="3.10.0-
>>>> 327.28.3.el7.x86_64",kubelet_version="v1.3.0+52492b4",kubeproxy_v
>>>> er
>>>> sion="v1.3.0+52492b4",node="origin",os_image="CentOS Linux 7
>>>> (Core)"} 1
>>>> 
>>>> # HELP kube_pod_status_scheduled Describes the status of the
>>>> scheduling process for the pod.
>>>> # TYPE kube_pod_status_scheduled gauge
>>>> kube_pod_status_scheduled{condition="true",namespace="cockpit",po
>>>> d=
>>>> "openshift-cockpit-1-0jub2"} 1
>>>> 
>>>> # HELP kube_pod_container_requested_cpu_cores The number of
>>>> requested cpu cores by a container.
>>>> # TYPE kube_pod_container_requested_cpu_cores gauge
>>>> kube_pod_container_requested_cpu_cores{container="registry",names
>>>> pa
>>>> ce="default",node="origin",pod="docker-registry-1-i37wn"} 0.1
>>>> 
>>>> If the agent was to create single metrics, the idea in my mind
>>>> would be as follows:
>>>> Add a new boolean flag to create unique metrics based on labels
>>>> (default=false, to keep functionality as-is)
>>>> Allow use of "id" field to contain value replacements based on
>>>> label value (probably need to be sanitised / escaped??)
>>>> i.e.:
>>>> 
>>>>> 
>>>>> metrics: 
>>>>> - name: kube_pod_container_requested_cpu_cores 
>>>>>   id:
>>>>> kube_pod_container_requested_cpu_cores_${container}_${node} 
>>>>>   unique_metric_per_label: true
>>>> Which would create a gauge called:
>>>> 
>>>>> 
>>>>> kube_pod_container_requested_cpu_cores_registry_origin
>>>> Does that sound feasible? and thoughts?
>>>> 
>>>> On Wed, Jan 4, 2017 at 1:06 PM, John Mazzitelli <mazz at redhat.com>
>>>> wrote:
>>>>> 
>>>>>> 
>>>>>> Here, query by tag means time series tags, not data points
>>>>> tags. That's
>>>>>> 
>>>>>> probably why you don't get anything. Maybe we should more
>>>>> clearly
>>>>>> 
>>>>>> disambiguate these two concepts as there's often some
>>>>> confusion.
>>>>> 
>>>>> :-) And that, I think, is what I meant by tags on "metric
>>>>> definitions" - I think the H-Metrics team calls them
>>>>> "timeseries
>>>>> tags".
>>>>> _______________________________________________
>>>>> hawkular-dev mailing list
>>>>> hawkular-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>>> 
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170119/b4c14da5/attachment.html 

From jmazzite at redhat.com  Thu Jan 19 15:35:05 2017
From: jmazzite at redhat.com (John Mazzitelli)
Date: Thu, 19 Jan 2017 15:35:05 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
	<CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
Message-ID: <1484858105.6232.29.camel@redhat.com>


On Thu, 2017-01-19 at 15:21 -0500, John Sanda wrote:
> I think I already asked this before, but does the tag based bucketing
> introduced in?https://issues.jboss.org/browse/HWKMETRICS-373?help
> here?

I do not know... that would be a question for?Gareth to answer.

From garethahealy at gmail.com  Thu Jan 19 16:02:11 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Thu, 19 Jan 2017 21:02:11 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <1484858105.6232.29.camel@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
	<CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
	<1484858105.6232.29.camel@redhat.com>
Message-ID: <CAHWT10aRrDjvmis-NjUgG2AK8JEbbV+h12aayRAF8JZmj3nzNg@mail.gmail.com>

@John S,

No, i dont get back what i expect.

Based on the below data (below is logs from HOSA):

I0113 17:24:09.138095       1 metrics_storage.go:154] TRACE: Stored [5]
[gauge] datapoints for metric named
[pod/ded071be-d9a6-11e6-8140-525400c583ad/custom/prometheus_MyCamel_MeanProcessingTime]:

[{2017-01-13 17:24:09.117190753 +0000 UTC 18 map[type:routes
name:"route1"]}
{2017-01-13 17:24:09.117190753 +0000 UTC 0 map[type:processors
name:"transform2"]}
{2017-01-13 17:24:09.117190753 +0000 UTC 17 map[type:processors
name:"choice1"]}
{2017-01-13 17:24:09.117190753 +0000 UTC 8 map[type:processors
name:"transform1"]}
{2017-01-13 17:24:09.117190753 +0000 UTC 18 map[name:"MyCamel"
type:context]}]


With the below cURL:

curl -k -X GET -H "Content-Type: application/json" -H "Hawkular-Tenant:
$HAWKULAR_TENANT" -H "Authorization: Bearer $(oc sa get-token -n
openshift-infra heapster)"
"$HAWKULAR_URL/gauges/stats?start=1484320827578&end=1484329973940&bucketDuration=10d&tags=type:routes"

I get no data back. From a reply i got from Joel (in this thread), its
because the stats endpoint is working on metric tags, but i am wanting to
look at datapoint tags.

On Thu, Jan 19, 2017 at 8:35 PM, John Mazzitelli <jmazzite at redhat.com>
wrote:

>
> On Thu, 2017-01-19 at 15:21 -0500, John Sanda wrote:
> > I think I already asked this before, but does the tag based bucketing
> > introduced in https://issues.jboss.org/browse/HWKMETRICS-373 help
> > here?
>
> I do not know... that would be a question for Gareth to answer.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170119/b8633ef7/attachment.html 

From jsanda at redhat.com  Thu Jan 19 16:45:21 2017
From: jsanda at redhat.com (John Sanda)
Date: Thu, 19 Jan 2017 16:45:21 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10aRrDjvmis-NjUgG2AK8JEbbV+h12aayRAF8JZmj3nzNg@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
	<CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
	<1484858105.6232.29.camel@redhat.com>
	<CAHWT10aRrDjvmis-NjUgG2AK8JEbbV+h12aayRAF8JZmj3nzNg@mail.gmail.com>
Message-ID: <953A2FBA-F384-4081-AE45-3E5803A3BBA2@redhat.com>

You are using the wrong endpoint. It should be:

$HAWKULAR_URL/gauges/{metric_id}/stats/tags/type/type:routes?start=1484320827578&end=1484329973940

where {metric_id} is pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%2Fprometheus_MyCamel_MeanProcessingTime

And for this endpoint the bucketDuration parameter is not supported. The buckets are determined by the tag filters. The endpoint URL has changed from the initial implementation (and needs to be documented) but the examples in the ticket of how the buckets are determined are still valid.

> On Jan 19, 2017, at 4:02 PM, Gareth Healy <garethahealy at gmail.com> wrote:
> 
> @John S,
> 
> No, i dont get back what i expect.
> 
> Based on the below data (below is logs from HOSA):
> 
> I0113 17:24:09.138095       1 metrics_storage.go:154] TRACE: Stored [5] [gauge] datapoints for metric named [pod/ded071be-d9a6-11e6-8140-525400c583ad/custom/prometheus_MyCamel_MeanProcessingTime]: 
> 
> [{2017-01-13 17:24:09.117190753 +0000 UTC 18 map[type:routes name:"route1"]} 
> {2017-01-13 17:24:09.117190753 +0000 UTC 0 map[type:processors name:"transform2"]} 
> {2017-01-13 17:24:09.117190753 +0000 UTC 17 map[type:processors name:"choice1"]} 
> {2017-01-13 17:24:09.117190753 +0000 UTC 8 map[type:processors name:"transform1"]} 
> {2017-01-13 17:24:09.117190753 +0000 UTC 18 map[name:"MyCamel" type:context]}]
> 
> With the below cURL:
> 
> curl -k -X GET -H "Content-Type: application/json" -H "Hawkular-Tenant: $HAWKULAR_TENANT" -H "Authorization: Bearer $(oc sa get-token -n openshift-infra heapster)" "$HAWKULAR_URL/gauges/stats?start=1484320827578&end=1484329973940&bucketDuration=10d&tags=type:routes"
> 
> I get no data back. From a reply i got from Joel (in this thread), its because the stats endpoint is working on metric tags, but i am wanting to look at datapoint tags.
> 
> On Thu, Jan 19, 2017 at 8:35 PM, John Mazzitelli <jmazzite at redhat.com <mailto:jmazzite at redhat.com>> wrote:
> 
> On Thu, 2017-01-19 at 15:21 -0500, John Sanda wrote:
> > I think I already asked this before, but does the tag based bucketing
> > introduced in https://issues.jboss.org/browse/HWKMETRICS-373 <https://issues.jboss.org/browse/HWKMETRICS-373> help
> > here?
> 
> I do not know... that would be a question for Gareth to answer.
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170119/00a7bfb2/attachment-0001.html 

From garethahealy at gmail.com  Fri Jan 20 04:21:34 2017
From: garethahealy at gmail.com (Gareth Healy)
Date: Fri, 20 Jan 2017 09:21:34 +0000
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <953A2FBA-F384-4081-AE45-3E5803A3BBA2@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<1982058557.7528867.1483029202597.JavaMail.zimbra@redhat.com>
	<CAHWT10ZTnT-VQb0cpi2G_NQJFgm0EtLrNoaydF1hFiseEiE53A@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
	<CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
	<1484858105.6232.29.camel@redhat.com>
	<CAHWT10aRrDjvmis-NjUgG2AK8JEbbV+h12aayRAF8JZmj3nzNg@mail.gmail.com>
	<953A2FBA-F384-4081-AE45-3E5803A3BBA2@redhat.com>
Message-ID: <CAHWT10YWzbkXwLyB+yXzWxRdM4iN-=_3o7ZNQ3qOS4jKzMKCCQ@mail.gmail.com>

Hi John S,

No, still dont get any data (204 error) back with that endpoint either.

I tweaked the endpoint, as you had /stats/tags/*type*/ - presume that was a
typo, since it gives me a 404 and i can't see it mentioned in the docs so
just removed it.

HAWKULAR_TENANT="fis2-monitoring-demo"
MERTIC_ID="pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%2Fprometheus_MyCamel_MeanProcessingTime"

curl -v -k -X GET \

-H "Content-Type: application/json" \

-H "Hawkular-Tenant: $HAWKULAR_TENANT" \

-H "Authorization: Bearer $(oc sa get-token -n openshift-infra heapster)" \

"$HAWKULAR_URL/gauges/$MERTIC_ID/stats/tags/name:*?start=1484320827578&
end=1484329973940"

Tried with both tag keys (name and type) and with the value set and
wild-carded.

The $HAWKULAR_URL/gauges/$MERTIC_ID/raw endpoint returns data points.

On Thu, Jan 19, 2017 at 9:45 PM, John Sanda <jsanda at redhat.com> wrote:

> You are using the wrong endpoint. It should be:
>
> $HAWKULAR_URL/gauges/{metric_id}/stats/tags/type/type:
> routes?start=1484320827578&end=1484329973940
>
> where {metric_id} is pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%
> 2Fprometheus_MyCamel_MeanProcessingTime
>
> And for this endpoint the bucketDuration parameter is not supported. The
> buckets are determined by the tag filters. The endpoint URL has changed
> from the initial implementation (and needs to be documented) but the
> examples in the ticket of how the buckets are determined are still valid.
>
> On Jan 19, 2017, at 4:02 PM, Gareth Healy <garethahealy at gmail.com> wrote:
>
> @John S,
>
> No, i dont get back what i expect.
>
> Based on the below data (below is logs from HOSA):
>
> I0113 17:24:09.138095       1 metrics_storage.go:154] TRACE: Stored [5]
> [gauge] datapoints for metric named [pod/ded071be-d9a6-11e6-8140-
> 525400c583ad/custom/prometheus_MyCamel_MeanProcessingTime]:
>
> [{2017-01-13 17:24:09.117190753 +0000 UTC 18 map[type:routes
> name:"route1"]}
> {2017-01-13 17:24:09.117190753 +0000 UTC 0 map[type:processors
> name:"transform2"]}
> {2017-01-13 17:24:09.117190753 +0000 UTC 17 map[type:processors
> name:"choice1"]}
> {2017-01-13 17:24:09.117190753 +0000 UTC 8 map[type:processors
> name:"transform1"]}
> {2017-01-13 17:24:09.117190753 +0000 UTC 18 map[name:"MyCamel"
> type:context]}]
>
>
> With the below cURL:
>
> curl -k -X GET -H "Content-Type: application/json" -H "Hawkular-Tenant:
> $HAWKULAR_TENANT" -H "Authorization: Bearer $(oc sa get-token -n
> openshift-infra heapster)" "$HAWKULAR_URL/gauges/stats?
> start=1484320827578&end=1484329973940&bucketDuration=10d&tags=type:routes"
>
> I get no data back. From a reply i got from Joel (in this thread), its
> because the stats endpoint is working on metric tags, but i am wanting to
> look at datapoint tags.
>
> On Thu, Jan 19, 2017 at 8:35 PM, John Mazzitelli <jmazzite at redhat.com>
> wrote:
>
>>
>> On Thu, 2017-01-19 at 15:21 -0500, John Sanda wrote:
>> > I think I already asked this before, but does the tag based bucketing
>> > introduced in https://issues.jboss.org/browse/HWKMETRICS-373 help
>> > here?
>>
>> I do not know... that would be a question for Gareth to answer.
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170120/c1f2c715/attachment.html 

From miburman at redhat.com  Fri Jan 20 08:09:29 2017
From: miburman at redhat.com (Michael Burman)
Date: Fri, 20 Jan 2017 15:09:29 +0200
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <CAHWT10YWzbkXwLyB+yXzWxRdM4iN-=_3o7ZNQ3qOS4jKzMKCCQ@mail.gmail.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
	<CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
	<1484858105.6232.29.camel@redhat.com>
	<CAHWT10aRrDjvmis-NjUgG2AK8JEbbV+h12aayRAF8JZmj3nzNg@mail.gmail.com>
	<953A2FBA-F384-4081-AE45-3E5803A3BBA2@redhat.com>
	<CAHWT10YWzbkXwLyB+yXzWxRdM4iN-=_3o7ZNQ3qOS4jKzMKCCQ@mail.gmail.com>
Message-ID: <40cfecd8-7fb2-8ef7-d4ba-c490f8318027@redhat.com>

Hi,

Your syntax is still incorrect. It's either metricId or tags, not both 
(at least not in this case).

You should call /gauges/stats?tags=name:*&start=..

https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L571

If you wish to use id/stats, then you can't have tags at all.

/gauges/$METRIC_ID/stats?start=..

https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L509

   - Micke

On 01/20/2017 11:21 AM, Gareth Healy wrote:
> Hi John S,
>
> No, still dont get any data (204 error) back with that endpoint either.
>
> I tweaked the endpoint, as you had /stats/tags/*type*/ - presume that 
> was a typo, since it gives me a 404 and i can't see it mentioned in 
> the docs so just removed it.
>
>     HAWKULAR_TENANT="fis2-monitoring-demo"
>     MERTIC_ID="pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%2Fprometheus_MyCamel_MeanProcessingTime"
>
>     curl -v -k -X GET \
>
>         -H "Content-Type: application/json" \
>
>         -H "Hawkular-Tenant: $HAWKULAR_TENANT" \
>
>         -H "Authorization: Bearer $(oc sa get-token -n openshift-infra
>         heapster)" \
>
>         "$HAWKULAR_URL/gauges/$MERTIC_ID/stats/tags/name:*?start=1484320827578&end=1484329973940"
>
> Tried with both tag keys (name and type) and with the value set and 
> wild-carded.
>
> The $HAWKULAR_URL/gauges/$MERTIC_ID/raw endpoint returns data points.
>
> On Thu, Jan 19, 2017 at 9:45 PM, John Sanda <jsanda at redhat.com 
> <mailto:jsanda at redhat.com>> wrote:
>
>     You are using the wrong endpoint. It should be:
>
>     $HAWKULAR_URL/gauges/{metric_id}/stats/tags/type/type:routes?start=1484320827578&end=1484329973940
>
>     where {metric_id}
>     is pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%2Fprometheus_MyCamel_MeanProcessingTime
>
>     And for this endpoint the bucketDuration parameter is not
>     supported. The buckets are determined by the tag filters. The
>     endpoint URL has changed from the initial implementation (and
>     needs to be documented) but the examples in the ticket of how the
>     buckets are determined are still valid.
>
>>     On Jan 19, 2017, at 4:02 PM, Gareth Healy <garethahealy at gmail.com
>>     <mailto:garethahealy at gmail.com>> wrote:
>>
>>     @John S,
>>
>>     No, i dont get back what i expect.
>>
>>     Based on the below data (below is logs from HOSA):
>>
>>         I0113 17:24:09.138095       1 metrics_storage.go:154] TRACE:
>>         Stored [5] [gauge] datapoints for metric named
>>         [pod/ded071be-d9a6-11e6-8140-525400c583ad/custom/prometheus_MyCamel_MeanProcessingTime]:
>>
>>
>>         [{2017-01-13 17:24:09.117190753 +0000 UTC 18 map[type:routes
>>         name:"route1"]}
>>         {2017-01-13 17:24:09.117190753 +0000 UTC 0
>>         map[type:processors name:"transform2"]}
>>         {2017-01-13 17:24:09.117190753 +0000 UTC 17
>>         map[type:processors name:"choice1"]}
>>         {2017-01-13 17:24:09.117190753 +0000 UTC 8
>>         map[type:processors name:"transform1"]}
>>         {2017-01-13 17:24:09.117190753 +0000 UTC 18
>>         map[name:"MyCamel" type:context]}]
>>
>>
>>     With the below cURL:
>>
>>         curl -k -X GET -H "Content-Type: application/json" -H
>>         "Hawkular-Tenant: $HAWKULAR_TENANT" -H "Authorization: Bearer
>>         $(oc sa get-token -n openshift-infra heapster)"
>>         "$HAWKULAR_URL/gauges/stats?start=1484320827578&end=1484329973940&bucketDuration=10d&tags=type:routes"
>>
>>     I get no data back. From a reply i got from Joel (in this
>>     thread), its because the stats endpoint is working on metric
>>     tags, but i am wanting to look at datapoint tags.
>>
>>     On Thu, Jan 19, 2017 at 8:35 PM, John Mazzitelli
>>     <jmazzite at redhat.com <mailto:jmazzite at redhat.com>> wrote:
>>
>>
>>         On Thu, 2017-01-19 at 15:21 -0500, John Sanda wrote:
>>         > I think I already asked this before, but does the tag based
>>         bucketing
>>         > introduced in
>>         https://issues.jboss.org/browse/HWKMETRICS-373
>>         <https://issues.jboss.org/browse/HWKMETRICS-373> help
>>         > here?
>>
>>         I do not know... that would be a question for Gareth to answer.
>>         _______________________________________________
>>         hawkular-dev mailing list
>>         hawkular-dev at lists.jboss.org
>>         <mailto:hawkular-dev at lists.jboss.org>
>>         https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>         <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>>
>>
>>     _______________________________________________
>>     hawkular-dev mailing list
>>     hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
>>     https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>     <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>
>
>     _______________________________________________
>     hawkular-dev mailing list
>     hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/hawkular-dev
>     <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>
>
>
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev


From jsanda at redhat.com  Fri Jan 20 09:35:44 2017
From: jsanda at redhat.com (John Sanda)
Date: Fri, 20 Jan 2017 09:35:44 -0500
Subject: [Hawkular-dev] Ability to group by datapoint tag in Grafana
In-Reply-To: <40cfecd8-7fb2-8ef7-d4ba-c490f8318027@redhat.com>
References: <CAHWT10YiNM4ma49yfu8SKrFL1Dkac=RwFYF+8UFjO2vp5PNjdA@mail.gmail.com>
	<2099209931.7580520.1483122543068.JavaMail.zimbra@redhat.com>
	<CAHWT10ZuqAo_zq7OSujuXkrr3bADBWq=wv9JR+dDdZjqKAWDkg@mail.gmail.com>
	<AFF2DCC5-8C2B-42D4-84C6-1D5609926100@redhat.com>
	<CAHWT10Z=ZZ9mzdH0L7xfmy7dxwJFYH-QaRDRgR_K3-uQJ4KbuA@mail.gmail.com>
	<CAJo5TFmq0L8=nh5DY04ntPZHhUStE8eN3xc4Zxg5fgy5bfMGKw@mail.gmail.com>
	<799472558.1443565.1483535199453.JavaMail.zimbra@redhat.com>
	<CAHWT10bRRmBWd1LB2d5Pweg6bwV7TmPai=EuRMRkGQ6B0qp-0g@mail.gmail.com>
	<CAHWT10YMDMdoWwSSX25bCviqyKJ53X1HRMpV+_9Ce-Ad0J2yQw@mail.gmail.com>
	<1484855053.6232.25.camel@redhat.com>
	<1484855897.6232.28.camel@redhat.com>
	<CC1D5D3B-487C-4610-92CC-C7FCCB24B3CC@redhat.com>
	<1484858105.6232.29.camel@redhat.com>
	<CAHWT10aRrDjvmis-NjUgG2AK8JEbbV+h12aayRAF8JZmj3nzNg@mail.gmail.com>
	<953A2FBA-F384-4081-AE45-3E5803A3BBA2@redhat.com>
	<CAHWT10YWzbkXwLyB+yXzWxRdM4iN-=_3o7ZNQ3qOS4jKzMKCCQ@mail.gmail.com>
	<40cfecd8-7fb2-8ef7-d4ba-c490f8318027@redhat.com>
Message-ID: <F6418151-4756-4E53-803E-1F7EDD8310AF@redhat.com>


> On Jan 20, 2017, at 8:09 AM, Michael Burman <miburman at redhat.com> wrote:
> 
> Hi,
> 
> Your syntax is still incorrect. It's either metricId or tags, not both 
> (at least not in this case).
> 
> You should call /gauges/stats?tags=name:*&start=..
> 
> https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L571 <https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L571>
> 
> If you wish to use id/stats, then you can't have tags at all.

That?s not right. Look at https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L628 <https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L628>. Maybe I had a typo in my previous example. The endpoint is:

GET /hawkular/metrics/gauges/{id}/stats/tags/{tags}

and the endpoint supports the standard start/end parameters.

> 
> /gauges/$METRIC_ID/stats?start=..
> 
> https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L509 <https://github.com/hawkular/hawkular-metrics/blob/master/api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/handler/GaugeHandler.java#L509>
> 
>   - Micke
> 
> On 01/20/2017 11:21 AM, Gareth Healy wrote:
>> Hi John S,
>> 
>> No, still dont get any data (204 error) back with that endpoint either.
>> 
>> I tweaked the endpoint, as you had /stats/tags/*type*/ - presume that 
>> was a typo, since it gives me a 404 and i can't see it mentioned in 
>> the docs so just removed it.
>> 
>>    HAWKULAR_TENANT="fis2-monitoring-demo"
>>    MERTIC_ID="pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%2Fprometheus_MyCamel_MeanProcessingTime"
>> 
>>    curl -v -k -X GET \
>> 
>>        -H "Content-Type: application/json" \
>> 
>>        -H "Hawkular-Tenant: $HAWKULAR_TENANT" \
>> 
>>        -H "Authorization: Bearer $(oc sa get-token -n openshift-infra
>>        heapster)" \
>> 
>>        "$HAWKULAR_URL/gauges/$MERTIC_ID/stats/tags/name:*?start=1484320827578&end=1484329973940"
>> 
>> Tried with both tag keys (name and type) and with the value set and 
>> wild-carded.
>> 
>> The $HAWKULAR_URL/gauges/$MERTIC_ID/raw endpoint returns data points.
>> 
>> On Thu, Jan 19, 2017 at 9:45 PM, John Sanda <jsanda at redhat.com 
>> <mailto:jsanda at redhat.com <mailto:jsanda at redhat.com>>> wrote:
>> 
>>    You are using the wrong endpoint. It should be:
>> 
>>    $HAWKULAR_URL/gauges/{metric_id}/stats/tags/type/type:routes?start=1484320827578&end=1484329973940
>> 
>>    where {metric_id}
>>    is pod%2Fded071be-d9a6-11e6-8140-525400c583ad%2Fcustom%2Fprometheus_MyCamel_MeanProcessingTime
>> 
>>    And for this endpoint the bucketDuration parameter is not
>>    supported. The buckets are determined by the tag filters. The
>>    endpoint URL has changed from the initial implementation (and
>>    needs to be documented) but the examples in the ticket of how the
>>    buckets are determined are still valid.
>> 
>>>    On Jan 19, 2017, at 4:02 PM, Gareth Healy <garethahealy at gmail.com <mailto:garethahealy at gmail.com>
>>>    <mailto:garethahealy at gmail.com <mailto:garethahealy at gmail.com>>> wrote:
>>> 
>>>    @John S,
>>> 
>>>    No, i dont get back what i expect.
>>> 
>>>    Based on the below data (below is logs from HOSA):
>>> 
>>>        I0113 17:24:09.138095       1 metrics_storage.go:154] TRACE:
>>>        Stored [5] [gauge] datapoints for metric named
>>>        [pod/ded071be-d9a6-11e6-8140-525400c583ad/custom/prometheus_MyCamel_MeanProcessingTime]:
>>> 
>>> 
>>>        [{2017-01-13 17:24:09.117190753 +0000 UTC 18 map[type:routes
>>>        name:"route1"]}
>>>        {2017-01-13 17:24:09.117190753 +0000 UTC 0
>>>        map[type:processors name:"transform2"]}
>>>        {2017-01-13 17:24:09.117190753 +0000 UTC 17
>>>        map[type:processors name:"choice1"]}
>>>        {2017-01-13 17:24:09.117190753 +0000 UTC 8
>>>        map[type:processors name:"transform1"]}
>>>        {2017-01-13 17:24:09.117190753 +0000 UTC 18
>>>        map[name:"MyCamel" type:context]}]
>>> 
>>> 
>>>    With the below cURL:
>>> 
>>>        curl -k -X GET -H "Content-Type: application/json" -H
>>>        "Hawkular-Tenant: $HAWKULAR_TENANT" -H "Authorization: Bearer
>>>        $(oc sa get-token -n openshift-infra heapster)"
>>>        "$HAWKULAR_URL/gauges/stats?start=1484320827578&end=1484329973940&bucketDuration=10d&tags=type:routes"
>>> 
>>>    I get no data back. From a reply i got from Joel (in this
>>>    thread), its because the stats endpoint is working on metric
>>>    tags, but i am wanting to look at datapoint tags.
>>> 
>>>    On Thu, Jan 19, 2017 at 8:35 PM, John Mazzitelli
>>>    <jmazzite at redhat.com <mailto:jmazzite at redhat.com> <mailto:jmazzite at redhat.com <mailto:jmazzite at redhat.com>>> wrote:
>>> 
>>> 
>>>        On Thu, 2017-01-19 at 15:21 -0500, John Sanda wrote:
>>>> I think I already asked this before, but does the tag based
>>>        bucketing
>>>> introduced in
>>>        https://issues.jboss.org/browse/HWKMETRICS-373 <https://issues.jboss.org/browse/HWKMETRICS-373>
>>>        <https://issues.jboss.org/browse/HWKMETRICS-373 <https://issues.jboss.org/browse/HWKMETRICS-373>> help
>>>> here?
>>> 
>>>        I do not know... that would be a question for Gareth to answer.
>>>        _______________________________________________
>>>        hawkular-dev mailing list
>>>        hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
>>>        <mailto:hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>>
>>>        https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>>>        <https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>>
>>> 
>>> 
>>>    _______________________________________________
>>>    hawkular-dev mailing list
>>>    hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org> <mailto:hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>>
>>>    https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>>>    <https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>>
>> 
>> 
>>    _______________________________________________
>>    hawkular-dev mailing list
>>    hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org> <mailto:hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>>
>>    https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>>    <https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>> 
>> 
>> 
>> 
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170120/35b2b6e9/attachment-0001.html 

From hrupp at redhat.com  Tue Jan 24 04:21:05 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Tue, 24 Jan 2017 10:21:05 +0100
Subject: [Hawkular-dev] Inventory and 'transient' servers
Message-ID: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>

Hey,

when WildFly connects to Inventory for the 1st time, we sync
the EAP information into inventory, which also includes the information
about which metrics are available.

Now when WildFly is deployed into Kubernetes or OpenShift, we
will see that WildFly is started, syncs and then dies at some point
in time, where k8s will not re-use the existing one, but start
a new one, which will have a different FeedId.

This will leave a WildFly in Inventory, that is later detected as
down in Metrics/Alerting, but the one in Inventory will stay
forever. Consequences are
- Inventory will get "full" with no-longer needed information
- Clients will retrieve data about non-"usable" servers

We need to figure out how to deal with this situation like e.g.:
- have inventory listen on k8s events and remove the server
  when k8s removes it (not when it is going down; stopped pods
 can stay around for some time)
- Have a different way of creating the feed id / server id so that
 the state is "re-used". Something like feedId/server name could
 be the name of the deployment config + version + k8s tenant.

Thoughts?


From jtakvori at redhat.com  Tue Jan 24 04:35:51 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Tue, 24 Jan 2017 10:35:51 +0100
Subject: [Hawkular-dev] Inventory and 'transient' servers
In-Reply-To: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
References: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
Message-ID: <CAJo5TFngepgAjaY+mNEC9DOzdb9O8S3H3_TQPQnQ7_8kn3W7xw@mail.gmail.com>

Re-using the state would mean there's always 1 or 0 running instance of
that server at a given time, but what happens if it's scaled up to 2
instances or more? Is it a possible scenario? If yes, it seems complicated
to use dc+version+tenant as the feed id.

I filed an issue in JIRA about introducing some kind of TTL :
https://issues.jboss.org/browse/HWKINVENT-205
Maybe those expired feeds fall in the scope of this issue.


On Tue, Jan 24, 2017 at 10:21 AM, Heiko W.Rupp <hrupp at redhat.com> wrote:

> Hey,
>
> when WildFly connects to Inventory for the 1st time, we sync
> the EAP information into inventory, which also includes the information
> about which metrics are available.
>
> Now when WildFly is deployed into Kubernetes or OpenShift, we
> will see that WildFly is started, syncs and then dies at some point
> in time, where k8s will not re-use the existing one, but start
> a new one, which will have a different FeedId.
>
> This will leave a WildFly in Inventory, that is later detected as
> down in Metrics/Alerting, but the one in Inventory will stay
> forever. Consequences are
> - Inventory will get "full" with no-longer needed information
> - Clients will retrieve data about non-"usable" servers
>
> We need to figure out how to deal with this situation like e.g.:
> - have inventory listen on k8s events and remove the server
>   when k8s removes it (not when it is going down; stopped pods
>  can stay around for some time)
> - Have a different way of creating the feed id / server id so that
>  the state is "re-used". Something like feedId/server name could
>  be the name of the deployment config + version + k8s tenant.
>
> Thoughts?
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170124/fe1c3111/attachment.html 

From jpkroehling at redhat.com  Tue Jan 24 05:13:33 2017
From: jpkroehling at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=)
Date: Tue, 24 Jan 2017 11:13:33 +0100
Subject: [Hawkular-dev] Inventory and 'transient' servers
In-Reply-To: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
References: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
Message-ID: <5381d5ff-66cb-9662-c293-646d3337762c@redhat.com>

On 01/24/2017 10:21 AM, Heiko W.Rupp wrote:
> Now when WildFly is deployed into Kubernetes or OpenShift, we
> will see that WildFly is started, syncs and then dies at some point
> in time, where k8s will not re-use the existing one, but start
> a new one, which will have a different FeedId.

It might be interesting to re-read the following threads, as some ideas 
were thrown around this subject already:

[Hawkular-dev] OpenShift Pet vs Cattle metaphor (10/13/2016)
[Hawkular-dev] Identification of WildFly in container in a 
Kube/Openshift env (07/03/2016)

- Juca.

From hrupp at redhat.com  Tue Jan 24 06:12:20 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Tue, 24 Jan 2017 12:12:20 +0100
Subject: [Hawkular-dev] Inventory and 'transient' servers
In-Reply-To: <5381d5ff-66cb-9662-c293-646d3337762c@redhat.com>
References: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
	<5381d5ff-66cb-9662-c293-646d3337762c@redhat.com>
Message-ID: <9964B895-711D-412E-8B76-99D2ADB6E533@redhat.com>

On 24 Jan 2017, at 11:13, Juraci Paix?o Kr?hling wrote:

> [Hawkular-dev] OpenShift Pet vs Cattle metaphor (10/13/2016)
> [Hawkular-dev] Identification of WildFly in container in a
> Kube/Openshift env (07/03/2016)

Thanks for the reminder - I had a feeling I wrote about that in the
past :)
And I missed the cited Pet vs Cattle thread even :-(
I think though, that requiring WildFly running in a PetSet for
this to work is a nono

From hrupp at redhat.com  Tue Jan 24 06:01:04 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Tue, 24 Jan 2017 12:01:04 +0100
Subject: [Hawkular-dev] Inventory and 'transient' servers
In-Reply-To: <CAJo5TFngepgAjaY+mNEC9DOzdb9O8S3H3_TQPQnQ7_8kn3W7xw@mail.gmail.com>
References: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
	<CAJo5TFngepgAjaY+mNEC9DOzdb9O8S3H3_TQPQnQ7_8kn3W7xw@mail.gmail.com>
Message-ID: <A960D439-BF2E-49D2-8826-40D927385659@redhat.com>

On 24 Jan 2017, at 10:35, Joel Takvorian wrote:

> Re-using the state would mean there's always 1 or 0 running instance of
> that server at a given time, but what happens if it's scaled up to 2
> instances or more? Is it a possible scenario? If yes, it seems

Yes of course it is possible. And the number of running
servers for a given DC can be scaled up and down "at will".

> complicated
> to use dc+version+tenant as the feed id.

Yes I am aware that this is not enough - was more meant
to be an example.

> I filed an issue in JIRA about introducing some kind of TTL :
> https://issues.jboss.org/browse/HWKINVENT-205
> Maybe those expired feeds fall in the scope of this issue.

Yes, a TTL could be a good solution. If a server re-connects
we reset the "last seen" value. We need to make sure to set the
TTL long enough that a short outage does not expire
servers that are not in containers.

Perhaps we need to also store some information in inventory
if the server is in containers or not (well, we need that
info anyway, see e.g.
https://issues.jboss.org/browse/HAWKULAR-1186 )
For the inventory use case we should perhaps expose
this more prominently than just 'hidden away' in some
property in some json.

From kavinkankeshwar at gmail.com  Tue Jan 24 14:23:41 2017
From: kavinkankeshwar at gmail.com (Kavin Kankeshwar)
Date: Tue, 24 Jan 2017 11:23:41 -0800
Subject: [Hawkular-dev] Hawkular drill down on calls
Message-ID: <CAAUT3g8Ui2tSZ8VqJ6OA2hJK+Sw7Mpg5=jYro7ZiZBMJHE3qPQ@mail.gmail.com>

Hi,

Iam using the Hawkular JVM agent to send metrics to Hawkular controller,
But the one thing which i cannot do is drill down into where the time was
spent. i.e. drill down to the class which was taking time. (something like
AppDynamics Agent)

So wanted to check if i am missing something or the feature is not yet
possible in hawkular.

Regards,
--
Kavin.Kankeshwar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170124/60246e0c/attachment.html 

From gbrown at redhat.com  Wed Jan 25 03:21:32 2017
From: gbrown at redhat.com (Gary Brown)
Date: Wed, 25 Jan 2017 03:21:32 -0500 (EST)
Subject: [Hawkular-dev] Hawkular drill down on calls
In-Reply-To: <CAAUT3g8Ui2tSZ8VqJ6OA2hJK+Sw7Mpg5=jYro7ZiZBMJHE3qPQ@mail.gmail.com>
References: <CAAUT3g8Ui2tSZ8VqJ6OA2hJK+Sw7Mpg5=jYro7ZiZBMJHE3qPQ@mail.gmail.com>
Message-ID: <585867639.19438790.1485332492451.JavaMail.zimbra@redhat.com>

Hi Kavin

Currently the only way to drill down into an instance is via the 'Distributed Tracing' tab. This will initially show the aggregated view of dependencies between service endpoints - you can then apply filters (including time span, properties, etc). Then at the top of the page you will see a button indicating the number of trace instances that were aggregated - pressing this button will show a table of those instances with more individual details (durations, properties) - you can then select them individually to see a trace instance.

This trace instance diagram will show the details of each node/span in the distributed call trace.

We are always looking at ways to improve the UI and how to access the information, so feel free to make suggestions or create a jira.

Regards
Gary

----- Original Message -----
> Hi,
> 
> Iam using the Hawkular JVM agent to send metrics to Hawkular controller, But
> the one thing which i cannot do is drill down into where the time was spent.
> i.e. drill down to the class which was taking time. (something like
> AppDynamics Agent)
> 
> So wanted to check if i am missing something or the feature is not yet
> possible in hawkular.
> 
> Regards,
> --
> Kavin.Kankeshwar
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From jshaughn at redhat.com  Wed Jan 25 08:19:21 2017
From: jshaughn at redhat.com (Jay Shaughnessy)
Date: Wed, 25 Jan 2017 08:19:21 -0500
Subject: [Hawkular-dev] Inventory and 'transient' servers
In-Reply-To: <9964B895-711D-412E-8B76-99D2ADB6E533@redhat.com>
References: <267C4346-30EF-43D5-B612-6112315D4584@redhat.com>
	<5381d5ff-66cb-9662-c293-646d3337762c@redhat.com>
	<9964B895-711D-412E-8B76-99D2ADB6E533@redhat.com>
Message-ID: <3b8b5abb-90ea-e9e7-6952-521d30bfe95b@redhat.com>


Another possible angle is to somehow involve alerting and a possible 
custom action.  We have the ability to do things when detecting down, or 
if we haven't seen an up avail in some period of time, or to respond to 
inventory events.

On 1/24/2017 6:12 AM, Heiko W.Rupp wrote:
> On 24 Jan 2017, at 11:13, Juraci Paix?o Kr?hling wrote:
>
>> [Hawkular-dev] OpenShift Pet vs Cattle metaphor (10/13/2016)
>> [Hawkular-dev] Identification of WildFly in container in a
>> Kube/Openshift env (07/03/2016)
> Thanks for the reminder - I had a feeling I wrote about that in the
> past :)
> And I missed the cited Pet vs Cattle thread even :-(
> I think though, that requiring WildFly running in a PetSet for
> this to work is a nono
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170125/aaf462da/attachment-0001.html 

From mazz at redhat.com  Wed Jan 25 22:13:34 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Wed, 25 Jan 2017 22:13:34 -0500 (EST)
Subject: [Hawkular-dev] hosa status endpoint now secured behind openshift
	secret
In-Reply-To: <2134878377.1191951.1485400039009.JavaMail.zimbra@redhat.com>
Message-ID: <570917481.1192189.1485400414476.JavaMail.zimbra@redhat.com>

If you are deploying HOSA using its makefile and you are using HOSA's status endpoint (heiko :-) you might want to update your blogs on this), just a heads up that the /status endpoint is now secured behind credentials defined in an openshift secret. So if you point your browser to the new route, for example, you'll see it asks you for username/password now.

By default, the status endpoint is disabled, but the yaml our Makefile uses will enable it and put it behind a secret that is created for you. The credentials are fixed in the secret the makefile creates (see the config.yaml example file to know what they are - its the same credentials that are in the secret) but you are free to base64 encode your own credentials in a secret and use that.

From hrupp at redhat.com  Mon Jan 30 05:50:31 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Mon, 30 Jan 2017 11:50:31 +0100
Subject: [Hawkular-dev] [metrics] Internal stats?
Message-ID: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>

Hey,

what internal stats of the Hawkular metrics do we currently collect?
I think Joel did some work for the C* part.

What I think we need is
- number of data points stored on a per tenant basis.
  Resolution could be something like "last minute" or
  "last 5 minutes" I.e. not realtime updates in the table.
- Total number of data points (i.e. sum over all tenants)

- Query stats. This is probably more complicated, as
  querying on metrics that are still in some buffer is
  cheaper than over 3 years of raw data.
  To get started I'd go with # of queries per tenant and global
  Those could perhaps be differentiated on
  - raw endpoint
  - stats endpoint

- What about alerting? More alert definitions certainly
  need more cpu, so number of alert definitions per tenant
 and total would be another pair.
 - does number of fired alerts also make sense?

The idea behind those is to get some usage figures of the
shared resource "Hawkular metrics" and then to be able to
charge them back onto individual tenants e.g. inside of
OpenShift.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170130/e7c55ad8/attachment.html 

From lponce at redhat.com  Mon Jan 30 06:05:07 2017
From: lponce at redhat.com (Lucas Ponce)
Date: Mon, 30 Jan 2017 06:05:07 -0500 (EST)
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
Message-ID: <359316928.2468301.1485774307888.JavaMail.zimbra@redhat.com>

> 
> What about alerting? More alert definitions certainly need more cpu, so
> number of alert definitions per tenant and total would be another pair.
>         * does number of fired alerts also make sense?
> 

Certainly, it could have some sense to have these figures.

These kind of data is "low hanging fruit" in the sense that adding an endpoint to query them is straightforward.

We can add a "/count" suffix to some endpoint to get the # instead the list or a similar.

Is there a preferred way to name the endpoint for this ?


> 
> The idea behind those is to get some usage figures of the
> shared resource "Hawkular metrics" and then to be able to
> charge them back onto individual tenants e.g. inside of
> OpenShift.
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 

From miburman at redhat.com  Mon Jan 30 08:10:03 2017
From: miburman at redhat.com (Michael Burman)
Date: Mon, 30 Jan 2017 15:10:03 +0200
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <359316928.2468301.1485774307888.JavaMail.zimbra@redhat.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
	<359316928.2468301.1485774307888.JavaMail.zimbra@redhat.com>
Message-ID: <e445b8e5-0299-03c4-2f99-e6b0108e17fe@redhat.com>

Hi,

Wouldn't it make sense in Metrics deployment to just send them to 
metrics? Then there's no need to create new endpoints, just fetch them 
as you would any other metric (from the internal admin tenant).

   - Micke


On 01/30/2017 01:05 PM, Lucas Ponce wrote:
>> What about alerting? More alert definitions certainly need more cpu, so
>> number of alert definitions per tenant and total would be another pair.
>>          * does number of fired alerts also make sense?
>>
> Certainly, it could have some sense to have these figures.
>
> These kind of data is "low hanging fruit" in the sense that adding an endpoint to query them is straightforward.
>
> We can add a "/count" suffix to some endpoint to get the # instead the list or a similar.
>
> Is there a preferred way to name the endpoint for this ?
>
>
>> The idea behind those is to get some usage figures of the
>> shared resource "Hawkular metrics" and then to be able to
>> charge them back onto individual tenants e.g. inside of
>> OpenShift.
>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev


From jtakvori at redhat.com  Mon Jan 30 09:15:20 2017
From: jtakvori at redhat.com (Joel Takvorian)
Date: Mon, 30 Jan 2017 15:15:20 +0100
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <e445b8e5-0299-03c4-2f99-e6b0108e17fe@redhat.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
	<359316928.2468301.1485774307888.JavaMail.zimbra@redhat.com>
	<e445b8e5-0299-03c4-2f99-e6b0108e17fe@redhat.com>
Message-ID: <CAJo5TF=2n5zbtT915DjiRf7XaTBKQFa70oDQUHpWAqC9bs6Xsw@mail.gmail.com>

C* metrics are pushed to hawkular-metrics through a dropwizard reporter,
there's no new endpoint as far as I can tell (it was mostly John's work,
actually, not mine).
Of course (and except if I'm missing something) metrics wouldn't be on a
per tenant basis here.


On Mon, Jan 30, 2017 at 2:10 PM, Michael Burman <miburman at redhat.com> wrote:

> Hi,
>
> Wouldn't it make sense in Metrics deployment to just send them to
> metrics? Then there's no need to create new endpoints, just fetch them
> as you would any other metric (from the internal admin tenant).
>
>    - Micke
>
>
> On 01/30/2017 01:05 PM, Lucas Ponce wrote:
> >> What about alerting? More alert definitions certainly need more cpu, so
> >> number of alert definitions per tenant and total would be another pair.
> >>          * does number of fired alerts also make sense?
> >>
> > Certainly, it could have some sense to have these figures.
> >
> > These kind of data is "low hanging fruit" in the sense that adding an
> endpoint to query them is straightforward.
> >
> > We can add a "/count" suffix to some endpoint to get the # instead the
> list or a similar.
> >
> > Is there a preferred way to name the endpoint for this ?
> >
> >
> >> The idea behind those is to get some usage figures of the
> >> shared resource "Hawkular metrics" and then to be able to
> >> charge them back onto individual tenants e.g. inside of
> >> OpenShift.
> >>
> >> _______________________________________________
> >> hawkular-dev mailing list
> >> hawkular-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >>
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170130/dfd67d9b/attachment.html 

From hrupp at redhat.com  Mon Jan 30 09:37:43 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Mon, 30 Jan 2017 15:37:43 +0100
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <CAJo5TF=2n5zbtT915DjiRf7XaTBKQFa70oDQUHpWAqC9bs6Xsw@mail.gmail.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
	<359316928.2468301.1485774307888.JavaMail.zimbra@redhat.com>
	<e445b8e5-0299-03c4-2f99-e6b0108e17fe@redhat.com>
	<CAJo5TF=2n5zbtT915DjiRf7XaTBKQFa70oDQUHpWAqC9bs6Xsw@mail.gmail.com>
Message-ID: <A276E431-19E0-46FC-9FBB-965BE8659889@redhat.com>

On 30 Jan 2017, at 15:15, Joel Takvorian wrote:

> Of course (and except if I'm missing something) metrics wouldn't be on a
> per tenant basis here.

The C* stats are probably more something for the "global"
admin tenant and not for the "user tenants".

For the per-tenant datapoint and query stats, I can imagine that
they are made available on two places
- per tenant in some "reserved" space ( e.g. /_hawkular-stats )
- on the agent tenant /stats/<tenant>/

With the global ones also at the agent tenant below /stats/<tenant>/_global
or similar.

The above locations are only meant for illustration purposes. ;-)

From jsanda at redhat.com  Mon Jan 30 10:12:48 2017
From: jsanda at redhat.com (John Sanda)
Date: Mon, 30 Jan 2017 10:12:48 -0500
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
Message-ID: <13B5CAB4-D3A7-4108-9C64-24A9C4B2C30A@redhat.com>

Not that I think this is a bad idea, but it is the first I have heard about the per tenant metrics. Where is this feature request/requirement coming from? I?d just like some context.
> On Jan 30, 2017, at 5:50 AM, Heiko W.Rupp <hrupp at redhat.com> wrote:
> 
> Hey,
> 
> what internal stats of the Hawkular metrics do we currently collect?
> I think Joel did some work for the C* part.
> 
> What I think we need is 
> - number of data points stored on a per tenant basis.
> Resolution could be something like "last minute" or
> "last 5 minutes" I.e. not realtime updates in the table.
> - Total number of data points (i.e. sum over all tenants)
> 
> Query stats. This is probably more complicated, as 
> querying on metrics that are still in some buffer is
> cheaper than over 3 years of raw data.
> To get started I'd go with # of queries per tenant and global
> Those could perhaps be differentiated on
> 
> raw endpoint
> stats endpoint
> What about alerting? More alert definitions certainly
> need more cpu, so number of alert definitions per tenant
> and total would be another pair.
> 
> does number of fired alerts also make sense?
> The idea behind those is to get some usage figures of the
> shared resource "Hawkular metrics" and then to be able to 
> charge them back onto individual tenants e.g. inside of 
> OpenShift.
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170130/9c811cfb/attachment-0001.html 

From mazz at redhat.com  Mon Jan 30 10:16:12 2017
From: mazz at redhat.com (John Mazzitelli)
Date: Mon, 30 Jan 2017 10:16:12 -0500 (EST)
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <13B5CAB4-D3A7-4108-9C64-24A9C4B2C30A@redhat.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
	<13B5CAB4-D3A7-4108-9C64-24A9C4B2C30A@redhat.com>
Message-ID: <913485585.42135.1485789372732.JavaMail.zimbra@redhat.com>



> Not that I think this is a bad idea, but it is the first I have heard about
> the per tenant metrics. Where is this feature request/requirement coming
> from? I?d just like some context.

Probably from OpenShift. The Hawkular OpenShift Agent (and the heapster stuff, too) stores data per tenant where tenant is the name of the OpenShift project (aka k8s namespace).

so I'm assuming this feature would be required for some kind of "charge back" - it can track how much metric data are stored for each OpenShift project.


From hrupp at redhat.com  Mon Jan 30 10:34:22 2017
From: hrupp at redhat.com (Heiko W.Rupp)
Date: Mon, 30 Jan 2017 16:34:22 +0100
Subject: [Hawkular-dev] [metrics] Internal stats?
In-Reply-To: <913485585.42135.1485789372732.JavaMail.zimbra@redhat.com>
References: <BA6C0106-664F-4951-81BF-F4E104C55286@redhat.com>
	<13B5CAB4-D3A7-4108-9C64-24A9C4B2C30A@redhat.com>
	<913485585.42135.1485789372732.JavaMail.zimbra@redhat.com>
Message-ID: <EBE18A14-36E4-446D-AFB0-A025F7162DAA@redhat.com>

On 30 Jan 2017, at 16:16, John Mazzitelli wrote:

> so I'm assuming this feature would be required for some kind of 
> "charge back" - it can track how much metric data are stored for each 
> OpenShift project.

Exactly. As I wrote:

---snip---
The idea behind those is to get some usage figures of the
shared resource "Hawkular metrics" and then to be able to
charge them back onto individual tenants e.g. inside of
OpenShift.