[Hawkular-dev] Garbage collection of outdated Servers/Metrics - especially with (orchestrated) containers

[Larry O'Leary]

If I understand how tags are getting used, I think tags will provide the
ability to keep track of history regardless of whether we are dealing with
a new container or an application which is no longer alive.

However, what is not clear to me is what happens when the old container
goes away. I am assuming that we will no longer be attempting to probe it
or gather data from it?

Ultimately what I, as a user, want to be able to do is change network
configuration or platform or container without changing what defines my
application. For example, if I have a helloworld.war deployed to an EAP
server and am monitoring it, I really don't care about the other stuff. If
it changes, so be it. But my helloworld application should still be
reflected the same.

If the database my helloworld application uses changes, so be it. I still
want to treat the datasource as the same datasource it always was even if
the underlying implementation changes and a new inventory object to
represent it gets created. My expectation is that _helloworld-datasource_
is _datasource-pg94_ and _datasource-pg95_ and _datasource_mysql_ combined.
Even though in reality I started with PostgreSQL 9.4; upgraded to
PostgreSQL 9.5; and decided to migrate to MySQL at some point. In the end,
I want to view the datasource metrics based on the entire life-cycle as
that datasource relates to my helloworld application.

The same could be said for my helloworld application's underlying
application server. Perhaps I have my WAR deployed to EAP 7.1 to start but
later I upgrade to EAP 8. It is still my helloworld application.

In the past I would suggest providing a "linking" capability that would
allow me to identify the EAP 8 instance as a continuation or upgrade of the
specific EAP 7.1 instance. Or the MySQL datasource as a continuation or
upgrade of the PostgreSQL 9.5 datasource and the 9.5 datasource an upgrade
or continuation of the PostgreSQL 9.4 datasource. However, if I understand
how tagging is getting implemented, tags would also accomplish this.

If I have separate objects in inventory, they would just get tagged as
helloworld application. When I view history, I am seeing the milestones
across the various inventory objects that make up what I am referring to as
helloworld application. For example, if I had to spin up a new container
every day for the last 7 days, then perhaps it would be interesting to see
that indicated in the history, but the metric values are considered to be a
single series as they relate to the all 7 containers.

But to take the burden of the user to tag each new container that
represents helloworld, I would hope that we can figure out how to do this
automatically and then allow the user to update/fix tags if they are wrong?

In any event, just my thoughts as this does relate to a very big pain point
with the legacy RHQ and JBoss ON implementation.

--
Larry O'Leary


On Fri, Mar 10, 2017 at 6:35 AM, Michael Burman <miburman at redhat.com> wrote:

> Hi,
>
> This is actually a solved issue if you want to track a single metric's
> history. Remember, metricId has no context information, it could (and it
> should have been so that people don't get confused) a randomly generated
> uuid.
>
> The way you track the container's historic CPU usage is by using the tags:
>
> For example, to get all the datapoints for a single container, you use:
>
> container_name=hawkular-metrics
>
> The podid does not make any difference here, it's just some random
> information. Now, on the other hand, if you want to  compare how the
> hawkular-metrics memory usage has changed between different pod versions
> or pod ids, you can do that also by actually using the tags of pod_id
> and container_version.
>
> Just use pod_id as one sorting method and container_version as one (I
> don't remember the exact names of these tags). Tags are the key to
> sorting and finding the information you want - not the metricId.
> MetricId is something you should completely forget, it's not relevant.
>
> The Kubernetes metrics have tags available that you can use for sorting,
> grouping and finding.
>
>    -  Micke
>
> On 03/09/2017 12:35 PM, Lukas Krejci wrote:
> > Isn't the question rather "how do we associate them with the
> > application(s)"?
> > Because if we want to track e.g. CPU load generated by an application in
> a
> > container - isn't that something users would want to look at history of?
> IMHO,
> > using the ephemeral "container id" as (part of) metric ids is a wrong
> thing to
> > do, because really the user isn't interested in the container itself,
> but the
> > applications that are running in it and their consumption of container's
> > resources.
> >
> >
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20170314/6cdab72f/attachment-0001.html