Re: [Hawkular-dev] OpenShift: Metric table out of sync issue and heapster
Hi,
That check is there because it greatly reduces the load on the
Hawkular-Metrics in an existing environment. Otherwise, after the
restart, the first time the metrics are sent it is going to make a huge
amount of writes to HWKMETRICS and this will take time (the operations
are slower than datapoint writes after all).
I'll make the change next week to allows deciding the behavior with a
command line parameter (in case we fix it in the metrics some other way
also - or this wasn't the real root cause for some reason).
- Micke
On 05/25/2017 01:24 AM, Matt Wringe wrote:
We are still running into issues on machines where the metric tables
getting out of sync bug has not been fixed for it yet
(https://issues.jboss.org/browse/HWKMETRICS-660).
@miburman: would it be possible to update our Heapster sink to always update metrics when
its started? Eg to remove this check
https://github.com/kubernetes/heapster/blob/master/metrics/sinks/hawkular...
I am not sure why we have that check in there in the first place, and I think we should
always just update the tag definition on each start. That way if someone has run into the
problem where the tags and metrics definitions get out of sync, they can just restart
heapster to get things back working again.
Any thoughts?
- Matt