[jopr-dev] reducing purge times

[Gregory Hinkle]

RT data could be a risk, but I'm not sure that as much about the
duration of storage as it is about how many apps its safe to watch.
I could see triming down defaults on that if it's show to help a
typical environment.


I really would not want trait or avail data to default to under 1 year.
These are both RLE and should not have huge amounts of data. Also, 
with our recent enhancement to the current avail attachement I think
a large availability table will be less of a concern to UI performance.
Why are there 4.5M rows in our test infrastructure? Perhaps because
we bounce the heck out of it (unlike a typical real world environment)?
Avail information is important for historical analysis... don't purge 
until necessary. (i'd like to see an alternative to purging it at all)


- Greg

----- "Joseph Marques" <jmarques at redhat.com> wrote:

> I think the system should should function irrespective of what the 
> default purge intervals are and irrespective of how long the database
> 
> and/or all servers have been down (and thus how much backlog there is
> to 
> purge / compress).  it's nice to give users reasonable out-of-the-box
> 
> defaults for this things, but IMO the more pressing concern is really
> 
> http://jira.rhq-project.org/browse/RHQ-1355
> 
> to answer your question specifically, i would, as a user of a sys mgmt
> 
> tool, like to keep data forever (so that i can correlate changes to
> data 
> over long periods of time to understand how my business is growing, 
> estimate future hardware needs, analyze the utilization of the virts
> in 
> my enterprise, etc etc).  2nd to that, i'd like to keep data for as
> long 
> as possible such that the monitoring tool still performs. 
> 
> but without RHQ-1355 in place i think the following are reasonable
> defaults:
> 
> RT data - on the one hand, i could see the benefit of mimicing
> whatever 
> our purge intervals for measurements are (1 yr, i believe?) so that
> the 
> monitor tab has consistency of data - i.e., i don't want to click on
> one 
> tab and see data and click over to another tab and see missing data. 
> RT 
> data is very similar metric data in that way.  however, RT differs
> from 
> it because it's not gracefully compressed over time like measurement 
> data is.  so, i'd probably want the RT purge interval to be equal to 
> however long we keep raw measurement data before compressing it - i 
> think that's 14 days?
> 
> Trait data - we could easily extend this to 1 year.  traits change
> very 
> infrequently, and many resources don't have any traits.  so, we can 
> easily bump this way up without impacting the system.
> 
> Availability data - since this is run-length encoded, we *should* be 
> able to keep this data for a long time.  i wouldn't think that we need
> 
> to set this to anything smaller than 31 days in order to keep the
> system 
> performing. 
> 
> John Mazzitelli wrote:
> > [I've been neglecting to send my dev comments to this list, will try
> 
> > to remember to use this mailing list]
> >
> > Can I ask people to review this:
> >
> > http://jira.rhq-project.org/browse/RHQ-1119
> > <mailto:jopr-dev at lists.jboss.org>
> > I think we need to reduce our purge times.
> > _______________________________________________
> > jopr-dev mailing list
> > jopr-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/jopr-dev
> 
> _______________________________________________
> jopr-dev mailing list
> jopr-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jopr-dev