[hibernate-dev] Pooled Optimiser Improvements

[Steve Ebersole]

Those tests tend to assert the increments.  We seem to agree that this
ThreadLocal one can skip gaps of values.  I'd look there first.


On Tue, Dec 15, 2015 at 11:34 AM Scott Marlow <smarlow at redhat.com> wrote:

> I'm trying to move the optimizer to PooledThreadLocalLoOptimizer [1].
> We are currently failing some new unit tests, which are cloned from
> existing PooledLoOptimizer tests which might be part of the problem.
>
> Scott
>
> [1]
> https://github.com/scottmarlow/hibernate-orm/tree/pooled-optimiser-hack
>
> On 12/14/2015 10:12 PM, Scott Marlow wrote:
> >
> >
> > On 12/11/2015 09:30 AM, Steve Ebersole wrote:
> >> It's hard to say without understanding the scenario where you are seeing
> >> this as a problem.  I have some guesses as to what may be the problem,
> >> but without understanding more about why you see this as a problem in
> >> the first place it is hard to give you an answer.  For example, I wonder
> >> if for environments not using multi-tenancy whether the recent changes
> >> for the generators to support multi-tenancy might be the culprit.  If
> >> that is the case, and those changes are in fact the underlying cause of
> >> the perf issues you see then I think there is actually a better
> >> solution.  But again, its hard to say unless we understand the reason
> >> this "shows up" as a perf problem for you.
> >
> > As best as I can tell from looking at the current PooledLoOptimizer,
> > versus the proposed change (to have a chunk of ids per thread), we went
> > from accessing a contented lock, to instead using per thread memory
> > (eliminating the contended lock on PooledLoOptimizer.generate()).
> >
> >>
> >> Until we hear more I think at this stage I'd vote for a separate
> >> optimizer.  And maybe even not one that is upstream.
> >>
> >> Also I agree with Scott that I am VERY leery of not cleaning up a
> >> ThreadLocal.
> >
> > My mistake, as Stuart pointed out, the TL is not static, so we shouldn't
> > introduce any leaks.
> >
> >>
> >> On Fri, Dec 11, 2015 at 7:55 AM Scott Marlow <smarlow at redhat.com
> >> <mailto:smarlow at redhat.com>> wrote:
> >>
> >>      Should this be a specialized pooled optimizer that is only used in
> >>      environments that do not suffer from leaving the ThreadLocal around
> >>      after the application is undeployed?  In other words, the
> expectation is
> >>      that classloader leaks with this pooled optimizer are expected
> (e.g.
> >>      user must restart the jvm to really undeploy the application
> >>      completely).
> >>
> >>      I am thinking that there are at least three typical situations:
> >>
> >>      1.  Applications are deployed in Java standalone edition.
> Generally,
> >>      when the app undeploys the jvm is shutting down.
> >>
> >>      2. Applications are deployed as part of some container (e.g. an EE
> >>      server) and the Hibernate jars are on the global classloader path
> (or
> >>      something like that).  On each shared container thread, there
> would be
> >>      one Optimizer for all deployed applications.  I wonder if instead,
> we
> >>      would want one Optimizer instance per Hibernate SessionFactory
> >>      associated with the many container threads?
> >>
> >>      3. Applications are deployed as part of some container (e.g. an EE
> >>      server) and the Hibernate jars are deployed with the application.
> The
> >>      ThreadLocals are associated with threads that are shared by
> different
> >>      deployed applications. The application classloader contains the
> >>      Hibernate classes. Each deployed application has its own Optimizer
> >>      threadlocal. On each shared container thread, there would be one
> >>      Optimizer per application (since each application has its
> Optimizer TL).
> >>         Like (2), there would be sharing of the same Optimizer with the
> many
> >>      application session factories.  Should we instead have an
> optimizer per
> >>      session factory?
> >>
> >>      Scott
> >>
> >>      On 12/10/2015 11:31 PM, Stuart Douglas wrote:
> >>       > Hello,
> >>       >
> >>       > I have been working on a change to the pooled optimizer that we
> >>      have been seeing good performance results with. Basically it hands
> >>      out blocks of ID's to a thread local, rather than having every
> >>      thread contend on the lock every time an ID is required.
> >>       >
> >>       >
> >>
> https://github.com/hibernate/hibernate-orm/compare/master...stuartwdouglas:pooled-optimiser-hack
> >>       >
> >>       > What would I need to do to get a change like this in? In
> particular:
> >>       >
> >>       > - Does it need to be a new type of optimizer, or is modifying
> the
> >>      existing one like I have done OK?
> >>       > - How should it be configured?
> >>       >
> >>       > I am happy to do up a PR for this, but I am just not really sure
> >>      what would be required to get it to a point where it would be
> >>      acceptable for inclusion.
> >>       >
> >>       > Stuart
> >>       > _______________________________________________
> >>       > hibernate-dev mailing list
> >>       > hibernate-dev at lists.jboss.org <mailto:
> hibernate-dev at lists.jboss.org>
> >>       > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >>       >
> >>      _______________________________________________
> >>      hibernate-dev mailing list
> >>      hibernate-dev at lists.jboss.org <mailto:
> hibernate-dev at lists.jboss.org>
> >>      https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >>
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
>