[hibernate-dev] Pooled Optimiser Improvements

[Scott Marlow]

I'm trying to move the optimizer to PooledThreadLocalLoOptimizer [1]. 
We are currently failing some new unit tests, which are cloned from 
existing PooledLoOptimizer tests which might be part of the problem.

Scott

[1] https://github.com/scottmarlow/hibernate-orm/tree/pooled-optimiser-hack

On 12/14/2015 10:12 PM, Scott Marlow wrote:
>
>
> On 12/11/2015 09:30 AM, Steve Ebersole wrote:
>> It's hard to say without understanding the scenario where you are seeing
>> this as a problem.  I have some guesses as to what may be the problem,
>> but without understanding more about why you see this as a problem in
>> the first place it is hard to give you an answer.  For example, I wonder
>> if for environments not using multi-tenancy whether the recent changes
>> for the generators to support multi-tenancy might be the culprit.  If
>> that is the case, and those changes are in fact the underlying cause of
>> the perf issues you see then I think there is actually a better
>> solution.  But again, its hard to say unless we understand the reason
>> this "shows up" as a perf problem for you.
>
> As best as I can tell from looking at the current PooledLoOptimizer,
> versus the proposed change (to have a chunk of ids per thread), we went
> from accessing a contented lock, to instead using per thread memory
> (eliminating the contended lock on PooledLoOptimizer.generate()).
>
>>
>> Until we hear more I think at this stage I'd vote for a separate
>> optimizer.  And maybe even not one that is upstream.
>>
>> Also I agree with Scott that I am VERY leery of not cleaning up a
>> ThreadLocal.
>
> My mistake, as Stuart pointed out, the TL is not static, so we shouldn't
> introduce any leaks.
>
>>
>> On Fri, Dec 11, 2015 at 7:55 AM Scott Marlow <smarlow at redhat.com
>> <mailto:smarlow at redhat.com>> wrote:
>>
>>      Should this be a specialized pooled optimizer that is only used in
>>      environments that do not suffer from leaving the ThreadLocal around
>>      after the application is undeployed?  In other words, the expectation is
>>      that classloader leaks with this pooled optimizer are expected (e.g.
>>      user must restart the jvm to really undeploy the application
>>      completely).
>>
>>      I am thinking that there are at least three typical situations:
>>
>>      1.  Applications are deployed in Java standalone edition.  Generally,
>>      when the app undeploys the jvm is shutting down.
>>
>>      2. Applications are deployed as part of some container (e.g. an EE
>>      server) and the Hibernate jars are on the global classloader path (or
>>      something like that).  On each shared container thread, there would be
>>      one Optimizer for all deployed applications.  I wonder if instead, we
>>      would want one Optimizer instance per Hibernate SessionFactory
>>      associated with the many container threads?
>>
>>      3. Applications are deployed as part of some container (e.g. an EE
>>      server) and the Hibernate jars are deployed with the application. The
>>      ThreadLocals are associated with threads that are shared by different
>>      deployed applications. The application classloader contains the
>>      Hibernate classes. Each deployed application has its own Optimizer
>>      threadlocal. On each shared container thread, there would be one
>>      Optimizer per application (since each application has its Optimizer TL).
>>         Like (2), there would be sharing of the same Optimizer with the many
>>      application session factories.  Should we instead have an optimizer per
>>      session factory?
>>
>>      Scott
>>
>>      On 12/10/2015 11:31 PM, Stuart Douglas wrote:
>>       > Hello,
>>       >
>>       > I have been working on a change to the pooled optimizer that we
>>      have been seeing good performance results with. Basically it hands
>>      out blocks of ID's to a thread local, rather than having every
>>      thread contend on the lock every time an ID is required.
>>       >
>>       >
>>      https://github.com/hibernate/hibernate-orm/compare/master...stuartwdouglas:pooled-optimiser-hack
>>       >
>>       > What would I need to do to get a change like this in? In particular:
>>       >
>>       > - Does it need to be a new type of optimizer, or is modifying the
>>      existing one like I have done OK?
>>       > - How should it be configured?
>>       >
>>       > I am happy to do up a PR for this, but I am just not really sure
>>      what would be required to get it to a point where it would be
>>      acceptable for inclusion.
>>       >
>>       > Stuart
>>       > _______________________________________________
>>       > hibernate-dev mailing list
>>       > hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
>>       > https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>       >
>>      _______________________________________________
>>      hibernate-dev mailing list
>>      hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
>>      https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>