[wildfly-dev] Pooling EJB Session Beans per default

[Andrig Miller]

----- Original Message -----
> From: "Jason Greene" <jason.greene at redhat.com>
> To: "Andrig Miller" <anmiller at redhat.com>
> Cc: "Bill Burke" <bburke at redhat.com>, wildfly-dev at lists.jboss.org
> Sent: Wednesday, August 6, 2014 11:48:53 AM
> Subject: Re: [wildfly-dev] Pooling EJB Session Beans per default
> 
> 
> On Aug 6, 2014, at 12:21 PM, Andrig Miller <anmiller at redhat.com>
> wrote:
> 
> > 
> > 
> > ----- Original Message -----
> >> From: "Jason Greene" <jason.greene at redhat.com>
> >> To: "Andrig Miller" <anmiller at redhat.com>
> >> Cc: "Bill Burke" <bburke at redhat.com>, wildfly-dev at lists.jboss.org
> >> Sent: Wednesday, August 6, 2014 11:08:02 AM
> >> Subject: Re: [wildfly-dev] Pooling EJB Session Beans per default
> >> 
> >> 
> >> On Aug 6, 2014, at 10:49 AM, Andrig Miller <anmiller at redhat.com>
> >> wrote:
> >> 
> >>> 
> >>> 
> >>> ----- Original Message -----
> >>>> From: "Bill Burke" <bburke at redhat.com>
> >>>> To: wildfly-dev at lists.jboss.org
> >>>> Sent: Wednesday, August 6, 2014 9:30:06 AM
> >>>> Subject: Re: [wildfly-dev] Pooling EJB Session Beans per default
> >>>> 
> >>> 
> 
> 
> >>> This conversation is a perfect example of misinformation that
> >>> causes us performance and scalability problems within our code
> >>> bases.
> >> 
> >> It’s just a surprising result. The pool saves a few allocations,
> >> but
> >> it also has the cost of concurrency usage which can trigger
> >> blocking, additional barriers, and busy looping on CAS. You also
> >> still have object churn in the underlying pool data structures
> >> that
> >> occurs per invocation since every invocation is a check-out and a
> >> check-in (requires a new node object instance), and if the
> >> semaphore
> >> blocks you have additional allocation for the entry in the wait
> >> queue. You factor in the remaining allocation savings relative to
> >> other allocations that are required for the invocation, and it
> >> should be a very small percentage. For that very small percentage
> >> to
> >> lead to several times a difference in performance to me hints at
> >> other factors being involved.
> >> 
> > 
> > All logically thought through.  At a 15% lower transaction rate
> > than we are doing now, we saw 4 Gigabytes per second of object
> > allocation.  We, with Sanne doing most of the work, managed to get
> > that down to 3 Gigabytes per second (I would have loved to get it
> > to 2).  Much of that was Hibernate allocations, and of course that
> > was with pooling on.  We have not spent the time to pinpoint the
> > exact differences, memory and other, between having pooling on vs.
> > off.  Our priority has been continue to scale the workload and fix
> > any problems we see as a result.  We have managed to increase the
> > transaction rate another 15% in the last couple of months, but
> > still have another 17+% to go on a single JVM before we start
> > looking at two JVM's for the testing.
> > 
> > Once we get to our goal, I would love to put this on our list of
> > tasks, so we can get the specific facts, and instead of talking
> > theory, we will no exactly what can and cannot be done, and
> > whether no pooling could ever match pooled.
> 
> Fair enough, and I certainly didn’t mean to imply that such work
> should be your team, I was just speaking generally. In any case,
> what I really really would like for us to achieve is a default
> implementation that performs generally well on all usage patterns,
> with no tuning required. Since we know that initialization can be
> costly for some applications usage of SLSB, such an implementation
> will definitely require a form of pooling.
> 

No problem, and I think we should do this, we just cannot afford to spend the time right now.

We share the same goal.  I too would love to have a default configuration, that didn't require tuning that performs and scales really well.

> I suspect that a thread local based design with the pooling tied to
> worker threads will give us this. Alternatively a shared pool which
> is auto-tuned to match might be worth looking into.
> 
> If there is anyone lurking who wishes to contribute in this area
> speak up, and I’ll worth with you on it. As doge would say “Such
> Fun. Much Glory” :)
> 
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
> 
>