----- Original Message -----
From: "Jason Greene" <jason.greene(a)redhat.com>
To: "Andrig Miller" <anmiller(a)redhat.com>
Cc: "Bill Burke" <bburke(a)redhat.com>, wildfly-dev(a)lists.jboss.org
Sent: Wednesday, August 6, 2014 11:48:53 AM
Subject: Re: [wildfly-dev] Pooling EJB Session Beans per default
On Aug 6, 2014, at 12:21 PM, Andrig Miller <anmiller(a)redhat.com>
wrote:
>
>
> ----- Original Message -----
>> From: "Jason Greene" <jason.greene(a)redhat.com>
>> To: "Andrig Miller" <anmiller(a)redhat.com>
>> Cc: "Bill Burke" <bburke(a)redhat.com>,
wildfly-dev(a)lists.jboss.org
>> Sent: Wednesday, August 6, 2014 11:08:02 AM
>> Subject: Re: [wildfly-dev] Pooling EJB Session Beans per default
>>
>>
>> On Aug 6, 2014, at 10:49 AM, Andrig Miller <anmiller(a)redhat.com>
>> wrote:
>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Bill Burke" <bburke(a)redhat.com>
>>>> To: wildfly-dev(a)lists.jboss.org
>>>> Sent: Wednesday, August 6, 2014 9:30:06 AM
>>>> Subject: Re: [wildfly-dev] Pooling EJB Session Beans per default
>>>>
>>>
>>> This conversation is a perfect example of misinformation that
>>> causes us performance and scalability problems within our code
>>> bases.
>>
>> It’s just a surprising result. The pool saves a few allocations,
>> but
>> it also has the cost of concurrency usage which can trigger
>> blocking, additional barriers, and busy looping on CAS. You also
>> still have object churn in the underlying pool data structures
>> that
>> occurs per invocation since every invocation is a check-out and a
>> check-in (requires a new node object instance), and if the
>> semaphore
>> blocks you have additional allocation for the entry in the wait
>> queue. You factor in the remaining allocation savings relative to
>> other allocations that are required for the invocation, and it
>> should be a very small percentage. For that very small percentage
>> to
>> lead to several times a difference in performance to me hints at
>> other factors being involved.
>>
>
> All logically thought through. At a 15% lower transaction rate
> than we are doing now, we saw 4 Gigabytes per second of object
> allocation. We, with Sanne doing most of the work, managed to get
> that down to 3 Gigabytes per second (I would have loved to get it
> to 2). Much of that was Hibernate allocations, and of course that
> was with pooling on. We have not spent the time to pinpoint the
> exact differences, memory and other, between having pooling on vs.
> off. Our priority has been continue to scale the workload and fix
> any problems we see as a result. We have managed to increase the
> transaction rate another 15% in the last couple of months, but
> still have another 17+% to go on a single JVM before we start
> looking at two JVM's for the testing.
>
> Once we get to our goal, I would love to put this on our list of
> tasks, so we can get the specific facts, and instead of talking
> theory, we will no exactly what can and cannot be done, and
> whether no pooling could ever match pooled.
Fair enough, and I certainly didn’t mean to imply that such work
should be your team, I was just speaking generally. In any case,
what I really really would like for us to achieve is a default
implementation that performs generally well on all usage patterns,
with no tuning required. Since we know that initialization can be
costly for some applications usage of SLSB, such an implementation
will definitely require a form of pooling.
No problem, and I think we should do this, we just cannot afford to spend the time right
now.
We share the same goal. I too would love to have a default configuration, that didn't
require tuning that performs and scales really well.
I suspect that a thread local based design with the pooling tied to
worker threads will give us this. Alternatively a shared pool which
is auto-tuned to match might be worth looking into.
If there is anyone lurking who wishes to contribute in this area
speak up, and I’ll worth with you on it. As doge would say “Such
Fun. Much Glory” :)
--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat