[jboss-as7-dev] Optimal MSC threadpool size?

[Brian Stansberry]

On 12/9/11 3:49 PM, Carlo de Wolf wrote:
> On 12/09/2011 10:34 PM, Brian Stansberry wrote:
>> On 12/9/11 3:03 PM, Dimitris Andreadis wrote:
>>> Brian, I am not sure I understand, especially the reconciliation bit.
>>>
>>> You are saying that whoever is producing the task list to feed to the pool is not breaking
>>> those tasks into pieces that can be parallelized?
>>>
>> In 7.0 the operations that build up the in-memory configuration model,
>> load extensions, call service constructors and and tell the MSC to
>> install the services are all done on a single thread, called the "boot
>> thread". MSC uses a thread pool to parallelize wiring together and
>> starting the services, but a lot of work is done on that single thread.
>>
>> This single threaded boot behavior is the way AS 7.0 always worked. For
>> 7.1 I added logic to execute a lot of that "boot thread" work in
>> parallel. But that logic is disabled if you set
>> org.jboss.server.bootstrap.maxThreads.
>
> Still the Controller Boot Thread is busy reading away the xml while
> other threads are waiting
> for something to do.
>
> Which is why I wrote https://github.com/wolfc/jboss-as/tree/stream-ops a
> long time ago.
> But this ran into stability issues with the ServiceVerificationHandler.
>
> Brian and I already asserted that doing streaming ops now is a pain the ...
> If we really want this we can take another look, but I don't see why.
> It's already #@*%ing fast.
>

If you don't execute the boot ops as a unit you get the 
ServiceVerificationHandler issues, since the SVH sees each individual op 
registering services and barfs it is finds unfilled dependencies. A 
streaming solution has to support executing the ops as a unit.

The current boot divides the operations into 2 big chunks each of which 
is executed like a composite operation, i.e. as a unit. The chunks are 
everything up to the last "extension add" and everything after that. 
(There is parllelization within the execution of those 2 chunks, but 
that's a detail.) This should allow streaming to work -- parsing thread 
streams ops, polling thread builds the first set of ops into a chunk and 
sends it off to an execution thread, and while those execute the polling 
thread builds the second chunk.

Whether that will save much time on a typical developer machine, I don't 
know. My SWAG is 50ms.  So, something to look at after 7.1 is done. I'm 
sure at some point we'll go off on another boot time reduction binge -- 
it's fun work!

Note that in 7.0 a lot of the parsing time was actually classloading 
time -- as each <extension> element is parsed, its module is loaded, the 
Extension service is loaded via ServiceLoader and the Extension is asked 
to register its parser. That is all done in parallel now -- unless 
org.jboss.server.bootstrap.maxThreads is set.

> Carlo
>
>>
>>> In any case, we seem to have many threads sitting there idle.
>>>
>>> I suppose, I need to look at the code in more detail.
>>>
>>> On 09/12/2011 22:02, Brian Stansberry wrote:
>>>> If you set org.jboss.server.bootstrap.maxThreads you disable the boot
>>>> time logic that tries to parallelize running all the operations that
>>>> load extensions and install subsystems subsystem.
>>>>
>>>> Why? Because that parallelization spawns quite a number of threads, and
>>>> the original intent of org.jboss.server.bootstrap.maxThreads was to
>>>> control thread counts. And there's been no time to reconcile the two.
>>>>
>>>> On 12/9/11 11:39 AM, Dimitris Andreadis wrote:
>>>>> I was experimenting a bit with the MSC threadpool size
>>>>> (org.jboss.server.bootstrap.maxThreads) vs boot time and I've put the results here:
>>>>>
>>>>> http://community.jboss.org/wiki/ModularServiceContainer-OptimalThreadpoolSize
>>>>>
>>>>> On my quad-core/dual-threaded machine with 8 reported processors, the default pool size is
>>>>> 16 (processors x 2). In most cases this looks like an overkill to me, i.e. you can get
>>>>> pretty much the same or better results with a thread pool of 4, or even 2.
>>>>>
>>>>> Another conclusion seems to be that not too much stuff gets parallelized under the hood?
>>>>> Even with 1 thread the default config boots pretty much in the same time, while some benefit
>>>>> kicks in as soon as you have some form of I/O in the full and ha configs. But even then,
>>>>> with a thread pool of 2 you achieve most of this gain.
>>>>>
>>>>> Do I miss something? Do you see different results?
>>>>>
>>>>> Just try a single threaded boot:
>>>>> standalone.bat -Dorg.jboss.server.bootstrap.maxThreads=1
>>>>>
>>>>> I am using the latest 7.1.0.CR1-SNAPSHOT
>>>>>
>>>>> /D
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>> jboss-as7-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


-- 
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat