For WildFly 12 I intend to introduce a new thread pool implementation [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor in the main codebases of WildFly and WildFly Core, and also in several support projects such as JBoss MSC and XNIO. The thread pool has the following desirable characteristics: • Idle threads are always reused before queueing tasks or starting new threads • The most recently used idle thread is always preferred, which allows the pool to shrink when it is not fully utilized • Useful core/max thread pool size distinction • API-equivalent to ThreadPoolExecutor • Performance parity with common ThreadPoolExecutor configurations • Suitable as a replacement for other jboss-threads thread pool types as well In addition, it presents the following features: • Configure via a builder API • Configurable exception handler • Integrated task queue with optional size limit • Full complement of standard metrics, which can be disabled • Configurable rejection handler in the form of a handoff executor • An optional growth-resistance algorithm which can be applied to larger pools to create growth resistance between the core and maximum pool size • An optional post-terminate task • A few system property based adjustments that can be applied for experimental purposes • A global flag which can be set via system property to recommend that the new thread pool should be disabled, allowing an emergency fallback in case of an unexpected problem Design-wise, the thread pool is based on a special lock-free/wait-free combination FIFO/LIFO queue algorithm. See [3] for a more complete explanation of internal operation, if you're curious about the gritty details. This change has a few implications with regards to the application server: • Thread pools with a core- and max-size which were locked together may now have these variables decoupled in a safe and useful manner • Thread pool configurations in the management model which previously featured only a max size can now have a core size attribute introduced Most of the thread pools we use are built and configured within the wildfly-core code base; that is where the first part of this change will be done. Later on there will be a follow-up change for WildFly proper, but this of course can not happen until the core change is merged and a core release done, so expect that part somewhat later. The initial WIP change to wildfly-core can be found here [4]. Note that no pull request will be opened until all the components are Final. I have done a good deal of testing already, but I will be doing more CI testing on other platforms and configurations before submitting the PR. I've made an attempt to link up all the JIRAs that are related to this effort from [1] (either directly or indirectly). If you find another JIRA that you think is related, or if you have questions or concerns, ask on this list, or you can ping me directly on HipChat or IRC. Thanks! [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread pool implementation [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128 [3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff- f5807a689506af0b70791d93ae81cc1cR67 [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool -- - DML _______________________________________________ wildfly-dev mailing list wildfly-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/wildfly-dev

The proposed change can be found here: https://github.com/dmlloyd/wildfly-core/tree/threadpool You can build that wildfly-core and then build a wildfly which uses the produced build. The whole process with -DskipTests should take under 10 minutes. I would be very happy to know what the performance team finds, particularly in high-contention scenarios. Thanks! On Fri, Nov 3, 2017 at 10:17 AM, Andrig Miller <anmiller(a)redhat.com&gt; wrote: > I think it would be wise for the performance team to test this out at scale > using our lab, as I'm sure you would agree. > > If you could point us at a particular build, or point us at what to build, > we can do that testing in parallel and give feedback on how this is working. > > Thanks. > > Andy > > On Fri, Nov 3, 2017 at 8:03 AM, David Lloyd <david.lloyd(a)redhat.com&gt; wrote: >> >> For WildFly 12 I intend to introduce a new thread pool implementation >> [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor >> in the main codebases of WildFly and WildFly Core, and also in several >> support projects such as JBoss MSC and XNIO. >> >> The thread pool has the following desirable characteristics: >> >> • Idle threads are always reused before queueing tasks or starting new >> threads >> • The most recently used idle thread is always preferred, which allows >> the pool to shrink when it is not fully utilized >> • Useful core/max thread pool size distinction >> • API-equivalent to ThreadPoolExecutor >> • Performance parity with common ThreadPoolExecutor configurations >> • Suitable as a replacement for other jboss-threads thread pool types as >> well >> >> In addition, it presents the following features: >> >> • Configure via a builder API >> • Configurable exception handler >> • Integrated task queue with optional size limit >> • Full complement of standard metrics, which can be disabled >> • Configurable rejection handler in the form of a handoff executor >> • An optional growth-resistance algorithm which can be applied to >> larger pools to create growth resistance between the core and maximum >> pool size >> • An optional post-terminate task >> • A few system property based adjustments that can be applied for >> experimental purposes >> • A global flag which can be set via system property to recommend that >> the new thread pool should be disabled, allowing an emergency fallback >> in case of an unexpected problem >> >> Design-wise, the thread pool is based on a special lock-free/wait-free >> combination FIFO/LIFO queue algorithm. See [3] for a more complete >> explanation of internal operation, if you're curious about the gritty >> details. >> >> This change has a few implications with regards to the application server: >> • Thread pools with a core- and max-size which were locked together >> may now have these variables decoupled in a safe and useful manner >> • Thread pool configurations in the management model which previously >> featured only a max size can now have a core size attribute introduced >> >> Most of the thread pools we use are built and configured within the >> wildfly-core code base; that is where the first part of this change >> will be done. Later on there will be a follow-up change for WildFly >> proper, but this of course can not happen until the core change is >> merged and a core release done, so expect that part somewhat later. >> The initial WIP change to wildfly-core can be found here [4]. Note >> that no pull request will be opened until all the components are >> Final. I have done a good deal of testing already, but I will be >> doing more CI testing on other platforms and configurations before >> submitting the PR. >> >> I've made an attempt to link up all the JIRAs that are related to this >> effort from [1] (either directly or indirectly). If you find another >> JIRA that you think is related, or if you have questions or concerns, >> ask on this list, or you can ping me directly on HipChat or IRC. >> >> Thanks! >> >> [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread >> pool implementation >> [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128 >> [3] >> https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff- f5807a689506af0b70791d93ae81cc1cR67 >> [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool >> >> -- >> - DML >> >> _______________________________________________ >> wildfly-dev mailing list >> wildfly-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/wildfly-dev > > > > > -- > Andrig (Andy) T. Miller > Global Platform Director, Middleware > Red Hat, Inc. -- - DML

On Nov 3, 2017, at 12:08 PM, Dimitris Andreadis <dandread(a)redhat.com&gt; wrote: Every time you re-design a new thread pool, a kitten dies. /me ducks :) > On 03/11/2017 15:03, David Lloyd wrote: > For WildFly 12 I intend to introduce a new thread pool implementation > [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor > in the main codebases of WildFly and WildFly Core, and also in several > support projects such as JBoss MSC and XNIO. > > The thread pool has the following desirable characteristics: > > • Idle threads are always reused before queueing tasks or starting new threads > • The most recently used idle thread is always preferred, which allows > the pool to shrink when it is not fully utilized > • Useful core/max thread pool size distinction > • API-equivalent to ThreadPoolExecutor > • Performance parity with common ThreadPoolExecutor configurations > • Suitable as a replacement for other jboss-threads thread pool types as well > > In addition, it presents the following features: > > • Configure via a builder API > • Configurable exception handler > • Integrated task queue with optional size limit > • Full complement of standard metrics, which can be disabled > • Configurable rejection handler in the form of a handoff executor > • An optional growth-resistance algorithm which can be applied to > larger pools to create growth resistance between the core and maximum > pool size > • An optional post-terminate task > • A few system property based adjustments that can be applied for > experimental purposes > • A global flag which can be set via system property to recommend that > the new thread pool should be disabled, allowing an emergency fallback > in case of an unexpected problem > > Design-wise, the thread pool is based on a special lock-free/wait-free > combination FIFO/LIFO queue algorithm. See [3] for a more complete > explanation of internal operation, if you're curious about the gritty > details. > > This change has a few implications with regards to the application server: > • Thread pools with a core- and max-size which were locked together > may now have these variables decoupled in a safe and useful manner > • Thread pool configurations in the management model which previously > featured only a max size can now have a core size attribute introduced > > Most of the thread pools we use are built and configured within the > wildfly-core code base; that is where the first part of this change > will be done. Later on there will be a follow-up change for WildFly > proper, but this of course can not happen until the core change is > merged and a core release done, so expect that part somewhat later. > The initial WIP change to wildfly-core can be found here [4]. Note > that no pull request will be opened until all the components are > Final. I have done a good deal of testing already, but I will be > doing more CI testing on other platforms and configurations before > submitting the PR. > > I've made an attempt to link up all the JIRAs that are related to this > effort from [1] (either directly or indirectly). If you find another > JIRA that you think is related, or if you have questions or concerns, > ask on this list, or you can ping me directly on HipChat or IRC. > > Thanks! > > [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread > pool implementation > [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128 > [3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807... > [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool > _______________________________________________ wildfly-dev mailing list wildfly-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/wildfly-dev