[wildfly-dev] Introducing a new thread pool implementation

For WildFly 12 I intend to introduce a new thread pool implementation [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor in the main codebases of WildFly and WildFly Core, and also in several support projects such as JBoss MSC and XNIO. The thread pool has the following desirable characteristics: • Idle threads are always reused before queueing tasks or starting new threads • The most recently used idle thread is always preferred, which allows the pool to shrink when it is not fully utilized • Useful core/max thread pool size distinction • API-equivalent to ThreadPoolExecutor • Performance parity with common ThreadPoolExecutor configurations • Suitable as a replacement for other jboss-threads thread pool types as well In addition, it presents the following features: • Configure via a builder API • Configurable exception handler • Integrated task queue with optional size limit • Full complement of standard metrics, which can be disabled • Configurable rejection handler in the form of a handoff executor • An optional growth-resistance algorithm which can be applied to larger pools to create growth resistance between the core and maximum pool size • An optional post-terminate task • A few system property based adjustments that can be applied for experimental purposes • A global flag which can be set via system property to recommend that the new thread pool should be disabled, allowing an emergency fallback in case of an unexpected problem Design-wise, the thread pool is based on a special lock-free/wait-free combination FIFO/LIFO queue algorithm. See [3] for a more complete explanation of internal operation, if you're curious about the gritty details. This change has a few implications with regards to the application server: • Thread pools with a core- and max-size which were locked together may now have these variables decoupled in a safe and useful manner • Thread pool configurations in the management model which previously featured only a max size can now have a core size attribute introduced Most of the thread pools we use are built and configured within the wildfly-core code base; that is where the first part of this change will be done. Later on there will be a follow-up change for WildFly proper, but this of course can not happen until the core change is merged and a core release done, so expect that part somewhat later. The initial WIP change to wildfly-core can be found here [4]. Note that no pull request will be opened until all the components are Final. I have done a good deal of testing already, but I will be doing more CI testing on other platforms and configurations before submitting the PR. I've made an attempt to link up all the JIRAs that are related to this effort from [1] (either directly or indirectly). If you find another JIRA that you think is related, or if you have questions or concerns, ask on this list, or you can ping me directly on HipChat or IRC. Thanks! [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread pool implementation [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128 [3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807... [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool -- - DML