[JBoss JIRA] (REMJMX-166) IllegalThreadStateException after idle jmx connection
by Brad Maxwell (Jira)
[ https://issues.redhat.com/browse/REMJMX-166?page=com.atlassian.jira.plugi... ]
Brad Maxwell reassigned REMJMX-166:
-----------------------------------
Assignee: Brad Maxwell (was: Darran Lofthouse)
> IllegalThreadStateException after idle jmx connection
> -----------------------------------------------------
>
> Key: REMJMX-166
> URL: https://issues.redhat.com/browse/REMJMX-166
> Project: Remoting JMX
> Issue Type: Bug
> Components: Connection
> Affects Versions: 3.0.2.Final, 3.0.3.Final
> Environment: org.jboss.remotingjmx:remoting-jmx:3.0.3.Final + java8
> Reporter: Märt Bakhoff
> Assignee: Brad Maxwell
> Priority: Blocker
> Attachments: REMJMX-160.jar, REMJMX-166.jar
>
>
> Start wildfly-17.0.1/bin/standalone.sh, then run this code snippet:
> {noformat}
> JMXServiceURL url = new JMXServiceURL("service:jmx:remote+http://127.0.0.1:9990");
> try (JMXConnector connector = new RemotingConnectorProvider().newJMXConnector(url, Collections.emptyMap())) {
> connector.connect();
> MBeanServerConnection beanServer = connector.getMBeanServerConnection();
> RuntimeMXBean bean = ManagementFactory.newPlatformMXBeanProxy(beanServer, ManagementFactory.RUNTIME_MXBEAN_NAME, RuntimeMXBean.class);
> Thread.sleep(70_000);
> System.out.println("uptime: " + bean.getUptime());
> }
> {noformat}
> The following exception is always thrown:
> {noformat}
> Exception in thread "XNIO-1 task-12" java.lang.IllegalThreadStateException
> at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867)
> at java.lang.Thread.init(Thread.java:405)
> at java.lang.Thread.init(Thread.java:349)
> at java.lang.Thread.<init>(Thread.java:599)
> at org.jboss.remotingjmx.protocol.v2.ClientExecutorManager$1.newThread(ClientExecutorManager.java:56)
> at java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:619)
> at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
> at org.jboss.remotingjmx.protocol.v2.ClientExecutorManager.execute(ClientExecutorManager.java:64)
> at org.jboss.remotingjmx.protocol.v2.ClientCommon$MessageReceiver.handleMessage(ClientCommon.java:118)
> at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430)
> at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:926)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The cause is in org.jboss.remotingjmx.protocol.v2.ClientExecutorManager.<init>. It creates a thread pool with Executors.newCachedThreadPool that has the default keepAliveTime of 60s.
> The thread factory is using a daemon thread group REMOTING_JMX that will self-destruct when the cached thread is terminated.
> The same code works when using older org.jboss.remotingjmx:remoting-jmx:3.0.1.Final. The regression is likely caused by commit https://github.com/jbossas/remoting-jmx/commit/2d6ae6c26da43304b752fc48f1...
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Affects Version/s: 13.0.0.Final
(was: 10.1.0.Final)
> ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
> ---------------------------------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 13.0.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Description:
On WF-13 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
was:
On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
> ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
> ---------------------------------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 13.0.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-13 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Steps to Reproduce:
* Configure a ManagedExecutorService with a limited pool and queuesize
* Let an application fill out the pool and queue with threads -> see the number of active requests stated by the RequestController is as expected
* Submit/Execute one more thread to get a RejectedExecutionException as expected -> see the number of active request is erronous incremented by each of such event.
was:
* Configure a ManagedExecutorService with a limited pool and queuesize
* Let an application fill out the pool and queue with threads -> see the number of active requests stated by the RequestController is as expected
* Submit/Execute one more thread to get a RejectedExecutionException as expected -> see the number of active request is erronously incremented by each of such event.
> ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
> ---------------------------------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 10.1.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Steps to Reproduce:
* Configure a ManagedExecutorService with a limited pool and queuesize
* Let an application fill out the pool and queue with threads -> see the number of active requests stated by the RequestController is as expected
* Submit/Execute one more thread to get a RejectedExecutionException as expected -> see the number of active request is erroneously incremented by each of such event.
was:
* Configure a ManagedExecutorService with a limited pool and queuesize
* Let an application fill out the pool and queue with threads -> see the number of active requests stated by the RequestController is as expected
* Submit/Execute one more thread to get a RejectedExecutionException as expected -> see the number of active request is erronous incremented by each of such event.
> ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
> ---------------------------------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 10.1.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Steps to Reproduce:
* Configure a ManagedExecutorService with a limited pool and queuesize
* Let an application fill out the pool and queue with threads -> see the number of active requests stated by the RequestController is as expected
* Submit/Execute one more thread to get a RejectedExecutionException as expected -> see the number of active request is erronously incremented by each of such event.
was:
* Configure a ManagedExecutorService with a limited pool and queuesize
* Let an application fill out the pool and queue with threads -> see the number of active requests stated by the RequestController is as expeted
* Submit/Execute one more thread to get a RejectedExecutionException as expected -> see the number of active request is errounously incremended by each of such event.
> ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
> ---------------------------------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 10.1.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Summary: ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException (was: ManagedExecutorService: Wrong RequestCount on RejectedExecutionException)
> ManagedExecutorService: Wrong activeRequestCount at RequestController on RejectedExecutionException
> ---------------------------------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 10.1.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (WFLY-13177) ManagedExecutorService: Wrong RequestCount on RejectedExecutionException
by Guido Jäkel (Jira)
[ https://issues.redhat.com/browse/WFLY-13177?page=com.atlassian.jira.plugi... ]
Guido Jäkel updated WFLY-13177:
-------------------------------
Summary: ManagedExecutorService: Wrong RequestCount on RejectedExecutionException (was: ManagedExecutorService: Wrong RequestCount on Exception )
> ManagedExecutorService: Wrong RequestCount on RejectedExecutionException
> ------------------------------------------------------------------------
>
> Key: WFLY-13177
> URL: https://issues.redhat.com/browse/WFLY-13177
> Project: WildFly
> Issue Type: Bug
> Components: EE, Server
> Affects Versions: 10.1.0.Final, 16.0.0.Final
> Reporter: Guido Jäkel
> Assignee: Brian Stansberry
> Priority: Major
>
> On WF-10 and WF-16 we observe a serious bug of the RequestCount of the RequestController while using ManagedExecutorService.submit() or ...execute() in the edge case of a full queue. In this case, the caller gets a RejectedExecutionException, but in the RequestController, the number of active requests is erroneously incremented.
> This will lead to a false and monotonously increasing number of active requests. And in case of a limitation configured by the maxRequestCount feature, which is best practice for production environments, over the time this will lead to deadlock of the RequestController and herewith the complete activity of the Wildfly at all.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months