[jboss-jira] [JBoss JIRA] (WFCORE-364) Hangs in mixed domain testsuite

[Brian Stansberry (JIRA)]

     [ https://issues.jboss.org/browse/WFCORE-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brian Stansberry resolved WFCORE-364.
-------------------------------------
    Fix Version/s:     (was: 2.0.0.CR1)
       Resolution: Out of Date


I haven't heard any cases of this for a long time.

> Hangs in mixed domain testsuite
> -------------------------------
>
>                 Key: WFCORE-364
>                 URL: https://issues.jboss.org/browse/WFCORE-364
>             Project: WildFly Core
>          Issue Type: Bug
>          Components: Domain Management
>            Reporter: Brian Stansberry
>            Assignee: Brian Stansberry
>         Attachments: testsuite-hang1-server.txt, testsuite-hang2-client.txt, testsuite-hang2-server.txt
>
>
> http://lightning.mw.lab.eng.bos.redhat.com/jenkins/job/wildfly-param-pull/7449/console
> https://dl.dropboxusercontent.com/u/712508/mixed-hang-debug.zip contains details.
> 30805.dump is the client and shows the problem is occurring in the @After cleanup of MixedDomainDeploymentTest as it removes deployments from the domain.
> "main" prio=10 tid=0xf7605c00 nid=0x7856 in Object.wait() [0xf776e000]
>    java.lang.Thread.State: WAITING (on object monitor)
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0xe50275f0> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> 	at java.lang.Object.wait(Object.java:503)
> 	at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
> 	- locked <0xe50275f0> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> 	at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
> 	- locked <0xe50275f0> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> 	at org.jboss.as.controller.client.impl.AbstractDelegatingAsyncFuture.get(AbstractDelegatingAsyncFuture.java:100)
> 	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:127)
> 	at org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:71)
> 	at org.jboss.as.controller.client.helpers.domain.impl.DomainClientImpl.execute(DomainClientImpl.java:81)
> 	at org.jboss.as.test.integration.domain.mixed.MixedDomainDeploymentTest.removeDeployment(MixedDomainDeploymentTest.java:441)
> 	at org.jboss.as.test.integration.domain.mixed.MixedDomainDeploymentTest.cleanDeployments(MixedDomainDeploymentTest.java:343)
> 	at org.jboss.as.test.integration.domain.mixed.MixedDomainDeploymentTest.confirmNoDeployments(MixedDomainDeploymentTest.java:163)
> 31894.dump shows the master HC. management-handler-thread is blocking on the way out (after sending the commit or rollback) waiting for a final result response from the slave.
> "management-handler-thread - 1" prio=10 tid=0xca0efc00 nid=0x7cdd in Object.wait() [0xc7469000]
>    java.lang.Thread.State: WAITING (on object monitor)
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0xeb6d6558> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> 	at java.lang.Object.wait(Object.java:503)
> 	at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
> 	- locked <0xeb6d6558> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> 	at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
> 	- locked <0xeb6d6558> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> 	at org.jboss.as.controller.client.impl.AbstractDelegatingAsyncFuture.get(AbstractDelegatingAsyncFuture.java:100)
> 	at org.jboss.as.domain.controller.operations.coordination.DomainSlaveHandler.finalizeOp(DomainSlaveHandler.java:215)
> 	at org.jboss.as.domain.controller.operations.coordination.DomainSlaveHandler.access$000(DomainSlaveHandler.java:58)
> 	at org.jboss.as.domain.controller.operations.coordination.DomainSlaveHandler$1.handleResult(DomainSlaveHandler.java:179)
> 	at org.jboss.as.controller.AbstractOperationContext$Step.handleRollback(AbstractOperationContext.java:805)
> 	at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:763)
> 	at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:738)
> (Note that the "handleRollback" method name is a red-herring -- the method should be renamed to a "handleResult" as it now is called no matter what the result.)
> 31986.dump is the slave HC. It shows it is blocking in stage DONE after having sent operationPrepared to the master, waiting for commit or rollback.
> "domain-connection-threads - 1" prio=10 tid=0xc91a8400 nid=0x7d1a waiting on condition [0xc87fe000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0xea1fd410> (a java.util.concurrent.CountDownLatch$Sync)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> 	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> 	at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ProxyOperationControlProxy.operationPrepared(TransactionalProtocolOperationHandler.java:173)
> 	at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:337)
> 	at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:211)
> 	at org.jboss.as.domain.controller.operations.coordination.ServerOperationsResolverHandler.execute(ServerOperationsResolverHandler.java:128)
> Indication is the commit or rollback message is getting lost. There are no threads shown sending or receiving it.



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)