[jboss-jira] [JBoss JIRA] (WFLY-11706) Deadlock in cleanup following DatabaseCertLoginModuleTestCase

[Brian Stansberry (Jira)]

Brian Stansberry created WFLY-11706:
---------------------------------------

             Summary: Deadlock in cleanup following DatabaseCertLoginModuleTestCase
                 Key: WFLY-11706
                 URL: https://issues.jboss.org/browse/WFLY-11706
             Project: WildFly
          Issue Type: Bug
          Components: Test Suite, Transactions
            Reporter: Brian Stansberry
            Assignee: Tom Jenkinson


A test hung due to a server-side deadlock.

https://ci.wildfly.org/viewLog.html?buildId=139407&buildTypeId=WFPR&tab=buildLog&branch_WF_PullRequest=pull%2F12062

Build log output stopped at this point:

[18:27:07][Step 2/3] [INFO] Running org.jboss.as.test.manualmode.security.OutboundLdapConnectionTestCase
[18:27:42][Step 2/3] [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.744 s - in org.jboss.as.test.manualmode.security.OutboundLdapConnectionTestCase
[18:27:43][Step 2/3] [INFO] Running org.jboss.as.test.manualmode.web.ssl.DatabaseCertLoginModuleTestCase

The 'main' thread of the test shows the test was shutting down the server:

{code}
"main" #1 prio=5 os_prio=0 tid=0xf6507c00 nid=0x205c in Object.wait() [0xf667d000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xc807a168> (a java.lang.UNIXProcess)
	at java.lang.Object.wait(Object.java:502)
	at java.lang.UNIXProcess.waitFor(UNIXProcess.java:395)
	- locked <0xc807a168> (a java.lang.UNIXProcess)
	at org.wildfly.core.launcher.ProcessHelper.destroyProcess(ProcessHelper.java:60)
	at org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.stopInternal(ManagedDeployableContainer.java:294)
	at org.jboss.as.arquillian.container.CommonDeployableContainer.stop(CommonDeployableContainer.java:135)
	at org.jboss.arquillian.container.impl.ContainerImpl.stop(ContainerImpl.java:217)
	at org.jboss.arquillian.container.impl.client.container.ContainerLifecycleController$9.perform(ContainerLifecycleController.java:178)
	at org.jboss.arquillian.container.impl.client.container.ContainerLifecycleController$9.perform(ContainerLifecycleController.java:172)
	at org.jboss.arquillian.container.impl.client.container.ContainerLifecycleController.forContainer(ContainerLifecycleController.java:255)
	at org.jboss.arquillian.container.impl.client.container.ContainerLifecycleController.stopContainer(ContainerLifecycleController.java:171)
{code}

The server-side thread dump shows the hang:

{code}
Found one Java-level deadlock:
=============================
"Periodic Recovery":
  waiting to lock monitor 0xbcc6dbc0 (object 0xc8f80918, a java.util.concurrent.atomic.AtomicInteger),
  which is held by "ServerService Thread Pool -- 8"
"ServerService Thread Pool -- 8":
  waiting to lock monitor 0xc1a6ffc8 (object 0xc8f64470, a com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule),
  which is held by "Periodic Recovery"

Java stack information for the threads listed above:
===================================================
"Periodic Recovery":
	at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.setScanState(XARecoveryModule.java:1088)
	- waiting to lock <0xc8f80918> (a java.util.concurrent.atomic.AtomicInteger)
	at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkSecondPass(XARecoveryModule.java:240)
	- locked <0xc8f64470> (a com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule)
	at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816)
	at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382)
"ServerService Thread Pool -- 8":
	at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.removeXAResourceRecoveryHelper(XARecoveryModule.java:119)
	- waiting to lock <0xc8f64470> (a com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule)
	- locked <0xc8f80918> (a java.util.concurrent.atomic.AtomicInteger)
	at com.arjuna.ats.jbossatx.jta.RecoveryManagerService.removeXAResourceRecovery(RecoveryManagerService.java:129)
	at org.jboss.jca.core.tx.jbossts.XAResourceRecoveryRegistryImpl.removeXAResourceRecovery(XAResourceRecoveryRegistryImpl.java:63)
	at org.jboss.as.connector.subsystems.datasources.XaDataSourceService.stopService(XaDataSourceService.java:66)
	- locked <0xc8f95a38> (a org.jboss.as.connector.subsystems.datasources.XaDataSourceService)
	at org.jboss.as.connector.subsystems.datasources.AbstractDataSourceService$1.run(AbstractDataSourceService.java:188)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.lang.Thread.run(Thread.java:748)
	at org.jboss.threads.JBossThread.run(JBossThread.java:485)

Found 1 deadlock.
{code}

I'll attach the full thread dump.



--
This message was sent by Atlassian Jira
(v7.12.1#712002)