[jbossts-issues] [JBoss JIRA] (JBTM-1951) Deadlock in JBOSS Transaction Reaper

[Tom Jenkinson (JIRA)]

    [ https://issues.jboss.org/browse/JBTM-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923224#comment-12923224 ] 

Tom Jenkinson commented on JBTM-1951:
-------------------------------------

Thanks Rico, I can give you some pointers. Transaction time out is processed in the reaper. To rollback a transaction (either reaped or normal) we need to lock the transaction. To rollback each resource, ironjacamar needs to lock the resource. For some reason Thread-503 has locked the resource before trying to rollback the transaction, hence the lock orders are reversed and the dead lock has happened.

What you are looking for is to understand why Thread-503 has needed to hold a lock on the resource before calling rollback on the transaction. I have looked through 1.0.21 and 1.0.9 and can't really see a scenario where it would have held the lock.

You could take a look in Jira and see if any issues might have been fixed for IronJacamar in this area:
https://issues.jboss.org/issues/?jql=project%20%3D%20JBJCA%20AND%20fixVersion%20%3C%3D%20%221.0.21.Final%22%20and%20fixversion%20%3E%20%221.0.9.Final%22

These ones look interesting:
https://issues.jboss.org/browse/JBJCA-942
https://issues.jboss.org/browse/JBJCA-922
I think this one although its in my search indicates that maybe the version ordering in the IJ project has some overlap on 1.1 and 1.0 version numbers, it sounded interesting as it looks to be trying to prevent this:
https://issues.jboss.org/browse/JBJCA-599

As Jesper mentions above, please do raise a discussion in the forums moving forward.

Thanks,
Tom
                
> Deadlock in JBOSS Transaction Reaper
> ------------------------------------
>
>                 Key: JBTM-1951
>                 URL: https://issues.jboss.org/browse/JBTM-1951
>             Project: JBoss Transaction Manager
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>    Affects Versions: 4.16.2
>            Reporter: Peter Klauser
>            Assignee: Tom Jenkinson
>
> Found one Java-level deadlock:
> =============================
> "Timer-93":
> waiting to lock monitor 0x00007f8fac190cf8 (object 0x00000000d6d50830, a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction),
> which is held by "Transaction Reaper Worker 0"
> "Transaction Reaper Worker 0":
> waiting for ownable synchronizer 0x00000000d6861408, (a java.util.concurrent.locks.ReentrantLock$FairSync),
> which is held by "Timer-93" 
> Java stack information for the threads listed above:
> ===================================================
> "Timer-93":
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1573)
> - waiting to lock <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:188)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1247)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:134)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.afterDelivery(MessageEndpointInvocationHandler.java:70)
> at sun.reflect.GeneratedMethodAccessor1756.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.handle(AbstractInvocationHandler.java:60)
> at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:136)
> at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)
> at com.sun.proxy.$Proxy852.afterDelivery(Unknown Source)
> at sunrise.ra.aq.inbound.AqServerSession.run(AqServerSession.java:141)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> "Transaction Reaper Worker 0":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000d6861408> (a java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock$FairSync.lock(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
> at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.lock(BaseWrapperManagedConnection.java:361)
> at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:290)
> at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:99)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:312)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2874)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2853)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1616)
> - locked <0x00000000d6d50830> (a com.arjuna.ats.internal.jta.transaction.arjunacore.AtomicAction)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:118)
> at com.arjuna.ats.arjuna.AtomicAction.cancel(AtomicAction.java:217)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.doCancellations(TransactionReaper.java:370)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:78) 
> Found 1 deadlock.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira