[jbosscache-issues] [JBoss JIRA] (JBCACHE-1613) Lock TimeoutException when using a JTS transaction manager under load

[Dennis Reed (JIRA)]

Dennis Reed created JBCACHE-1613:
------------------------------------

             Summary: Lock TimeoutException when using a JTS transaction manager under load
                 Key: JBCACHE-1613
                 URL: https://issues.jboss.org/browse/JBCACHE-1613
             Project: JBoss Cache
          Issue Type: Bug
      Security Level: Public (Everyone can see)
          Components: Transactions
    Affects Versions: 3.2.8.GA
            Reporter: Dennis Reed
            Assignee: Dennis Reed


After switching to JTS for transactions, "random" lock timeouts occur:

    org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/foo] after [15000] milliseconds for requestor [GlobalTransaction:<10.1.2.3:12345>:5]! Lock held by [null]
        at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:157)

The actual cause is only logged at TRACE level:

    TRACE [org.jboss.cache.lock.MVCCLockManager] (RequestProcessor-97) Attempting to unlock /foo
    TRACE [org.jboss.cache.lock.MVCCLockManager] (RequestProcessor-97) Caught exception and ignoring.
java.lang.IllegalMonitorStateException: org.jboss.cache.util.concurrent.locks.OwnableReentrantLock at 4d93ddfa[State = 1, empty queue][Locked by GlobalTransaction:<10.1.2.3:12345>:4]
        at org.jboss.cache.util.concurrent.locks.OwnableReentrantLock.tryRelease(OwnableReentrantLock.java:139)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1239)
        at org.jboss.cache.util.concurrent.locks.OwnableReentrantLock.unlock(OwnableReentrantLock.java:105)
        at org.jboss.cache.util.concurrent.locks.PerElementLockContainer.releaseLock(PerElementLockContainer.java:112)
        at org.jboss.cache.lock.MVCCLockManager.unlock(MVCCLockManager.java:147)
        at org.jboss.cache.interceptors.MVCCLockingInterceptor.cleanupLocks(MVCCLockingInterceptor.java:375)
        at org.jboss.cache.interceptors.MVCCLockingInterceptor.transactionalCleanup(MVCCLockingInterceptor.java:400)
        at org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePrepareCommand(MVCCLockingInterceptor.java:332)
        ...

The root cause is that JTS commits transactions in a different thread than the one calling commit.
Under load, it even uses different threads for each call during the same transaction commit (beforeCompletion/afterCompletion).

JBoss Cache stores some data in ThreadLocals, and when different threads call beforeCompletion and afterCompletion for the same transaction,
it causes it to mix the data between the transactions causing data corruption.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira