[jboss-jira] [JBoss JIRA] (WFLY-5128) TimeoutException: Could not acquire lock: Waiting to complete tx

[Michal Vinkler (JIRA)]

     [ https://issues.jboss.org/browse/WFLY-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michal Vinkler updated WFLY-5128:
---------------------------------
    Affects Version/s: 10.0.0.Beta1
                       10.0.0.Alpha6


> TimeoutException: Could not acquire lock: Waiting to complete tx
> ----------------------------------------------------------------
>
>                 Key: WFLY-5128
>                 URL: https://issues.jboss.org/browse/WFLY-5128
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering
>    Affects Versions: 10.0.0.Alpha6, 10.0.0.Beta1
>            Reporter: Michal Vinkler
>            Assignee: Paul Ferraro
>
> Seen in these scenarios: 
> ejbservlet - shutdown/undeploy
> http session - shutdown/undeploy
> Setup: 4 node cluster, one node at time is shutdown, while standalone clients keep calling the application.
> Approx. 30 seconds after failing one node, one or more nodes intermittently log this error:
> {code}
> [JBossINF] [0m[31m16:25:28,774 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p3-t13) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on Jc-A2RkwALPqP0gqsUZuC4PWTd_UUdeRtFDD7gv5 on behalf of transaction GlobalTransaction:<perf20>:19672:remote. Waiting to complete tx: RemoteTransaction{modifications=[ReplaceCommand{key=Jc-A2RkwALPqP0gqsUZuC4PWTd_UUdeRtFDD7gv5, oldValue=null, newValue=org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry at 63261b4e, metadata=EmbeddedMetadata{version=null}, flags=[FORCE_WRITE_LOCK, IGNORE_RETURN_VALUES], successful=true, valueMatcher=MATCH_ALWAYS}], lookedUpEntries={Jc-A2RkwALPqP0gqsUZuC4PWTd_UUdeRtFDD7gv5=RepeatableReadEntry(3ac1d699){key=Jc-A2RkwALPqP0gqsUZuC4PWTd_UUdeRtFDD7gv5, value=org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionCacheEntry at 63261b4e, oldValue=null, isCreated=false, isChanged=false, isRemoved=false, isValid=true, skipRemoteGet=false, metadata=EmbeddedMetadata{version=null}}}, lockedKeys=[Jc-A2RkwALPqP0gqsUZuC4PWTd_UUdeRtFDD7gv5], backupKeyLocks=null, lookedUpEntriesTopology=9, isMarkedForRollback=false, tx=GlobalTransaction:<perf20>:18402:remote, state=null}.
> [JBossINF] 	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.newTimeoutException(AbstractTxLockingInterceptor.java:217)
> [JBossINF] 	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:209)
> [JBossINF] 	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:173)
> [JBossINF] 	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] 	at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] 	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] 	at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] 	at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] 	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] 	at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] 	at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] 	at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] 	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] 	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] 	at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] 	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] 	at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] 	at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] 	at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] 	at java.lang.Thread.run(Thread.java:745)
> {code}
> This error may be the root cause of https://issues.jboss.org/browse/JBEAP-505 (server ends up with Operation ("deploy") failed when restarted after failover).
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-session-shutdown-dist-sync/4/console-perf21/



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)