[infinispan-issues] [JBoss JIRA] Commented: (ISPN-819) Deadlock detection seem to not detect deadlock case between two nodes on the same cache.

[Mircea Markus (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570306#comment-12570306 ] 

Mircea Markus commented on ISPN-819:
------------------------------------

I've just tried to run the same. The test stops indeed, but received exception is:

2010-12-15 11:36:36,644 INFO  [ThreeNodesDeadlocks] (main) Before LOCK...
2010-12-15 11:36:36,876 ERROR [InvocationContextInterceptor] (main) Execution error: 
org.infinispan.util.concurrent.locks.DeadlockDetectedException: Deadlock found and we DldGlobalTransaction{coinToss=3780946122068384327, isMarkedForRollback=false, lockIntention=counter_key, affectedKeys=[counter_key], locksAtOrigin=[]} GlobalTransaction:<nodeDeadLock-51901>:1:local shall not continue. Other tx is DldGlobalTransaction{coinToss=6506972304784572625, isMarkedForRollback=false, lockIntention=null, affectedKeys=[], locksAtOrigin=[]} GlobalTransaction:<nodeDeadLock-40589>:1:remote
	at org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager.lockAndRecord(DeadlockDetectingLockManager.java:84)
	at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:220)
	at org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:155)
	at org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:107)
	at org.infinispan.interceptors.LockingInterceptor.lockKeysForLockCommand(LockingInterceptor.java:215)
	at org.infinispan.interceptors.LockingInterceptor.visitLockControlCommand(LockingInterceptor.java:182)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.DeadlockDetectingInterceptor.handleDataCommand(DeadlockDetectingInterceptor.java:91)
	at org.infinispan.interceptors.DeadlockDetectingInterceptor.visitLockControlCommand(DeadlockDetectingInterceptor.java:73)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:169)
	at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:125)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:87)
	at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:63)
	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:143)
	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:273)
	at org.infinispan.CacheDelegate.lock(CacheDelegate.java:311)
	at org.infinispan.CacheDelegate.lock(CacheDelegate.java:303)
	at ThreeNodesDeadlocks.launchOperation(ThreeNodesDeadlocks.java:61)
	at ThreeNodesDeadlocks.main(ThreeNodesDeadlocks.java:98)

This is to point out that a deadlock was found, and it's correct/expected. Do you receive timeoutexceptions (these might indicate a problem indeed) ? Otherwise this is expected behavior, as even with deadlock enabled one of the processes does give up. It's only that it gives up earlier and allows the other one to continue. 

> Deadlock detection seem to not detect deadlock case between two nodes on the same cache.
> ----------------------------------------------------------------------------------------
>
>                 Key: ISPN-819
>                 URL: https://issues.jboss.org/browse/ISPN-819
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Locking and Concurrency
>    Affects Versions: 4.2.0.CR3
>         Environment: Windows XP, 4.2.0-CR3
>            Reporter: Thomas Jean
>            Assignee: Mircea Markus
>            Priority: Critical
>             Fix For: 4.2.0.CR4, 4.2.0.Final
>
>         Attachments: AgentManager_InfiniSpan_Config_Generic.xml, log4j_Generic.properties, nodeDeadLock-43429.log, nodeDeadLock-45547.log, ThreeNodesDeadlocks.java, TwoNodesDeadlocks.java
>
>
> Hello,
> after activating deadlock detection with 500 ms of spin duration, i launch a classic deadlock scenario.
> Node 1 acquire local lock.
> Node 1 try to acquire cluster wide lock.
> Node 2 acquire local lock.
> Node 2 try to acquire cluster wide lock. 
> Then deadlock is not detected and the TimeoutException occurs:
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [20 seconds] on key [counter_key] for requestor [DldGlobalTrans
> action{coinToss=3136454835479173852, isMarkedForRollback=false, lockIntention=counter_key, affectedKeys=[counter_key], locksAtOrigin=[]} Glob
> alTransaction:<nodeDeadLock-11465>:5:local]! Lock held by [DldGlobalTransaction{coinToss=3164596253027655679, isMarkedForRollback=false, lock
> Intention=null, affectedKeys=[], locksAtOrigin=[]} GlobalTransaction:<nodeDeadLock-16603>:1:remote].
> I give my configuration file for ISPN and a Class TwoNodesDeadlocks.java  to reproduce the problem.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira