[infinispan-issues] [JBoss JIRA] (ISPN-2291) Tx rollback during state transfer has stale locks

[Erik Salter (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717994#comment-12717994 ] 

Erik Salter commented on ISPN-2291:
-----------------------------------

I used the following to check for stale keys:

for k in "$1" ; 
do zcat  lock/*/server.log*.gz | grep -F $k | egrep "Unable to acquire|Successfully acquired lock|Attempting to unlock" > $1 ; done
cat $1 | sed -n 's/\(.*\) \(DEBUG\|TRACE\) .*\(Fail\|acquired\|unlock\).*/\1 \3/p' | less

A sample key is ServiceGroupKey[edgeDeviceId=4,serviceGroupNo=435].  (All these *[edgeDeviceId=*] keys use edgeDeviceId as their "group").

You can find the described flow above on lock/10.30.12.85/server.gz on (OOB-7,erm-cluster,phl-dg3-41250)

                
> Tx rollback during state transfer has stale locks
> -------------------------------------------------
>
>                 Key: ISPN-2291
>                 URL: https://issues.jboss.org/browse/ISPN-2291
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Core API
>    Affects Versions: 5.2.0.Alpha3
>            Reporter: Erik Salter
>            Assignee: Mircea Markus
>
> There are stale locks that happened when a transaction failed due to a replication timeout to its peer node.  The transaction contained grouped keys that were submitted to the primary owner.  During execution of one such transaction, there was a state transfer, which changed ownership of these keys.  
> Here's an example flow:
> The task executed and started a transaction.  Since it was not the primary owner, it sent a LockControlCommand to the new owner.  This call timed out, and a rollback was issued.  The local transaction is completed, but subsequent attempts to lock this key fail.
> The logs can be found here:
> http://dl.dropbox.com/u/50401510/5.2.0.ALPHA3/lock/10.30.12.83/server.log.gz
> http://dl.dropbox.com/u/50401510/5.2.0.ALPHA3/lock/10.30.12.84/server.log.gz
> http://dl.dropbox.com/u/50401510/5.2.0.ALPHA3/lock/10.30.12.85/server.log.gz

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira