Dan Berindei created ISPN-3029:
----------------------------------
Summary: IllegalMonitorStateException in LockSupportCacheStore.loadAllKeys
Key: ISPN-3029
URL:
https://issues.jboss.org/browse/ISPN-3029
Project: Infinispan
Issue Type: Bug
Components: Loaders and Stores
Affects Versions: 5.2.5.Final
Reporter: Dan Berindei
Assignee: Mircea Markus
Priority: Critical
Fix For: 5.3.0.Final
Most LockSupportCacheStore methods that call {{acquireGlobalLock()}} ignore its return
value and proceed as if the lock was acquired on all the buckets. ISPN-2378 partially
fixed this by only attempting to unlock the global lock if the lock was actually acquired,
but the processing that was supposed to be protected by the global lock is still executed
even if the lock acquisition failed.
In {{loadAllKeys}}, this doesn't usually cause any problems. But if the cache store
contains expired entries, it will try to upgrade a bucket lock to a write lock in order to
update the bucket on disk, and the upgrade will fail with a IllegalMonitorStateException:
{noformat}
20:41:36,960 ERROR
[org.infinispan.statetransfer.OutboundTransferTask] (undefined) Failed to execute outbound
transfer: java.lang.IllegalMonitorStateException: attempt to unlock read lock, not locked
by current thread
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:447)
[rt.jar:1.7.0_09-icedtea]
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:431)
[rt.jar:1.7.0_09-icedtea]
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1340)
[rt.jar:1.7.0_09-icedtea]
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:883)
[rt.jar:1.7.0_09-icedtea]
at
org.infinispan.util.concurrent.locks.StripedLock.upgradeLock(StripedLock.java:140)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.LockSupportCacheStore.upgradeLock(LockSupportCacheStore.java:106)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.bucket.BucketBasedCacheStore.access$000(BucketBasedCacheStore.java:49)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.bucket.BucketBasedCacheStore$CollectionGeneratingBucketHandler.handle(BucketBasedCacheStore.java:159)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.file.FileCacheStore.loopOverBuckets(FileCacheStore.java:102)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.bucket.BucketBasedCacheStore.loadAllKeysLockSafe(BucketBasedCacheStore.java:219)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.LockSupportCacheStore.loadAllKeys(LockSupportCacheStore.java:179)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.loaders.decorators.AbstractDelegatingStore.loadAllKeys(AbstractDelegatingStore.java:140)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at org.infinispan.loaders.decorators.AsyncStore.loadKeys(AsyncStore.java:184)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at org.infinispan.loaders.decorators.AsyncStore.loadAllKeys(AsyncStore.java:205)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at
org.infinispan.statetransfer.OutboundTransferTask.run(OutboundTransferTask.java:163)
[infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[rt.jar:1.7.0_09-icedtea]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[rt.jar:1.7.0_09-icedtea]
at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
[jboss-threads-2.0.0.GA-redhat-2.jar:2.0.0.GA-redhat-2]
{noformat}
A simple solution would be to throw an exception any time the global lock acquisition
failed, but the current global lock acquisition algorithm might need to change because it
seems very deadlock-prone at the moment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:
http://www.atlassian.com/software/jira