[JBoss JIRA] (ISPN-2522) The org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest (StateTransferFunctionTest) tests fail periodically
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2522?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2522:
--------------------------------
Assignee: Dan Berindei (was: Adrian Nistor)
> The org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest (StateTransferFunctionTest) tests fail periodically
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-2522
> URL: https://issues.jboss.org/browse/ISPN-2522
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite
> Reporter: Anna Manukyan
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 6.0.0.Alpha1
>
>
> The following tests of org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest class are failing periodically during JDG 6.1.0.ER3 build testing:
> org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest.testSTWithWritingNonTxThread
> org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest.testSTWithWritingTxThread
> You can find the failure logs here:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite...
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite...
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite...
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite...
> The same issue happens to org.infinispan.statetranser.StateTransferFunctionTest tests(the parent test class).
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite...
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-testsuite...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
- 1
- 0
[JBoss JIRA] (ISPN-3029) IllegalMonitorStateException in LockSupportCacheStore.loadAllKeys
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3029?page=com.atlassian.jira.plugin.... ]
Mircea Markus commented on ISPN-3029:
-------------------------------------
{quote}Most LockSupportCacheStore methods that call acquireGlobalLock() ignore its return value and proceed as if the lock was acquired on all the buckets.{quote}
Better to throw an checked exception (java.util.concurrent.TimeoutException?) form "acquireGlobalLock" in order to force the callers to handle this situation.
> IllegalMonitorStateException in LockSupportCacheStore.loadAllKeys
> -----------------------------------------------------------------
>
> Key: ISPN-3029
> URL: https://issues.jboss.org/browse/ISPN-3029
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 5.2.5.Final
> Reporter: Dan Berindei
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: onboard
> Fix For: 6.0.0.Final
>
>
> Most LockSupportCacheStore methods that call {{acquireGlobalLock()}} ignore its return value and proceed as if the lock was acquired on all the buckets. ISPN-2378 partially fixed this by only attempting to unlock the global lock if the lock was actually acquired, but the processing that was supposed to be protected by the global lock is still executed even if the lock acquisition failed.
> In {{loadAllKeys}}, this doesn't usually cause any problems. But if the cache store contains expired entries, it will try to upgrade a bucket lock to a write lock in order to update the bucket on disk, and the upgrade will fail with a IllegalMonitorStateException:
> {noformat}
> > 20:41:36,960 ERROR [org.infinispan.statetransfer.OutboundTransferTask] (undefined) Failed to execute outbound transfer: java.lang.IllegalMonitorStateException: attempt to unlock read lock, not locked by current thread
> > at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:447) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:431) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1340) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:883) [rt.jar:1.7.0_09-icedtea]
> > at org.infinispan.util.concurrent.locks.StripedLock.upgradeLock(StripedLock.java:140) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.LockSupportCacheStore.upgradeLock(LockSupportCacheStore.java:106) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.bucket.BucketBasedCacheStore.access$000(BucketBasedCacheStore.java:49) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.bucket.BucketBasedCacheStore$CollectionGeneratingBucketHandler.handle(BucketBasedCacheStore.java:159) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.file.FileCacheStore.loopOverBuckets(FileCacheStore.java:102) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.bucket.BucketBasedCacheStore.loadAllKeysLockSafe(BucketBasedCacheStore.java:219) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.LockSupportCacheStore.loadAllKeys(LockSupportCacheStore.java:179) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.decorators.AbstractDelegatingStore.loadAllKeys(AbstractDelegatingStore.java:140) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.decorators.AsyncStore.loadKeys(AsyncStore.java:184) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.decorators.AsyncStore.loadAllKeys(AsyncStore.java:205) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.statetransfer.OutboundTransferTask.run(OutboundTransferTask.java:163) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_09-icedtea]
> > at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
> > at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA-redhat-2.jar:2.0.0.GA-redhat-2]
> {noformat}
> A simple solution would be to throw an exception any time the global lock acquisition failed, but the current global lock acquisition algorithm might need to change because it seems very deadlock-prone at the moment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
- 1
- 0
[JBoss JIRA] (ISPN-1195) Rehashing and state transfer to use digesting
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-1195?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-1195:
--------------------------------
Fix Version/s: (was: 6.0.0.Final)
> Rehashing and state transfer to use digesting
> ---------------------------------------------
>
> Key: ISPN-1195
> URL: https://issues.jboss.org/browse/ISPN-1195
> Project: Infinispan
> Issue Type: Feature Request
> Components: Distributed Cache, State transfer
> Reporter: Manik Surtani
> Assignee: Galder Zamarreño
> Priority: Critical
> Labels: rehash, state_transfer, versioning
>
> Rather than pushing entire state to new joiners, a mechanism of digests should be used to minimise network transfers. For example, Merkle trees can be used to determine which entries are out of date and need pushing, in the case of nodes that persist to a local cache store, and is restarted but still has access to some state on disk.
> This would require versioned entries though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
- 1
- 0