[JBoss JIRA] (ISPN-2718) MinViewIdCalculusTest.testMinViewId1 randomly failing
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-2718?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-2718:
-------------------------------
Status: Pull Request Sent (was: Reopened)
Git Pull Request: https://github.com/infinispan/infinispan/pull/1898
> MinViewIdCalculusTest.testMinViewId1 randomly failing
> -----------------------------------------------------
>
> Key: ISPN-2718
> URL: https://issues.jboss.org/browse/ISPN-2718
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Reporter: Galder Zamarreño
> Assignee: Adrian Nistor
> Labels: testsuite_stability
> Fix For: 5.2.2.Final, 5.3.0.Alpha1
>
> Attachments: testMinViewId1-4.tgz
>
>
> {code}testMinViewId1(org.infinispan.lock.singlelock.MinViewIdCalculusTest) Time elapsed: 0.002 sec <<< FAILURE!
> java.lang.AssertionError: expected [4] but found [3]
> at org.testng.Assert.fail(Assert.java:89)
> at org.testng.Assert.failNotEquals(Assert.java:489)
> at org.testng.Assert.assertEquals(Assert.java:118)
> at org.testng.Assert.assertEquals(Assert.java:365)
> at org.testng.Assert.assertEquals(Assert.java:375)
> at org.infinispan.lock.singlelock.MinViewIdCalculusTest.testMinViewId1(MinViewIdCalculusTest.java:81)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:715)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:907)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1237)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680){code}
> Is this related to ISPN-2114?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] (ISPN-2982) CLONE - State transfer does not end because some segments are erroneously reported as unreceived
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2982?page=com.atlassian.jira.plugin.... ]
Mircea Markus commented on ISPN-2982:
-------------------------------------
[~ittay] not sure your problem is related. Please open a new bug with a config + logs if the problem still happens in your environment.
> CLONE - State transfer does not end because some segments are erroneously reported as unreceived
> ------------------------------------------------------------------------------------------------
>
> Key: ISPN-2982
> URL: https://issues.jboss.org/browse/ISPN-2982
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.2.1.Final
> Reporter: Ittay Dror
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.3.0.Final
>
> Attachments: jgroups.xml
>
>
> Hard to reproduce. I lost the last log where this was visible but still have a stack trace:
> org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:879)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:650)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:639)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:542)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:197)
> at org.infinispan.CacheImpl.start(CacheImpl.java:517)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:689)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:652)
> at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:126)
> at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:574)
> Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache LuceneIndexesMetadata on PersistentStateTransferQueryDistributedIndexTest-NodeC-6067
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:199)
> at sun.reflect.GeneratedMethodAccessor139.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] (ISPN-2982) CLONE - State transfer does not end because some segments are erroneously reported as unreceived
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2982?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2982:
--------------------------------
Fix Version/s: 5.3.0.Final
> CLONE - State transfer does not end because some segments are erroneously reported as unreceived
> ------------------------------------------------------------------------------------------------
>
> Key: ISPN-2982
> URL: https://issues.jboss.org/browse/ISPN-2982
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.2.1.Final
> Reporter: Ittay Dror
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.3.0.Final
>
> Attachments: jgroups.xml
>
>
> Hard to reproduce. I lost the last log where this was visible but still have a stack trace:
> org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:879)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:650)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:639)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:542)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:197)
> at org.infinispan.CacheImpl.start(CacheImpl.java:517)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:689)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:652)
> at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:126)
> at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:574)
> Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache LuceneIndexesMetadata on PersistentStateTransferQueryDistributedIndexTest-NodeC-6067
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:199)
> at sun.reflect.GeneratedMethodAccessor139.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] (ISPN-2982) CLONE - State transfer does not end because some segments are erroneously reported as unreceived
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2982?page=com.atlassian.jira.plugin.... ]
Mircea Markus resolved ISPN-2982.
---------------------------------
Resolution: Done
> CLONE - State transfer does not end because some segments are erroneously reported as unreceived
> ------------------------------------------------------------------------------------------------
>
> Key: ISPN-2982
> URL: https://issues.jboss.org/browse/ISPN-2982
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.2.1.Final
> Reporter: Ittay Dror
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.3.0.Final
>
> Attachments: jgroups.xml
>
>
> Hard to reproduce. I lost the last log where this was visible but still have a stack trace:
> org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:879)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:650)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:639)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:542)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:197)
> at org.infinispan.CacheImpl.start(CacheImpl.java:517)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:689)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:652)
> at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:126)
> at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:574)
> Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache LuceneIndexesMetadata on PersistentStateTransferQueryDistributedIndexTest-NodeC-6067
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:199)
> at sun.reflect.GeneratedMethodAccessor139.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] (ISPN-3029) IllegalMonitorStateException in LockSupportCacheStore.loadAllKeys
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3029?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3029:
--------------------------------
Assignee: Pedro Ruivo (was: Mircea Markus)
> IllegalMonitorStateException in LockSupportCacheStore.loadAllKeys
> -----------------------------------------------------------------
>
> Key: ISPN-3029
> URL: https://issues.jboss.org/browse/ISPN-3029
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 5.2.5.Final
> Reporter: Dan Berindei
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: onboard
> Fix For: 6.0.0.Final
>
>
> Most LockSupportCacheStore methods that call {{acquireGlobalLock()}} ignore its return value and proceed as if the lock was acquired on all the buckets. ISPN-2378 partially fixed this by only attempting to unlock the global lock if the lock was actually acquired, but the processing that was supposed to be protected by the global lock is still executed even if the lock acquisition failed.
> In {{loadAllKeys}}, this doesn't usually cause any problems. But if the cache store contains expired entries, it will try to upgrade a bucket lock to a write lock in order to update the bucket on disk, and the upgrade will fail with a IllegalMonitorStateException:
> {noformat}
> > 20:41:36,960 ERROR [org.infinispan.statetransfer.OutboundTransferTask] (undefined) Failed to execute outbound transfer: java.lang.IllegalMonitorStateException: attempt to unlock read lock, not locked by current thread
> > at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:447) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:431) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1340) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:883) [rt.jar:1.7.0_09-icedtea]
> > at org.infinispan.util.concurrent.locks.StripedLock.upgradeLock(StripedLock.java:140) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.LockSupportCacheStore.upgradeLock(LockSupportCacheStore.java:106) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.bucket.BucketBasedCacheStore.access$000(BucketBasedCacheStore.java:49) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.bucket.BucketBasedCacheStore$CollectionGeneratingBucketHandler.handle(BucketBasedCacheStore.java:159) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.file.FileCacheStore.loopOverBuckets(FileCacheStore.java:102) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.bucket.BucketBasedCacheStore.loadAllKeysLockSafe(BucketBasedCacheStore.java:219) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.LockSupportCacheStore.loadAllKeys(LockSupportCacheStore.java:179) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.decorators.AbstractDelegatingStore.loadAllKeys(AbstractDelegatingStore.java:140) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.decorators.AsyncStore.loadKeys(AsyncStore.java:184) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.loaders.decorators.AsyncStore.loadAllKeys(AsyncStore.java:205) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at org.infinispan.statetransfer.OutboundTransferTask.run(OutboundTransferTask.java:163) [infinispan-core-5.2.4.Final-redhat-2.jar:5.2.4.Final-redhat-2]
> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_09-icedtea]
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_09-icedtea]
> > at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
> > at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA-redhat-2.jar:2.0.0.GA-redhat-2]
> {noformat}
> A simple solution would be to throw an exception any time the global lock acquisition failed, but the current global lock acquisition algorithm might need to change because it seems very deadlock-prone at the moment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months