[
https://issues.jboss.org/browse/ISPN-7996?page=com.atlassian.jira.plugin....
]
Dan Berindei commented on ISPN-7996:
------------------------------------
Random failures in {{RehashWithSharedStoreTest.testRehashes\[SCATTERED_SYNC, tx=false,
numOwners=1, l1=false\]}} probably have the same cause:
{noformat}
11:47:55,248 ERROR (transport-thread-Test-NodeB-p150-t6:[Topology-dist])
[LocalTopologyManagerImpl] ISPN000230: Failed to start rebalance for cache dist
java.lang.IllegalStateException: Segment 71 is in state OWNED
at
org.infinispan.scattered.impl.ScatteredVersionManagerImpl.registerSegment(ScatteredVersionManagerImpl.java:248)
~[classes/:?]
at
org.infinispan.scattered.impl.ScatteredStateConsumerImpl.beforeTopologyInstalled(ScatteredStateConsumerImpl.java:125)
~[classes/:?]
at
org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:336)
~[classes/:?]
at
org.infinispan.scattered.impl.ScatteredStateConsumerImpl.onTopologyUpdate(ScatteredStateConsumerImpl.java:98)
~[classes/:?]
at
org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:215)
~[classes/:?]
at
org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:53)
~[classes/:?]
at
org.infinispan.statetransfer.StateTransferManagerImpl$1.rebalance(StateTransferManagerImpl.java:138)
~[classes/:?]
at
org.infinispan.topology.LocalTopologyManagerImpl.doHandleRebalance(LocalTopologyManagerImpl.java:530)
~[classes/:?]
at
org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleRebalance$3(LocalTopologyManagerImpl.java:490)
~[classes/:?]
at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
[classes/:?]
at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
[classes/:?]
at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
[classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
11:48:54,864 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed:
org.infinispan.distribution.rehash.RehashWithSharedStoreTest.testRehashes[SCATTERED_SYNC,
tx=false, numOwners=1, l1=false]
java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node
Test-NodeB-32389, current topology is CacheTopology{id=8, rebalanceId=5,
currentCH=PartitionerConsistentHash:ScatteredConsistentHash{ns=256, rebalanced=false,
owners = (2)[Test-NodeB-32389: 85, Test-NodeC-55673: 85]}, pendingCH=null, unionCH=null,
phase=NO_REBALANCE, actualMembers=[Test-NodeB-32389, Test-NodeC-55673],
persistentUUIDs=[c86368c2-10f2-49a0-88e8-f975a68ba0cc,
da487490-24d6-4b28-a4f8-6fb7efbe0ce0]}. rebalanceInProgress=false,
currentChIsBalanced=false
at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:383)
~[test-classes/:?]
at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:419)
~[test-classes/:?]
at
org.infinispan.distribution.rehash.RehashWithSharedStoreTest.testRehashes(RehashWithSharedStoreTest.java:82)
~[test-classes/:?]
{noformat}
ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC, tx=false] random
failures
---------------------------------------------------------------------------------------------------
Key: ISPN-7996
URL:
https://issues.jboss.org/browse/ISPN-7996
Project: Infinispan
Issue Type: Bug
Components: Core, Test Suite - Core
Reporter: Dan Berindei
Assignee: Radim Vansa
Priority: Critical
Labels: testsuite_stability
Attachments: ClusterTopologyManagerTest_master_20170628.log.gz
The test first logs a few exceptions:
{noformat}
23:16:53,126 ERROR (transport-thread-Test-NodeE-p55594-t1:[Topology-___defaultcache])
[LocalTopologyManagerImpl] ISPN000230: Failed to start rebalance for cache
___defaultcache
java.lang.IllegalStateException: Segment 7 is in state OWNED
at
org.infinispan.scattered.impl.ScatteredVersionManagerImpl.registerSegment(ScatteredVersionManagerImpl.java:248)
~[classes/:?]
at
org.infinispan.scattered.impl.ScatteredStateConsumerImpl.beforeTopologyInstalled(ScatteredStateConsumerImpl.java:125)
~[classes/:?]
at
org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:336)
~[classes/:?]
at
org.infinispan.scattered.impl.ScatteredStateConsumerImpl.onTopologyUpdate(ScatteredStateConsumerImpl.java:98)
~[classes/:?]
at
org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:215)
~[classes/:?]
at
org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:53)
~[classes/:?]
at
org.infinispan.statetransfer.StateTransferManagerImpl$1.rebalance(StateTransferManagerImpl.java:138)
~[classes/:?]
at
org.infinispan.topology.LocalTopologyManagerImpl.doHandleRebalance(LocalTopologyManagerImpl.java:526)
~[classes/:?]
at
org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleRebalance$3(LocalTopologyManagerImpl.java:486)
~[classes/:?]
at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
[classes/:?]
{noformat}
And then it times out:
{noformat}
23:17:52,949 ERROR (testng-Test:[]) [TestingUtil] Timed out waiting for rebalancing to
complete on node Test-NodeE-8881, expected member list is [Test-NodeE-8881], current
member list is [Test-NodeE-8881, Test-NodeF-9273]!
23:17:52,949 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed:
org.infinispan.statetransfer.ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC,
tx=false]
java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node
Test-NodeE-8881, expected member list is [Test-NodeE-8881], current member list is
[Test-NodeE-8881, Test-NodeF-9273]!
at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:383)
~[test-classes/:?]
at
org.infinispan.statetransfer.ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2(ClusterTopologyManagerTest.java:433)
~[test-classes/:?]
{noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)