[infinispan-issues] [JBoss JIRA] (ISPN-7996) ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC, tx=false] random failures

[Radim Vansa (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430004#comment-13430004 ] 

Radim Vansa commented on ISPN-7996:
-----------------------------------

OK, so what happens:

1) NodeB is rebalancing because of view [NodeB, NodeC]
2) requests key transfer from C, sets segment state to KEY_TRANSFER
3) View with NodeB only comes and installs new CH that does not contain the new segments
4) the new topology marks the segment as NOT_OWNED
5) Key transfer fails and marks the segment as OWNED (and this is correct, what else could it do, we want to proceed)
6) There is another rebalance which attemps to move the segment to BLOCKED state, but this fails as the segment is OWNED.

> ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC, tx=false] random failures
> ---------------------------------------------------------------------------------------------------
>
>                 Key: ISPN-7996
>                 URL: https://issues.jboss.org/browse/ISPN-7996
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Core, Test Suite - Core
>            Reporter: Dan Berindei
>            Assignee: Radim Vansa
>            Priority: Critical
>              Labels: testsuite_stability
>         Attachments: ClusterTopologyManagerTest_master_20170628.log.gz
>
>
> The test first logs a few exceptions:
> {noformat}
> 23:16:53,126 ERROR (transport-thread-Test-NodeE-p55594-t1:[Topology-___defaultcache]) [LocalTopologyManagerImpl] ISPN000230: Failed to start rebalance for cache ___defaultcache
> java.lang.IllegalStateException: Segment 7 is in state OWNED
> 	at org.infinispan.scattered.impl.ScatteredVersionManagerImpl.registerSegment(ScatteredVersionManagerImpl.java:248) ~[classes/:?]
> 	at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.beforeTopologyInstalled(ScatteredStateConsumerImpl.java:125) ~[classes/:?]
> 	at org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:336) ~[classes/:?]
> 	at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.onTopologyUpdate(ScatteredStateConsumerImpl.java:98) ~[classes/:?]
> 	at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:215) ~[classes/:?]
> 	at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:53) ~[classes/:?]
> 	at org.infinispan.statetransfer.StateTransferManagerImpl$1.rebalance(StateTransferManagerImpl.java:138) ~[classes/:?]
> 	at org.infinispan.topology.LocalTopologyManagerImpl.doHandleRebalance(LocalTopologyManagerImpl.java:526) ~[classes/:?]
> 	at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleRebalance$3(LocalTopologyManagerImpl.java:486) ~[classes/:?]
> 	at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> {noformat}
> And then it times out:
> {noformat}
> 23:17:52,949 ERROR (testng-Test:[]) [TestingUtil] Timed out waiting for rebalancing to complete on node Test-NodeE-8881, expected member list is [Test-NodeE-8881], current member list is [Test-NodeE-8881, Test-NodeF-9273]!
> 23:17:52,949 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.statetransfer.ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC, tx=false]
> java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node Test-NodeE-8881, expected member list is [Test-NodeE-8881], current member list is [Test-NodeE-8881, Test-NodeF-9273]!
> 	at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:383) ~[test-classes/:?]
> 	at org.infinispan.statetransfer.ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2(ClusterTopologyManagerTest.java:433) ~[test-classes/:?]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)