ConcurrentJoinTest random failures ---------------------------------- Key: ISPN-6984 URL: https://issues.jboss.org/browse/ISPN-6984 Project: Infinispan Issue Type: Bug Components: Core, Test Suite - Core Affects Versions: 9.0.0.Alpha4 Reporter: Dan Berindei Assignee: Dan Berindei Labels: testsuite_stability Fix For: 9.0.0.Beta1, 9.0.0.Final {{RehashTestBase.testTransactional}} changes the cache membership (in {{ConcurrentJoinTest}}, that means it starts 4 nodes) and waits for the rebalance to finish, then asserts that the keys are owned by the proper nodes. Waiting for the rebalance to finish is not enough, however, because entries that are no longer owned are removed asynchronously, *after* the rebalance is finished. {noformat} 12:00:20,346 TRACE (transport-thread-CJT-NodeL-p1952-t3:[Topology-dist]) [StateConsumerImpl] Received new topology for cache dist, isRebalance = false, isMember = true, topology = CacheTopology{id=11, rebalanceId=6, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (8)[CJT-NodeI-14064: 34+28, CJT-NodeJ-23687: 34+32, CJT-NodeK-45647: 30+32, CJT-NodeL-44214: 29+31, CJT-NodeM-42056: 34+33, CJT-NodeN-9760: 34+36, CJT-NodeO-43833: 27+29, CJT-NodeP-5141: 34+35]}, pendingCH=null, unionCH=null, actualMembers=[CJT-NodeI-14064, CJT-NodeJ-23687, CJT-NodeK-45647, CJT-NodeL-44214, CJT-NodeM-42056, CJT-NodeN-9760, CJT-NodeO-43833, CJT-NodeP-5141], persistentUUIDs=[5dc1f506-81b5-4c26-bf9a-f1fb14d54d26, 8943aaa4-2cd1-484a-a6ba-45269e376620, b8cb3e0e-cc6d-4768-8825-8275b06b571e, 5c46f8ac-2007-4286-a64e-580fbc4308eb, 42964274-543b-4c8c-ac39-b74485124aff, cc489f64-345f-4f75-bc60-60fedba26b47, 85b51a75-d9d3-497b-8922-588f8cfa95d3, a5aca3ae-7812-45e9-acc4-392102c7d1da]} 12:00:20,346 TRACE (transport-thread-CJT-NodeL-p1952-t3:[Topology-dist]) [StateTransferLockImpl] Signalling topology 11 is installed 12:00:20,372 DEBUG (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [StateConsumerImpl] Removing no longer owned entries for cache dist 12:00:20,451 ERROR (testng-CJT:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.rehash.CJT.testTransactional java.lang.AssertionError: Fail on non-owner cache CJT-NodeJ-23687: dc.get(MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}) returned ImmortalCacheEntry{key=MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}, value=v2}! at org.infinispan.distribution.BaseDistFunctionalTest.assertOwnershipAndNonOwnership(BaseDistFunctionalTest.java:192) ~[test-classes/:?] at org.infinispan.distribution.rehash.RehashTestBase.testTransactional(RehashTestBase.java:143) ~[test-classes/:?] 12:00:20,459 TRACE (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [InvalidateCommand] Invalidating keys [MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}] 12:00:20,465 TRACE (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [ReadCommittedEntry] Updating entry (key=MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687} removed=true valid=false changed=true created=false value=v2 metadata=EmbeddedMetadata{version=null}, providedMetadata=null) {noformat}