[JBoss JIRA] (ISPN-6984) ConcurrentJoinTest random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-6984?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-6984:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4538
> ConcurrentJoinTest random failures
> ----------------------------------
>
> Key: ISPN-6984
> URL: https://issues.jboss.org/browse/ISPN-6984
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 9.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1
>
>
> {{RehashTestBase.testTransactional}} changes the cache membership (in {{ConcurrentJoinTest}}, that means it starts 4 nodes) and waits for the rebalance to finish, then asserts that the keys are owned by the proper nodes.
> Waiting for the rebalance to finish is not enough, however, because entries that are no longer owned are removed asynchronously, *after* the rebalance is finished.
> {noformat}
> 12:00:20,346 TRACE (transport-thread-CJT-NodeL-p1952-t3:[Topology-dist]) [StateConsumerImpl] Received new topology for cache dist, isRebalance = false, isMember = true, topology = CacheTopology{id=11, rebalanceId=6, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (8)[CJT-NodeI-14064: 34+28, CJT-NodeJ-23687: 34+32, CJT-NodeK-45647: 30+32, CJT-NodeL-44214: 29+31, CJT-NodeM-42056: 34+33, CJT-NodeN-9760: 34+36, CJT-NodeO-43833: 27+29, CJT-NodeP-5141: 34+35]}, pendingCH=null, unionCH=null, actualMembers=[CJT-NodeI-14064, CJT-NodeJ-23687, CJT-NodeK-45647, CJT-NodeL-44214, CJT-NodeM-42056, CJT-NodeN-9760, CJT-NodeO-43833, CJT-NodeP-5141], persistentUUIDs=[5dc1f506-81b5-4c26-bf9a-f1fb14d54d26, 8943aaa4-2cd1-484a-a6ba-45269e376620, b8cb3e0e-cc6d-4768-8825-8275b06b571e, 5c46f8ac-2007-4286-a64e-580fbc4308eb, 42964274-543b-4c8c-ac39-b74485124aff, cc489f64-345f-4f75-bc60-60fedba26b47, 85b51a75-d9d3-497b-8922-588f8cfa95d3, a5aca3ae-7812-45e9-acc4-392102c7d1da]}
> 12:00:20,346 TRACE (transport-thread-CJT-NodeL-p1952-t3:[Topology-dist]) [StateTransferLockImpl] Signalling topology 11 is installed
> 12:00:20,372 DEBUG (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [StateConsumerImpl] Removing no longer owned entries for cache dist
> 12:00:20,451 ERROR (testng-CJT:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.rehash.CJT.testTransactional
> java.lang.AssertionError: Fail on non-owner cache CJT-NodeJ-23687: dc.get(MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}) returned ImmortalCacheEntry{key=MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}, value=v2}!
> at org.infinispan.distribution.BaseDistFunctionalTest.assertOwnershipAndNonOwnership(BaseDistFunctionalTest.java:192) ~[test-classes/:?]
> at org.infinispan.distribution.rehash.RehashTestBase.testTransactional(RehashTestBase.java:143) ~[test-classes/:?]
> 12:00:20,459 TRACE (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [InvalidateCommand] Invalidating keys [MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}]
> 12:00:20,465 TRACE (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [ReadCommittedEntry] Updating entry (key=MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687} removed=true valid=false changed=true created=false value=v2 metadata=EmbeddedMetadata{version=null}, providedMetadata=null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 7 months
[JBoss JIRA] (ISPN-6983) ThreeNodesSplitAndMergeTest random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-6983?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-6983:
-------------------------------
Status: Open (was: New)
> ThreeNodesSplitAndMergeTest random failures
> -------------------------------------------
>
> Key: ISPN-6983
> URL: https://issues.jboss.org/browse/ISPN-6983
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 9.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1
>
>
> When merging the partitions, the stable topology is updated last. So when the merge is done and the cache is balanced, we cannot assume that the stable topology was updated already:
> {noformat}
> 10:31:19,576 INFO (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [CLUSTER] ISPN000336: Finished cluster-wide rebalance for cache ___defaultcache, topology id = 12
> 10:31:19,578 DEBUG (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [ClusterTopologyManagerImpl] Updating cluster-wide current topology for cache ___defaultcache, topology = CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}, availability mode = AVAILABLE
> 10:31:19,579 DEBUG (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [ClusterTopologyManagerImpl] Updating cluster-wide stable topology for cache ___defaultcache, topology = CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> 10:31:19,579 TRACE (transport-thread-TNRSAMT-NodeD-p47037-t3:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> 10:31:19,581 TRACE (OOB-2,TNRSAMT-NodeE-56471:[]) [GlobalInboundInvocationHandler] Attempting to execute non-CacheRpcCommand: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=TNRSAMT-NodeD-51650, joinInfo=null, topologyId=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, availabilityMode=AVAILABLE, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], throwable=null, viewId=15} [sender=TNRSAMT-NodeD-51650]
> 10:31:19,590 TRACE (OOB-2,TNRSAMT-NodeE-56471:[]) [GlobalInboundInvocationHandler] Attempting to execute non-CacheRpcCommand: CacheTopologyControlCommand{cache=___defaultcache, type=STABLE_TOPOLOGY_UPDATE, sender=TNRSAMT-NodeD-51650, joinInfo=null, topologyId=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, availabilityMode=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], throwable=null, viewId=15} [sender=TNRSAMT-NodeD-51650]
> 10:31:19,591 ERROR (testng-TNRSAMT:[]) [TestSuiteProgress] Test failed: org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge1
> java.lang.AssertionError: Sets differ: expected [TNRSAMT-NodeD-51650, TNRSAMT-NodeE-56471, TNRSAMT-NodeF-19367] but got [TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367]
> at org.testng.Assert.fail(Assert.java:94) ~[testng-6.8.8.jar:?]
> at org.testng.Assert.assertEquals(Assert.java:806) ~[testng-6.8.8.jar:?]
> at org.testng.Assert.assertEquals(Assert.java:784) ~[testng-6.8.8.jar:?]
> at org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge(ThreeNodesSplitAndMergeTest.java:106) ~[test-classes/:?]
> at org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge1(ThreeNodesSplitAndMergeTest.java:29) ~[test-classes/:?]
> 10:31:19,595 TRACE (transport-thread-TNRSAMT-NodeF-p47149-t1:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> 10:31:19,596 TRACE (transport-thread-TNRSAMT-NodeE-p47049-t6:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 7 months
[JBoss JIRA] (ISPN-6983) ThreeNodesSplitAndMergeTest random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-6983?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-6983:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4538
> ThreeNodesSplitAndMergeTest random failures
> -------------------------------------------
>
> Key: ISPN-6983
> URL: https://issues.jboss.org/browse/ISPN-6983
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 9.0.0.Alpha4
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1
>
>
> When merging the partitions, the stable topology is updated last. So when the merge is done and the cache is balanced, we cannot assume that the stable topology was updated already:
> {noformat}
> 10:31:19,576 INFO (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [CLUSTER] ISPN000336: Finished cluster-wide rebalance for cache ___defaultcache, topology id = 12
> 10:31:19,578 DEBUG (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [ClusterTopologyManagerImpl] Updating cluster-wide current topology for cache ___defaultcache, topology = CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}, availability mode = AVAILABLE
> 10:31:19,579 DEBUG (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [ClusterTopologyManagerImpl] Updating cluster-wide stable topology for cache ___defaultcache, topology = CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> 10:31:19,579 TRACE (transport-thread-TNRSAMT-NodeD-p47037-t3:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> 10:31:19,581 TRACE (OOB-2,TNRSAMT-NodeE-56471:[]) [GlobalInboundInvocationHandler] Attempting to execute non-CacheRpcCommand: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=TNRSAMT-NodeD-51650, joinInfo=null, topologyId=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, availabilityMode=AVAILABLE, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], throwable=null, viewId=15} [sender=TNRSAMT-NodeD-51650]
> 10:31:19,590 TRACE (OOB-2,TNRSAMT-NodeE-56471:[]) [GlobalInboundInvocationHandler] Attempting to execute non-CacheRpcCommand: CacheTopologyControlCommand{cache=___defaultcache, type=STABLE_TOPOLOGY_UPDATE, sender=TNRSAMT-NodeD-51650, joinInfo=null, topologyId=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, availabilityMode=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], throwable=null, viewId=15} [sender=TNRSAMT-NodeD-51650]
> 10:31:19,591 ERROR (testng-TNRSAMT:[]) [TestSuiteProgress] Test failed: org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge1
> java.lang.AssertionError: Sets differ: expected [TNRSAMT-NodeD-51650, TNRSAMT-NodeE-56471, TNRSAMT-NodeF-19367] but got [TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367]
> at org.testng.Assert.fail(Assert.java:94) ~[testng-6.8.8.jar:?]
> at org.testng.Assert.assertEquals(Assert.java:806) ~[testng-6.8.8.jar:?]
> at org.testng.Assert.assertEquals(Assert.java:784) ~[testng-6.8.8.jar:?]
> at org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge(ThreeNodesSplitAndMergeTest.java:106) ~[test-classes/:?]
> at org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge1(ThreeNodesSplitAndMergeTest.java:29) ~[test-classes/:?]
> 10:31:19,595 TRACE (transport-thread-TNRSAMT-NodeF-p47149-t1:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> 10:31:19,596 TRACE (transport-thread-TNRSAMT-NodeE-p47049-t6:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 7 months
[JBoss JIRA] (ISPN-6984) ConcurrentJoinTest random failures
by Dan Berindei (JIRA)
Dan Berindei created ISPN-6984:
----------------------------------
Summary: ConcurrentJoinTest random failures
Key: ISPN-6984
URL: https://issues.jboss.org/browse/ISPN-6984
Project: Infinispan
Issue Type: Bug
Components: Core, Test Suite - Core
Affects Versions: 9.0.0.Alpha4
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 9.0.0.Beta1
{{RehashTestBase.testTransactional}} changes the cache membership (in {{ConcurrentJoinTest}}, that means it starts 4 nodes) and waits for the rebalance to finish, then asserts that the keys are owned by the proper nodes.
Waiting for the rebalance to finish is not enough, however, because entries that are no longer owned are removed asynchronously, *after* the rebalance is finished.
{noformat}
12:00:20,346 TRACE (transport-thread-CJT-NodeL-p1952-t3:[Topology-dist]) [StateConsumerImpl] Received new topology for cache dist, isRebalance = false, isMember = true, topology = CacheTopology{id=11, rebalanceId=6, currentCH=PartitionerConsistentHash:DefaultConsistentHash{ns=256, owners = (8)[CJT-NodeI-14064: 34+28, CJT-NodeJ-23687: 34+32, CJT-NodeK-45647: 30+32, CJT-NodeL-44214: 29+31, CJT-NodeM-42056: 34+33, CJT-NodeN-9760: 34+36, CJT-NodeO-43833: 27+29, CJT-NodeP-5141: 34+35]}, pendingCH=null, unionCH=null, actualMembers=[CJT-NodeI-14064, CJT-NodeJ-23687, CJT-NodeK-45647, CJT-NodeL-44214, CJT-NodeM-42056, CJT-NodeN-9760, CJT-NodeO-43833, CJT-NodeP-5141], persistentUUIDs=[5dc1f506-81b5-4c26-bf9a-f1fb14d54d26, 8943aaa4-2cd1-484a-a6ba-45269e376620, b8cb3e0e-cc6d-4768-8825-8275b06b571e, 5c46f8ac-2007-4286-a64e-580fbc4308eb, 42964274-543b-4c8c-ac39-b74485124aff, cc489f64-345f-4f75-bc60-60fedba26b47, 85b51a75-d9d3-497b-8922-588f8cfa95d3, a5aca3ae-7812-45e9-acc4-392102c7d1da]}
12:00:20,346 TRACE (transport-thread-CJT-NodeL-p1952-t3:[Topology-dist]) [StateTransferLockImpl] Signalling topology 11 is installed
12:00:20,372 DEBUG (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [StateConsumerImpl] Removing no longer owned entries for cache dist
12:00:20,451 ERROR (testng-CJT:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.rehash.CJT.testTransactional
java.lang.AssertionError: Fail on non-owner cache CJT-NodeJ-23687: dc.get(MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}) returned ImmortalCacheEntry{key=MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}, value=v2}!
at org.infinispan.distribution.BaseDistFunctionalTest.assertOwnershipAndNonOwnership(BaseDistFunctionalTest.java:192) ~[test-classes/:?]
at org.infinispan.distribution.rehash.RehashTestBase.testTransactional(RehashTestBase.java:143) ~[test-classes/:?]
12:00:20,459 TRACE (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [InvalidateCommand] Invalidating keys [MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687}]
12:00:20,465 TRACE (transport-thread-CJT-NodeJ-p1808-t3:[Topology-dist]) [ReadCommittedEntry] Updating entry (key=MagicKey#k2{58/0399D54B/204@CJT-NodeJ-23687} removed=true valid=false changed=true created=false value=v2 metadata=EmbeddedMetadata{version=null}, providedMetadata=null)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 7 months
[JBoss JIRA] (ISPN-6983) ThreeNodesSplitAndMergeTest random failures
by Dan Berindei (JIRA)
Dan Berindei created ISPN-6983:
----------------------------------
Summary: ThreeNodesSplitAndMergeTest random failures
Key: ISPN-6983
URL: https://issues.jboss.org/browse/ISPN-6983
Project: Infinispan
Issue Type: Bug
Components: Core, Test Suite - Core
Affects Versions: 9.0.0.Alpha4
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 9.0.0.Beta1
When merging the partitions, the stable topology is updated last. So when the merge is done and the cache is balanced, we cannot assume that the stable topology was updated already:
{noformat}
10:31:19,576 INFO (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [CLUSTER] ISPN000336: Finished cluster-wide rebalance for cache ___defaultcache, topology id = 12
10:31:19,578 DEBUG (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [ClusterTopologyManagerImpl] Updating cluster-wide current topology for cache ___defaultcache, topology = CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}, availability mode = AVAILABLE
10:31:19,579 DEBUG (remote-thread-TNRSAMT-NodeD-p47035-t6:[___defaultcache]) [ClusterTopologyManagerImpl] Updating cluster-wide stable topology for cache ___defaultcache, topology = CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
10:31:19,579 TRACE (transport-thread-TNRSAMT-NodeD-p47037-t3:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
10:31:19,581 TRACE (OOB-2,TNRSAMT-NodeE-56471:[]) [GlobalInboundInvocationHandler] Attempting to execute non-CacheRpcCommand: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=TNRSAMT-NodeD-51650, joinInfo=null, topologyId=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, availabilityMode=AVAILABLE, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], throwable=null, viewId=15} [sender=TNRSAMT-NodeD-51650]
10:31:19,590 TRACE (OOB-2,TNRSAMT-NodeE-56471:[]) [GlobalInboundInvocationHandler] Attempting to execute non-CacheRpcCommand: CacheTopologyControlCommand{cache=___defaultcache, type=STABLE_TOPOLOGY_UPDATE, sender=TNRSAMT-NodeD-51650, joinInfo=null, topologyId=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, availabilityMode=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], throwable=null, viewId=15} [sender=TNRSAMT-NodeD-51650]
10:31:19,591 ERROR (testng-TNRSAMT:[]) [TestSuiteProgress] Test failed: org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge1
java.lang.AssertionError: Sets differ: expected [TNRSAMT-NodeD-51650, TNRSAMT-NodeE-56471, TNRSAMT-NodeF-19367] but got [TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367]
at org.testng.Assert.fail(Assert.java:94) ~[testng-6.8.8.jar:?]
at org.testng.Assert.assertEquals(Assert.java:806) ~[testng-6.8.8.jar:?]
at org.testng.Assert.assertEquals(Assert.java:784) ~[testng-6.8.8.jar:?]
at org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge(ThreeNodesSplitAndMergeTest.java:106) ~[test-classes/:?]
at org.infinispan.partitionhandling.TNRSAMT.testSplitAndMerge1(ThreeNodesSplitAndMergeTest.java:29) ~[test-classes/:?]
10:31:19,595 TRACE (transport-thread-TNRSAMT-NodeF-p47149-t1:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
10:31:19,596 TRACE (transport-thread-TNRSAMT-NodeE-p47049-t6:[Topology-___defaultcache]) [LocalTopologyManagerImpl] Updating stable topology for cache ___defaultcache: CacheTopology{id=13, rebalanceId=5, currentCH=ReplicatedConsistentHash{ns = 256, owners = (3)[TNRSAMT-NodeD-51650: 82, TNRSAMT-NodeF-19367: 90, TNRSAMT-NodeE-56471: 84]}, pendingCH=null, unionCH=null, actualMembers=[TNRSAMT-NodeD-51650, TNRSAMT-NodeF-19367, TNRSAMT-NodeE-56471], persistentUUIDs=[2ce986af-9b26-4b1b-9b0c-bdd51fc591d3, ebe7eef6-3916-4061-b6f7-ce4fb35253f1, 8f6ef9d0-c5a8-4ac2-af54-4c7d07726f83]}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 7 months