[JBoss JIRA] (ISPN-8329) ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2 random failures with scattered cache
by Dan Berindei (Jira)
[ https://issues.jboss.org/browse/ISPN-8329?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8329:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/7094
> ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2 random failures with scattered cache
> ----------------------------------------------------------------------------------------------
>
> Key: ISPN-8329
> URL: https://issues.jboss.org/browse/ISPN-8329
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core
> Reporter: Tristan Tarrant
> Assignee: Dan Berindei
> Priority: Major
> Labels: testsuite_stability
>
> Error Message
> Timed out waiting for rebalancing to complete on node ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665, current topology is CacheTopology{id=8, rebalanceId=4, currentCH=PartitionerConsistentHash:ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665: 85]}, pendingCH=null, unionCH=null, phase=NO_REBALANCE, actualMembers=[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665], persistentUUIDs=[63b3a997-f229-475b-a14c-9c892f608ba0]}. rebalanceInProgress=false, currentChIsBalanced=false
> Stacktrace
> java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665, current topology is CacheTopology{id=8, rebalanceId=4, currentCH=PartitionerConsistentHash:ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665: 85]}, pendingCH=null, unionCH=null, phase=NO_REBALANCE, actualMembers=[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665], persistentUUIDs=[63b3a997-f229-475b-a14c-9c892f608ba0]}. rebalanceInProgress=false, currentChIsBalanced=false
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:386)
> at org.infinispan.statetransfer.ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2(ClusterTopologyManagerTest.java:430)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ... Removed 16 stack frames
>
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months
[JBoss JIRA] (ISPN-8329) ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2 random failures with scattered cache
by Dan Berindei (Jira)
[ https://issues.jboss.org/browse/ISPN-8329?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8329:
-------------------------------
Summary: ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2 random failures with scattered cache (was: ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus random failures with scattered cache)
> ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2 random failures with scattered cache
> ----------------------------------------------------------------------------------------------
>
> Key: ISPN-8329
> URL: https://issues.jboss.org/browse/ISPN-8329
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core
> Reporter: Tristan Tarrant
> Priority: Major
> Labels: testsuite_stability
>
> Error Message
> Timed out waiting for rebalancing to complete on node ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665, current topology is CacheTopology{id=8, rebalanceId=4, currentCH=PartitionerConsistentHash:ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665: 85]}, pendingCH=null, unionCH=null, phase=NO_REBALANCE, actualMembers=[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665], persistentUUIDs=[63b3a997-f229-475b-a14c-9c892f608ba0]}. rebalanceInProgress=false, currentChIsBalanced=false
> Stacktrace
> java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665, current topology is CacheTopology{id=8, rebalanceId=4, currentCH=PartitionerConsistentHash:ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665: 85]}, pendingCH=null, unionCH=null, phase=NO_REBALANCE, actualMembers=[ClusterTopologyManagerTest[SCATTERED_SYNC, tx=false]-NodeB-47665], persistentUUIDs=[63b3a997-f229-475b-a14c-9c892f608ba0]}. rebalanceInProgress=false, currentChIsBalanced=false
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:386)
> at org.infinispan.statetransfer.ClusterTopologyManagerTest.testAbruptLeaveAfterGetStatus2(ClusterTopologyManagerTest.java:430)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ... Removed 16 stack frames
>
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months
[JBoss JIRA] (ISPN-9257) ClustertopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC, tx=false] random failures
by Dan Berindei (Jira)
[ https://issues.jboss.org/browse/ISPN-9257?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-9257:
------------------------------------
The problem is that the coordinator (NodeA) is killed with a regular {{EmbeddedCacheManager.stop()}}, and has time to rebalance the cache with members \[NodeB, NodeC\] before stopping.
It is not a problem in the DIST_SYNC config once the rebalance with \[NodeB, NodeC\] is done, killing one more node doesn't start another rebalance. But scattered cache needs a second rebalance to assign primary owners to all the segments, and this rebalance is discarded by the test instead of the \[NodeB, NodeC\] rebalance.
> ClustertopologyManagerTest.testAbruptLeaveAfterGetStatus2[SCATTERED_SYNC, tx=false] random failures
> ---------------------------------------------------------------------------------------------------
>
> Key: ISPN-9257
> URL: https://issues.jboss.org/browse/ISPN-9257
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.3.0.CR1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Minor
> Labels: testsuite_stability
> Fix For: 10.0.0.Final
>
> Attachments: ISPN-8731_wrong_topology_2018-05-18_ClusterTopologyManagerTest-infinispan-core.log.gz
>
>
> The test kills the coordinator NodeA, then while NodeB is trying to recover the caches it also kills NodeC. It expects NodeB to start a rebalance with 2 nodes and discards it, in order to test that it can process the 1-node rebalance first:
> {noformat}
> 00:34:06,582 DEBUG (transport-thread-test-NodeB-p12-t6:[testCache]) [ClusterTopologyManagerTest] Discarding rebalance command CacheTopology{id=8, phase=TRANSITORY, rebalanceId=5, currentCH=ScatteredConsistentHash{ns=256, rebalanced=false, owners = (2)[test-NodeB-49590: 85, test-NodeC-58596: 85]}, pendingCH=ScatteredConsistentHash{ns=256, rebalanced=true, owners = (2)[test-NodeB-49590: 128, test-NodeC-58596: 128]}, unionCH=null, actualMembers=[test-NodeB-49590, test-NodeC-58596], persistentUUIDs=[6b96414e-15d8-4350-aa3c-4fb4fc34e888, d47dc4a9-2a95-4bb1-a83b-bb8a27c9999f]}
> 00:34:06,609 DEBUG (transport-thread-test-NodeB-p12-t2:[Topology-testCache]) [LocalTopologyManagerImpl] Updating local topology for cache testCache: CacheTopology{id=9, phase=TRANSITORY, rebalanceId=5, currentCH=ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[test-NodeB-49590: 85]}, pendingCH=ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[test-NodeB-49590: 128]}, unionCH=null, actualMembers=[test-NodeB-49590], persistentUUIDs=[6b96414e-15d8-4350-aa3c-4fb4fc34e888]}
> 00:34:06,609 DEBUG (transport-thread-test-NodeB-p12-t2:[Topology-testCache]) [LocalTopologyManagerImpl] Installing fake cache topology CacheTopology{id=8, phase=NO_REBALANCE, rebalanceId=4, currentCH=ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[test-NodeB-49590: 85]}, pendingCH=null, unionCH=null, actualMembers=[test-NodeB-49590], persistentUUIDs=[6b96414e-15d8-4350-aa3c-4fb4fc34e888]} for cache testCache
> {noformat}
> Unfortunately {{PreferAvailabilityStrategy}} has changed a bit and the rebalance ids don't always match the expectations of the test, so that the 1-node rebalance is discarded instead:
> {noformat}
> 09:46:10,530 DEBUG (transport-thread-Test-NodeB-p54539-t3:[testCache]) [Test] Discarding rebalance command CacheTopology{id=9, phase=TRANSITORY, rebalanceId=5, currentCH=ScatteredConsistentHash{ns=256, rebalanced=false, owners = (1)[Test-NodeB-62039: 85]}, pendingCH=ScatteredConsistentHash{ns=256, rebalanced=true, owners = (1)[Test-NodeB-62039: 256]}, unionCH=null, actualMembers=[Test-NodeB-62039], persistentUUIDs=[0ed7be74-4485-489b-baee-28c461c9e5de]}
> {noformat}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months
[JBoss JIRA] (ISPN-10360) Expiration with stores should be non blocking
by Will Burns (Jira)
Will Burns created ISPN-10360:
---------------------------------
Summary: Expiration with stores should be non blocking
Key: ISPN-10360
URL: https://issues.jboss.org/browse/ISPN-10360
Project: Infinispan
Issue Type: Enhancement
Components: Expiration, Loaders and Stores
Reporter: Will Burns
Expiration currently occurs while holding the lock in the data container using the compute method. This is inherently blocking. This needs to be redone to allow for this to be done in a non blocking way. Most likely we will have to acquire the lock from the LockManager to perform such operations in the future.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months
[JBoss JIRA] (ISPN-10312) Evaluate locations that use CompletableFuture#join
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10312?page=com.atlassian.jira.plugin... ]
Will Burns updated ISPN-10312:
------------------------------
Status: Open (was: New)
> Evaluate locations that use CompletableFuture#join
> --------------------------------------------------
>
> Key: ISPN-10312
> URL: https://issues.jboss.org/browse/ISPN-10312
> Project: Infinispan
> Issue Type: Sub-task
> Reporter: Will Burns
> Priority: Major
>
> Invoking join or get on a CompletableFuture is a code stink and should be evaluated if it can be refactored to something else. Quite possibly this JIRA may be to just create other JIRAs but any simple refactorings can be done here.
> Invocations to watch for {code}CompletionStages.join(){code}, {code}CompletableFuture.join(){code}, {code}CompletableFuture.get(){code}, {code}CompletableFuture.get(int, TimeUnit){code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months
[JBoss JIRA] (ISPN-10312) Evaluate locations that use CompletableFuture#join
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10312?page=com.atlassian.jira.plugin... ]
Work on ISPN-10312 started by Will Burns.
-----------------------------------------
> Evaluate locations that use CompletableFuture#join
> --------------------------------------------------
>
> Key: ISPN-10312
> URL: https://issues.jboss.org/browse/ISPN-10312
> Project: Infinispan
> Issue Type: Sub-task
> Reporter: Will Burns
> Assignee: Will Burns
> Priority: Major
>
> Invoking join or get on a CompletableFuture is a code stink and should be evaluated if it can be refactored to something else. Quite possibly this JIRA may be to just create other JIRAs but any simple refactorings can be done here.
> Invocations to watch for {code}CompletionStages.join(){code}, {code}CompletableFuture.join(){code}, {code}CompletableFuture.get(){code}, {code}CompletableFuture.get(int, TimeUnit){code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months
[JBoss JIRA] (ISPN-10359) Consolidate Thread Pools
by Will Burns (Jira)
Will Burns created ISPN-10359:
---------------------------------
Summary: Consolidate Thread Pools
Key: ISPN-10359
URL: https://issues.jboss.org/browse/ISPN-10359
Project: Infinispan
Issue Type: Sub-task
Components: Core
Reporter: Will Burns
We should be able to consolidate the thread pools down to 2 main ones: CPU bound thread pool and a blocking thread pool. The former should be sized equal to the # of cpus and the latter should be arbitrarily large. We will also need additional pool(s) for schedulers.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
4 years, 11 months