[infinispan-issues] [JBoss JIRA] (ISPN-7172) Total order caches can hang during join

[Dan Berindei (JIRA)]

Dan Berindei created ISPN-7172:
----------------------------------

             Summary: Total order caches can hang during join
                 Key: ISPN-7172
                 URL: https://issues.jboss.org/browse/ISPN-7172
             Project: Infinispan
          Issue Type: Bug
          Components: Core, Test Suite - Core
    Affects Versions: 9.0.0.Alpha4
            Reporter: Dan Berindei
            Assignee: Dan Berindei
            Priority: Critical
             Fix For: 9.0.0.Beta1


When two nodes join a cache in quick sequence, it's possible for a {{CacheTopologyControlCommand}} to arrive between {{LocalTopologyManagerImpl}} adding the {{LocalCacheStatus}} to the {{runningCaches}} map, but before it calling {{cacheStatus.getTopologyUpdatesExecutor().executeAsync(() -> joinFuture)}} to block topology updates.

This exposes a bug in the way {{LimitedExecutor}} wraps a {{WithinThreadExecutor}}, which causes {{SyncPrepareUseSynchronizationTotalOrderTest}} (and possibly other total order tests) to hang:

{noformat}
"testng-SyncPrepareUseSynchronizationTotalOrderTest" #15 prio=5 os_prio=0 tid=0x00007fc674b74000 nid=0x2f3f waiting on condition [0x00007fc626682000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000829d03a0> (a java.util.concurrent.CompletableFuture$Signaller)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1687)
	at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3320)
	at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1767)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1907)
	at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:82)
	at org.infinispan.executors.LimitedExecutor.execute(LimitedExecutor.java:70)
	at org.infinispan.executors.LimitedExecutor.executeAsync(LimitedExecutor.java:96)
	at org.infinispan.topology.LocalTopologyManagerImpl.join(LocalTopologyManagerImpl.java:141)
	at org.infinispan.statetransfer.StateTransferManagerImpl.start(StateTransferManagerImpl.java:121)
	at sun.reflect.GeneratedMethodAccessor146.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:867)
	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:633)
	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:622)
	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:547)
	- locked <0x00000000829d0520> (a org.infinispan.factories.ComponentRegistry)
	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:231)
	at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:808)
	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:639)
	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:590)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:422)
	at org.infinispan.test.MultipleCacheManagersTest.getCaches(MultipleCacheManagersTest.java:248)
	at org.infinispan.test.MultipleCacheManagersTest.waitForClusterToForm(MultipleCacheManagersTest.java:257)
	at org.infinispan.test.MultipleCacheManagersTest.waitForClusterToForm(MultipleCacheManagersTest.java:266)
	at org.infinispan.tx.totalorder.simple.BaseSimpleTotalOrderTest.createCacheManagers(BaseSimpleTotalOrderTest.java:326)
	at org.infinispan.test.MultipleCacheManagersTest.callCreateCacheManagers(MultipleCacheManagersTest.java:109)
	at org.infinispan.test.MultipleCacheManagersTest.createBeforeMethod(MultipleCacheManagersTest.java:119)
{noformat}




--
This message was sent by Atlassian JIRA
(v7.2.3#72005)