AbstractComponentRegistry.stop() can hang if called concurrently ---------------------------------------------------------------- Key: ISPN-9979 URL: https://issues.jboss.org/browse/ISPN-9979 Project: Infinispan Issue Type: Bug Affects Versions: 9.4.3.Final Reporter: Philippe Julien Assignee: Dan Berindei Priority: Major I believe that there is a bug in org.infinispan.factories.AbstractComponentRegistry.stop() Our Wildfly 15 nodes often hang on shutdown on the following: {noformat} "MSC service thread 1-2" #37 prio=5 os_prio=0 tid=0x0000000003807000 nid=0xf32d in Object.wait() [0x00007f0012c6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000005cc795fa8> (a org.infinispan.factories.ComponentRegistry) at java.lang.Object.wait(Object.java:502) at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:359) - locked <0x00000005cc795fa8> (a org.infinispan.factories.ComponentRegistry) at org.infinispan.cache.impl.CacheImpl.performImmediateShutdown(CacheImpl.java:1159) at org.infinispan.cache.impl.CacheImpl.stop(CacheImpl.java:1124) at org.infinispan.cache.impl.AbstractDelegatingCache.stop(AbstractDelegatingCache.java:520) at org.infinispan.manager.DefaultCacheManager.terminate(DefaultCacheManager.java:734) at org.infinispan.manager.DefaultCacheManager.stopCaches(DefaultCacheManager.java:786) at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:762) at org.jboss.as.clustering.infinispan.subsystem.CacheContainerServiceConfigurator.accept(CacheContainerServiceConfigurator.java:114) at org.jboss.as.clustering.infinispan.subsystem.CacheContainerServiceConfigurator.accept(CacheContainerServiceConfigurator.java:70) at org.wildfly.clustering.service.FunctionalService.stop(FunctionalService.java:77) at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1794) at org.jboss.msc.service.ServiceControllerImpl$StopTask.execute(ServiceControllerImpl.java:1763) at org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1558) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1364) at java.lang.Thread.run(Thread.java:748) {noformat} There are no other threads that are working with 0x00000005cc795fa8 in the thread dump. I checked in a heap dump and the 0x00000005cc795fa8 org.infinispan.factories.ComponentRegistry object state is TERMINATED. I think that the finally block of AbstractComponentRegistry.stop() is missing a notifyAll(). Shouldn't this: {code} ... } finally { synchronized (this) { state = ComponentStatus.TERMINATED; } } {code} Be: {code} ... } finally { synchronized (this) { state = ComponentStatus.TERMINATED; notifyAll(); } } {code} This way, if two thread try to stop the same cache, the one that wins will notify the one that is waiting letting its stop complete.