[JBoss JIRA] (ISPN-8724) HotRodRemoteStreamingIT hangs
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8724?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8724:
------------------------------------
Git Pull Request: https://github.com/infinispan/infinispan/pull/5699, https://github.com/infinispan/infinispan/pull/5702 (was: https://github.com/infinispan/infinispan/pull/5699)
> HotRodRemoteStreamingIT hangs
> -----------------------------
>
> Key: ISPN-8724
> URL: https://issues.jboss.org/browse/ISPN-8724
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.0.CR1
> Reporter: Gustavo Fernandes
> Priority: Critical
> Attachments: s1, s2, test
>
>
> This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
> The surefire process waits forever for the two servers to stop.
> One of the servers is waiting for the initial state transfer to complete:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The other server is running a series of operations after the server stops, such
> as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
> MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
> {noformat}
> "MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
> at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
> at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
> - locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
> at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
> - locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> For some reason, also in the other server, the CounterModule lifecycle is reacting to a
> cache manager start event, and it's hanged as well:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8722) Impossible to create replicated/distributed cache configuration based on scattered cache configuration
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/ISPN-8722?page=com.atlassian.jira.plugin.... ]
Paul Ferraro updated ISPN-8722:
-------------------------------
Description:
Consider an existing scattered cache configuration.
Configuration scatteredConfig = ...;
ConfigurationBuilder builder = new ConfigurationBuilder().read(scatteredConfig);
builder.clustering().cacheMode(CacheMode.DIST_SYNC);
Configuration distConfig = builder.build();
This throws:
{noformat}
org.infinispan.commons.CacheConfigurationException: ISPN000468: Invalidation batch size configuration options applies only to scattered caches.
at org.infinispan.configuration.cache.ClusteringConfigurationBuilder.validate(ClusteringConfigurationBuilder.java:135)
at org.infinispan.configuration.cache.ConfigurationBuilder.validate(ConfigurationBuilder.java:226)
at org.infinispan.configuration.cache.ConfigurationBuilder.build(ConfigurationBuilder.java:283)
at org.infinispan.configuration.cache.ConfigurationBuilder.build(ConfigurationBuilder.java:273)
{noformat}
The validation logic of ClusteringConfigurationBuilder specifically requires the invalidationBatchSize attribute to be unset. However, there is no way to "unset" an attribute.
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
The same issue applies to the biasAcquisition and biasLifespan attributes in 9.2.x.
Any reason not to expose an public AttributeSet attributes() method to all configuration builders? It looks like the AttributeDefintions are all public, as are the Attribute and AttributeSet objects. That way any configuration builder can easily reset a configuration attribute to its default. WDYT?
was:
Consider an existing scattered cache configuration.
Configuration scatteredConfig = ...;
ConfigurationBuilder builder = new ConfigurationBuilder().read(scatteredConfig);
builder.clustering().cacheMode(CacheMode.DIST_SYNC);
Configuration distConfig = builder.build();
This throws:
{noformat}
org.infinispan.commons.CacheConfigurationException: ISPN000468: Invalidation batch size configuration options applies only to scattered caches.
at org.infinispan.configuration.cache.ClusteringConfigurationBuilder.validate(ClusteringConfigurationBuilder.java:135)
at org.infinispan.configuration.cache.ConfigurationBuilder.validate(ConfigurationBuilder.java:226)
at org.infinispan.configuration.cache.ConfigurationBuilder.build(ConfigurationBuilder.java:283)
at org.infinispan.configuration.cache.ConfigurationBuilder.build(ConfigurationBuilder.java:273)
{noformat}
The validation logic of ClusteringConfigurationBuilder specifically requires the invalidationBatchSize attribute to be unset. However, there is no way to "unset" an attribute.
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
The same issue applies to the biasAcquisition and biasLifespan attributes in 9.2.x.
> Impossible to create replicated/distributed cache configuration based on scattered cache configuration
> ------------------------------------------------------------------------------------------------------
>
> Key: ISPN-8722
> URL: https://issues.jboss.org/browse/ISPN-8722
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.0.CR1, 9.1.4.Final
> Reporter: Paul Ferraro
>
> Consider an existing scattered cache configuration.
> Configuration scatteredConfig = ...;
> ConfigurationBuilder builder = new ConfigurationBuilder().read(scatteredConfig);
> builder.clustering().cacheMode(CacheMode.DIST_SYNC);
> Configuration distConfig = builder.build();
> This throws:
> {noformat}
> org.infinispan.commons.CacheConfigurationException: ISPN000468: Invalidation batch size configuration options applies only to scattered caches.
> at org.infinispan.configuration.cache.ClusteringConfigurationBuilder.validate(ClusteringConfigurationBuilder.java:135)
> at org.infinispan.configuration.cache.ConfigurationBuilder.validate(ConfigurationBuilder.java:226)
> at org.infinispan.configuration.cache.ConfigurationBuilder.build(ConfigurationBuilder.java:283)
> at org.infinispan.configuration.cache.ConfigurationBuilder.build(ConfigurationBuilder.java:273)
> {noformat}
> The validation logic of ClusteringConfigurationBuilder specifically requires the invalidationBatchSize attribute to be unset. However, there is no way to "unset" an attribute.
> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> The same issue applies to the biasAcquisition and biasLifespan attributes in 9.2.x.
> Any reason not to expose an public AttributeSet attributes() method to all configuration builders? It looks like the AttributeDefintions are all public, as are the Attribute and AttributeSet objects. That way any configuration builder can easily reset a configuration attribute to its default. WDYT?
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8724) HotRodRemoteStreamingIT hangs
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8724?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8724:
------------------------------------
Description:
This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
The surefire process waits forever for the two servers to stop.
One of the servers is waiting for the initial state transfer to complete:
{noformat}
"Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The other server is running a series of operations after the server stops, such
as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
{noformat}
"MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
- locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
- locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
For some reason, also in the other server, the CounterModule lifecycle is reacting to a
cache manager start event, and it's hanged as well:
{noformat}
"Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{noformat}
was:
This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
The surefire process waits forever for the two servers to stop.
One of the servers is waiting for the initial state transfer to complete:
{noformat}
"Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The other server is running a series of operations after the server stops, such
as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
{noformat}
"MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
- locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
- locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
For some reason, also in the other server, the CounterModule lifecycle is reacting to a
cache start event, and it's hanged as well:
{noformat}
"Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{noformat}
> HotRodRemoteStreamingIT hangs
> -----------------------------
>
> Key: ISPN-8724
> URL: https://issues.jboss.org/browse/ISPN-8724
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.0.CR1
> Reporter: Gustavo Fernandes
> Priority: Critical
> Attachments: s1, s2, test
>
>
> This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
> The surefire process waits forever for the two servers to stop.
> One of the servers is waiting for the initial state transfer to complete:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The other server is running a series of operations after the server stops, such
> as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
> MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
> {noformat}
> "MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
> at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
> at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
> - locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
> at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
> - locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> For some reason, also in the other server, the CounterModule lifecycle is reacting to a
> cache manager start event, and it's hanged as well:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8724) HotRodRemoteStreamingIT hangs
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8724?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8724:
------------------------------------
Status: Open (was: New)
> HotRodRemoteStreamingIT hangs
> -----------------------------
>
> Key: ISPN-8724
> URL: https://issues.jboss.org/browse/ISPN-8724
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.0.CR1
> Reporter: Gustavo Fernandes
> Priority: Critical
> Attachments: s1, s2, test
>
>
> This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
> The surefire process waits forever for the two servers to stop.
> One of the servers is waiting for the initial state transfer to complete:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The other server is running a series of operations after the server stops, such
> as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
> MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
> {noformat}
> "MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
> at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
> at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
> - locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
> at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
> - locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> For some reason, also in the other server, the CounterModule lifecycle is reacting to a
> cache start event, and it's hanged as well:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8724) HotRodRemoteStreamingIT hangs
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8724?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8724:
------------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/5699
> HotRodRemoteStreamingIT hangs
> -----------------------------
>
> Key: ISPN-8724
> URL: https://issues.jboss.org/browse/ISPN-8724
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.0.CR1
> Reporter: Gustavo Fernandes
> Priority: Critical
> Attachments: s1, s2, test
>
>
> This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
> The surefire process waits forever for the two servers to stop.
> One of the servers is waiting for the initial state transfer to complete:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The other server is running a series of operations after the server stops, such
> as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
> MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
> {noformat}
> "MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
> at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
> at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
> - locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
> at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
> - locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> For some reason, also in the other server, the CounterModule lifecycle is reacting to a
> cache start event, and it's hanged as well:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8706) HotRodMergeTest Timed out waiting for rebalancing to complete
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8706?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-8706:
-----------------------------------
[~ryanemerson] I've found {{HotRodMergeTest.testNewTopologySentAfterOverlappingMerge}} to be timing out as well, and this is a stack trace I've captured when waiting for it:
{code}
"transport-thread-HotRodMergeTest[DIST_SYNC]-NodeA-p309-t6" #1172 daemon prio=5 os_prio=0 tid=0x00007efdb8011000 nid=0x23bd waiting on condition [0x00007efd56c59000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c5383230> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1695)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1775)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.infinispan.conflict.impl.DefaultConflictManager$ReplicaSpliterator.tryAdvance(DefaultConflictManager.java:452)
at java.util.Spliterator.forEachRemaining(Spliterator.java:326)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.infinispan.conflict.impl.DefaultConflictManager.doResolveConflicts(DefaultConflictManager.java:257)
at org.infinispan.conflict.impl.DefaultConflictManager.resolveConflicts(DefaultConflictManager.java:244)
at org.infinispan.topology.ClusterCacheStatus.updateTopologiesAfterMerge(ClusterCacheStatus.java:193)
- locked <0x00000000c56202a8> (a org.infinispan.topology.ClusterCacheStatus)
at org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy.onPartitionMerge(PreferAvailabilityStrategy.java:218)
at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:597)
- locked <0x00000000c56202a8> (a org.infinispan.topology.ClusterCacheStatus)
at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$6(ClusterTopologyManagerImpl.java:519)
at org.infinispan.topology.ClusterTopologyManagerImpl$$Lambda$966/1179375334.run(Unknown Source)
at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
> HotRodMergeTest Timed out waiting for rebalancing to complete
> -------------------------------------------------------------
>
> Key: ISPN-8706
> URL: https://issues.jboss.org/browse/ISPN-8706
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 9.2.0.Alpha2
> Reporter: Ryan Emerson
> Assignee: Ryan Emerson
>
> The HotRodMergeTest is now consistently failing with the following error:
> {code}
> [OK: 321, KO: 1, SKIP: 0] Test failed: org.infinispan.server.hotrod.HotRodMergeTest.testNewTopologySentAfterCleanMerge[DIST_SYNC]
> java.lang.RuntimeException: Timed out waiting for rebalancing to complete on node HotRodMergeTest[DIST_SYNC]-NodeA-31095, expected member list is [HotRodMergeTest[DIST_SYNC]-NodeA-31095, HotRodMergeTest[DIST_SYNC]-NodeB-33866], current member list is [HotRodMergeTest[DIST_SYNC]-NodeA-31095]!
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:388)
> at org.infinispan.test.TestingUtil.waitForNoRebalance(TestingUtil.java:424)
> at org.infinispan.partitionhandling.BasePartitionHandlingTest$Partition.waitForPartitionToForm(BasePartitionHandlingTest.java:282)
> at org.infinispan.partitionhandling.BasePartitionHandlingTest$Partition.merge(BasePartitionHandlingTest.java:258)
> at org.infinispan.partitionhandling.BasePartitionHandlingTest$Partition.merge(BasePartitionHandlingTest.java:246)
> at org.infinispan.server.hotrod.HotRodMergeTest.testNewTopologySentAfterCleanMerge(HotRodMergeTest.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8724) HotRodRemoteStreamingIT hangs
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8724?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8724:
------------------------------------
Attachment: s1
s2
test
> HotRodRemoteStreamingIT hangs
> -----------------------------
>
> Key: ISPN-8724
> URL: https://issues.jboss.org/browse/ISPN-8724
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.0.CR1
> Reporter: Gustavo Fernandes
> Priority: Critical
> Attachments: s1, s2, test
>
>
> This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
> The surefire process waits forever for the two servers to stop.
> One of the servers is waiting for the initial state transfer to complete:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The other server is running a series of operations after the server stops, such
> as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
> MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
> {noformat}
> "MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
> at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
> at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
> at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
> - locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
> at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
> at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
> - locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> For some reason, also in the other server, the CounterModule lifecycle is reacting to a
> cache start event, and it's hanged as well:
> {noformat}
> "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
> at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
> at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8724) HotRodRemoteStreamingIT hangs
by Gustavo Fernandes (JIRA)
Gustavo Fernandes created ISPN-8724:
---------------------------------------
Summary: HotRodRemoteStreamingIT hangs
Key: ISPN-8724
URL: https://issues.jboss.org/browse/ISPN-8724
Project: Infinispan
Issue Type: Bug
Affects Versions: 9.2.0.CR1
Reporter: Gustavo Fernandes
Priority: Critical
This was observed during the execution of test {{HotRodRemoteStreamingIT}}, Line 499 on method stopServer().
The surefire process waits forever for the two servers to stop.
One of the servers is waiting for the initial state transfer to complete:
{noformat}
"Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The other server is running a series of operations after the server stops, such
as {{EmbeddedCacheManager#undefineConfiguration}}, this can be seen on 3 threads:
MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3
{noformat}
"MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
- locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
- locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
For some reason, also in the other server, the CounterModule lifecycle is reacting to a
cache start event, and it's hanged as well:
{noformat}
"Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
at org.infinispan.security.Security.doPrivileged(Security.java:76)
at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8400) Adjust merge policies for JDG Online Services
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-8400?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-8400.
--------------------------------
Fix Version/s: (was: 9.2.0.CR2)
Resolution: Done
> Adjust merge policies for JDG Online Services
> ---------------------------------------------
>
> Key: ISPN-8400
> URL: https://issues.jboss.org/browse/ISPN-8400
> Project: Infinispan
> Issue Type: Feature Request
> Components: Cloud
> Reporter: Sebastian Łaskawiec
> Assignee: Ryan Emerson
>
> Both Shared Memory and Caching Service require custom merge policies.
> In Caching Service we need to clear out all conflicted entries upon split brain. Shared Memory service might be a little bit more tricky and we might want to use some different strategy (but that needs to be checked out).
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0
[JBoss JIRA] (ISPN-8717) JCache.put calls RemoteCache in forceReturnValues configuration
by Alon Greenland (JIRA)
[ https://issues.jboss.org/browse/ISPN-8717?page=com.atlassian.jira.plugin.... ]
Alon Greenland updated ISPN-8717:
---------------------------------
Summary: JCache.put calls RemoteCache in forceReturnValues configuration (was: Cache.put calls RemoteCache in forceReturnValues configuration)
> JCache.put calls RemoteCache in forceReturnValues configuration
> ---------------------------------------------------------------
>
> Key: ISPN-8717
> URL: https://issues.jboss.org/browse/ISPN-8717
> Project: Infinispan
> Issue Type: Bug
> Components: JCache
> Affects Versions: 9.2.0.CR1, 9.1.4.Final
> Reporter: Alon Greenland
>
> In JCache Hot Rod client, Cache.put() operation, which doesn't return previous value, calls underlying Hot Rod RemoteCache in force return values mode.
> This is inefficient.
> In the code, I can see that in JCache class there is [RemoteCache<K, V> cacheWithoutReturnValue field|https://github.com/infinispan/infinispan/blob/master/jcache/remote/...], but it is never being used!
>
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
- 1
- 0