[JBoss JIRA] (ISPN-10206) Cache fails to stop due to unreleased persistence manager semaphore permits
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10206?page=com.atlassian.jira.plugin... ]
Will Burns updated ISPN-10206:
------------------------------
Git Pull Request: https://github.com/infinispan/infinispan/pull/6940, https://github.com/infinispan/infinispan/pull/6941 (was: https://github.com/infinispan/infinispan/pull/6940)
> Cache fails to stop due to unreleased persistence manager semaphore permits
> ---------------------------------------------------------------------------
>
> Key: ISPN-10206
> URL: https://issues.jboss.org/browse/ISPN-10206
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Loaders and Stores
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Environment: Windows 10 x64
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
> Wildfly 15.0.1
> Reporter: Paul Ferraro
> Assignee: Will Burns
> Priority: Major
> Fix For: 10.0.0.Beta4, 9.4.14.Final
>
>
> Wildfly 15.0.1 fails to stop on shutdown command due to Infinispan subsystem failing to stop it's persisted caches successfully due to a possible bug in Infinispan's PersistenceManagerImpl.publishEntries not releasing Semaphore permit after finishing, so that the stop method waits forever because it lacks one more permit.
> !third_semaphore.png|thumbnail!
> !forth_semaphore.png|thumbnail!
> Use following application to reproduce the issue:
> [^web-java-wildfly.zip]
> standalone Infinispan caches:
> {code}
> <cache-container name="testMetaData">
> <local-cache name="testCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> <local-cache name="testComponentCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testComponentCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> </cache-container>
> {code}
> {code}
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:03:17,748 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0236: Suspending server with no timeout.
> 11:03:17,750 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete
> 11:03:17,752 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown'
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb1
> 11:03:17,785 INFO [org.jboss.as.mail.extension] (MSC service thread 1-7) WFLYMAIL0002: Unbound mail session [java:/mail/myproject]
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb2
> 11:03:17,787 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb3
> 11:03:17,789 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 88) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:/ConnectionFactory
> 11:03:17,790 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
> 11:03:17,790 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb4
> 11:03:17,797 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0019: Stopped Driver service with driver-name = ojdbc.jar
> 11:03:17,800 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 101) WFLYUT0022: Unregistered web context: '/wildfly-helloworld' from server 'default-server'
> 11:03:17,821 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 101) MODCLUSTER000002: Initiating mod_cluster shutdown
> 11:03:17,823 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping
> 11:03:17,840 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0119: Unbinding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory
> 11:03:17,841 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-4) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA]
> 11:03:17,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> 11:03:17,842 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
> 11:03:17,846 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 101) WFLYCLINF0003: Stopped client-mappings cache from ejb container
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow AJP listener ajp suspending
> 11:03:17,870 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0028: Stopped deployment ojdbc.jar (runtime-name: ojdbc.jar) in 108ms
> 11:03:17,883 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment wildfly-helloworld.war (runtime-name: wildfly-helloworld.war) in 120ms
> 11:03:17,875 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 192.168.1.2:8009
> 11:03:17,874 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 192.168.1.2:8080
> 11:03:17,872 INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 104) AMQ151003: resource adaptor stopped
> 11:03:17,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 104) AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.6.3.jbossorg-00014 [7b985d04-761e-11e9-bb4e-0a0027000008] stopped, uptime 1 minute
> 11:03:17,933 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTPS listener default-ssl suspending
> 11:03:17,934 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTPS listener default-ssl stopped, was bound to 192.168.1.2:8443
> 11:03:17,935 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 2.0.15.Final stopping
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10206) Cache fails to stop due to unreleased persistence manager semaphore permits
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10206?page=com.atlassian.jira.plugin... ]
Will Burns updated ISPN-10206:
------------------------------
Fix Version/s: 10.0.0.Beta4
9.4.14.Final
> Cache fails to stop due to unreleased persistence manager semaphore permits
> ---------------------------------------------------------------------------
>
> Key: ISPN-10206
> URL: https://issues.jboss.org/browse/ISPN-10206
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Loaders and Stores
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Environment: Windows 10 x64
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
> Wildfly 15.0.1
> Reporter: Paul Ferraro
> Assignee: Will Burns
> Priority: Major
> Fix For: 10.0.0.Beta4, 9.4.14.Final
>
>
> Wildfly 15.0.1 fails to stop on shutdown command due to Infinispan subsystem failing to stop it's persisted caches successfully due to a possible bug in Infinispan's PersistenceManagerImpl.publishEntries not releasing Semaphore permit after finishing, so that the stop method waits forever because it lacks one more permit.
> !third_semaphore.png|thumbnail!
> !forth_semaphore.png|thumbnail!
> Use following application to reproduce the issue:
> [^web-java-wildfly.zip]
> standalone Infinispan caches:
> {code}
> <cache-container name="testMetaData">
> <local-cache name="testCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> <local-cache name="testComponentCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testComponentCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> </cache-container>
> {code}
> {code}
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:03:17,748 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0236: Suspending server with no timeout.
> 11:03:17,750 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete
> 11:03:17,752 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown'
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb1
> 11:03:17,785 INFO [org.jboss.as.mail.extension] (MSC service thread 1-7) WFLYMAIL0002: Unbound mail session [java:/mail/myproject]
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb2
> 11:03:17,787 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb3
> 11:03:17,789 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 88) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:/ConnectionFactory
> 11:03:17,790 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
> 11:03:17,790 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb4
> 11:03:17,797 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0019: Stopped Driver service with driver-name = ojdbc.jar
> 11:03:17,800 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 101) WFLYUT0022: Unregistered web context: '/wildfly-helloworld' from server 'default-server'
> 11:03:17,821 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 101) MODCLUSTER000002: Initiating mod_cluster shutdown
> 11:03:17,823 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping
> 11:03:17,840 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0119: Unbinding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory
> 11:03:17,841 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-4) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA]
> 11:03:17,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> 11:03:17,842 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
> 11:03:17,846 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 101) WFLYCLINF0003: Stopped client-mappings cache from ejb container
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow AJP listener ajp suspending
> 11:03:17,870 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0028: Stopped deployment ojdbc.jar (runtime-name: ojdbc.jar) in 108ms
> 11:03:17,883 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment wildfly-helloworld.war (runtime-name: wildfly-helloworld.war) in 120ms
> 11:03:17,875 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 192.168.1.2:8009
> 11:03:17,874 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 192.168.1.2:8080
> 11:03:17,872 INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 104) AMQ151003: resource adaptor stopped
> 11:03:17,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 104) AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.6.3.jbossorg-00014 [7b985d04-761e-11e9-bb4e-0a0027000008] stopped, uptime 1 minute
> 11:03:17,933 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTPS listener default-ssl suspending
> 11:03:17,934 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTPS listener default-ssl stopped, was bound to 192.168.1.2:8443
> 11:03:17,935 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 2.0.15.Final stopping
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10093) PersistenceManagerImpl stop deadlock with topology update
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10093?page=com.atlassian.jira.plugin... ]
Will Burns commented on ISPN-10093:
-----------------------------------
I am pretty sure this is the same issue as https://issues.jboss.org/browse/ISPN-10206
> PersistenceManagerImpl stop deadlock with topology update
> ---------------------------------------------------------
>
> Key: ISPN-10093
> URL: https://issues.jboss.org/browse/ISPN-10093
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 10.0.0.Beta3
> Reporter: Dan Berindei
> Assignee: Will Burns
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Beta4
>
> Attachments: threaddump.txt
>
>
> {{DistSyncStoreNotSharedTest.clearContent}} hanged in CI recently:
> {noformat}
> "testng-DistSyncStoreNotSharedTest" #16 prio=5 os_prio=0 cpu=11511.26ms elapsed=435.14s tid=0x00007fdb710b6000 nid=0x3222 waiting on condition [0x00007fdb352d3000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11/Native Method)
> - parking to wait for <0x00000000c8a22450> (a java.util.concurrent.Semaphore$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(java.base@11/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11/AbstractQueuedSynchronizer.java:885)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(java.base@11/AbstractQueuedSynchronizer.java:1009)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(java.base@11/AbstractQueuedSynchronizer.java:1324)
> at java.util.concurrent.Semaphore.acquireUninterruptibly(java.base@11/Semaphore.java:504)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.stop(PersistenceManagerImpl.java:222)
> at jdk.internal.reflect.GeneratedMethodAccessor72.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11/Method.java:566)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$237/0x0000000100661c40.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:181)
> at org.infinispan.factories.impl.BasicComponentRegistryImpl.performStop(BasicComponentRegistryImpl.java:601)
> at org.infinispan.factories.impl.BasicComponentRegistryImpl.stopWrapper(BasicComponentRegistryImpl.java:590)
> at org.infinispan.factories.impl.BasicComponentRegistryImpl.stop(BasicComponentRegistryImpl.java:461)
> at org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:431)
> at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:366)
> at org.infinispan.cache.impl.CacheImpl.performImmediateShutdown(CacheImpl.java:1160)
> at org.infinispan.cache.impl.CacheImpl.stop(CacheImpl.java:1125)
> at org.infinispan.cache.impl.AbstractDelegatingCache.stop(AbstractDelegatingCache.java:521)
> at org.infinispan.manager.DefaultCacheManager.terminate(DefaultCacheManager.java:747)
> at org.infinispan.manager.DefaultCacheManager.stopCaches(DefaultCacheManager.java:799)
> at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:775)
> at org.infinispan.test.TestingUtil.killCacheManagers(TestingUtil.java:846)
> at org.infinispan.test.MultipleCacheManagersTest.clearContent(MultipleCacheManagersTest.java:158)
> "persistence-thread-DistSyncStoreNotSharedTest-NodeB-p16432-t1" #53654 daemon prio=5 os_prio=0 cpu=1.26ms elapsed=301.93s tid=0x00007fdb3c3d8000 nid=0x8ef waiting on condition [0x00007fdb00055000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11/Native Method)
> - parking to wait for <0x00000000c8b1fb88> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(java.base@11/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11/AbstractQueuedSynchronizer.java:885)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(java.base@11/AbstractQueuedSynchronizer.java:1009)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(java.base@11/AbstractQueuedSynchronizer.java:1324)
> at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(java.base@11/ReentrantReadWriteLock.java:738)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.pollStoreAvailability(PersistenceManagerImpl.java:196)
> at org.infinispan.persistence.manager.PersistenceManagerImpl$$Lambda$492/0x00000001007fb440.run(Unknown Source)
> at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11/Executors.java:515)
> at java.util.concurrent.FutureTask.runAndReset(java.base@11/FutureTask.java:305)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@11/ScheduledThreadPoolExecutor.java:305)
> "transport-thread-DistSyncStoreNotSharedTest-NodeB-p16424-t5" #53646 daemon prio=5 os_prio=0 cpu=3.15ms elapsed=301.94s tid=0x00007fdb2007a000 nid=0x8e8 waiting on condition [0x00007fdb0b406000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11/Native Method)
> - parking to wait for <0x00000000c8d2abb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(java.base@11/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11/AbstractQueuedSynchronizer.java:2081)
> at io.reactivex.internal.operators.flowable.BlockingFlowableIterable$BlockingFlowableIterator.hasNext(BlockingFlowableIterable.java:94)
> at io.reactivex.Flowable.blockingForEach(Flowable.java:5682)
> at org.infinispan.statetransfer.StateConsumerImpl.removeStaleData(StateConsumerImpl.java:1011)
> at org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:453)
> at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:202)
> at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:58)
> at org.infinispan.statetransfer.StateTransferManagerImpl$1.updateConsistentHash(StateTransferManagerImpl.java:114)
> at org.infinispan.topology.LocalTopologyManagerImpl.resetLocalTopologyBeforeRebalance(LocalTopologyManagerImpl.java:437)
> at org.infinispan.topology.LocalTopologyManagerImpl.doHandleRebalance(LocalTopologyManagerImpl.java:519)
> - locked <0x00000000c8b30b30> (a org.infinispan.topology.LocalCacheStatus)
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleRebalance$3(LocalTopologyManagerImpl.java:484)
> at org.infinispan.topology.LocalTopologyManagerImpl$$Lambda$574/0x000000010089a040.run(Unknown Source)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:175){noformat}
> [Full thread dump|https://ci.infinispan.org/job/Infinispan/job/master/1133/artifact/core/]
> Somehow the producer thread for the transport-thread iteration is blocked, but without waiting for the persistence mutex. Maybe it's waiting for a topology? Not sure if it's relevant, but the last test to run was {{testClearWithFlag}}, so the data container was empty and the store had 5 entries.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10093) PersistenceManagerImpl stop deadlock with topology update
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10093?page=com.atlassian.jira.plugin... ]
Will Burns resolved ISPN-10093.
-------------------------------
Resolution: Duplicate Issue
> PersistenceManagerImpl stop deadlock with topology update
> ---------------------------------------------------------
>
> Key: ISPN-10093
> URL: https://issues.jboss.org/browse/ISPN-10093
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 10.0.0.Beta3
> Reporter: Dan Berindei
> Assignee: Will Burns
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Beta4
>
> Attachments: threaddump.txt
>
>
> {{DistSyncStoreNotSharedTest.clearContent}} hanged in CI recently:
> {noformat}
> "testng-DistSyncStoreNotSharedTest" #16 prio=5 os_prio=0 cpu=11511.26ms elapsed=435.14s tid=0x00007fdb710b6000 nid=0x3222 waiting on condition [0x00007fdb352d3000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11/Native Method)
> - parking to wait for <0x00000000c8a22450> (a java.util.concurrent.Semaphore$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(java.base@11/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11/AbstractQueuedSynchronizer.java:885)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(java.base@11/AbstractQueuedSynchronizer.java:1009)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(java.base@11/AbstractQueuedSynchronizer.java:1324)
> at java.util.concurrent.Semaphore.acquireUninterruptibly(java.base@11/Semaphore.java:504)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.stop(PersistenceManagerImpl.java:222)
> at jdk.internal.reflect.GeneratedMethodAccessor72.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11/Method.java:566)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions$$Lambda$237/0x0000000100661c40.run(Unknown Source)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:181)
> at org.infinispan.factories.impl.BasicComponentRegistryImpl.performStop(BasicComponentRegistryImpl.java:601)
> at org.infinispan.factories.impl.BasicComponentRegistryImpl.stopWrapper(BasicComponentRegistryImpl.java:590)
> at org.infinispan.factories.impl.BasicComponentRegistryImpl.stop(BasicComponentRegistryImpl.java:461)
> at org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:431)
> at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:366)
> at org.infinispan.cache.impl.CacheImpl.performImmediateShutdown(CacheImpl.java:1160)
> at org.infinispan.cache.impl.CacheImpl.stop(CacheImpl.java:1125)
> at org.infinispan.cache.impl.AbstractDelegatingCache.stop(AbstractDelegatingCache.java:521)
> at org.infinispan.manager.DefaultCacheManager.terminate(DefaultCacheManager.java:747)
> at org.infinispan.manager.DefaultCacheManager.stopCaches(DefaultCacheManager.java:799)
> at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:775)
> at org.infinispan.test.TestingUtil.killCacheManagers(TestingUtil.java:846)
> at org.infinispan.test.MultipleCacheManagersTest.clearContent(MultipleCacheManagersTest.java:158)
> "persistence-thread-DistSyncStoreNotSharedTest-NodeB-p16432-t1" #53654 daemon prio=5 os_prio=0 cpu=1.26ms elapsed=301.93s tid=0x00007fdb3c3d8000 nid=0x8ef waiting on condition [0x00007fdb00055000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11/Native Method)
> - parking to wait for <0x00000000c8b1fb88> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(java.base@11/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11/AbstractQueuedSynchronizer.java:885)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(java.base@11/AbstractQueuedSynchronizer.java:1009)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(java.base@11/AbstractQueuedSynchronizer.java:1324)
> at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(java.base@11/ReentrantReadWriteLock.java:738)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.pollStoreAvailability(PersistenceManagerImpl.java:196)
> at org.infinispan.persistence.manager.PersistenceManagerImpl$$Lambda$492/0x00000001007fb440.run(Unknown Source)
> at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11/Executors.java:515)
> at java.util.concurrent.FutureTask.runAndReset(java.base@11/FutureTask.java:305)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@11/ScheduledThreadPoolExecutor.java:305)
> "transport-thread-DistSyncStoreNotSharedTest-NodeB-p16424-t5" #53646 daemon prio=5 os_prio=0 cpu=3.15ms elapsed=301.94s tid=0x00007fdb2007a000 nid=0x8e8 waiting on condition [0x00007fdb0b406000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11/Native Method)
> - parking to wait for <0x00000000c8d2abb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(java.base@11/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11/AbstractQueuedSynchronizer.java:2081)
> at io.reactivex.internal.operators.flowable.BlockingFlowableIterable$BlockingFlowableIterator.hasNext(BlockingFlowableIterable.java:94)
> at io.reactivex.Flowable.blockingForEach(Flowable.java:5682)
> at org.infinispan.statetransfer.StateConsumerImpl.removeStaleData(StateConsumerImpl.java:1011)
> at org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:453)
> at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:202)
> at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:58)
> at org.infinispan.statetransfer.StateTransferManagerImpl$1.updateConsistentHash(StateTransferManagerImpl.java:114)
> at org.infinispan.topology.LocalTopologyManagerImpl.resetLocalTopologyBeforeRebalance(LocalTopologyManagerImpl.java:437)
> at org.infinispan.topology.LocalTopologyManagerImpl.doHandleRebalance(LocalTopologyManagerImpl.java:519)
> - locked <0x00000000c8b30b30> (a org.infinispan.topology.LocalCacheStatus)
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleRebalance$3(LocalTopologyManagerImpl.java:484)
> at org.infinispan.topology.LocalTopologyManagerImpl$$Lambda$574/0x000000010089a040.run(Unknown Source)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:175){noformat}
> [Full thread dump|https://ci.infinispan.org/job/Infinispan/job/master/1133/artifact/core/]
> Somehow the producer thread for the transport-thread iteration is blocked, but without waiting for the persistence mutex. Maybe it's waiting for a topology? Not sure if it's relevant, but the last test to run was {{testClearWithFlag}}, so the data container was empty and the store had 5 entries.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10206) Cache fails to stop due to unreleased persistence manager semaphore permits
by Will Burns (Jira)
[ https://issues.jboss.org/browse/ISPN-10206?page=com.atlassian.jira.plugin... ]
Will Burns updated ISPN-10206:
------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/6940
> Cache fails to stop due to unreleased persistence manager semaphore permits
> ---------------------------------------------------------------------------
>
> Key: ISPN-10206
> URL: https://issues.jboss.org/browse/ISPN-10206
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Loaders and Stores
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Environment: Windows 10 x64
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
> Wildfly 15.0.1
> Reporter: Paul Ferraro
> Assignee: Will Burns
> Priority: Major
>
> Wildfly 15.0.1 fails to stop on shutdown command due to Infinispan subsystem failing to stop it's persisted caches successfully due to a possible bug in Infinispan's PersistenceManagerImpl.publishEntries not releasing Semaphore permit after finishing, so that the stop method waits forever because it lacks one more permit.
> !third_semaphore.png|thumbnail!
> !forth_semaphore.png|thumbnail!
> Use following application to reproduce the issue:
> [^web-java-wildfly.zip]
> standalone Infinispan caches:
> {code}
> <cache-container name="testMetaData">
> <local-cache name="testCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> <local-cache name="testComponentCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testComponentCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> </cache-container>
> {code}
> {code}
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:03:17,748 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0236: Suspending server with no timeout.
> 11:03:17,750 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete
> 11:03:17,752 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown'
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb1
> 11:03:17,785 INFO [org.jboss.as.mail.extension] (MSC service thread 1-7) WFLYMAIL0002: Unbound mail session [java:/mail/myproject]
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb2
> 11:03:17,787 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb3
> 11:03:17,789 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 88) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:/ConnectionFactory
> 11:03:17,790 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
> 11:03:17,790 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb4
> 11:03:17,797 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0019: Stopped Driver service with driver-name = ojdbc.jar
> 11:03:17,800 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 101) WFLYUT0022: Unregistered web context: '/wildfly-helloworld' from server 'default-server'
> 11:03:17,821 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 101) MODCLUSTER000002: Initiating mod_cluster shutdown
> 11:03:17,823 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping
> 11:03:17,840 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0119: Unbinding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory
> 11:03:17,841 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-4) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA]
> 11:03:17,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> 11:03:17,842 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
> 11:03:17,846 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 101) WFLYCLINF0003: Stopped client-mappings cache from ejb container
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow AJP listener ajp suspending
> 11:03:17,870 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0028: Stopped deployment ojdbc.jar (runtime-name: ojdbc.jar) in 108ms
> 11:03:17,883 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment wildfly-helloworld.war (runtime-name: wildfly-helloworld.war) in 120ms
> 11:03:17,875 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 192.168.1.2:8009
> 11:03:17,874 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 192.168.1.2:8080
> 11:03:17,872 INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 104) AMQ151003: resource adaptor stopped
> 11:03:17,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 104) AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.6.3.jbossorg-00014 [7b985d04-761e-11e9-bb4e-0a0027000008] stopped, uptime 1 minute
> 11:03:17,933 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTPS listener default-ssl suspending
> 11:03:17,934 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTPS listener default-ssl stopped, was bound to 192.168.1.2:8443
> 11:03:17,935 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 2.0.15.Final stopping
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10206) Cache fails to stop due to unreleased persistence manager semaphore permits
by Paul Ferraro (Jira)
[ https://issues.jboss.org/browse/ISPN-10206?page=com.atlassian.jira.plugin... ]
Paul Ferraro commented on ISPN-10206:
-------------------------------------
Needs backporting to 9.4.x (at a minimum).
> Cache fails to stop due to unreleased persistence manager semaphore permits
> ---------------------------------------------------------------------------
>
> Key: ISPN-10206
> URL: https://issues.jboss.org/browse/ISPN-10206
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Loaders and Stores
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Environment: Windows 10 x64
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
> Wildfly 15.0.1
> Reporter: Paul Ferraro
> Assignee: Will Burns
> Priority: Major
>
> Wildfly 15.0.1 fails to stop on shutdown command due to Infinispan subsystem failing to stop it's persisted caches successfully due to a possible bug in Infinispan's PersistenceManagerImpl.publishEntries not releasing Semaphore permit after finishing, so that the stop method waits forever because it lacks one more permit.
> !third_semaphore.png|thumbnail!
> !forth_semaphore.png|thumbnail!
> Use following application to reproduce the issue:
> [^web-java-wildfly.zip]
> standalone Infinispan caches:
> {code}
> <cache-container name="testMetaData">
> <local-cache name="testCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> <local-cache name="testComponentCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testComponentCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> </cache-container>
> {code}
> {code}
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:03:17,748 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0236: Suspending server with no timeout.
> 11:03:17,750 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete
> 11:03:17,752 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown'
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb1
> 11:03:17,785 INFO [org.jboss.as.mail.extension] (MSC service thread 1-7) WFLYMAIL0002: Unbound mail session [java:/mail/myproject]
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb2
> 11:03:17,787 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb3
> 11:03:17,789 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 88) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:/ConnectionFactory
> 11:03:17,790 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
> 11:03:17,790 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb4
> 11:03:17,797 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0019: Stopped Driver service with driver-name = ojdbc.jar
> 11:03:17,800 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 101) WFLYUT0022: Unregistered web context: '/wildfly-helloworld' from server 'default-server'
> 11:03:17,821 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 101) MODCLUSTER000002: Initiating mod_cluster shutdown
> 11:03:17,823 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping
> 11:03:17,840 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0119: Unbinding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory
> 11:03:17,841 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-4) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA]
> 11:03:17,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> 11:03:17,842 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
> 11:03:17,846 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 101) WFLYCLINF0003: Stopped client-mappings cache from ejb container
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow AJP listener ajp suspending
> 11:03:17,870 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0028: Stopped deployment ojdbc.jar (runtime-name: ojdbc.jar) in 108ms
> 11:03:17,883 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment wildfly-helloworld.war (runtime-name: wildfly-helloworld.war) in 120ms
> 11:03:17,875 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 192.168.1.2:8009
> 11:03:17,874 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 192.168.1.2:8080
> 11:03:17,872 INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 104) AMQ151003: resource adaptor stopped
> 11:03:17,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 104) AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.6.3.jbossorg-00014 [7b985d04-761e-11e9-bb4e-0a0027000008] stopped, uptime 1 minute
> 11:03:17,933 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTPS listener default-ssl suspending
> 11:03:17,934 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTPS listener default-ssl stopped, was bound to 192.168.1.2:8443
> 11:03:17,935 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 2.0.15.Final stopping
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10206) Cache fails to stop due to unreleased persistence manager semaphore permits
by Paul Ferraro (Jira)
[ https://issues.jboss.org/browse/ISPN-10206?page=com.atlassian.jira.plugin... ]
Paul Ferraro updated ISPN-10206:
--------------------------------
Status: Open (was: Pull Request Sent)
Git Pull Request: (was: https://github.com/infinispan/infinispan/pull/6937, https://github.com/infinispan/infinispan/pull/6938)
> Cache fails to stop due to unreleased persistence manager semaphore permits
> ---------------------------------------------------------------------------
>
> Key: ISPN-10206
> URL: https://issues.jboss.org/browse/ISPN-10206
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Loaders and Stores
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Environment: Windows 10 x64
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
> Wildfly 15.0.1
> Reporter: Paul Ferraro
> Assignee: Paul Ferraro
> Priority: Major
>
> Wildfly 15.0.1 fails to stop on shutdown command due to Infinispan subsystem failing to stop it's persisted caches successfully due to a possible bug in Infinispan's PersistenceManagerImpl.publishEntries not releasing Semaphore permit after finishing, so that the stop method waits forever because it lacks one more permit.
> !third_semaphore.png|thumbnail!
> !forth_semaphore.png|thumbnail!
> Use following application to reproduce the issue:
> [^web-java-wildfly.zip]
> standalone Infinispan caches:
> {code}
> <cache-container name="testMetaData">
> <local-cache name="testCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> <local-cache name="testComponentCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testComponentCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> </cache-container>
> {code}
> {code}
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:03:17,748 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0236: Suspending server with no timeout.
> 11:03:17,750 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete
> 11:03:17,752 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown'
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb1
> 11:03:17,785 INFO [org.jboss.as.mail.extension] (MSC service thread 1-7) WFLYMAIL0002: Unbound mail session [java:/mail/myproject]
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb2
> 11:03:17,787 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb3
> 11:03:17,789 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 88) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:/ConnectionFactory
> 11:03:17,790 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
> 11:03:17,790 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb4
> 11:03:17,797 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0019: Stopped Driver service with driver-name = ojdbc.jar
> 11:03:17,800 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 101) WFLYUT0022: Unregistered web context: '/wildfly-helloworld' from server 'default-server'
> 11:03:17,821 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 101) MODCLUSTER000002: Initiating mod_cluster shutdown
> 11:03:17,823 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping
> 11:03:17,840 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0119: Unbinding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory
> 11:03:17,841 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-4) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA]
> 11:03:17,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> 11:03:17,842 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
> 11:03:17,846 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 101) WFLYCLINF0003: Stopped client-mappings cache from ejb container
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow AJP listener ajp suspending
> 11:03:17,870 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0028: Stopped deployment ojdbc.jar (runtime-name: ojdbc.jar) in 108ms
> 11:03:17,883 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment wildfly-helloworld.war (runtime-name: wildfly-helloworld.war) in 120ms
> 11:03:17,875 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 192.168.1.2:8009
> 11:03:17,874 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 192.168.1.2:8080
> 11:03:17,872 INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 104) AMQ151003: resource adaptor stopped
> 11:03:17,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 104) AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.6.3.jbossorg-00014 [7b985d04-761e-11e9-bb4e-0a0027000008] stopped, uptime 1 minute
> 11:03:17,933 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTPS listener default-ssl suspending
> 11:03:17,934 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTPS listener default-ssl stopped, was bound to 192.168.1.2:8443
> 11:03:17,935 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 2.0.15.Final stopping
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years
[JBoss JIRA] (ISPN-10206) Cache fails to stop due to unreleased persistence manager semaphore permits
by Paul Ferraro (Jira)
[ https://issues.jboss.org/browse/ISPN-10206?page=com.atlassian.jira.plugin... ]
Paul Ferraro reassigned ISPN-10206:
-----------------------------------
Assignee: Will Burns (was: Paul Ferraro)
> Cache fails to stop due to unreleased persistence manager semaphore permits
> ---------------------------------------------------------------------------
>
> Key: ISPN-10206
> URL: https://issues.jboss.org/browse/ISPN-10206
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Loaders and Stores
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Environment: Windows 10 x64
> java version "1.8.0_212"
> Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
> Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)
> Wildfly 15.0.1
> Reporter: Paul Ferraro
> Assignee: Will Burns
> Priority: Major
>
> Wildfly 15.0.1 fails to stop on shutdown command due to Infinispan subsystem failing to stop it's persisted caches successfully due to a possible bug in Infinispan's PersistenceManagerImpl.publishEntries not releasing Semaphore permit after finishing, so that the stop method waits forever because it lacks one more permit.
> !third_semaphore.png|thumbnail!
> !forth_semaphore.png|thumbnail!
> Use following application to reproduce the issue:
> [^web-java-wildfly.zip]
> standalone Infinispan caches:
> {code}
> <cache-container name="testMetaData">
> <local-cache name="testCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> <local-cache name="testComponentCache">
> <locking concurrency-level="3"/>
> <transaction locking="OPTIMISTIC" mode="FULL_XA"/>
> <object-memory size="1000000"/>
> <file-store path="testCH/testComponentCache" passivation="false" preload="false" purge="false"/>
> </local-cache>
> </cache-container>
> {code}
> {code}
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:02:00,103 INFO [stdout] (default task-1) bla blanull
> 11:03:17,748 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0236: Suspending server with no timeout.
> 11:03:17,750 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete
> 11:03:17,752 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown'
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb1
> 11:03:17,785 INFO [org.jboss.as.mail.extension] (MSC service thread 1-7) WFLYMAIL0002: Unbound mail session [java:/mail/myproject]
> 11:03:17,785 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb2
> 11:03:17,787 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb3
> 11:03:17,789 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 88) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:/ConnectionFactory
> 11:03:17,790 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
> 11:03:17,790 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-8) WFLYJCA0099: Unbound non-transactional data source: java:jboss/datasources/myprojdb4
> 11:03:17,797 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0019: Stopped Driver service with driver-name = ojdbc.jar
> 11:03:17,800 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 101) WFLYUT0022: Unregistered web context: '/wildfly-helloworld' from server 'default-server'
> 11:03:17,821 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 101) MODCLUSTER000002: Initiating mod_cluster shutdown
> 11:03:17,823 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping
> 11:03:17,840 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0119: Unbinding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory
> 11:03:17,841 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-4) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA]
> 11:03:17,842 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
> 11:03:17,842 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
> 11:03:17,846 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 101) WFLYCLINF0003: Stopped client-mappings cache from ejb container
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
> 11:03:17,864 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow AJP listener ajp suspending
> 11:03:17,870 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0028: Stopped deployment ojdbc.jar (runtime-name: ojdbc.jar) in 108ms
> 11:03:17,883 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment wildfly-helloworld.war (runtime-name: wildfly-helloworld.war) in 120ms
> 11:03:17,875 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 192.168.1.2:8009
> 11:03:17,874 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 192.168.1.2:8080
> 11:03:17,872 INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 104) AMQ151003: resource adaptor stopped
> 11:03:17,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 104) AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.6.3.jbossorg-00014 [7b985d04-761e-11e9-bb4e-0a0027000008] stopped, uptime 1 minute
> 11:03:17,933 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTPS listener default-ssl suspending
> 11:03:17,934 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTPS listener default-ssl stopped, was bound to 192.168.1.2:8443
> 11:03:17,935 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 2.0.15.Final stopping
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
7 years