[JBoss JIRA] (JGRP-1984) NPE while creating a dynamic FORK channel
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/JGRP-1984?page=com.atlassian.jira.plugin.... ]
Steven Hawkins commented on JGRP-1984:
--------------------------------------
This was addressed by the recent commit: https://github.com/belaban/JGroups/commit/673d6593e9b41a9df95582987581738...
It looks like this means that the create if absent logic won't work without the fix, so the FORK protocol must already be in the configuration as a workaround.
> NPE while creating a dynamic FORK channel
> -----------------------------------------
>
> Key: JGRP-1984
> URL: https://issues.jboss.org/browse/JGRP-1984
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.6.4
> Reporter: Ramesh Reddy
> Assignee: Bela Ban
> Fix For: 3.6.7
>
> Attachments: RpcForkDispatcherTest.java, tcp-shared.xml
>
>
> I am trying to create a ForkChannel based on the example in docs with code, when FORK protocol not present in the main stack
> {code}
> channel=new JChannel("tcp.xml"));
> channel.connect("RpcDispatcherTestGroup");
> ForkChannel forkChannel = new ForkChannel(channel, "test-rpc", name, true, ProtocolStack.BELOW, FRAG2.class);
> forkChannel.connect("test-rpc");
> {code}
> this ends with
> {code}
> Exception in thread "main" java.lang.NullPointerException
> at org.jgroups.protocols.FORK.createForkStack(FORK.java:199)
> at org.jgroups.fork.ForkChannel.<init>(ForkChannel.java:79)
> at org.teiid.systemmodel.RpcDispatcherTest.start(RpcDispatcherTest.java:32)
> at org.teiid.systemmodel.RpcDispatcherTest.main(RpcDispatcherTest.java:54
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months
[JBoss JIRA] (WFLY-5441) Received unsolicited state from node X for segment Y of cache Z
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5441?page=com.atlassian.jira.plugin.... ]
Michal Vinkler updated WFLY-5441:
---------------------------------
Affects Version/s: 10.0.0.CR5
> Received unsolicited state from node X for segment Y of cache Z
> ---------------------------------------------------------------
>
> Key: WFLY-5441
> URL: https://issues.jboss.org/browse/WFLY-5441
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.CR5
> Reporter: Richard Janík
> Assignee: Paul Ferraro
> Priority: Minor
>
> Hi,
> we're occasionally seeing these warnings in the server logs:
> {code}
> [JBossINF] [0m[33m08:20:45,364 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p11-t36) Received unsolicited state from node perf20 for segment 48 of cache clusterbench-ee7.ear/clusterbench-ee7-ejb.jar
> ...
> [JBossINF] [0m[33m08:21:02,793 WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p11-t22) Discarding received cache entries for segment 48 of cache dist because they do not belong to this node.
> {code}
> The warnings seem to be harmless.
> This is the equivalent of https://bugzilla.redhat.com/show_bug.cgi?id=923789 which has been (with EAP6) followed by IllegalStateExceptions and has later been closed as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=922699. However, no IllegalStateExceptions occur with EAP7 and thus these two messages seem to be unrelated.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months
[JBoss JIRA] (WFLY-5770) TimeoutException on CacheRegistry.close()
by Radoslav Husar (JIRA)
[ https://issues.jboss.org/browse/WFLY-5770?page=com.atlassian.jira.plugin.... ]
Radoslav Husar updated WFLY-5770:
---------------------------------
Summary: TimeoutException on CacheRegistry.close() (was: Investigate TimeoutException on CacheRegistry.close())
> TimeoutException on CacheRegistry.close()
> -----------------------------------------
>
> Key: WFLY-5770
> URL: https://issues.jboss.org/browse/WFLY-5770
> Project: WildFly
> Issue Type: Task
> Components: Clustering
> Affects Versions: 10.0.0.CR4
> Reporter: Radoslav Husar
> Assignee: Richard Achmatowicz
>
> Just happened on node shutdown when testing locally.
> {noformat}
> 19:22:21,114 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0004: Undertow 1.3.7.Final stopping
> 19:22:21,114 ERROR [stderr] (ServerService Thread Pool -- 82) Exception in thread "ServerService Thread Pool -- 82" org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on node2 in behalf of transaction GlobalTransaction:<node2>:4223:local. Current owner GlobalTransaction:<node2>:4217:local.
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.timeout(DefaultPendingLockManager.java:224)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.checkForPendingLock(DefaultPendingLockManager.java:196)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.awaitPendingTransactionsForKey(DefaultPendingLockManager.java:115)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:190)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataWriteCommand(PessimisticLockingInterceptor.java:121)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitRemoveCommand(AbstractLockingInterceptor.java:71)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:366)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:230)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:304)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:286)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:123)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1634)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:558)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:549)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:452)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:296)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:101)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.service.concurrent.StampedLockServiceExecutor.close(StampedLockServiceExecutor.java:55)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.close(CacheRegistry.java:96)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistryFactory$1.close(CacheRegistryFactory.java:56)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.RegistryBuilder.stop(RegistryBuilder.java:88)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.service.AsynchronousServiceBuilder$2.run(AsynchronousServiceBuilder.java:130)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at java.lang.Thread.run(Thread.java:745)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) Suppressed: org.infinispan.commons.CacheException: javax.transaction.RollbackException: Transaction marked as rollback only.
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.ee.infinispan.ActiveTransactionBatch.close(ActiveTransactionBatch.java:50)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:102)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) ... 9 more
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) Caused by: javax.transaction.RollbackException: Transaction marked as rollback only.
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.transaction.tm.DummyTransaction.setRollbackOnly(DummyTransaction.java:146)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:374)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:230)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:304)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:286)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:123)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1634)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:558)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:549)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:452)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:296)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:101)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) ... 9 more
> 19:22:31,124 WARN [org.infinispan.transaction.impl.TransactionTable] (ServerService Thread Pool -- 88) ISPN000100: Stopping, but there are 1 local transactions and 0 remote transactions that did not finish in time.
> 19:22:31,125 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 88) WFLYCLINF0003: Stopped routing cache from web container
> 19:22:31,128 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000080: Disconnecting JGroups channel web
> 19:22:31,128 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000082: Stopping the RpcDispatcher for channel web
> 19:22:31,141 INFO [org.jboss.as] (MSC service thread 1-5) WFLYSRV0050: WildFly Full 10.0.0.CR5-SNAPSHOT (WildFly Core 2.0.3.Final) stopped in 25086ms
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months
[JBoss JIRA] (WFLY-5770) Investigate TimeoutException on CacheRegistry.close()
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5770?page=com.atlassian.jira.plugin.... ]
Michal Vinkler commented on WFLY-5770:
--------------------------------------
Also seen in failover testing, see
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...
> Investigate TimeoutException on CacheRegistry.close()
> -----------------------------------------------------
>
> Key: WFLY-5770
> URL: https://issues.jboss.org/browse/WFLY-5770
> Project: WildFly
> Issue Type: Task
> Components: Clustering
> Affects Versions: 10.0.0.CR4
> Reporter: Radoslav Husar
> Assignee: Richard Achmatowicz
>
> Just happened on node shutdown when testing locally.
> {noformat}
> 19:22:21,114 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0004: Undertow 1.3.7.Final stopping
> 19:22:21,114 ERROR [stderr] (ServerService Thread Pool -- 82) Exception in thread "ServerService Thread Pool -- 82" org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on node2 in behalf of transaction GlobalTransaction:<node2>:4223:local. Current owner GlobalTransaction:<node2>:4217:local.
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.timeout(DefaultPendingLockManager.java:224)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.checkForPendingLock(DefaultPendingLockManager.java:196)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.awaitPendingTransactionsForKey(DefaultPendingLockManager.java:115)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:190)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataWriteCommand(PessimisticLockingInterceptor.java:121)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitRemoveCommand(AbstractLockingInterceptor.java:71)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:366)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:230)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:304)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:286)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:123)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1634)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:558)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:549)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:452)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:296)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:101)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.service.concurrent.StampedLockServiceExecutor.close(StampedLockServiceExecutor.java:55)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.close(CacheRegistry.java:96)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistryFactory$1.close(CacheRegistryFactory.java:56)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.RegistryBuilder.stop(RegistryBuilder.java:88)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.service.AsynchronousServiceBuilder$2.run(AsynchronousServiceBuilder.java:130)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at java.lang.Thread.run(Thread.java:745)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) Suppressed: org.infinispan.commons.CacheException: javax.transaction.RollbackException: Transaction marked as rollback only.
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.ee.infinispan.ActiveTransactionBatch.close(ActiveTransactionBatch.java:50)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:102)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) ... 9 more
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) Caused by: javax.transaction.RollbackException: Transaction marked as rollback only.
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.transaction.tm.DummyTransaction.setRollbackOnly(DummyTransaction.java:146)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:374)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:230)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:304)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:286)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:123)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1634)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:558)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:549)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:452)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:296)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:101)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) ... 9 more
> 19:22:31,124 WARN [org.infinispan.transaction.impl.TransactionTable] (ServerService Thread Pool -- 88) ISPN000100: Stopping, but there are 1 local transactions and 0 remote transactions that did not finish in time.
> 19:22:31,125 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 88) WFLYCLINF0003: Stopped routing cache from web container
> 19:22:31,128 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000080: Disconnecting JGroups channel web
> 19:22:31,128 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000082: Stopping the RpcDispatcher for channel web
> 19:22:31,141 INFO [org.jboss.as] (MSC service thread 1-5) WFLYSRV0050: WildFly Full 10.0.0.CR5-SNAPSHOT (WildFly Core 2.0.3.Final) stopped in 25086ms
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months
[JBoss JIRA] (WFLY-5770) Investigate TimeoutException on CacheRegistry.close()
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5770?page=com.atlassian.jira.plugin.... ]
Michal Vinkler updated WFLY-5770:
---------------------------------
Comment: was deleted
(was: Also seen in failover testing, see
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...)
> Investigate TimeoutException on CacheRegistry.close()
> -----------------------------------------------------
>
> Key: WFLY-5770
> URL: https://issues.jboss.org/browse/WFLY-5770
> Project: WildFly
> Issue Type: Task
> Components: Clustering
> Affects Versions: 10.0.0.CR4
> Reporter: Radoslav Husar
> Assignee: Richard Achmatowicz
>
> Just happened on node shutdown when testing locally.
> {noformat}
> 19:22:21,114 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0004: Undertow 1.3.7.Final stopping
> 19:22:21,114 ERROR [stderr] (ServerService Thread Pool -- 82) Exception in thread "ServerService Thread Pool -- 82" org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on node2 in behalf of transaction GlobalTransaction:<node2>:4223:local. Current owner GlobalTransaction:<node2>:4217:local.
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.timeout(DefaultPendingLockManager.java:224)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.checkForPendingLock(DefaultPendingLockManager.java:196)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.awaitPendingTransactionsForKey(DefaultPendingLockManager.java:115)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:190)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataWriteCommand(PessimisticLockingInterceptor.java:121)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitRemoveCommand(AbstractLockingInterceptor.java:71)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,115 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:366)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:230)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:304)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:286)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:123)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,116 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1634)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:558)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:549)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:452)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:296)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:101)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.service.concurrent.StampedLockServiceExecutor.close(StampedLockServiceExecutor.java:55)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.close(CacheRegistry.java:96)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistryFactory$1.close(CacheRegistryFactory.java:56)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.RegistryBuilder.stop(RegistryBuilder.java:88)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.service.AsynchronousServiceBuilder$2.run(AsynchronousServiceBuilder.java:130)
> 19:22:21,117 ERROR [stderr] (ServerService Thread Pool -- 82) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at java.lang.Thread.run(Thread.java:745)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) Suppressed: org.infinispan.commons.CacheException: javax.transaction.RollbackException: Transaction marked as rollback only.
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.ee.infinispan.ActiveTransactionBatch.close(ActiveTransactionBatch.java:50)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:102)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) ... 9 more
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) Caused by: javax.transaction.RollbackException: Transaction marked as rollback only.
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.transaction.tm.DummyTransaction.setRollbackOnly(DummyTransaction.java:146)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:374)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:230)
> 19:22:21,118 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:304)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:286)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.statetransfer.StateTransferInterceptor.visitRemoveCommand(StateTransferInterceptor.java:123)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:48)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:59)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1634)
> 19:22:21,119 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.removeInternal(CacheImpl.java:558)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:549)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:452)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:296)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) at org.wildfly.clustering.server.registry.CacheRegistry.lambda$close$0(CacheRegistry.java:101)
> 19:22:21,120 ERROR [stderr] (ServerService Thread Pool -- 82) ... 9 more
> 19:22:31,124 WARN [org.infinispan.transaction.impl.TransactionTable] (ServerService Thread Pool -- 88) ISPN000100: Stopping, but there are 1 local transactions and 0 remote transactions that did not finish in time.
> 19:22:31,125 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 88) WFLYCLINF0003: Stopped routing cache from web container
> 19:22:31,128 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000080: Disconnecting JGroups channel web
> 19:22:31,128 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000082: Stopping the RpcDispatcher for channel web
> 19:22:31,141 INFO [org.jboss.as] (MSC service thread 1-5) WFLYSRV0050: WildFly Full 10.0.0.CR5-SNAPSHOT (WildFly Core 2.0.3.Final) stopped in 25086ms
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months
[JBoss JIRA] (WFLY-5785) EJB lookup fails with "No cluster context available" in failover tests
by Radoslav Husar (JIRA)
[ https://issues.jboss.org/browse/WFLY-5785?page=com.atlassian.jira.plugin.... ]
Radoslav Husar reassigned WFLY-5785:
------------------------------------
Assignee: Richard Achmatowicz (was: Paul Ferraro)
> EJB lookup fails with "No cluster context available" in failover tests
> ----------------------------------------------------------------------
>
> Key: WFLY-5785
> URL: https://issues.jboss.org/browse/WFLY-5785
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.CR5
> Reporter: Michal Vinkler
> Assignee: Richard Achmatowicz
>
> Seen in our failover tests for remote stateful EJBs - scenarios:
> ejb-ejbremote-shutdown-dist-async
> ejb-ejbremote-shutdown-dist-sync
> ejb-ejbremote-undeploy-dist-async
> After failing a node, occasionally EJB lookup starts failing - client starts logging these error messages:
> {code}
> 2015/12/03 04:46:47:078 EST [ERROR][Runner - 9] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error getting response. <java.lang.IllegalStateException: EJBCLIENT000029: No cluster context available for cluster named ejb>
> java.lang.IllegalStateException: EJBCLIENT000029: No cluster context available for cluster named ejb
> at org.jboss.ejb.client.EJBClientContext.requireClusterEJBReceiverContext(EJBClientContext.java:1063)
> at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:84)
> at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)
> at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:255)
> at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:200)
> at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)
> at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
> at com.sun.proxy.$Proxy2.getSerialAndIncrement(Unknown Source)
> at org.jboss.smartfrog.clustering.ejb3.StatefulSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(StatefulSBProcessorFactoryImpl.java:84)
> at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
> at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It stops logging these messages only after the failed node is restarted and joins the cluster again.
> Link (this job was configured to use only 100 sessions in order to keep the log size small)
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months
[JBoss JIRA] (WFLY-5727) After day or two Wildfly hangs and consumes 100% CPU
by Stuart Douglas (JIRA)
[ https://issues.jboss.org/browse/WFLY-5727?page=com.atlassian.jira.plugin.... ]
Stuart Douglas resolved WFLY-5727.
----------------------------------
Resolution: Out of Date
A read timeout should stop it, however if it hits this bug it may still have periods where it hits 100% CPU till the timeout occurs.
> After day or two Wildfly hangs and consumes 100% CPU
> ----------------------------------------------------
>
> Key: WFLY-5727
> URL: https://issues.jboss.org/browse/WFLY-5727
> Project: WildFly
> Issue Type: Bug
> Components: IO, Web (Undertow)
> Affects Versions: 8.2.0.Final, 9.0.1.Final
> Environment: Wildfly 9.01 Final, Wildfly 8.2
> CentOS Linux release 7.1.1503 (Core)
> java version "1.8.0_66"
> Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
> Reporter: Petri Tuomaala
> Assignee: Stuart Douglas
> Priority: Blocker
> Labels: Linux, io, ssl, undertow, wildfly, xnio
> Fix For: 9.x.x TBD
>
> Attachments: jstack.22263.102919.280420771, jstack.22263.102925.032513958, jstack.22263.102930.660792783, jstack.22263.102936.697001192, jstack.22263.102942.535009068, jstack.22263.102948.169314576, jstack.22263.102954.185475598, jstack.22263.102959.848031929, jstack.22263.103005.599986837, jstack.22263.103011.261542540, monitor.png, pid.png
>
>
> First disclaimer, this looks pointing to undertow/xnio but I am not sure for the root cause so I'm filing this for Wildfly.
> After a day or two Wildfly just hangs and consumes the cpu. We have noticed that the site traffic itself has no impact on this: it has happened during peak traffic and also almost in idle situations.
> We are handling SSL in Wildfly and example this https://issues.jboss.org/browse/UNDERTOW-282 looks similar to this but it has been resolved over year ago so it should be in Wildfly 9.01.
> Our SSL conf in standalone.xml:
> <https-listener name="https" socket-binding="https" security-realm="UndertowRealm" enabled-cipher-suites="TLS_DHE_DSS_WITH_AES_128_CBC_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA" max-parameters="5000"/>
> I have narrowed down the thread that hangs. Or at least tried, my first time to study thread dumps. PID for thread that is jammed is 22599 => 0x5847 as hex and it points to the thread end of this description. I attach the actual thread dumps to this ticket. Any workaround for this is preciated, we currently need to reboot the servers when the Wildfly hangs (almost every nigth).
> "default task-20" #285 prio=5 os_prio=0 tid=0x00007f862c328800 nid=0x5847 runnable [0x00007f85f5ad6000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:192)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
> - locked <0x00000006f6065210> (a java.lang.Object)
> at org.xnio.nio.NioSocketConduit.read(NioSocketConduit.java:282)
> at org.xnio.ssl.JsseSslConduitEngine.handleUnwrapResult(JsseSslConduitEngine.java:737)
> - locked <0x00000006f606b708> (a org.xnio.ByteBufferSlicePool$PooledByteBuffer)
> at org.xnio.ssl.JsseSslConduitEngine.unwrap(JsseSslConduitEngine.java:620)
> - locked <0x00000006f606b708> (a org.xnio.ByteBufferSlicePool$PooledByteBuffer)
> at org.xnio.ssl.JsseSslConduitEngine.unwrap(JsseSslConduitEngine.java:574)
> at org.xnio.ssl.JsseSslStreamSourceConduit.read(JsseSslStreamSourceConduit.java:89)
> at org.xnio.conduits.AbstractStreamSourceConduit.read(AbstractStreamSourceConduit.java:51)
> at io.undertow.conduits.ReadDataStreamSourceConduit.read(ReadDataStreamSourceConduit.java:67)
> at io.undertow.conduits.FixedLengthStreamSourceConduit.read(FixedLengthStreamSourceConduit.java:249)
> at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127)
> at io.undertow.channels.DetachableStreamSourceChannel.read(DetachableStreamSourceChannel.java:209)
> at io.undertow.server.HttpServerExchange$ReadDispatchChannel.read(HttpServerExchange.java:2136)
> at io.undertow.server.handlers.form.FormEncodedDataDefinition$FormEncodedDataParser.doParse(FormEncodedDataDefinition.java:133)
> at io.undertow.server.handlers.form.FormEncodedDataDefinition$FormEncodedDataParser.parseBlocking(FormEncodedDataDefinition.java:251)
> at io.undertow.servlet.spec.HttpServletRequestImpl.parseFormData(HttpServletRequestImpl.java:753)
> at io.undertow.servlet.spec.HttpServletRequestImpl.getParameter(HttpServletRequestImpl.java:627)
> at org.apache.myfaces.context.servlet.RequestParameterMap.getAttribute(RequestParameterMap.java:45)
> at org.apache.myfaces.context.servlet.RequestParameterMap.getAttribute(RequestParameterMap.java:33)
> at org.apache.myfaces.util.AbstractAttributeMap.containsKey(AbstractAttributeMap.java:62)
> at org.apache.myfaces.renderkit.html.HtmlResponseStateManager.isPostback(HtmlResponseStateManager.java:303)
> at org.apache.myfaces.lifecycle.DefaultRestoreViewSupport.isPostback(DefaultRestoreViewSupport.java:242)
> at org.apache.myfaces.lifecycle.RestoreViewExecutor.execute(RestoreViewExecutor.java:120)
> at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:170)
> at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:117)
> at javax.faces.webapp.FacesServlet.service(FacesServlet.java:197)
> at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)
> at fi.elbit.skills.loginview.DefaultsFilter.doFilter(DefaultsFilter.java:117)
> at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
> at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:357)
> at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
> at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
> at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)
> at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51)
> at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56)
> at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
> at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Locked ownable synchronizers:
> - <0x00000006f20bb6c8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 7 months