[ https://issues.jboss.org/browse/ISPN-1754?page=com.atlassian.jira.plugin.... ] Michal Linhard updated ISPN-1754: --------------------------------- Summary: IllegalStateException during cluster shutdown (was: IllegalStateException in surviving nodes during node crash in cluster) Affects Version/s: 5.1.0.CR4 Description: I get these during stress/failover test while shutting down the servers {code} 05:44:54,051 ERROR [org.infinispan.transaction.TransactionCoordinator] ISPN000098: Exception while rollback: java.lang.IllegalStateException: Cache 'testCache' is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so i.. at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:100) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:76) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.commands.AbstractVisitor.visitRollbackCommand(AbstractVisitor.java:117) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.commands.tx.RollbackCommand.acceptVisitor(RollbackCommand.java:54) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.transaction.TransactionCoordinator.rollbackInternal(TransactionCoordinator.java:208) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.transaction.TransactionCoordinator.rollback(TransactionCoordinator.java:176) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:89) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:117) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:403) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:104) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:159) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1162) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:119) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:951) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.CacheImpl.put(CacheImpl.java:654) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.DecoratedCache.put(DecoratedCache.java:179) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.AbstractDelegatingCache.put(AbstractDelegatingCache.java:119) [infinispan-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:187) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.server.core.AbstractProtocolDecoder.decodeValue(AbstractProtocolDecoder.scala:141) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:71) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:44) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.jboss.netty.handler.codec.replay.CustomReplayingDecoder.callDecode(CustomReplayingDecoder.java:250) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.jboss.netty.handler.codec.replay.CustomReplayingDecoder.messageReceived(CustomReplayingDecoder.java:223) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.infinispan.server.core.AbstractProtocolDecoder.messageReceived(AbstractProtocolDecoder.scala:351) [infinispan-server-core-5.1.0.CR4.jar:5.1.0.CR4] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282) [netty-3.2.6.Final.jar:] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202) [netty-3.2.6.Final.jar:] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.2.6.Final.jar:] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44) [netty-3.2.6.Final.jar:] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_29] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_29] at java.lang.Thread.run(Thread.java:662) [:1.6.0_29] {code} Component/s: Distributed Cache -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1754?page=com.atlassian.jira.plugin.... ] Michal Linhard commented on ISPN-1754: -------------------------------------- I don't consider this a serious problem, my tests are ignoring it (they ignore all exceptions happening in shutdown phase) I wouldn't call it IllegalStateException though, since it's nothing extraordinary. I just thought all IllegalStateException should be reported :-) just for the record. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1754?page=com.atlassian.jira.plugin.... ] Work on ISPN-1754 started by Manik Surtani. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1754?page=com.atlassian.jira.plugin.... ] Dan Berindei commented on ISPN-1754: ------------------------------------ @Michal, it is extraordinary at a low level (can't execute any command on a cache that's shutting down) but not at the high level (the server is also shutting down, so it doesn't care about all the commands that aren't going to execute). @Manik, sounds good. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1754?page=com.atlassian.jira.plugin.... ] Dan Berindei reopened ISPN-1754: -------------------------------- There was a more serious issue with StateTransferLock.waitForStateTransferToEnd(), when it failed to re-acquire the lock we would still try to release it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ISPN-1754?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-1754: ------------------------------- Status: Pull Request Sent (was: Reopened) Git Pull Request: https://github.com/infinispan/infinispan/pull/862, https://github.com/infinispan/infinispan/pull/875 (was: https://github.com/infinispan/infinispan/pull/862) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira