[jboss-jira] [JBoss JIRA] (AS7-4144) Failures in cluster cause significat disruption (IllegalStateException, "Rolling back to cache view Y, but last committed view is Z", TOs)
Radoslav Husar (JIRA)
jira-events at lists.jboss.org
Fri Apr 13 09:20:48 EDT 2012
[ https://issues.jboss.org/browse/AS7-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radoslav Husar closed AS7-4144.
-------------------------------
Resolution: Done
This has been fixed! Nice.
This was fixed by one of the large Paul's PR but I cant find it now (https://github.com/jbossas/jboss-as/pulls/pferraro).
Closing.
> Failures in cluster cause significat disruption (IllegalStateException, "Rolling back to cache view Y, but last committed view is Z", TOs)
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: AS7-4144
> URL: https://issues.jboss.org/browse/AS7-4144
> Project: Application Server 7
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 7.1.1.Final
> Environment: Linux
> Reporter: Radoslav Husar
> Assignee: Paul Ferraro
> Priority: Critical
> Labels: ISPN000071, ISPN000109, ISPN000152, ISPN000164, ISPN000170, eap6_need_triage, failover_testing
> Fix For: 7.1.2.Final-redhat1
>
>
> Negative failover scenarios (server crashing) cause significant distruptions in the cluster.
> {noformat}
> [JBossINF] 17:19:13,423 INFO [org.jboss.weld.deployer] (MSC service thread 1-6) JBAS016002: Processing weld deployment clusterbench-ee6-web.war
> [JBossINF] 17:19:13,433 INFO [org.jboss.weld.deployer] (MSC service thread 1-15) JBAS016005: Starting Services for CDI deployment: clusterbench-ee6.ear
> [JBossINF] 17:19:13,527 INFO [org.jboss.weld.Version] (MSC service thread 1-15) WELD-000900 2012-02-22 16:46
> [JBossINF] 17:19:13,552 INFO [org.jboss.weld.deployer] (MSC service thread 1-12) JBAS016008: Starting weld service for deployment clusterbench-ee6.ear
> [JBossINF] 17:19:13,865 INFO [stdout] (pool-13-thread-1)
> [JBossINF] 17:19:13,865 INFO [stdout] (pool-13-thread-1) -------------------------------------------------------------------
> [JBossINF] 17:19:13,865 INFO [stdout] (pool-13-thread-1) GMS: address=perf18/web, cluster=web, physical address=10.16.90.54:55200
> [JBossINF] 17:19:13,866 INFO [stdout] (pool-13-thread-1) -------------------------------------------------------------------
> [JBossINF] 17:19:13,869 INFO [stdout] (pool-19-thread-1)
> [JBossINF] 17:19:13,869 INFO [stdout] (pool-19-thread-1) -------------------------------------------------------------------
> [JBossINF] 17:19:13,869 INFO [stdout] (pool-19-thread-1) GMS: address=perf18/ejb, cluster=ejb, physical address=10.16.90.54:55200
> [JBossINF] 17:19:13,870 INFO [stdout] (pool-19-thread-1) -------------------------------------------------------------------
> [JBossINF] 17:19:14,195 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-12) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:19:14,195 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-11) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:19:14,268 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-15-thread-1) ISPN000078: Starting JGroups Channel
> [JBossINF] 17:19:14,268 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-21-thread-1) ISPN000078: Starting JGroups Channel
> [JBossINF] 17:19:14,276 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-21-thread-1) ISPN000094: Received new cluster view: [perf19/ejb|5] [perf19/ejb, perf21/ejb, perf20/ejb, perf18/ejb]
> [JBossINF] 17:19:14,276 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-15-thread-1) ISPN000094: Received new cluster view: [perf19/web|5] [perf19/web, perf21/web, perf20/web, perf18/web]
> [JBossINF] 17:19:14,277 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-15-thread-1) ISPN000079: Cache local address is perf18/web, physical addresses are [10.16.90.54:55200]
> [JBossINF] 17:19:14,277 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (pool-21-thread-1) ISPN000079: Cache local address is perf18/ejb, physical addresses are [10.16.90.54:55200]
> [JBossINF] 17:19:14,283 INFO [org.infinispan.factories.GlobalComponentRegistry] (pool-21-thread-1) ISPN000128: Infinispan version: Infinispan 'Brahma' 5.1.2.FINAL
> [JBossINF] 17:19:14,283 INFO [org.infinispan.factories.GlobalComponentRegistry] (pool-15-thread-1) ISPN000128: Infinispan version: Infinispan 'Brahma' 5.1.2.FINAL
> [JBossINF] 17:19:14,284 INFO [org.infinispan.config.ConfigurationValidatingVisitor] (pool-15-thread-1) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:19:14,422 INFO [org.infinispan.jmx.CacheJmxRegistration] (pool-21-thread-1) ISPN000031: MBeans were successfully registered to the platform mbean server.
> [JBossINF] 17:19:14,442 INFO [org.infinispan.jmx.CacheJmxRegistration] (pool-20-thread-1) ISPN000031: MBeans were successfully registered to the platform mbean server.
> [JBossINF] 17:19:14,447 INFO [org.infinispan.jmx.CacheJmxRegistration] (pool-15-thread-1) ISPN000031: MBeans were successfully registered to the platform mbean server.
> [JBossINF] 17:19:14,503 INFO [org.jboss.as.clustering.infinispan] (pool-21-thread-1) JBAS010281: Started remote-connector-client-mappings cache from ejb container
> [JBossINF] 17:19:14,503 INFO [org.jboss.as.clustering.infinispan] (pool-15-thread-1) JBAS010281: Started repl cache from web container
> [JBossINF] 17:19:14,503 INFO [org.jboss.as.clustering.infinispan] (pool-20-thread-1) JBAS010281: Started repl cache from ejb container
> [JBossINF] 17:19:14,519 INFO [org.jboss.as.clustering.impl.CoreGroupCommunicationService.web] (MSC service thread 1-2) JBAS010206: Number of cluster members: 4
> [JBossINF] 17:19:14,519 INFO [org.jboss.as.clustering.impl.CoreGroupCommunicationService.ejb] (MSC service thread 1-12) JBAS010206: Number of cluster members: 4
> [JBossINF] 17:19:14,555 INFO [org.jboss.ejb.client.ClusterContext] (ejb-client-cluster-node-connection-creation-3-thread-1) Added a new EJB receiver in cluster context ejb for node perf18. Total nodes in cluster context = 1
> [JBossINF] 17:19:14,573 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-5) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:19:14,576 INFO [org.infinispan.config.ConfigurationValidatingVisitor] (MSC service thread 1-5) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:19:14,584 INFO [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-5) ISPN000031: MBeans were successfully registered to the platform mbean server.
> [JBossINF] 17:19:14,589 INFO [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-9) ISPN000031: MBeans were successfully registered to the platform mbean server.
> [JBossINF] 17:19:14,592 INFO [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-12) ISPN000031: MBeans were successfully registered to the platform mbean server.
> [JBossINF] 17:19:14,605 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-12) JBAS010281: Started org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSBImpl cache from ejb container
> [JBossINF] 17:19:14,606 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-9) JBAS010281: Started org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB cache from ejb container
> 2012/03/09 17:20:09:956 EST [DEBUG][Thread-33] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - JBossStartup, server started!
> [JBossINF] 17:20:13,087 INFO [org.jboss.as.server] (DeploymentScanner-threads - 2) JBAS015870: Deploy of deployment "clusterbench-ee6.ear" was rolled back with failure message Operation cancelled
> [JBossINF] 17:20:13,087 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015052: Did not receive a response to the deployment operation within the allowed timeout period [60 seconds]. Check the server configuration file and the server logs to find more about the status of the deployment.
> [JBossINF] 17:20:13,095 INFO [org.infinispan.eviction.PassivationManagerImpl] (MSC service thread 1-9) ISPN000029: Passivating all entries to disk
> [JBossINF] 17:20:13,096 INFO [org.infinispan.eviction.PassivationManagerImpl] (MSC service thread 1-16) ISPN000029: Passivating all entries to disk
> [JBossINF] 17:20:13,098 INFO [org.infinispan.eviction.PassivationManagerImpl] (MSC service thread 1-9) ISPN000030: Passivated 0 entries in 3 milliseconds
> [JBossINF] 17:20:13,098 INFO [org.infinispan.eviction.PassivationManagerImpl] (MSC service thread 1-16) ISPN000030: Passivated 0 entries in 2 milliseconds
> [JBossINF] 17:20:13,106 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-9) JBAS010282: Stopped org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB cache from ejb container
> [JBossINF] 17:20:13,106 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-16) JBAS010282: Stopped org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSBImpl cache from ejb container
> [JBossINF] 17:20:14,587 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-5) JBAS010281: Started //default-host//clusterbench cache from web container
> [JBossINF] 17:20:14,596 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-5) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:20:14,597 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-5) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be pasivated.
> [JBossINF] 17:20:14,844 INFO [org.jboss.as.clustering.impl.CoreGroupCommunicationService.web] (Incoming-18,null) JBAS010248: New cluster view for partition web: 6 (org.jboss.as.clustering.impl.CoreGroupCommunicationService$GroupView at 1a6fa30c delta: -1, merge: false)
> [JBossINF] 17:20:14,844 INFO [org.jboss.as.clustering.impl.CoreGroupCommunicationService.ejb] (Incoming-11,null) JBAS010248: New cluster view for partition ejb: 6 (org.jboss.as.clustering.impl.CoreGroupCommunicationService$GroupView at 17397663 delta: -1, merge: false)
> [JBossINF] 17:20:14,846 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-18,null) ISPN000094: Received new cluster view: [perf21/web|6] [perf21/web, perf20/web, perf18/web]
> [JBossINF] 17:20:14,849 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-11,null) ISPN000094: Received new cluster view: [perf21/ejb|6] [perf21/ejb, perf20/ejb, perf18/ejb]
> [JBossINF] 17:20:14,974 INFO [org.infinispan.cacheviews.CacheViewsManagerImpl] (OOB-19,null) ISPN000170: Our last committed view (CacheView{viewId=6, members=[perf19/ejb, perf21/ejb, perf20/ejb, perf18/ejb]}) is not the same as the coordinator's last committed view (CacheView{viewId=7, members=[perf19/ejb, perf21/ejb, perf20/ejb]}). This is normal during a merge
> [JBossINF] 17:20:14,977 WARN [org.infinispan.commands.control.CacheViewControlCommand] (OOB-15,null) ISPN000071: Caught exception when handling command CacheViewControlCommand{cache=//default-host//clusterbench, type=PREPARE_VIEW, sender=perf21/web, newViewId=8, newMembers=[perf21/web, perf20/web, perf18/web], oldViewId=7, oldMembers=[perf19/web, perf21/web, perf20/web]}: java.lang.IllegalStateException: Cannot prepare new view CacheView{viewId=8, members=[perf21/web, perf20/web, perf18/web]} on cache //default-host//clusterbench, we are currently preparing view CacheView{viewId=6, members=[perf19/web, perf21/web, perf20/web, perf18/web]}
> [JBossINF] at org.infinispan.cacheviews.CacheViewInfo.prepareView(CacheViewInfo.java:102) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:482) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.control.CacheViewControlCommand.perform(CacheViewControlCommand.java:125) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:95) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:217) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:197) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:456) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:363) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderAwareUpHandler.up(ClassLoaderAwareUpHandler.java:56) [jboss-as-clustering-jgroups-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.jgroups.MuxChannel$ClassLoaderAwareMuxUpHandler.up(MuxChannel.java:64) [jboss-as-clustering-jgroups-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jgroups.JChannel.up(JChannel.java:716) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:383) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:697) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:559) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.BARRIER.up(BARRIER.java:126) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:355) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:358) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1174) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> [JBossINF]
> [JBossINF] 17:20:14,975 INFO [org.infinispan.cacheviews.CacheViewsManagerImpl] (OOB-17,null) ISPN000170: Our last committed view (CacheView{viewId=6, members=[perf19/ejb, perf21/ejb, perf20/ejb, perf18/ejb]}) is not the same as the coordinator's last committed view (CacheView{viewId=7, members=[perf19/ejb, perf21/ejb, perf20/ejb]}). This is normal during a merge
> [JBossINF] 17:20:14,990 WARN [org.infinispan.commands.control.CacheViewControlCommand] (OOB-19,null) ISPN000071: Caught exception when handling command CacheViewControlCommand{cache=org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSBImpl, type=PREPARE_VIEW, sender=perf21/ejb, newViewId=11, newMembers=[perf20/ejb, perf18/ejb, perf21/ejb], oldViewId=7, oldMembers=[perf19/ejb, perf21/ejb, perf20/ejb]}: java.lang.IllegalStateException: org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSBImpl: Received cache view prepare request after the local node has already shut down
> [JBossINF] at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:488) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.control.CacheViewControlCommand.perform(CacheViewControlCommand.java:125) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:95) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:217) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:197) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:456) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:363) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderAwareUpHandler.up(ClassLoaderAwareUpHandler.java:56) [jboss-as-clustering-jgroups-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.jgroups.MuxChannel$ClassLoaderAwareMuxUpHandler.up(MuxChannel.java:64) [jboss-as-clustering-jgroups-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jgroups.JChannel.up(JChannel.java:716) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:383) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:697) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:559) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.BARRIER.up(BARRIER.java:126) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:355) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:358) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1174) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> [JBossINF]
> [JBossINF] 17:20:15,021 WARN [org.infinispan.commands.control.CacheViewControlCommand] (OOB-17,null) ISPN000071: Caught exception when handling command CacheViewControlCommand{cache=org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB, type=PREPARE_VIEW, sender=perf21/ejb, newViewId=11, newMembers=[perf20/ejb, perf18/ejb, perf21/ejb], oldViewId=7, oldMembers=[perf19/ejb, perf21/ejb, perf20/ejb]}: java.lang.IllegalStateException: org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB: Received cache view prepare request after the local node has already shut down
> [JBossINF] at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:488) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.control.CacheViewControlCommand.perform(CacheViewControlCommand.java:125) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:95) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:217) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:197) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:456) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:363) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderAwareUpHandler.up(ClassLoaderAwareUpHandler.java:56) [jboss-as-clustering-jgroups-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.jgroups.MuxChannel$ClassLoaderAwareMuxUpHandler.up(MuxChannel.java:64) [jboss-as-clustering-jgroups-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jgroups.JChannel.up(JChannel.java:716) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:383) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:697) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:559) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.BARRIER.up(BARRIER.java:126) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:355) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:358) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1174) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704) [jgroups-3.0.6.Final-redhat-1.jar:3.0.6.Final-redhat-1]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> [JBossINF]
> [JBossINF] 17:20:15,072 ERROR [org.infinispan.cacheviews.CacheViewInfo] (OOB-17,null) ISPN000164: Rolling back to cache view 7, but last committed view is 6
> [JBossINF] 17:20:15,084 ERROR [org.infinispan.cacheviews.CacheViewInfo] (OOB-17,null) ISPN000164: Rolling back to cache view 7, but last committed view is 6
> [JBossINF] 17:20:15,197 INFO [javax.enterprise.resource.webcontainer.jsf.config] (MSC service thread 1-5) Initializing Mojarra 2.1.5 (SNAPSHOT 20120217) for context '/clusterbench'
> [JBossINF] 17:20:16,638 INFO [org.hibernate.validator.util.Version] (MSC service thread 1-5) Hibernate Validator 4.2.0.Final-redhat-1
> [JBossINF] 17:20:16,856 INFO [org.jboss.web] (MSC service thread 1-5) JBAS018210: Registering web context: /clusterbench
> [JBossINF] 17:21:15,196 WARN [org.infinispan.statetransfer.BaseStateTransferManagerImpl] (OOB-14,null) ISPN000167: Rejecting state pushed by node perf21/web for view 8, there is no state transfer in progress (we are at view 9)
> [JBossINF] 17:21:15,196 WARN [org.infinispan.statetransfer.BaseStateTransferManagerImpl] (OOB-20,null) ISPN000167: Rejecting state pushed by node perf20/web for view 8, there is no state transfer in progress (we are at view 9)
> [JBossINF] 17:21:16,864 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MSC service thread 1-13) ISPN000136: Execution error: org.infinispan.statetransfer.StateTransferInProgressException: Timed out waiting for the state transfer lock, state transfer in progress for view 10
> [JBossINF] at org.infinispan.interceptors.StateTransferLockInterceptor.signalStateTransferInProgress(StateTransferLockInterceptor.java:199) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.StateTransferLockInterceptor.visitPrepareCommand(StateTransferLockInterceptor.java:80) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:130) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:89) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:86) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager$ClassLoaderAwareCommandInterceptor.handleDefault(DefaultEmbeddedCacheManager.java:410) [jboss-as-clustering-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:140) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:122) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:223) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:234) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:85) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:100) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.CacheImpl.endBatch(CacheImpl.java:580) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75) [infinispan-core-5.1.2.FINAL-redhat-1.jar:5.1.2.FINAL-redhat-1]
> [JBossINF] at org.jboss.as.clustering.infinispan.invoker.BatchOperation.invoke(BatchOperation.java:26) [jboss-as-clustering-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:61) [jboss-as-clustering-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager$ForceSynchronousCacheInvoker.invoke(DistributedCacheManager.java:522) [jboss-as-clustering-web-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.invoke(DistributedCacheManager.java:490) [jboss-as-clustering-web-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.batch(DistributedCacheManager.java:486) [jboss-as-clustering-web-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.removeSession(DistributedCacheManager.java:277) [jboss-as-clustering-web-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.removeSessionLocal(DistributedCacheManager.java:266) [jboss-as-clustering-web-infinispan-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.clearSessions(DistributableSessionManager.java:507)
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.stop(DistributableSessionManager.java:445)
> [JBossINF] at org.apache.catalina.core.StandardContext.stop(StandardContext.java:3995)
> [JBossINF] at org.jboss.as.web.deployment.WebDeploymentService.stop(WebDeploymentService.java:108)
> [JBossINF] at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1911) [jboss-msc-1.0.2.GA-redhat-1.jar:1.0.2.GA-redhat-1]
> [JBossINF] at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1874) [jboss-msc-1.0.2.GA-redhat-1.jar:1.0.2.GA-redhat-1]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> [JBossINF]
> [JBossINF] 17:21:16,889 ERROR [org.infinispan.transaction.TransactionCoordinator] (MSC service thread 1-13) Error while processing prepare: org.infinispan.statetransfer.StateTransferInProgressException: Timed out waiting for the state transfer lock, state transfer in progress for view 10
> [JBossINF] at org.infinispan.interceptors.StateTransferLockInterceptor.signalStateTransferInProgress(StateTransferLockInterceptor.java:199)
> [JBossINF] at org.infinispan.interceptors.StateTransferLockInterceptor.visitPrepareCommand(StateTransferLockInterceptor.java:80)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:130)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:89)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> [JBossINF] at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:86)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
> [JBossINF] at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager$ClassLoaderAwareCommandInterceptor.handleDefault(DefaultEmbeddedCacheManager.java:410)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:113)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:131)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345)
> [JBossINF] at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:140)
> [JBossINF] at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:122)
> [JBossINF] at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:223)
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:234)
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:85)
> [JBossINF] at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:100)
> [JBossINF] at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123)
> [JBossINF] at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105)
> [JBossINF] at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86)
> [JBossINF] at org.infinispan.CacheImpl.endBatch(CacheImpl.java:580)
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75)
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75)
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75)
> [JBossINF] at org.jboss.as.clustering.infinispan.invoker.BatchOperation.invoke(BatchOperation.java:26)
> [JBossINF] at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:61)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager$ForceSynchronousCacheInvoker.invoke(DistributedCacheManager.java:522)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.invoke(DistributedCacheManager.java:490)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.batch(DistributedCacheManager.java:486)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.removeSession(DistributedCacheManager.java:277)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.removeSessionLocal(DistributedCacheManager.java:266)
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.clearSessions(DistributableSessionManager.java:507) [jboss-as-web-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.stop(DistributableSessionManager.java:445) [jboss-as-web-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.apache.catalina.core.StandardContext.stop(StandardContext.java:3995) [jbossweb-7.0.10.Final-redhat-1.jar:]
> [JBossINF] at org.jboss.as.web.deployment.WebDeploymentService.stop(WebDeploymentService.java:108) [jboss-as-web-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1911)
> [JBossINF] at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1874)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> [JBossINF]
> [JBossINF] 17:21:16,906 ERROR [org.infinispan.transaction.tm.DummyTransaction] (MSC service thread 1-13) ISPN000109: beforeCompletion() failed for SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, transaction=DummyTransaction{xid=DummyXid{id=6}, status=1}, lockedKeys=null, backupKeyLocks=null, viewId=6} org.infinispan.transaction.synchronization.SyncLocalTransaction at 5} org.infinispan.transaction.synchronization.SynchronizationAdapter at 24: org.infinispan.CacheException: Could not prepare.
> [JBossINF] at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.notifyBeforeCompletion(DummyTransaction.java:223)
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.runPrepare(DummyTransaction.java:234)
> [JBossINF] at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:85)
> [JBossINF] at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:100)
> [JBossINF] at org.infinispan.batch.BatchContainer.resolveTransaction(BatchContainer.java:123)
> [JBossINF] at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:105)
> [JBossINF] at org.infinispan.batch.BatchContainer.endBatch(BatchContainer.java:86)
> [JBossINF] at org.infinispan.CacheImpl.endBatch(CacheImpl.java:580)
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75)
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75)
> [JBossINF] at org.infinispan.AbstractDelegatingCache.endBatch(AbstractDelegatingCache.java:75)
> [JBossINF] at org.jboss.as.clustering.infinispan.invoker.BatchOperation.invoke(BatchOperation.java:26)
> [JBossINF] at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:61)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager$ForceSynchronousCacheInvoker.invoke(DistributedCacheManager.java:522)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.invoke(DistributedCacheManager.java:490)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.batch(DistributedCacheManager.java:486)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.removeSession(DistributedCacheManager.java:277)
> [JBossINF] at org.jboss.as.clustering.web.infinispan.DistributedCacheManager.removeSessionLocal(DistributedCacheManager.java:266)
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.clearSessions(DistributableSessionManager.java:507) [jboss-as-web-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.as.web.session.DistributableSessionManager.stop(DistributableSessionManager.java:445) [jboss-as-web-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.apache.catalina.core.StandardContext.stop(StandardContext.java:3995) [jbossweb-7.0.10.Final-redhat-1.jar:]
> [JBossINF] at org.jboss.as.web.deployment.WebDeploymentService.stop(WebDeploymentService.java:108) [jboss-as-web-7.1.0.Final-redhat-1.jar:7.1.0.Final-redhat-1]
> [JBossINF] at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1911)
> [JBossINF] at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1874)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
> [JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
> [JBossINF] Caused by: javax.transaction.xa.XAException
> [JBossINF] at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:160)
> [JBossINF] at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:122)
> [JBossINF] at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> [JBossINF] ... 27 more
> [JBossINF]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
More information about the jboss-jira
mailing list