[jboss-jira] [JBoss JIRA] (WFLY-9957) ISPN000481: Cannot create remote transaction GlobalTx:X, the originator is not in the cluster view
Paul Ferraro (Jira)
issues at jboss.org
Thu Apr 4 15:30:01 EDT 2019
[ https://issues.jboss.org/browse/WFLY-9957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718240#comment-13718240 ]
Paul Ferraro commented on WFLY-9957:
------------------------------------
[~tommaso-borgato] Can you revalidate this against a recent release?
> ISPN000481: Cannot create remote transaction GlobalTx:X, the originator is not in the cluster view
> --------------------------------------------------------------------------------------------------
>
> Key: WFLY-9957
> URL: https://issues.jboss.org/browse/WFLY-9957
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 12.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
> Priority: Major
>
> Seen in scenario: stress-granular-dist-sync
> In the beggining of the test, all 4 servers form a cluster:
> {code}
> [JBossINF] [0m[0m09:38:07,301 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN000094: Received new cluster view for channel ejb: [dev212|3] (4) [dev212, dev213, dev214, dev215]
> [JBossINF] [0m[0m09:38:07,301 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN100000: Node dev215 joined the cluster
> [JBossINF] [0m[0m09:38:07,301 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN000094: Received new cluster view for channel ejb: [dev212|3] (4) [dev212, dev213, dev214, dev215]
> [JBossINF] [0m[0m09:38:07,302 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN100000: Node dev215 joined the cluster
> [JBossINF] [0m[0m09:38:07,303 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN000094: Received new cluster view for channel ejb: [dev212|3] (4) [dev212, dev213, dev214, dev215]
> [JBossINF] [0m[0m09:38:07,303 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN100000: Node dev215 joined the cluster
> [JBossINF] [0m[0m09:38:07,303 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN000094: Received new cluster view for channel ejb: [dev212|3] (4) [dev212, dev213, dev214, dev215]
> [JBossINF] [0m[0m09:38:07,303 INFO [org.infinispan.CLUSTER] (thread-10,ejb,dev212) ISPN100000: Node dev215 joined the cluster
> {code}
> After less than 1 minute of running test, servers start logging:
> {code}
> [JBossINF] [0m[31m10:36:11,844 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (thread-259,ejb,dev212) ISPN000136: Error executing command PrepareCommand, writing keys [SessionAccessMetaDataKey(gqatwr7cNgtvGW9Z60E3ug2v9sSoZZbGl40r1smV), SessionCreationMetaDataKey(gqatwr7cNgtvGW9Z60E3ug2v9sSoZZbGl40r1smV), SessionAttributeKey(gqatwr7cNgtvGW9Z60E3ug2v9sSoZZbGl40r1smV[1])]: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from dev215, see cause for remote stack trace
> [JBossINF] at org.infinispan.remoting.transport.ResponseCollectors.wrapRemoteException(ResponseCollectors.java:27)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.SyncMapResponseCollector.addException(SyncMapResponseCollector.java:48)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.SyncMapResponseCollector.addException(SyncMapResponseCollector.java:26)
> [JBossINF] at org.infinispan.remoting.transport.ValidResponseCollector.addResponse(ValidResponseCollector.java:29)
> [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onResponse(MultiTargetRequest.java:91)
> [JBossINF] at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:53)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1328)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1238)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:121)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1366)
> [JBossINF] at org.jgroups.JChannel.up(JChannel.java:819)
> [JBossINF] at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:134)
> [JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:340)
> [JBossINF] at org.jgroups.protocols.FORK.up(FORK.java:134)
> [JBossINF] at org.jgroups.protocols.FRAG3.up(FRAG3.java:171)
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> [JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:864)
> [JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:240)
> [JBossINF] at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1002)
> [JBossINF] at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:728)
> [JBossINF] at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:383)
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
> [JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:119)
> [JBossINF] at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:199)
> [JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:252)
> [JBossINF] at org.jgroups.protocols.MERGE3.up(MERGE3.java:276)
> [JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:267)
> [JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1248)
> [JBossINF] at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> [JBossINF] Caused by: org.infinispan.commons.CacheException: ISPN000481: Cannot create remote transaction GlobalTx:dev212:63375657, the originator is not in the cluster view
> [JBossINF] at org.infinispan.transaction.impl.TransactionTable.getOrCreateRemoteTransaction(TransactionTable.java:375)
> [JBossINF] at org.infinispan.transaction.impl.TransactionTable.getOrCreateRemoteTransaction(TransactionTable.java:360)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.createContext(PrepareCommand.java:120)
> [JBossINF] at org.infinispan.commands.tx.PrepareCommand.invokeAsync(PrepareCommand.java:101)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:102)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.invoke(BaseBlockingRunnable.java:99)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runAsync(BaseBlockingRunnable.java:71)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:40)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> [JBossINF] ... 1 more
> {code}
> Also it is interesting, that clients in this test log either stale data:
> {code}
> 2018/02/16 10:36:16:596 EST [WARN ][Runner - 19] SFCORE_LOG - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 31196, received 31195, Runner: 19>
> org.jboss.smartfrog.loaddriver.RequestProcessingException: Stale session data received. Expected 31196, received 31195, Runner: 19
> at org.jboss.smartfrog.loaddriver.http.AbstractSerialNumberValidatorFactoryImpl$SerialNumberValidator.processRequest(AbstractSerialNumberValidatorFactoryImpl.java:133)
> at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
> at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> or 503 Service Temporarily Unavailable
> {code}
> 2018/02/16 10:37:24:837 EST [WARN ][Runner - 822] HOST dev220.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: <org.jboss.smartfrog.loaddriver.RequestProcessingException: Invalid response code: 503 Content: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>503 Service Temporarily Unavailable</title>
> </head><body>
> <h1>Service Temporarily Unavailable</h1>
> <p>The server is temporarily unable to service your
> request due to maintenance downtime or capacity
> problems. Please try again later.</p>
> <hr>
> <address>Apache/2.2.26 (@VENDOR@) Server at dev221 Port 8080</address>
> </body></html>
> >
> {code}
> Logs:
> client http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-granular-dist-sync/41/console-dev221/
> server http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-granular-dist-sync/41/console-dev212/
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
More information about the jboss-jira
mailing list