[jboss-jira] [JBoss JIRA] (WFLY-10763) ISPN000476: Timed out waiting for responses for request

tommaso borgato (JIRA) issues at jboss.org
Mon Jul 30 09:22:00 EDT 2018 

tommaso borgato created WFLY-10763:
--------------------------------------

             Summary: ISPN000476: Timed out waiting for responses for request
                 Key: WFLY-10763
                 URL: https://issues.jboss.org/browse/WFLY-10763
             Project: WildFly
          Issue Type: Bug
          Components: Clustering
            Reporter: tommaso borgato
            Assignee: Paul Ferraro


The error was observed in scenario {{*[eap-7x-failover-ejb-ejbservlet-jvmkill-dist-sync|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/view/EAP7/view/EAP7-Clustering_JJB/view/clustering-ejb-ejbservlet-tests/job/eap-7x-failover-ejb-ejbservlet-jvmkill-dist-sync_perf27_JJB/1/]*}}: a 4 nodes cluster with a mod_jk load balancer where fail-over is introduced by killing the server jvm.

The error was observed on node [dev215|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/view/EAP7/view/EAP7-Clustering_JJB/view/clustering-ejb-ejbservlet-tests/job/eap-7x-failover-ejb-ejbservlet-jvmkill-dist-sync_perf27_JJB/1/console-dev215/] :

{noformat}
[JBossINF] 04:10:27,889 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-178) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 40700 from dev213
[JBossINF] 	at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259)
[JBossINF] 	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137)
[JBossINF] 	at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155)
[JBossINF] 	at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:127)
[JBossINF] 	at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68)
[JBossINF] 	at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419)
[JBossINF] 	at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466)
[JBossINF] 	at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335)
[JBossINF] 	at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110)
[JBossINF] 	at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97)
[JBossINF] 	at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:91)
[JBossINF] 	at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945)
[JBossINF] 	at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:339)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
[JBossINF] 	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
[JBossINF] 	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
[JBossINF] 	at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
[JBossINF] 	at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 	at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 	at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 	at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
[JBossINF] 	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
[JBossINF] 	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
[JBossINF] 	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
[JBossINF] 	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
[JBossINF] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
[JBossINF] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1349)
[JBossINF] 	at java.lang.Thread.run(Thread.java:748)
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 40700 from dev213
[JBossINF] 	at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167)
[JBossINF] 	at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87)
[JBossINF] 	at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
[JBossINF] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] 	... 1 more
[JBossINF] 	Suppressed: org.infinispan.util.logging.TraceException
[JBossINF] 		at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41)
[JBossINF] 		at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250)
[JBossINF] 		at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137)
[JBossINF] 		at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:155)
[JBossINF] 		at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:127)
[JBossINF] 		at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68)
[JBossINF] 		at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419)
[JBossINF] 		at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466)
[JBossINF] 		at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335)
[JBossINF] 		at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110)
[JBossINF] 		at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97)
[JBossINF] 		at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:91)
[JBossINF] 		at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945)
[JBossINF] 		at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:339)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
[JBossINF] 		at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
[JBossINF] 		at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
[JBossINF] 		at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
[JBossINF] 		at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 		at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 		at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 		at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] 		at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
[JBossINF] 		at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
[JBossINF] 		at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
[JBossINF] 		at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
[JBossINF] 		at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
[JBossINF] 		at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
[JBossINF] 		at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1349)
[JBossINF] 		... 1 more
{noformat}

{quote}
Please note that this scenario is the counterpart of scenario {{*[eap-7x-failover-ejb-ejbservlet-jvmkill-dist-sync|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/view/EAP7/view/EAP7-Clustering_JJB/view/clustering-ejb-ejbservlet-tests/job/eap-7x-failover-ejb-ejbservlet-jvmkill-dist-sync_JJB/]*}} with the only difference of being executed on a different segment (perf27 instead of perf17). This other scenario has zero sampling errors and doesn't exhibit this error. 
{quote}

{quote}
Please also note that this errors was observed *right after* other error described in [WFLY-10758|https://issues.jboss.org/browse/WFLY-10758] so they may be related. Please refer to that issue for the sequence of events that led to the error.
{quote}






--
This message was sent by Atlassian JIRA
(v7.5.0#75005)

More information about the jboss-jira mailing list