[jboss-jira] [JBoss JIRA] (WFLY-9901) Timed out waiting for responses for request X from Y

Richard Janík created WFLY-9901: ----------------------------------- Summary: Timed out waiting for responses for request X from Y Key: WFLY-9901 URL: https://issues.jboss.org/browse/WFLY-9901 Project: WildFly Issue Type: Bug Components: Clustering Affects Versions: 12.0.0.Beta1 Reporter: Richard Janík Assignee: Paul Ferraro Seen in our clustering tests in scenarios with EJBs: {noformat} [JBossINF] 08:56:32,059 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-30) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 68008 from perf21 [JBossINF] at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) [JBossINF] at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:166) [JBossINF] at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:126) [JBossINF] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:446) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:493) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335) [JBossINF] at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) [JBossINF] at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) [JBossINF] at org.jboss.as.ejb3.cache.distributable.DistributableCache.release(DistributableCache.java:154) [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:200) [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.handleAfterCompletion(StatefulSessionSynchronizationInterceptor.java:287) [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:260) [JBossINF] at org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList.afterCompletion(JCAOrderedLastSynchronizationList.java:147) [JBossINF] at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:196) [JBossINF] at org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization.afterCompletion(AbstractTransaction.java:279) [JBossINF] at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96) [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:542) [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101) [JBossINF] at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289) [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [JBossINF] at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) [JBossINF] at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73) [JBossINF] at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71) [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:90) [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:277) [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:330) [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:238) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) [JBossINF] at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81) [JBossINF] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:89) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:57) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) [JBossINF] at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:619) [JBossINF] at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) [JBossINF] at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) [JBossINF] at org.wildfly.security.auth.server.SecurityIdentity.runAsFunctionEx(SecurityIdentity.java:380) [JBossINF] at org.jboss.as.ejb3.remote.AssociationImpl.invokeWithIdentity(AssociationImpl.java:541) [JBossINF] at org.jboss.as.ejb3.remote.AssociationImpl.invokeMethod(AssociationImpl.java:536) [JBossINF] at org.jboss.as.ejb3.remote.AssociationImpl.lambda$receiveInvocationRequest$0(AssociationImpl.java:194) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1979) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1481) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1374) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 68008 from perf21 [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] ... 1 more {noformat} Example: [shutdown-repl-sync|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/j...] Looks similar to WFLY-6356 (that maybe should be closed as the clone was verified), but the cause is different and this is not restricted to *REPL* scenarios. Otherwise it behaves the same - seen after a node rejoins a cluster and doesn't result in invocation failures. -- This message was sent by Atlassian JIRA (v7.5.0#75005)