Michal Vinkler created WFLY-10160: ------------------------------------- Summary: "ISPN000476: Timed out waiting for responses for request X from node Y" immediately after node Y rejoins the cluster (failover) Key: WFLY-10160 URL: https://issues.jboss.org/browse/WFLY-10160 Project: WildFly Issue Type: Bug Components: Clustering Affects Versions: 12.0.0.Final Reporter: Michal Vinkler Assignee: Paul Ferraro Follow-up on WFLY-9890, this time it does not affect the clients. Seen in SF failover tests - presumably in the most of the scenarios, for example: failover-http-session-jvmkill-repl-sync Description: In the failover test, each server is shut down and after some time it rejoins the cluster. After joining the cluster again, the load is distributed to this node. It seems that first request for some session handled by that node produces ERROR in the log of some other server, the errors keep usually occurring until the very end of the test (i.e. not only to another cluster topology change). Please notice in the following stacktrace (taken from perf19) - as soon as perf20 rejoins the cluster, it starts logging the TimeoutException errors. It stops only after the test is stopped. Nothing happens (i.e. no errror) at this time in the log of perf20. stacktrace: {code} [JBossINF] [0m[0m09:01:14,076 INFO [org.infinispan.CLUSTER] (thread-59,ejb,perf19) ISPN100000: Node perf20 joined the cluster [JBossINF] [0m[31m09:01:24,568 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p10-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionAccessMetaDataKey(HP8SQsg6CF1loZZb-kQ0r7qeOIl78bDk85wvRQm1), SessionCreationMetaDataKey(HP8SQsg6CF1loZZb-kQ0r7qeOIl78bDk85wvRQm1), SessionAttributesKey(HP8SQsg6CF1loZZb-kQ0r7qeOIl78bDk85wvRQm1)]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 13116 from perf20 [JBossINF] at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:163) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:86) [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:21) [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] {code} Link: perf19: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa... perf20: http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-fa... The clients are not affected. -- This message was sent by Atlassian JIRA (v7.5.0#75005)