[jboss-jira] [JBoss JIRA] (WFLY-5158) Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for node_name

[Paul Ferraro (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118692#comment-13118692 ] 

Paul Ferraro commented on WFLY-5158:
------------------------------------

The PR I submitted only impacts the cache used for web sessions.  I see that you've disabled awaitInitialTransfer - this can certainly cause replication timeouts if the state to be transferred is large, since cache gets will always go remote until the transfer finishes.

> Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for node_name
> ---------------------------------------------------------------------------------------------------
>
>                 Key: WFLY-5158
>                 URL: https://issues.jboss.org/browse/WFLY-5158
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering
>    Affects Versions: 10.0.0.Beta1, 10.0.0.CR2
>            Reporter: Michal Vinkler
>            Assignee: Paul Ferraro
>
> Seen in ejb-ejbservlet and http-session scenarios intermittently (no matter what failover type or cache is used). 
> When node perf18 is restarted after failover other servers log this error several times: 
> {code}
> [JBossINF] [0m[31m16:11:43,595 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-107) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf18
> [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:752)
> [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$6(JGroupsTransport.java:599)
> [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport$$Lambda$34/238012590.apply(Unknown Source)
> [JBossINF] 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> [JBossINF] 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> [JBossINF] 	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> [JBossINF] 	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1954)
> [JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.timeout(RspListFuture.java:40)
> [JBossINF] 	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$$Lambda$32/2073718099.run(Unknown Source)
> [JBossINF] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [JBossINF] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] 	at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-ejbservlet-jvmkill-repl-async/4/console-perf19/
> In this particular test run, after perf18 restarted , perf19 logged the first error in 2 seconds, perf20 in 30 seconds, perf21 in 10 seconds.
> timeline:
> {code}
> perf18: [JBossINF] [0m[0m16:11:42,361 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: EAP 7.0.0.Alpha1 (WildFly Core 2.0.0.Beta1) started in 20244ms - Started 747 of 993 services (424 services are lazy, passive or on-demand)
> perf19: [JBossINF] [0m[31m16:11:43,595 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-107) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf18
> perf20: [JBossINF] [0m[31m16:12:12,836 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-51) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf18
> perf21: [JBossINF] [0m[31m16:11:52,826 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-22) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf18
> {code}
> This error also intermittently appears after server is shutdown.
> Total number of errors for this particular test run: 1183



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)