]
Michal Vinkler commented on WFLY-5158:
--------------------------------------
Still present in EAP 7.0.0.DR11:
{code}
[JBossINF] [0m[31m16:44:33,553 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default task-128) ISPN000136:
Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for
perf20
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:755)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$81(JGroupsTransport.java:602)
[JBossINF] at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
[JBossINF] at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
[JBossINF] at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[JBossINF] at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
{code}
Also, this WARN message is related:
{code}
[JBossINF] [0m[33m16:44:33,555 WARN [org.jboss.as.txn] (default task-89) WFLYTX0027:
The pre-jca synchronization
org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization@775d5d66
associated with tx TransactionImple < ac, BasicAction:
0:ffff0a105a36:-2ed0eb04:560af7f1:ae5 status: ActionStatus.COMMITTED > failed during
after completion: org.infinispan.util.concurrent.TimeoutException: Replication timeout for
perf21
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:755)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$81(JGroupsTransport.java:602)
[JBossINF] at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
[JBossINF] at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
[JBossINF] at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[JBossINF] at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
{code}
Server log:
Execution error: org.infinispan.util.concurrent.TimeoutException:
Replication timeout for node_name
---------------------------------------------------------------------------------------------------
Key: WFLY-5158
URL:
https://issues.jboss.org/browse/WFLY-5158
Project: WildFly
Issue Type: Bug
Components: Clustering
Affects Versions: 10.0.0.Beta1, 10.0.0.CR2
Reporter: Michal Vinkler
Assignee: Paul Ferraro
Seen in ejb-ejbservlet and http-session scenarios intermittently (no matter what failover
type or cache is used).
When node perf18 is restarted after failover other servers log this error several times:
{code}
[JBossINF] [0m[31m16:11:43,595 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default task-107) ISPN000136:
Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for
perf18
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:752)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$6(JGroupsTransport.java:599)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport$$Lambda$34/238012590.apply(Unknown
Source)
[JBossINF] at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
[JBossINF] at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
[JBossINF] at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[JBossINF] at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1954)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.RspListFuture.timeout(RspListFuture.java:40)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$$Lambda$32/2073718099.run(Unknown
Source)
[JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
In this particular test run, after perf18 restarted , perf19 logged the first error in 2
seconds, perf20 in 30 seconds, perf21 in 10 seconds.
timeline:
{code}
perf18: [JBossINF] [0m[0m16:11:42,361 INFO [org.jboss.as] (Controller Boot Thread)
WFLYSRV0025: EAP 7.0.0.Alpha1 (WildFly Core 2.0.0.Beta1) started in 20244ms - Started 747
of 993 services (424 services are lazy, passive or on-demand)
perf19: [JBossINF] [0m[31m16:11:43,595 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default task-107) ISPN000136:
Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for
perf18
perf20: [JBossINF] [0m[31m16:12:12,836 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default task-51) ISPN000136:
Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for
perf18
perf21: [JBossINF] [0m[31m16:11:52,826 ERROR
[org.infinispan.interceptors.InvocationContextInterceptor] (default task-22) ISPN000136:
Execution error: org.infinispan.util.concurrent.TimeoutException: Replication timeout for
perf18
{code}
This error also intermittently appears after server is shutdown.
Total number of errors for this particular test run: 1183