[jboss-jira] [JBoss JIRA] (WFLY-5307) CacheException: Initial state transfer timed out for cache
Michal Karm Babacek (JIRA)
issues at jboss.org
Sun Oct 4 14:25:00 EDT 2015
[ https://issues.jboss.org/browse/WFLY-5307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114811#comment-13114811 ]
Michal Karm Babacek commented on WFLY-5307:
-------------------------------------------
I can hit this on any slightly latency-prone network; e.g. with [weave|https://github.com/weaveworks/weave] network overlay across 6 docker containers on different hosts; having: {{.clustering().cacheMode(CacheMode.DIST_ASYNC)}} and the default {{jgroups-udp.xml}}.
> CacheException: Initial state transfer timed out for cache
> ----------------------------------------------------------
>
> Key: WFLY-5307
> URL: https://issues.jboss.org/browse/WFLY-5307
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Beta1, 10.0.0.CR2
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
>
> EAP 7.0.0.DR9
> Scenario: failover-ejb-ejbremote-undeploy-dist-async
> During application deployment (after failover), perf18 logged this error (besides the Replication timeout WARN message):
> {code}
> [JBossINF] [0m[0m15:29:55,511 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 85) WFLYCLINF0002: Started dist cache from ejb container
> [JBossINF] [0m[0m15:29:55,513 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 86) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-default.war cache from web container
> [JBossINF] [0m[0m15:29:55,516 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 87) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-passivating.war cache from web container
> [JBossINF] [0m[0m15:29:55,538 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 87) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m15:29:55,539 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 87) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m15:29:56,499 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 87) WFLYCLINF0002: Started clusterbench-ee7.ear/clusterbench-ee7-ejb.jar cache from ejb container
> [JBossINF] [0m[0m15:30:00,507 INFO [org.jboss.test.clusterbench.common.ejb.CommonStatefulSBImpl] (EJB default - 1) New SFSB created: org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSBImpl at 5293c097.
> [JBossINF] [0m[0m15:30:01,323 INFO [org.jboss.test.clusterbench.common.ejb.CommonStatefulSBImpl] (EJB default - 2) New SFSB created: org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSBImpl at 4dfb0a03.
> [JBossINF] [0m[33m15:30:55,461 WARN [org.infinispan.statetransfer.StateConsumerImpl] (ServerService Thread Pool -- 84) ISPN000286: Issue when retrieving cluster listeners from perf19: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf19
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:755)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$79(JGroupsTransport.java:589)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport$$Lambda$22/1199733093.apply(Unknown Source)
> [JBossINF] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> [JBossINF] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> [JBossINF] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> [JBossINF] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1954)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.call(SingleResponseFuture.java:46)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.call(SingleResponseFuture.java:17)
> [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF]
> [JBossINF] [0m[31m15:31:55,467 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 84) MSC000001: Failed to start service jboss.infinispan.web.dist: org.jboss.msc.service.StartException in service jboss.infinispan.web.dist: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.Exception on object of type StateTransferManagerImpl
> [JBossINF] at org.wildfly.clustering.service.AsynchronousServiceBuilder$1.run(AsynchronousServiceBuilder.java:107)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> [JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.Exception on object of type StateTransferManagerImpl
> [JBossINF] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:172)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> [JBossINF] at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:218)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:839)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:629)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:580)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:445)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:464)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:455)
> [JBossINF] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:120)
> [JBossINF] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:111)
> [JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.start(CacheBuilder.java:80)
> [JBossINF] at org.wildfly.clustering.service.AsynchronousServiceBuilder$1.run(AsynchronousServiceBuilder.java:102)
> [JBossINF] ... 4 more
> [JBossINF] Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache dist on perf18
> [JBossINF] at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:225)
> [JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [JBossINF] at java.lang.reflect.Method.invoke(Method.java:497)
> [JBossINF] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> [JBossINF] ... 19 more
> [JBossINF]
> [JBossINF] [0m[31m15:31:55,476 ERROR [org.jboss.as.server] (management-handler-thread - 2) WFLYSRV0021: Deploy of deployment "clusterbench-ee7.ear" was rolled back with the following failure message: undefined
> {code}
> Which ends up with WFLYSRV0021: *Deploy of deployment "clusterbench-ee7.ear" was rolled back*.
> In this very same testrun, there was another issue with application deployment on perf20, may be related: JBEAP-1043
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-ejbremote-undeploy-dist-async/5/console-perf18/
> This issue is similar to this EAP6 bug: [BZ927948|https://bugzilla.redhat.com/show_bug.cgi?id=927948]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
More information about the jboss-jira
mailing list