[
https://issues.jboss.org/browse/ISPN-2572?page=com.atlassian.jira.plugin....
]
Radoslav Husar commented on ISPN-2572:
--------------------------------------
Hi Dan,
lets look at the logs. Looks to me that ClusterTopologyManager picks up the topology, so
that doesnt seem to be an issue here.
{noformat}
15:46:57,442 TRACE [org.infinispan.topology.ClusterTopologyManagerImpl]
(notification-thread-0) Received new cluster view: 11
15:46:57,442 TRACE [org.infinispan.topology.ClusterTopologyManagerImpl]
(notification-thread-0) Updating cluster members for all the caches. New list is
[node-0/ejb, node-1/ejb]
{noformat}
Whats interesting, node1 says
{noformat}15:46:57,476 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl]
(ServerService Thread Pool -- 15) Waiting for initial state transfer to finish for cache
repl on node-1/ejb
{noformat}
node0 says
{noformat}
15:46:57,474 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher]
(OOB-13,null) Attempting to execute non-CacheRpcCommand command:
CacheTopologyControlCommand{cache=repl, type=JOIN, sender=node-1/ejb,
joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@43e3ac,
hashFunction=org.infinispan.commons.hash.MurmurHash3@118618d, numSegments=60, numOwners=2,
timeout=60000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=11}
[sender=node-1/ejb]
15:46:57,474 TRACE [org.infinispan.topology.DefaultRebalancePolicy] (OOB-13,null) Cache
repl status changed: joiners=[node-1/ejb], topology=CacheTopology{id=7,
currentCH=ReplicatedConsistentHash{members=[node-0/ejb, node-1/ejb]},
pendingCH=ReplicatedConsistentHash{members=[node-0/ejb, node-1/ejb]}}
15:46:57,475 TRACE [org.infinispan.topology.DefaultRebalancePolicy] (OOB-13,null) Not
triggering rebalance for cache %s, a rebalance is already in progress
{noformat}
So we end up with "Not triggering rebalance for cache %s, a rebalance is already in
progress" and then timeout -- Not sure which cache so I submitted:
https://github.com/infinispan/infinispan/pull/1506
Could it be the problem is the rebalance?
Full logs are here
http://teamcity.cafe-babe.org/repository/download/bt2/1755:id/testsuite/i...
http://teamcity.cafe-babe.org/repository/download/bt2/1755:id/testsuite/i...
http://teamcity.cafe-babe.org/viewLog.html?buildId=1755&buildTypeId=bt2
"CacheException: Initial state transfer timed out for
cache" reliably on AS7 testsuite
--------------------------------------------------------------------------------------
Key: ISPN-2572
URL:
https://issues.jboss.org/browse/ISPN-2572
Project: Infinispan
Issue Type: Bug
Components: State transfer
Affects Versions: 5.2.0.Beta4
Reporter: Radoslav Husar
Assignee: Dan Berindei
Priority: Blocker
Fix For: 5.2.0.Beta6
While running AS7 testsuite with speedups implemented in my branch
(
https://github.com/jbossas/jboss-as/pull/3381) we are contantly seeing (log below) on
Windows 2008.
Run:
http://teamcity.cafe-babe.org/viewLog.html?buildId=1689&tab=buildResu...
{code}
16:34:46,092 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 13)
MSC00001: Failed to start service jboss.infinispan.ejb.remote-connector-client-mappings:
org.jboss.msc.service.StartException in service
jboss.infinispan.ejb.remote-connector-client-mappings: org.infinispan.CacheException:
Unable to invoke method public void
org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
throws java.lang.InterruptedException on object of type StateTransferManagerImpl
at
org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:87)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[rt.jar:1.6.0_32]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[rt.jar:1.6.0_32]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_32]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
[jboss-threads-2.0.0.GA.jar:2.0.0.GA]
Caused by: org.infinispan.CacheException: Unable to invoke method public void
org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
throws java.lang.InterruptedException on object of type StateTransferManagerImpl
at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
at
org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:883)
at
org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:654)
at
org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:643)
at
org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:546)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:199)
at org.infinispan.CacheImpl.start(CacheImpl.java:520)
at
org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:690)
at
org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:653)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:549)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:563)
at
org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:107)
at
org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:98)
at
org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
at
org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:82)
... 4 more
Caused by: org.infinispan.CacheException: Initial state transfer timed out for cache
remote-connector-client-mappings on node-1/ejb
at
org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:209)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_32]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[rt.jar:1.6.0_32]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[rt.jar:1.6.0_32]
at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_32]
at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
... 18 more
{code}
Affected version -- current master (say 7dc531002539b078e429418d8ef204e401beafd1).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:
http://www.atlassian.com/software/jira