[JBoss JIRA] (ISPN-2960) '[ClusterTopologyManagerImpl] Failed to start rebalance: ... IllegalStateException: transport was closed' on cache stop
11:42 a.m.
[
https://issues.jboss.org/browse/ISPN-2960?page=com.atlassian.jira.plugin....
]
Radoslav Husar commented on ISPN-2960:
--------------------------------------
The other nodes generates this message (below). Afterwards, there are some failures
(missing/stale entries) but I am unable to isolate this particular failure as there are
quite a few things failing altogether.
{code}
[JBossINF] [0m[33m07:01:44,585 WARN
[org.infinispan.topology.CacheTopologyControlCommand] (transport-thread-8) ISPN000071:
Caught exception when handling command CacheTopologyControlCommand{cache=dist,
type=REBALANCE_CONFIRM, sender=perf21/web, joinInfo=null, topologyId=13, currentCH=null,
pendingCH=null, throwable=null, viewId=6}: org.infinispan.CacheException: Received invalid
rebalance confirmation from perf21/web for cache dist, we don't have a rebalance in
progress
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl.handleRebalanceCompleted(ClusterTopologyManagerImpl.java:206)
[JBossINF] at
org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:160)
[JBossINF] at
org.infinispan.topology.CacheTopologyControlCommand.perform(CacheTopologyControlCommand.java:137)
[JBossINF] at
org.infinispan.topology.LocalTopologyManagerImpl$1.call(LocalTopologyManagerImpl.java:291)
[JBossINF] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[rt.jar:1.6.0_43]
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_43]
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[rt.jar:1.6.0_43]
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[rt.jar:1.6.0_43]
[JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_43]
[JBossINF]
[JBossINF] [0m[33m07:01:44,584 WARN
[org.infinispan.topology.CacheTopologyControlCommand] (OOB-19,shared=udp) ISPN000071:
Caught exception when handling command
CacheTopologyControlCommand{cache=default-host/clusterbench, type=REBALANCE_CONFIRM,
sender=perf20/web, joinInfo=null, topologyId=13, currentCH=null, pendingCH=null,
throwable=null, viewId=6}: org.infinispan.CacheException: Received invalid rebalance
confirmation from perf20/web for cache default-host/clusterbench, we don't have a
rebalance in progress
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl.handleRebalanceCompleted(ClusterTopologyManagerImpl.java:206)
[JBossINF] at
org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:160)
[JBossINF] at
org.infinispan.topology.CacheTopologyControlCommand.perform(CacheTopologyControlCommand.java:137)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:253)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:220)
[JBossINF] at
org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:484)
[JBossINF] at
org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:391)
[JBossINF] at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:249)
[JBossINF] at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:598)
[JBossINF] at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
[JBossINF] at org.jgroups.JChannel.up(JChannel.java:707)
[JBossINF] at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
[JBossINF] at org.jgroups.protocols.RSVP.up(RSVP.java:172)
[JBossINF] at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
[JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
[JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
[JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
[JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
[JBossINF] at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:765)
[JBossINF] at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:420)
[JBossINF] at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:606)
[JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
[JBossINF] at org.jgroups.protocols.FD.up(FD.java:253)
[JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
[JBossINF] at org.jgroups.protocols.MERGE3.up(MERGE3.java:290)
[JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:359)
[JBossINF] at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2616)
[JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
[JBossINF] at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
[JBossINF] at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1798)
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[rt.jar:1.6.0_43]
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[rt.jar:1.6.0_43]
[JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_43]
[JBossINF]
{code}
'[ClusterTopologyManagerImpl] Failed to start rebalance: ...
IllegalStateException: transport was closed' on cache stop
-----------------------------------------------------------------------------------------------------------------------
Key: ISPN-2960
URL: https://issues.jboss.org/browse/ISPN-2960
Project: Infinispan
Issue Type: Bug
Components: State transfer
Affects Versions: 5.2.5.Final
Reporter: Radoslav Husar
Assignee: Dan Berindei
Components are still not keeping track whether the cache is stopping down and holf off
stopping the channel. This is biting us in AS on undeloy/shutdown.
{code}
[JBossINF] [0m[31m07:07:58,044 ERROR
[org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread-8) Failed to start
rebalance: org.infinispan.CacheException: Remote (perf18/web) failed unexpectedly:
java.util.concurrent.ExecutionException: org.infinispan.CacheException: Remote
(perf18/web) failed unexpectedly
[JBossINF] at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
[rt.jar:1.6.0_43]
[JBossINF] at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_43]
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl.executeOnClusterSync(ClusterTopologyManagerImpl.java:549)
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl.broadcastRebalanceStart(ClusterTopologyManagerImpl.java:392)
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl.startRebalance(ClusterTopologyManagerImpl.java:382)
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl.access$000(ClusterTopologyManagerImpl.java:66)
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl$1.call(ClusterTopologyManagerImpl.java:128)
[JBossINF] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[rt.jar:1.6.0_43]
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[rt.jar:1.6.0_43]
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[rt.jar:1.6.0_43]
[JBossINF] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[rt.jar:1.6.0_43]
[JBossINF] at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_43]
[JBossINF] Caused by: org.infinispan.CacheException: Remote (perf18/web) failed
unexpectedly
[JBossINF] at
org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:99)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:541)
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl$2.call(ClusterTopologyManagerImpl.java:531)
[JBossINF] at
org.infinispan.topology.ClusterTopologyManagerImpl$2.call(ClusterTopologyManagerImpl.java:528)
[JBossINF] ... 5 more
[JBossINF] Caused by: java.lang.IllegalStateException: transport was closed
[JBossINF] at org.jgroups.blocks.GroupRequest.transportClosed(GroupRequest.java:273)
[JBossINF] at org.jgroups.blocks.RequestCorrelator.stop(RequestCorrelator.java:269)
[JBossINF] at org.jgroups.blocks.MessageDispatcher.stop(MessageDispatcher.java:152)
[JBossINF] at
org.jgroups.blocks.MessageDispatcher.channelDisconnected(MessageDispatcher.java:455)
[JBossINF] at org.jgroups.Channel.notifyChannelDisconnected(Channel.java:507)
[JBossINF] at org.jgroups.JChannel.disconnect(JChannel.java:363)
[JBossINF] at
org.infinispan.remoting.transport.jgroups.JGroupsTransport.stop(JGroupsTransport.java:258)
[JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[rt.jar:1.6.0_43]
[JBossINF] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[rt.jar:1.6.0_43]
[JBossINF] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[rt.jar:1.6.0_43]
[JBossINF] at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_43]
[JBossINF] at
org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
[JBossINF] at
org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:886)
[JBossINF] at
org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:693)
[JBossINF] at
org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:571)
[JBossINF] at
org.infinispan.factories.GlobalComponentRegistry.stop(GlobalComponentRegistry.java:260)
[JBossINF] at
org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:742)
[JBossINF] at
org.infinispan.manager.AbstractDelegatingEmbeddedCacheManager.stop(AbstractDelegatingEmbeddedCacheManager.java:179)
[JBossINF] at
org.jboss.as.clustering.infinispan.subsystem.EmbeddedCacheManagerService.stop(EmbeddedCacheManagerService.java:76)
[JBossINF] at
org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1911)
[jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
[JBossINF] at
org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1874)
[jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
[JBossINF] ... 3 more
{code}
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-http-...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
4287
days inactive
4287
days old
infinispan-issues@lists.jboss.org
0 comments
1 participants
participants (1)
-
Radoslav Husar (JIRA)