[infinispan-dev] Hung testsuite

Tue Oct 2 12:03:11 EDT 2012

Cool, thanks Bela!

On Tue, Oct 2, 2012 at 7:02 PM, Bela Ban <bban at redhat.com> wrote:

> I know what the issue is, I'll look into it tomorrow
>
> On 10/2/12 1:43 PM, Dan Berindei wrote:
> > On Tue, Oct 2, 2012 at 12:41 PM, Bela Ban<bban at redhat.com>  wrote:
> >
> >>
> >>
> >> On 10/2/12 11:27 AM, Dan Berindei wrote:
> >>> Hey Sanne
> >>>
> >>> Funny, Tristan just pinged me about the same problem in
> >>> StateTransferFunctionalTest...
> >>>
> >>> I haven't reproduced it yet, but it could be because I don't have the
> >> very
> >>> latest updates. At first I thought it was related to
> >>> https://issues.jboss.org/browse/ISPN-2357, but then in your thread
> dump
> >> all
> >>> test threads are stuck in JGroupsTransport.waitForChannelToConnect so
> it
> >>> looks like a JGroups thing.
> >>
> >>
> >> JGroupsTransport.waitForChannelToConnect() is Infinispan code, not
> >> JGroups code. IMO unneeded code, too, as I don't understand why we're
> >> waiting on a latch until we get a view after calling JChannel.connect().
> >> When JChannel.connect() returns, you're guaranteed to have a view
> >> installed, so this code can (should !) be removed, as it makes things
> >> unnecessarily complex (and error prone ?).
> >>
> >
> > True, it's Infinispan code - my code, actually ;)
> >
> > The reason behind the latch was probably to avoid concurrent updates of
> the
> > members list from the main thread and from the JGroups thread that calls
> > viewAccepted(). I agree it's probably unnecessary, as we already use
> > synchronization in our viewAccepted() implementation, but I don't see why
> > it shouldn't work.
> >
> > We have attached the JGroupsTransport as a membership listener to the
> > MessageDispatcher (CommandAwareRpcDispatcher, actually) and we have
> > attached the MessageDispatcher as a channel listener to the channel
> before
> > calling connect(), so JGroups should call viewAccepted() for the initial
> > view just as it does for every view.
> >
> > I just got the same test to hang on my machine, and I found this
> exception
> > in the log:
> >
> > 12:26:25,098 DEBUG (CacheStarter-Cache4:nbst) [GMS]
> > exception=java.lang.IllegalStateException: channel is not connected,
> > retrying
> > java.lang.IllegalStateException: channel is not connected
> >      at
> >
> org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:621)
> >      at
> >
> org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:535)
> >      at
> >
> org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:390)
> >      at
> > org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:248)
> >      at
> >
> org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:604)
> >      at org.jgroups.JChannel.up(JChannel.java:715)
> >      at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
> >      at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
> >      at org.jgroups.protocols.FC.up(FC.java:479)
> >      at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
> >      at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
> >      at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:432)
> >      at
> org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:722)
> >      at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:570)
> >      at
> >
> org.jgroups.protocols.pbcast.NAKACK2.flushBecomeServerQueue(NAKACK2.java:841)
> >      at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:490)
> >      at org.jgroups.protocols.UNICAST2.down(UNICAST2.java:523)
> >      at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:307)
> >      at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:637)
> >      at
> >
> org.jgroups.protocols.pbcast.ClientGmsImpl.installView(ClientGmsImpl.java:248)
> >      at
> >
> org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:182)
> >      at
> > org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:37)
> >      at org.jgroups.protocols.pbcast.GMS.down(GMS.java:938)
> >      at org.jgroups.protocols.FC.down(FC.java:435)
> >      at org.jgroups.protocols.FRAG2.down(FRAG2.java:147)
> >      at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1025)
> >      at org.jgroups.JChannel.down(JChannel.java:729)
> >      at org.jgroups.JChannel.connect(JChannel.java:291)
> >      at org.jgroups.JChannel.connect(JChannel.java:262)
> >      at
> >
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:206)
> >      at
> >
> org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:197)
> >      at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source)
> >      at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >      at java.lang.reflect.Method.invoke(Method.java:601)
> >      at
> >
> org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:236)
> >      at
> >
> org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:900)
> >      at
> >
> org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:650)
> >      at
> >
> org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:639)
> >      at
> >
> org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:542)
> >      at
> >
> org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:218)
> >      at
> >
> org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:680)
> >      at
> >
> org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:652)
> >      at
> >
> org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:548)
> >      at
> > org.infinispan.statetransfer.JoiningNode.getCache(JoiningNode.java:54)
> >      at
> >
> org.infinispan.statetransfer.StateTransferFunctionalTest$2.run(StateTransferFunctionalTest.java:234)
> >      at java.lang.Thread.run(Thread.java:722)
> >
> >
> > Cheers
> > Dan
> >
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20121002/891a9c3f/attachment-0001.html 