[infinispan-issues] [JBoss JIRA] (ISPN-5573) Node shutting down can throw IllegalLifecycleStateException wrapped multiple times

Wed Jun 24 07:48:03 EDT 2015

William Burns created ISPN-5573:
-----------------------------------

             Summary: Node shutting down can throw IllegalLifecycleStateException wrapped multiple times
                 Key: ISPN-5573
                 URL: https://issues.jboss.org/browse/ISPN-5573
             Project: Infinispan
          Issue Type: Bug
          Components: Core
    Affects Versions: 8.0.0.Alpha2
            Reporter: William Burns
            Assignee: William Burns
             Fix For: 8.0.0.Beta1

When working on some new work I have a stress test that constantly stops and starts a cache to test the resiliency of my code.

I found that every once in a while if I have something that takes a little longer to serialize it could throw back a non SuspectException to the originator.  It looks like the underlying issue is that IllegalLifecycleStateException is wrapped by multiple exceptions and thus it comes back as a simple CacheException instead which treats the exception properly.

{code}
15:31:48,053 TRACE (OOB-2,main-NodeD-61136:) [ClusterStreamManagerImpl] Encounted exception for cf133b23-9bbf-49ee-823d-f2f650f39d9c from main-NodeH-22716
java.util.concurrent.CompletionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from main-NodeH-22716, see cause for remote stack trace
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
	at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1954)
	at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.futureDone(SingleResponseFuture.java:27)
	at org.jgroups.blocks.Request.checkCompletion(Request.java:169)
	at org.jgroups.blocks.UnicastRequest.receiveResponse(UnicastRequest.java:83)
	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:398)
	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:250)
	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:675)
	at org.jgroups.JChannel.up(JChannel.java:739)
	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1029)
	at org.jgroups.protocols.RSVP.up(RSVP.java:201)
	at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
	at org.jgroups.protocols.tom.TOA.up(TOA.java:121)
	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1042)
	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
	at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
	at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
	at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
	at org.jgroups.protocols.Discovery.up(Discovery.java:291)
	at org.jgroups.protocols.TP.passMessageUp(TP.java:1577)
	at org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from main-NodeH-22716, see cause for remote stack trace
	at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:747)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$76(JGroupsTransport.java:586)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport$$Lambda$5/1861463967.apply(Unknown Source)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	... 26 more
Caused by: org.infinispan.commons.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.rethrowAsCacheException(CommandAwareRpcDispatcher.java:127)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:119)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotelyAsync(JGroupsTransport.java:565)
	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotelyAsync(RpcManagerImpl.java:180)
	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:222)
	at org.infinispan.stream.impl.LocalStreamManagerImpl.streamOperationRehashAware(LocalStreamManagerImpl.java:240)
	at org.infinispan.stream.impl.StreamRequestCommand.perform(StreamRequestCommand.java:93)
	at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
	at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
	... 3 more
Caused by: java.lang.RuntimeException: Failure to marshal argument(s)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:238)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:252)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:116)
	... 10 more
Caused by: org.infinispan.IllegalLifecycleStateException: Cache marshaller has been stopped
	at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.getObjectWriter(JBossMarshaller.java:146)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:132)
	at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
	at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
	at org.infinispan.commons.marshall.MarshallUtil.marshallMap(MarshallUtil.java:36)
	at org.infinispan.marshall.exts.MapExternalizer.writeObject(MapExternalizer.java:60)
	at org.infinispan.marshall.exts.MapExternalizer.writeObject(MapExternalizer.java:28)
	at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:442)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:138)
	at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
	at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
	at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:58)
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.marshallParameters(CacheRpcCommandExternalizer.java:128)
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:112)
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:68)
	at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:442)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:138)
	at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
	at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
	at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectToObjectStream(AbstractJBossMarshaller.java:72)
	at org.infinispan.marshall.core.VersionAwareMarshaller.objectToBuffer(VersionAwareMarshaller.java:77)
	at org.infinispan.commons.marshall.AbstractMarshaller.objectToBuffer(AbstractMarshaller.java:41)
	at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectToBuffer(AbstractDelegatingMarshaller.java:85)
	at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectToBuffer(MarshallerAdapter.java:23)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:236)
	... 12 more
Caused by: an exception which occurred:
	in object java.lang.Integer at 20d5
	in object java.util.HashMap at 0
	in object org.infinispan.stream.impl.StreamResponseCommand at 66d77895
{code}

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)