[infinispan-issues] [JBoss JIRA] (ISPN-6424) Distributed Executor get can propagate exception if interrupt returns before cancellation

Dan Berindei (JIRA) issues at jboss.org
Wed Mar 23 10:20:00 EDT 2016 

    [ https://issues.jboss.org/browse/ISPN-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181074#comment-13181074 ] 

Dan Berindei edited comment on ISPN-6424 at 3/23/16 10:19 AM:
--------------------------------------------------------------

The problem seems to me to be in the remote node's {{BasePerCacheInboundInvocationHandler}}, not in the distributed executor:

{noformat}
org.testng.TestException: 
Expected exception java.util.concurrent.CancellationException but got java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ReplSyncDistributedExecutorWithTopologyAwareNodesTest-NodeZ-44352, see cause for remote stack trace
    at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1497)
Caused by: java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ReplSyncDistributedExecutorWithTopologyAwareNodesTest-NodeZ-44352, see cause for remote stack trace
    at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
    at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1907)
    at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.getResult(DefaultExecutorService.java:876)
    at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.innerGet(DefaultExecutorService.java:850)
    at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.get(DefaultExecutorService.java:831)
    at org.infinispan.distexec.DistributedExecutorTest.testCancelAndGet(DistributedExecutorTest.java:216)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
    ... 14 more
Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ReplSyncDistributedExecutorWithTopologyAwareNodesTest-NodeZ-44352, see cause for remote stack trace
    at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:792)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$42(JGroupsTransport.java:629)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport$$Lambda$38/748415771.apply(Unknown Source)
    at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
    at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1954)
    at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.futureDone(SingleResponseFuture.java:30)
    at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
    at org.jgroups.blocks.UnicastRequest.receiveResponse(UnicastRequest.java:81)
    at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
    at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
    at org.jgroups.JChannel.up(JChannel.java:738)
    at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
    at org.jgroups.protocols.RSVP.up(RSVP.java:201)
    at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
    at org.jgroups.protocols.tom.TOA.up(TOA.java:121)
    at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
    at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
    at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
    at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
    at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
    at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
    at org.jgroups.protocols.Discovery.up(Discovery.java:296)
    at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
    at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
    ... 3 more
Caused by: org.infinispan.commons.CacheException: Cache is shutting down
    at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.interruptedException(BasePerCacheInboundInvocationHandler.java:121)
    at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:38)
    ... 3 more
{noformat}


was (Author: dan.berindei):
The problem seems to me to be in the remote node's {{BasePerCacheInboundInvocationHandler}}, not in the distributed executor:

{noformat}
org.testng.TestException: 
Expected exception java.util.concurrent.CancellationException but got java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ReplSyncDistributedExecutorWithTopologyAwareNodesTest-NodeZ-44352, see cause for remote stack trace
    at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1497)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:754)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
    at org.testng.TestRunner.privateRun(TestRunner.java:767)
    at org.testng.TestRunner.run(TestRunner.java:617)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
    at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
    at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
    at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ReplSyncDistributedExecutorWithTopologyAwareNodesTest-NodeZ-44352, see cause for remote stack trace
    at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
    at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1907)
    at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.getResult(DefaultExecutorService.java:876)
    at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.innerGet(DefaultExecutorService.java:850)
    at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.get(DefaultExecutorService.java:831)
    at org.infinispan.distexec.DistributedExecutorTest.testCancelAndGet(DistributedExecutorTest.java:216)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
    ... 14 more
Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from ReplSyncDistributedExecutorWithTopologyAwareNodesTest-NodeZ-44352, see cause for remote stack trace
    at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:792)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$42(JGroupsTransport.java:629)
    at org.infinispan.remoting.transport.jgroups.JGroupsTransport$$Lambda$38/748415771.apply(Unknown Source)
    at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
    at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1954)
    at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.futureDone(SingleResponseFuture.java:30)
    at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
    at org.jgroups.blocks.UnicastRequest.receiveResponse(UnicastRequest.java:81)
    at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
    at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
    at org.jgroups.JChannel.up(JChannel.java:738)
    at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
    at org.jgroups.protocols.RSVP.up(RSVP.java:201)
    at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
    at org.jgroups.protocols.tom.TOA.up(TOA.java:121)
    at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
    at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
    at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
    at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
    at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
    at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
    at org.jgroups.protocols.Discovery.up(Discovery.java:296)
    at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
    at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
    ... 3 more
Caused by: org.infinispan.commons.CacheException: Cache is shutting down
    at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.interruptedException(BasePerCacheInboundInvocationHandler.java:121)
    at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:38)
    ... 3 more
{noformat}

> Distributed Executor get can propagate exception if interrupt returns before cancellation
> -----------------------------------------------------------------------------------------
>
>                 Key: ISPN-6424
>                 URL: https://issues.jboss.org/browse/ISPN-6424
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Distributed Execution and Map/Reduce
>            Reporter: William Burns
>            Assignee: William Burns
>             Fix For: 9.0.0.Alpha1
>
>
> This can cause get method to return an exception of the operation being interrupted instead of a CancellationException as it should.
> This causes a test failure http://ci.infinispan.org/viewLog.html?buildId=37756&tab=buildResultsDiv&buildTypeId=Infinispan_MasterHotspotJdk8#testNameId-3420236380418240325



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)

More information about the infinispan-issues mailing list