[JBoss JIRA] (ISPN-6721) SuspectException when stopping and starting nodes in an embedded cluster using invalidation
by Karl von Randow (JIRA)
[ https://issues.jboss.org/browse/ISPN-6721?page=com.atlassian.jira.plugin.... ]
Karl von Randow updated ISPN-6721:
----------------------------------
Description:
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
We are using synchronous invalidation.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
I note in this wiki article https://github.com/infinispan/infinispan/wiki/Consistency-guarantees-in-I... there is a note "TODO Create an issue in JIRA to ignore suspect exceptions.". I'm not sure if this is related!
There are two stacktraces below. I note that the shutdown example fails in {{JGroupsTransport.java:798}} which is when the response was suspected. While the startup example fails in {{JGroupsTransport.java:795}}, which is a {{CacheNotFoundResponse}} response.
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
was:
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
There are two stacktraces below. I note that the shutdown example fails in {{JGroupsTransport.java:798}} which is when the response was suspected. While the startup example fails in {{JGroupsTransport.java:795}}, which is a {{CacheNotFoundResponse}} response.
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
> SuspectException when stopping and starting nodes in an embedded cluster using invalidation
> -------------------------------------------------------------------------------------------
>
> Key: ISPN-6721
> URL: https://issues.jboss.org/browse/ISPN-6721
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 8.2.2.Final
> Reporter: Karl von Randow
>
> We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
> We are using synchronous invalidation.
> This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
> I note in this wiki article https://github.com/infinispan/infinispan/wiki/Consistency-guarantees-in-I... there is a note "TODO Create an issue in JIRA to ignore suspect exceptions.". I'm not sure if this is related!
> There are two stacktraces below. I note that the shutdown example fails in {{JGroupsTransport.java:798}} which is when the response was suspected. While the startup example fails in {{JGroupsTransport.java:795}}, which is a {{CacheNotFoundResponse}} response.
> Here is a shutdown exception:
> {code:java}
> May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
> SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
> at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
> at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
> at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
> at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
> at org.jgroups.stack.Protocol.up(Protocol.java:417)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
> at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
> at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
> at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:296)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
> at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> And a startup exception:
> {code:java}
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
> at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
> at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
> at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:296)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
> at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6721) SuspectException when stopping and starting nodes in an embedded cluster using invalidation
by Karl von Randow (JIRA)
[ https://issues.jboss.org/browse/ISPN-6721?page=com.atlassian.jira.plugin.... ]
Karl von Randow updated ISPN-6721:
----------------------------------
Description:
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
There are two stacktraces below. I note that the shutdown example fails in {{JGroupsTransport.java:798}} which is when the response was suspected. While the startup example fails in {{JGroupsTransport.java:795}}, which is a {{CacheNotFoundResponse}} response.
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
was:
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
> SuspectException when stopping and starting nodes in an embedded cluster using invalidation
> -------------------------------------------------------------------------------------------
>
> Key: ISPN-6721
> URL: https://issues.jboss.org/browse/ISPN-6721
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 8.2.2.Final
> Reporter: Karl von Randow
>
> We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
> This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
> There are two stacktraces below. I note that the shutdown example fails in {{JGroupsTransport.java:798}} which is when the response was suspected. While the startup example fails in {{JGroupsTransport.java:795}}, which is a {{CacheNotFoundResponse}} response.
> Here is a shutdown exception:
> {code:java}
> May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
> SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
> at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
> at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
> at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
> at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
> at org.jgroups.stack.Protocol.up(Protocol.java:417)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
> at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
> at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
> at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:296)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
> at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> And a startup exception:
> {code:java}
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
> at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
> at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
> at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:296)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
> at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6721) SuspectException when stopping and starting nodes in an embedded cluster using invalidation
by Karl von Randow (JIRA)
[ https://issues.jboss.org/browse/ISPN-6721?page=com.atlassian.jira.plugin.... ]
Karl von Randow updated ISPN-6721:
----------------------------------
Description:
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
was:
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code:java}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code:java}
> SuspectException when stopping and starting nodes in an embedded cluster using invalidation
> -------------------------------------------------------------------------------------------
>
> Key: ISPN-6721
> URL: https://issues.jboss.org/browse/ISPN-6721
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 8.2.2.Final
> Reporter: Karl von Randow
>
> We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
> This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
> Here is a shutdown exception:
> {code:java}
> May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
> SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
> at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
> at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
> at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
> at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
> at org.jgroups.stack.Protocol.up(Protocol.java:417)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
> at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
> at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
> at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:296)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
> at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> And a startup exception:
> {code:java}
> org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
> at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
> at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
> at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:296)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
> at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6721) SuspectException when stopping and starting nodes in an embedded cluster using invalidation
by Karl von Randow (JIRA)
Karl von Randow created ISPN-6721:
-------------------------------------
Summary: SuspectException when stopping and starting nodes in an embedded cluster using invalidation
Key: ISPN-6721
URL: https://issues.jboss.org/browse/ISPN-6721
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 8.2.2.Final
Reporter: Karl von Randow
We run a cluster of four app servers on Tomcat with embedded Infinispan for Hibernate L2. When we bring down one of the app servers it shuts down that cache and then exits. On other nodes we frequently, although not consistently, receive SuspectExceptions from other nodes stating that the cache isn't running on the shutting down node. Similarly when starting a new app server we get the same.
This feels like a race condition during startup and shutdown of the caches. Does that sound likely?
Here is a shutdown exception:
{code:java}
May 27, 2016 3:05:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-12786
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:798)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.viewChange(GroupRequest.java:250)
at org.jgroups.blocks.RequestCorrelator.receiveView(RequestCorrelator.java:316)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:229)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:733)
at org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleViewChange(ParticipantGmsImpl.java:140)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:923)
at org.jgroups.stack.Protocol.up(Protocol.java:417)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:474)
at org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:982)
at org.jgroups.protocols.pbcast.NAKACK2.removeAndPassUp(NAKACK2.java:912)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:846)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:618)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code:java}
And a startup exception:
{code:java}
org.infinispan.remoting.transport.jgroups.SuspectException: Cache not running on node app1-46933
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:46)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:795)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.RspListFuture.futureDone(RspListFuture.java:31)
at org.jgroups.blocks.Request.checkCompletion(Request.java:162)
at org.jgroups.blocks.GroupRequest.receiveResponse(GroupRequest.java:136)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:373)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:237)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:695)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:392)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1043)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:296)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1590)
at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code:java}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6720) Module tests depend on both testng and junit
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6720?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6720:
----------------------------------
Status: Open (was: New)
> Module tests depend on both testng and junit
> --------------------------------------------
>
> Key: ISPN-6720
> URL: https://issues.jboss.org/browse/ISPN-6720
> Project: Infinispan
> Issue Type: Bug
> Components: Build process
> Affects Versions: 9.0.0.Alpha2
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
> Fix For: 9.0.0.Alpha3
>
>
> The parent pom adds default dependencies to testng and junit (as well as mockito). Because of this many modules use a mix of test utility classes.
> We should drop the default dependencies and force each module to choose the testing libraries it needs.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6720) Module tests depend on both testng and junit
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6720?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6720:
----------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4377
> Module tests depend on both testng and junit
> --------------------------------------------
>
> Key: ISPN-6720
> URL: https://issues.jboss.org/browse/ISPN-6720
> Project: Infinispan
> Issue Type: Bug
> Components: Build process
> Affects Versions: 9.0.0.Alpha2
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
> Fix For: 9.0.0.Alpha3
>
>
> The parent pom adds default dependencies to testng and junit (as well as mockito). Because of this many modules use a mix of test utility classes.
> We should drop the default dependencies and force each module to choose the testing libraries it needs.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6720) Module tests depend on both testng and junit
by Tristan Tarrant (JIRA)
Tristan Tarrant created ISPN-6720:
-------------------------------------
Summary: Module tests depend on both testng and junit
Key: ISPN-6720
URL: https://issues.jboss.org/browse/ISPN-6720
Project: Infinispan
Issue Type: Bug
Components: Build process
Affects Versions: 9.0.0.Alpha2
Reporter: Tristan Tarrant
Assignee: Tristan Tarrant
Fix For: 9.0.0.Alpha3
The parent pom adds default dependencies to testng and junit (as well as mockito). Because of this many modules use a mix of test utility classes.
We should drop the default dependencies and force each module to choose the testing libraries it needs.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6719) Add class attributes to element on JDBC store configuration for testing
by Jiří Holuša (JIRA)
[ https://issues.jboss.org/browse/ISPN-6719?page=com.atlassian.jira.plugin.... ]
Jiří Holuša reassigned ISPN-6719:
---------------------------------
Assignee: Vladimir Blagojevic (was: Jiří Holuša)
> Add class attributes to element on JDBC store configuration for testing
> -----------------------------------------------------------------------
>
> Key: ISPN-6719
> URL: https://issues.jboss.org/browse/ISPN-6719
> Project: Infinispan
> Issue Type: Enhancement
> Components: Console
> Affects Versions: 8.2.2.Final
> Reporter: Jiří Holuša
> Assignee: Vladimir Blagojevic
>
> In the cache configuration page, on the JDBC store tab, there are several inputs/selects. To be able to test is automatically, it's very helpful to have the inputs/selects marked with classes just like with File store or LevelDB tabs.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6719) Add class attributes to element on JDBC store configuration for testing
by Jiří Holuša (JIRA)
Jiří Holuša created ISPN-6719:
---------------------------------
Summary: Add class attributes to element on JDBC store configuration for testing
Key: ISPN-6719
URL: https://issues.jboss.org/browse/ISPN-6719
Project: Infinispan
Issue Type: Enhancement
Components: Console
Affects Versions: 8.2.2.Final
Reporter: Jiří Holuša
Assignee: Jiří Holuša
In the cache configuration page, on the JDBC store tab, there are several inputs/selects. To be able to test is automatically, it's very helpful to have the inputs/selects marked with classes just like with File store or LevelDB tabs.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0
[JBoss JIRA] (ISPN-6718) Improve error messages
by Jiří Holuša (JIRA)
Jiří Holuša created ISPN-6718:
---------------------------------
Summary: Improve error messages
Key: ISPN-6718
URL: https://issues.jboss.org/browse/ISPN-6718
Project: Infinispan
Issue Type: Enhancement
Components: Console
Affects Versions: 8.2.2.Final
Reporter: Jiří Holuša
Assignee: Vladimir Blagojevic
Priority: Minor
When editing cache configuration, I try to put some invalid value, for example on "Transactions" tab into "Stop timeout" field, when I put a string "xyz", which is trivially invalid since it's not a number, I get an error dialog with text:
"domain-failure-description
WFLYCTL0097: Wrong type for stop-timeout. Expected [LONG] but was STRING"
The text is very descriptive, which is fine. I would suggest removing the "domain-failure-description" string and maybe even the WildFly error code, to make it more user friendly.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
- 1
- 0