[jboss-jira] [JBoss JIRA] (WFLY-6168) Huge burst of java.lang.IllegalArgumentException: Segments [18] are not owned by <slave-node>:<server-name>
Stijn de Witt (JIRA)
issues at jboss.org
Tue Apr 12 18:00:01 EDT 2016
[ https://issues.jboss.org/browse/WFLY-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190849#comment-13190849 ]
Stijn de Witt commented on WFLY-6168:
-------------------------------------
I'm seeing this issue on WildFly 10.0.0.Final.
I am (was) running two WildFly 10 nodes ('gears') on OpenShift.
I have set the web cache to do optimistic locking. That is about the only change from the default config OpenShift's WildFly 10 cartridge comes with that I made.
Below is a snippet from the logs. The servers had gotten into an infinite loop of printing this exception until the logs drained available cartridge space.
{{2016-04-12 14:59:48,168 WARN [org.infinispan.statetransfer.StateConsumerImpl] (transport-thread--p16-t2) ISPN000209: Failed to retrieve transactions for segments [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79] of cache routing from node 5708481e2d52714ecd000166-brautschloss.rhcloud.com: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from 5708481e2d52714ecd000166-brautschloss.rhcloud.com, see cause for remote stack trace
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:760)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$72(JGroupsTransport.java:599)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.futureDone(SingleResponseFuture.java:30)
at org.jgroups.blocks.Request.checkCompletion(Request.java:169)
at org.jgroups.blocks.UnicastRequest.receiveResponse(UnicastRequest.java:83)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:398)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:250)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:684)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:119)
at org.jgroups.stack.Protocol.up(Protocol.java:374)
at org.jgroups.protocols.FORK.up(FORK.java:114)
at org.jgroups.protocols.RSVP.up(RSVP.java:201)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1045)
at org.jgroups.protocols.AUTH.up(AUTH.java:148)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD.up(FD.java:260)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:295)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1577)
at org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Node 5708481e2d52714ecd000166-brautschloss.rhcloud.com is not a member
at org.infinispan.distribution.ch.impl.DefaultConsistentHash.getSegmentsForOwner(DefaultConsistentHash.java:115)
at org.infinispan.distribution.group.GroupingConsistentHash.getSegmentsForOwner(GroupingConsistentHash.java:67)
at org.infinispan.statetransfer.StateProviderImpl.getTransactionsForSegments(StateProviderImpl.java:163)
at org.infinispan.statetransfer.StateRequestCommand.perform(StateRequestCommand.java:67)
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
... 3 more
}}
> Huge burst of java.lang.IllegalArgumentException: Segments [18] are not owned by <slave-node>:<server-name>
> -----------------------------------------------------------------------------------------------------------
>
> Key: WFLY-6168
> URL: https://issues.jboss.org/browse/WFLY-6168
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.CR4
> Reporter: Thiago Presa
> Assignee: Paul Ferraro
>
> We have a Wildfly 10 CR4 cluster in domain mode with 2 slaves, and we're getting a huge burst of the following log entry:
> 10:57:31,065 WARN [org.infinispan.statetransfer.StateConsumerImpl] (transport-thread--p13-t16) ISPN000209: Failed to retrieve transactions for segments [18] of cache dist from node <slave-host>:<server-name>: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from <slave-host>:<server-name>, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:750)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$80(JGroupsTransport.java:589)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.futureDone(SingleResponseFuture.java:30)
> at org.jgroups.blocks.Request.checkCompletion(Request.java:169)
> at org.jgroups.blocks.UnicastRequest.receiveResponse(UnicastRequest.java:83)
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:398)
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:250)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:684)
> at org.jgroups.JChannel.up(JChannel.java:738)
> at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:119)
> at org.jgroups.stack.Protocol.up(Protocol.java:374)
> at org.jgroups.protocols.FORK.up(FORK.java:114)
> at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1045)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
> at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
> at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
> at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
> at org.jgroups.protocols.FD.up(FD.java:260)
> at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
> at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
> at org.jgroups.protocols.Discovery.up(Discovery.java:295)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1577)
> at org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Segments [18] are not owned by <slave-host>:<server-name>
> at org.infinispan.statetransfer.StateProviderImpl.getTransactionsForSegments(StateProviderImpl.java:166)
> at org.infinispan.statetransfer.StateRequestCommand.perform(StateRequestCommand.java:67)
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> ... 3 more
> It's funny because server-name is not the name of the server where this log entry is found. What may be causing this?
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
More information about the jboss-jira
mailing list