[jboss-jira] [JBoss JIRA] (WFLY-10274) NPE during server shutdown when using scattered cache
Michal Vinkler (JIRA)
issues at jboss.org
Fri Apr 20 08:06:00 EDT 2018
[ https://issues.jboss.org/browse/WFLY-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michal Vinkler updated WFLY-10274:
----------------------------------
Description:
We hit NPE when running tests for RFE EAP7-867.
EAP distribution was built from https://github.com/pferraro/wildfly/tree/scattered .
Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.
During clean server shutdown in the end of the test, server logged NPE and got stuck:
{code}
[JBossINF] [0m[31m07:55:57,643 ERROR [org.infinispan.scattered.impl.ScatteredStateConsumerImpl] (thread-200,ejb,dev214) ISPN000471: Failed processing values received from remote node during rebalance.: java.lang.NullPointerException
[JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.applyValues(ScatteredStateConsumerImpl.java:505)
[JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.lambda$getValuesAndApply$8(ScatteredStateConsumerImpl.java:475)
[JBossINF] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
[JBossINF] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
[JBossINF] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[JBossINF] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:66)
[JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:56)
[JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.onResponse(SingleTargetRequest.java:35)
[JBossINF] at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:53)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1304)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1207)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:123)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1342)
[JBossINF] at org.jgroups.JChannel.up(JChannel.java:819)
[JBossINF] at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:134)
[JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:340)
[JBossINF] at org.jgroups.protocols.FORK.up(FORK.java:134)
[JBossINF] at org.jgroups.protocols.FRAG3.up(FRAG3.java:166)
[JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
[JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
[JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:864)
[JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:240)
[JBossINF] at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1002)
[JBossINF] at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:728)
[JBossINF] at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:383)
[JBossINF] at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
[JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:119)
[JBossINF] at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:199)
[JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:252)
[JBossINF] at org.jgroups.protocols.MERGE3.up(MERGE3.java:276)
[JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:267)
[JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1248)
[JBossINF] at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF]
{code}
Scattered cache was configured with bias-lifespan="0".
Server configuration:
http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/14/artifact/report/config/jboss-dev214/standalone-ha.xml
Server link:
http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/14/console-dev214/
was:
We hit NPE when running tests for RFE EAP7-867.
EAP distribution was built from https://github.com/pferraro/wildfly/tree/scattered .
Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.
During clean server shutdown in the end of the test, server logged NPE and got stuck:
{code}
[JBossINF] [0m[31m07:55:57,643 ERROR [org.infinispan.scattered.impl.ScatteredStateConsumerImpl] (thread-200,ejb,dev214) ISPN000471: Failed processing values received from remote node during rebalance.: java.lang.NullPointerException
[JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.applyValues(ScatteredStateConsumerImpl.java:505)
[JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.lambda$getValuesAndApply$8(ScatteredStateConsumerImpl.java:475)
[JBossINF] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
[JBossINF] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
[JBossINF] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[JBossINF] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
[JBossINF] at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:66)
[JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:56)
[JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.onResponse(SingleTargetRequest.java:35)
[JBossINF] at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:53)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1304)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1207)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:123)
[JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1342)
[JBossINF] at org.jgroups.JChannel.up(JChannel.java:819)
[JBossINF] at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:134)
[JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:340)
[JBossINF] at org.jgroups.protocols.FORK.up(FORK.java:134)
[JBossINF] at org.jgroups.protocols.FRAG3.up(FRAG3.java:166)
[JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
[JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
[JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:864)
[JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:240)
[JBossINF] at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1002)
[JBossINF] at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:728)
[JBossINF] at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:383)
[JBossINF] at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
[JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:119)
[JBossINF] at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:199)
[JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:252)
[JBossINF] at org.jgroups.protocols.MERGE3.up(MERGE3.java:276)
[JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:267)
[JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1248)
[JBossINF] at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF]
{code}
Server link:
https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/14/console-dev214/
> NPE during server shutdown when using scattered cache
> -----------------------------------------------------
>
> Key: WFLY-10274
> URL: https://issues.jboss.org/browse/WFLY-10274
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 13.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
>
> We hit NPE when running tests for RFE EAP7-867.
> EAP distribution was built from https://github.com/pferraro/wildfly/tree/scattered .
> Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.
> During clean server shutdown in the end of the test, server logged NPE and got stuck:
> {code}
> [JBossINF] [0m[31m07:55:57,643 ERROR [org.infinispan.scattered.impl.ScatteredStateConsumerImpl] (thread-200,ejb,dev214) ISPN000471: Failed processing values received from remote node during rebalance.: java.lang.NullPointerException
> [JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.applyValues(ScatteredStateConsumerImpl.java:505)
> [JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.lambda$getValuesAndApply$8(ScatteredStateConsumerImpl.java:475)
> [JBossINF] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> [JBossINF] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> [JBossINF] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> [JBossINF] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:66)
> [JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:56)
> [JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.onResponse(SingleTargetRequest.java:35)
> [JBossINF] at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:53)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1304)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1207)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:123)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1342)
> [JBossINF] at org.jgroups.JChannel.up(JChannel.java:819)
> [JBossINF] at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:134)
> [JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:340)
> [JBossINF] at org.jgroups.protocols.FORK.up(FORK.java:134)
> [JBossINF] at org.jgroups.protocols.FRAG3.up(FRAG3.java:166)
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> [JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:864)
> [JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:240)
> [JBossINF] at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1002)
> [JBossINF] at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:728)
> [JBossINF] at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:383)
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
> [JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:119)
> [JBossINF] at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:199)
> [JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:252)
> [JBossINF] at org.jgroups.protocols.MERGE3.up(MERGE3.java:276)
> [JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:267)
> [JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1248)
> [JBossINF] at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> [JBossINF]
> {code}
> Scattered cache was configured with bias-lifespan="0".
> Server configuration:
> http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/14/artifact/report/config/jboss-dev214/standalone-ha.xml
> Server link:
> http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/14/console-dev214/
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
More information about the jboss-jira
mailing list