[jboss-jira] [JBoss JIRA] (WFLY-3839) EjbClient: No cluster node manager found for node XY during server restart

[Tomas Hofman (JIRA)]

     [ https://issues.jboss.org/browse/WFLY-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tomas Hofman reassigned WFLY-3839:
----------------------------------

    Assignee: Tomas Hofman  (was: Richard Achmatowicz)


>  EjbClient: No cluster node manager found for node XY during server restart
> ---------------------------------------------------------------------------
>
>                 Key: WFLY-3839
>                 URL: https://issues.jboss.org/browse/WFLY-3839
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering, EJB
>    Affects Versions: 8.1.0.Final
>            Reporter: Dominik Pospisil
>            Assignee: Tomas Hofman
>
> Situation: 4-node cluster, one node is failed at a time, during the whole test, clients are accessing stateless clustered EJB3. The failure type in this particular case is simulating server crash: JVM is killed using kill -9.
> When the server was starting, being brought back, we saw this error: 
> 16:05:29,966 ERROR [org.jboss.ejb.client.ClusterContext] (ejb-client-cluster-node-connection-creation-2-thread-17) Cannot create EJBReceiver since no cluster node manager found for node perf20 in cluster context for cluster ejb
> Here is the log output again with context, the server was starting: 
> {noformat}
> 16:05:29,966 INFO  [org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler] (Remoting "config-based-ejb-client-endpoint" task-2) Retrying invocation which failed on node perf20 with exception:
> javax.ejb.NoSuchEJBException: No such EJB[appname=clusterbench-ee6,modulename=clusterbench-ee6-ejb,distinctname=,beanname=RemoteStatelessSBImpl]
> 	at org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler.processMessage(NoSuchEJBExceptionResponseHandler.java:64)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation.processResponse(ChannelAssociation.java:366)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation$ResponseReceiver.handleMessage(ChannelAssociation.java:458)
> 	at org.jboss.remoting3.remote.RemoteConnectionChannel$4.run(RemoteConnectionChannel.java:373)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> 	at java.lang.Thread.run(Thread.java:662)
> 16:05:29,966 ERROR [org.jboss.ejb.client.ClusterContext] (ejb-client-cluster-node-connection-creation-2-thread-17) Cannot create EJBReceiver since no cluster node manager found for node perf20 in cluster context for cluster ejb
> 16:05:29,974 INFO  [org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler] (Remoting "config-based-ejb-client-endpoint" task-2) Retrying invocation which failed on node perf20 with exception:
> javax.ejb.NoSuchEJBException: No such EJB[appname=clusterbench-ee6,modulename=clusterbench-ee6-ejb,distinctname=,beanname=RemoteStatelessSBImpl]
> 	at org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler.processMessage(NoSuchEJBExceptionResponseHandler.java:64)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation.processResponse(ChannelAssociation.java:366)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation$ResponseReceiver.handleMessage(ChannelAssociation.java:458)
> 	at org.jboss.remoting3.remote.RemoteConnectionChannel$4.run(RemoteConnectionChannel.java:373)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> 	at java.lang.Thread.run(Thread.java:662)
> 16:05:29,974 ERROR [org.jboss.ejb.client.ClusterContext] (ejb-client-cluster-node-connection-creation-2-thread-18) Cannot create EJBReceiver since no cluster node manager found for node perf20 in cluster context for cluster ejb
> 16:05:29,975 INFO  [org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler] (Remoting "config-based-ejb-client-endpoint" task-2) Retrying invocation which failed on node perf20 with exception:
> javax.ejb.NoSuchEJBException: No such EJB[appname=clusterbench-ee6,modulename=clusterbench-ee6-ejb,distinctname=,beanname=RemoteStatelessSBImpl]
> 	at org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler.processMessage(NoSuchEJBExceptionResponseHandler.java:64)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation.processResponse(ChannelAssociation.java:366)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation$ResponseReceiver.handleMessage(ChannelAssociation.java:458)
> 	at org.jboss.remoting3.remote.RemoteConnectionChannel$4.run(RemoteConnectionChannel.java:373)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> 	at java.lang.Thread.run(Thread.java:662)
> 16:05:29,975 INFO  [org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler] (Remoting "config-based-ejb-client-endpoint" task-2) Retrying invocation which failed on node perf20 with exception:
> javax.ejb.NoSuchEJBException: No such EJB[appname=clusterbench-ee6,modulename=clusterbench-ee6-ejb,distinctname=,beanname=RemoteStatelessSBImpl]
> 	at org.jboss.ejb.client.remoting.NoSuchEJBExceptionResponseHandler.processMessage(NoSuchEJBExceptionResponseHandler.java:64)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation.processResponse(ChannelAssociation.java:366)
> 	at org.jboss.ejb.client.remoting.ChannelAssociation$ResponseReceiver.handleMessage(ChannelAssociation.java:458)
> 	at org.jboss.remoting3.remote.RemoteConnectionChannel$4.run(RemoteConnectionChannel.java:373)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> {noformat}
> During the whole test, there were 4 server crashes and startups (1 for each node), but only 3 occurences of the above mentioned error. These cluster nodes are perf18, perf19, perf20, perf21, but this error was seen only for perf20 (two occurences) and perf18 (one occurence).
> I did not find anything suspicious in the server.log of perf20.
> Cache: REPL_ASYNC
> Versions: EAP 6.1.0.ER6, ejb-client 1.0.19.Final
> Link to hudson job: 
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbstateless-jvmkill-repl-async/15/
> Server log:
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-failover-ejb-ejbstateless-jvmkill-repl-async/15/artifact/report/config/jboss-perf20/server.log



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)