[jboss-jira] [JBoss JIRA] (WFLY-10642) javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed

[ https://issues.jboss.org/browse/WFLY-10642?page=com.atlassian.jira.plugin... ] tommaso borgato updated WFLY-10642: ----------------------------------- Description: Affected scenario: eap-7x-failover-ejb-ejbservlet-undeploy-repl-sync. Right after *clusterbench* has been undeployed, we see these exceptions: {noformat} javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed {noformat} They come right after these exceptions: {noformat} org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down {noformat} Here is the relevant part of the logs (complete logs [here|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap...]): {code:java} 018/05/21 13:31:54:915 EDT [DEBUG][RMI TCP Connection(56)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Going to undeploy: 'clusterbench-ee7.ear' from 'perf18' now. 13:31:55,034 INFO [org.jboss.threads] (RMI TCP Connection(56)-10.16.90.52) JBoss Threads version 2.3.2.Final 2018/05/21 13:31:55:196 EDT [DEBUG][RMI TCP Connection(56)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Requesting: { "operation" : "undeploy", "address" : [{ "deployment" : "clusterbench-ee7.ear" }] } 13:31:55,266 INFO [org.jboss.remoting] (RMI TCP Connection(56)-10.16.90.52) JBoss Remoting version 5.0.7.Final 13:31:55,401 INFO [org.xnio] (RMI TCP Connection(56)-10.16.90.52) XNIO version 3.6.3.Final 13:31:55,412 INFO [org.xnio.nio] (RMI TCP Connection(56)-10.16.90.52) XNIO NIO Implementation Version 3.6.3.Final 13:31:55,534 INFO [org.wildfly.security] (RMI TCP Connection(56)-10.16.90.52) ELY00001: WildFly Elytron version 1.3.2.Final [JBossINF] 13:31:56,658 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 82) WFLYUT0022: Unregistered web context: '/clusterbench-passivating' from server 'default-server' [JBossINF] 13:31:56,673 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 84) WFLYUT0022: Unregistered web context: '/clusterbench-granular' from server 'default-server' [JBossINF] 13:31:56,673 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 83) WFLYUT0022: Unregistered web context: '/clusterbench' from server 'default-server' [JBossINF] 13:31:56,662 ERROR [io.undertow.request] (default task-10) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down [JBossINF] 13:31:56,661 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down [repeated 6 times] [JBossINF] 13:31:56,690 ERROR [io.undertow.request] (default task-11) UT005023: Exception handling request to /clusterbench/ejbservlet: javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed: StatefulSessionComponent LocalStatefulSB [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.getBusinessObject(StatefulSessionObjectReferenceImpl.java:83) [JBossINF] at org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:129) [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [JBossINF] at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:67) [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] [JBossINF] 13:31:56,695 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /clusterbench/ejbservlet: javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed: StatefulSessionComponent LocalStatefulSB [JBossINF] 13:31:57,148 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment clusterbench-ee7.ear (runtime-name: clusterbench-ee7.ear) in 523ms [JBossINF] 13:31:57,251 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0009: Undeployed "clusterbench-ee7.ear" (runtime-name: "clusterbench-ee7.ear") 2018/05/21 13:31:57:267 EDT [DEBUG][RMI TCP Connection(56)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Got result: {"outcome" : "success"} 2018/05/21 13:32:57:291 EDT [DEBUG][RMI TCP Connection(60)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Going to deploy back: 'clusterbench-ee7.ear' on 'perf18' now. 2018/05/21 13:32:57:292 EDT [DEBUG][RMI TCP Connection(60)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Requesting: { "operation" : "deploy", "address" : [{ "deployment" : "clusterbench-ee7.ear" }] } {code} was: Affected scenario: eap-7x-failover-ejb-ejbservlet-undeploy-repl-sync. Right after the clusterbench has been undeployed, we see these exceptions: {noformat} javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed {noformat} They come right after these exceptions: {noformat} org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down {noformat} Here is the relevant part of the logs (complete logs [here|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap...]): {code:java} 018/05/21 13:31:54:915 EDT [DEBUG][RMI TCP Connection(56)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Going to undeploy: 'clusterbench-ee7.ear' from 'perf18' now. 13:31:55,034 INFO [org.jboss.threads] (RMI TCP Connection(56)-10.16.90.52) JBoss Threads version 2.3.2.Final 2018/05/21 13:31:55:196 EDT [DEBUG][RMI TCP Connection(56)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Requesting: { "operation" : "undeploy", "address" : [{ "deployment" : "clusterbench-ee7.ear" }] } 13:31:55,266 INFO [org.jboss.remoting] (RMI TCP Connection(56)-10.16.90.52) JBoss Remoting version 5.0.7.Final 13:31:55,401 INFO [org.xnio] (RMI TCP Connection(56)-10.16.90.52) XNIO version 3.6.3.Final 13:31:55,412 INFO [org.xnio.nio] (RMI TCP Connection(56)-10.16.90.52) XNIO NIO Implementation Version 3.6.3.Final 13:31:55,534 INFO [org.wildfly.security] (RMI TCP Connection(56)-10.16.90.52) ELY00001: WildFly Elytron version 1.3.2.Final [JBossINF] 13:31:56,658 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 82) WFLYUT0022: Unregistered web context: '/clusterbench-passivating' from server 'default-server' [JBossINF] 13:31:56,673 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 84) WFLYUT0022: Unregistered web context: '/clusterbench-granular' from server 'default-server' [JBossINF] 13:31:56,673 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 83) WFLYUT0022: Unregistered web context: '/clusterbench' from server 'default-server' [JBossINF] 13:31:56,662 ERROR [io.undertow.request] (default task-10) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down [JBossINF] 13:31:56,661 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down [repeated 6 times] [JBossINF] 13:31:56,690 ERROR [io.undertow.request] (default task-11) UT005023: Exception handling request to /clusterbench/ejbservlet: javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed: StatefulSessionComponent LocalStatefulSB [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.getBusinessObject(StatefulSessionObjectReferenceImpl.java:83) [JBossINF] at org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:129) [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [JBossINF] at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:67) [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [JBossINF] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [JBossINF] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1514) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [JBossINF] at java.lang.Thread.run(Thread.java:748) [JBossINF] [JBossINF] 13:31:56,695 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /clusterbench/ejbservlet: javax.ejb.NoSuchEJBException: WFLYWELDEJB0001: EJB has been removed: StatefulSessionComponent LocalStatefulSB [JBossINF] 13:31:57,148 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment clusterbench-ee7.ear (runtime-name: clusterbench-ee7.ear) in 523ms [JBossINF] 13:31:57,251 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0009: Undeployed "clusterbench-ee7.ear" (runtime-name: "clusterbench-ee7.ear") 2018/05/21 13:31:57:267 EDT [DEBUG][RMI TCP Connection(56)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Got result: {"outcome" : "success"} 2018/05/21 13:32:57:291 EDT [DEBUG][RMI TCP Connection(60)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Going to deploy back: 'clusterbench-ee7.ear' on 'perf18' now. 2018/05/21 13:32:57:292 EDT [DEBUG][RMI TCP Connection(60)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Requesting: { "operation" : "deploy", "address" : [{ "deployment" : "clusterbench-ee7.ear" }] } {code} -- This message was sent by Atlassian JIRA (v7.5.0#75005)