Michal Vinkler created WFLY-5208: ------------------------------------ Summary: UT005023: Exception handling request: NullPointerException Key: WFLY-5208 URL: https://issues.jboss.org/browse/WFLY-5208 Project: WildFly Issue Type: Bug Components: CDI / Weld Reporter: Michal Vinkler Assignee: Stuart Douglas Setup: 4-node cluster, one node at time either shuts down or undeploys application (clusterbench-ee7.ear), and then recovers after period of time, while standalone clients keep calling the application. This error was seen during clustering failover testing in the following scenarios: ejb-ejbservlet-shutdown-repl-async (failover of HTTP traffic accessing a servlet that locally invokes a stateful clustered EJB, failover type: server shutdown, with asynchronously replicated cache) ejb-ejbservlet-shutdown-repl-sync (the same as above, with synchronously replicated cache) ejb-ejbservlet-undeploy-dist-sync (failover type: application undeploy, with distributed cache which is replicated synchronously) It occurs in these two situations (intermittently): 1. server startup after previous shutdown {code} [JBossINF] [0m[0m17:23:14,240 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started repl cache from web container [JBossINF] [0m[0m17:23:14,247 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-default.war cache from web container [JBossINF] [0m[0m17:23:14,374 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. [JBossINF] [0m[0m17:23:14,375 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 65) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench' [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 71) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench-passivating' [JBossINF] [0m[0m17:23:16,784 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 71) WFLYUT0021: Registered web context: /clusterbench-passivating [JBossINF] [0m[0m17:23:17,198 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 65) WFLYUT0021: Registered web context: /clusterbench [JBossINF] [0m[31m17:23:18,931 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133) [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122) [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86) [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72) [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [JBossINF] at java.lang.Thread.run(Thread.java:745) {code} Server log: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e... 2. Server shutdown/application undeploy {code} [JBossINF] [0m[0m18:04:35,977 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0211: Suspending server [JBossINF] [0m[0m18:04:35,981 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested. 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - [SHUTDOWN] JBossShutdown command executed successfully. 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Waiting for server to shutdown. [JBossINF] [0m[0m18:04:36,048 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0022: Unregistered web context: /clusterbench-passivating [JBossINF] [0m[0m18:04:36,063 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 81) WFLYUT0022: Unregistered web context: /clusterbench [JBossINF] [0m[0m18:04:36,072 INFO [org.infinispan.eviction.impl.PassivationManagerImpl] (ServerService Thread Pool -- 82) ISPN000029: Passivating all entries to disk [JBossINF] [0m[31m18:04:36,106 ERROR [io.undertow.request] (default task-63) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133) [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122) [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source) [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86) [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72) [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [JBossINF] at java.lang.Thread.run(Thread.java:745) {code} Server log: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e... During server shutdown, it occurred also with different stacktrace: {code} [JBossINF] [0m[31m17:19:11,728 ERROR [io.undertow.request] (default task-128) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException [JBossINF] at org.jboss.weld.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:143) [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:216) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:281) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80) [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172) [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199) [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778) [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [JBossINF] at java.lang.Thread.run(Thread.java:745) [JBossINF] {code} Server log: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e... -- This message was sent by Atlassian JIRA (v6.3.15#6346)