6:47 p.m.
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started? Here's
a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:50 p.m.
New subject: [JBoss JIRA] Commented: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Brian Stansberry commented on EJBTHREE-1894:
--------------------------------------------
Hmm, so complete bypasses the interceptors. :(
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:05 p.m.
New subject: [JBoss JIRA] Commented: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Andrew Lee Rubinger commented on EJBTHREE-1894:
-----------------------------------------------
The true fix should be the introduction of a graceful shutdown mechanism supplied by AS.
In this case the EJB container will limit new invocations when in a
"prepareShutdown" phase, so this NPE should never take place.
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:01 a.m.
New subject: [JBoss JIRA] Updated: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Brian Stansberry updated EJBTHREE-1894:
---------------------------------------
Priority: Critical (was: Major)
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Priority: Critical
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:27 a.m.
New subject: [JBoss JIRA] Assigned: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Paul Ferraro reassigned EJBTHREE-1894:
--------------------------------------
Assignee: Paul Ferraro
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Assignee: Paul Ferraro
Priority: Critical
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:48 a.m.
New subject: [JBoss JIRA] Updated: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Paul Ferraro updated EJBTHREE-1894:
-----------------------------------
Attachment: patch.txt
Patch for review.
Summary of changes:
1. Remove lazy loading from StatefulContainer.getCache()
2. Remove extraneous StatefulCache.get(...) from StatefulContainer.dynamicInvoke(...);
move NoSuchEJBException catch block around invokeNext()
3. Remove assertion from ContainerMethodInvocation.getBeanContext(), since this is no
longer valid as a consequence of #2.
4. Do not unblock invocations in EJBContainer.destroy() - prevents remote invocation
threads from passing through BlockContainerShutdownInterceptor that were dispatched just
before container was stopped.
Changes are verifiable against ejbthree1116 clean shutdown test case.
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Assignee: Paul Ferraro
Priority: Critical
Attachments: patch.txt
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:16 p.m.
New subject: [JBoss JIRA] Assigned: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Paul Ferraro reassigned EJBTHREE-1894:
--------------------------------------
Assignee: Carlo de Wolf (was: Paul Ferraro)
Reassigning to carlo to assess attached patch and fix version.
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Assignee: Carlo de Wolf
Priority: Critical
Attachments: patch.txt
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:48 a.m.
New subject: [JBoss JIRA] Commented: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Radoslav Husar commented on EJBTHREE-1894:
------------------------------------------
Awesome! The patch seems to have:
* improved the total sync replication (which is still suffering supposingly from timeout
exceptions. We can now further look at the total sync.)
* fixed the buddy sync replication
* fixed buddy async replication
* fixed total async replication
When can we make this into the EAP?
Test:
https://volga.englab.brq.redhat.com:8443/hudson/job/eap5-failover-matrix-...
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Assignee: Carlo de Wolf
Priority: Critical
Attachments: patch.txt
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:17 a.m.
New subject: [JBoss JIRA] Resolved: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
Paul Ferraro resolved EJBTHREE-1894.
------------------------------------
Fix Version/s: 1.1.18
Resolution: Done
Assignee: Paul Ferraro (was: Carlo de Wolf)
Patch applied (with dependency updates).
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Assignee: Paul Ferraro
Priority: Critical
Fix For: 1.1.18
Attachments: patch.txt
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:01 a.m.
New subject: [JBoss JIRA] Updated: (EJBTHREE-1894) NPE on client invoking remote SFSB method during shutdown
[
https://jira.jboss.org/jira/browse/EJBTHREE-1894?page=com.atlassian.jira....
]
jaikiran pai updated EJBTHREE-1894:
-----------------------------------
Component Fix Version(s): jboss-ejb3-core:1.1.18 jboss-ejb3-interceptors:1.0.5
Component/s: interceptors
Fix Version/s: Plugin 1.0.15
NPE on client invoking remote SFSB method during shutdown
---------------------------------------------------------
Key: EJBTHREE-1894
URL: https://jira.jboss.org/jira/browse/EJBTHREE-1894
Project: EJB 3.0
Issue Type: Bug
Components: Clustering, core, interceptors
Affects Versions: 1.1.13
Reporter: Paul Ferraro
Assignee: Paul Ferraro
Priority: Critical
Fix For: 1.1.18, Plugin 1.0.15
Attachments: patch.txt
Scenario encountered during clustered REPL_ASYNC sfsb clean failover testing:
Exception occurs just after shutdown is initiated on target node.
java.lang.NullPointerException
at org.jboss.cache.Fqn.<init>(Fqn.java:218)
at org.jboss.cache.Fqn.<init>(Fqn.java:184)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.getFqn(StatefulTreeCache.java:579)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:154)
at org.jboss.ejb3.cache.tree.StatefulTreeCache.get(StatefulTreeCache.java:148)
at org.jboss.ejb3.stateful.StatefulContainer.dynamicInvoke(StatefulContainer.java:556)
at
org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:53)
at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
at
org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)
at
org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)
at
org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)
at
org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)
at org.jboss.remoting.Client.invoke(Client.java:1724)
at org.jboss.remoting.Client.invoke(Client.java:629)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
at
org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.remoting.ClusterChooserInterceptor.invoke(ClusterChooserInterceptor.java:84)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.security.client.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:65)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at
org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy4.invoke(Unknown Source)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
at
org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
at $Proxy3.getSerial(Unknown Source)
at
org.jboss.smartfrog.ejb3.as5.SFSBProcessorFactoryImpl$EJB3RequestProcessor.processRequest(SFSBProcessorFactoryImpl.java:72)
at
org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:33)
at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:89)
at java.lang.Thread.run(Thread.java:619)
The NPE occurs when the Fqn created in StatefulTreeCache.getFqn(...) tries to create an
Fqn with a null cacheNode. The reference to cacheNode is created in
StatefulTreeCache.start(), and is not dereferenced anywhere, so the NPE indicates that the
Cache instance used in dynamicInvoke(...) was never started. Interestingly the Cache
instances used in dynamicInvoke(...) are always obtained via StatefulContainer.getCache(),
which lazily creates and starts the container's cache if it either does not yet exist
or is not started. At first glance, it would appear that the cache returned by getCache()
should always be started. The problem, I think, is a concurrency issue with getCache()
that occurs when multiple remote requests on the container reach dynamicInvoke(...) when a
shutdown is in progress.
So how can StatefulContainer.getCache() return a cache that was never started?
Here's a possible scenario:
* Container shutdown initiated, cache is stopped.
* Asynchronous replication from a previous invocation kicks in calling
StatefulContainer.getCache(). Due to lazy loading in getCache(), this causes a new cache
to be created and started (BAD)
* A second remote invocation reaches StatefulContainer.dynamicInvoke(Invocation) and
simultaneously calls StatefulContainer.getCache().
* Either thread may end up returning a new instance of a cache that has not yet started,
since the cache validated as started by StatefulContainer.createAndStartCache() may not
necessarily be the same instance that is returned by the getCache() method.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5595
days inactive
5625
days old
9 comments
5 participants
participants (5)
-
Andrew Lee Rubinger (JIRA) -
Brian Stansberry (JIRA)
-
jaikiran pai (JIRA)
-
Paul Ferraro (JIRA)
-
Radoslav Husar (JIRA)