[jboss-jira] [JBoss JIRA] (WFLY-4697) EJBCLient access to SFSBs does not play well with clean shutdown

[Richard Achmatowicz (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13076945#comment-13076945 ] 

Richard Achmatowicz commented on WFLY-4697:
-------------------------------------------

The clean shutdown mechainism has two phases: suspend the server, then shutdown the server. 

suspend phase:
- the EJB interfaces are locked off by way of the EjbSuspendInterceptor, preventing further invocations from getting through, and allowing active EJB requests some time to complete. Any invocations arriving after the EjbSuspendInterceptor has been locked off receive EJBComponentNotAvailableException as a response.
shutdown phase: 
- the server starts its regular shutdown process.

When an EJB Client invocation arrives at the EJBRemoteConnector, and before the EJB invocation is made, the following happens:
- it undergoes some processing to unpack the message and look up the EJB in the DeploymentRepository
- the method invocation and parameters are setup
- the method is invoked by reflection on the EJB interface and the interceptors are executed (one of which looks up the SFSB instance state in the cache)

On clean shutdown, this invocation pre-processing is not involved in the suspend phase not the shutdown phase. 
So as the server is being suspended, and then later shutdown, EJBClient invocations are arriving at the EJBRemoteConnector and being processed. 
If the server is in the suspend phase, the EjbSuspenbdInterceptor will case EJBComponentNotAvailableException to be returned as the result of an invocation.
If the server has been suspended and is now shutting down, services such as the SFSB cache service, the DeploymentRepository and eventually Channels will get stopped, in that order, while EJBClient invocations are still arriving at the interface. 

This results in exceptions, like the one in the example above.   

 

 




> EJBCLient access to SFSBs does not play well with clean shutdown
> ----------------------------------------------------------------
>
>                 Key: WFLY-4697
>                 URL: https://issues.jboss.org/browse/WFLY-4697
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering, EJB, Remoting
>    Affects Versions: 10.0.0.Alpha1
>            Reporter: Richard Achmatowicz
>            Assignee: Richard Achmatowicz
>
> The clean shutdown mechanism allows EJB and web applications to make use of shutdown interceptors to allow the application server to refuse requests when a server is in the process of shutting down. These interceptors are tied to the processing of EJB invocations and web requests.
> In the case of EJBCLient invocations, they arrive at the remoting connector, and undergo some preliminary processing before being sent to the EJB interface in question. When a server is shutting down, EJBCLient invocations can arrive at the RemoteConnector and start processing, even when the EJB interface has been locked down, so to speak.
> I am seeing various types of exceptions arising from this preliminary processing (e.g. NPE on DeploymentRepository lookups) and these get returned to the client as exceptions on the SFSB invocation, before even reaching the SFSB interceptors.
> If the EJBCLient is running in a managed transaction context, these returned exceptions will case the SFSB to be discarded (as the SFSB invocation is considered failed with the exception returned) and the transaction will attempt to rollback. If the rollback processing fails (because the original node in the transaction is down the bean gets removed anyway. The SFSB session state is lost, even if there is another node in the cluster which can support the invocation.
> In short, it looks as though the clean shutdown mechanism needs to be used to also lock down the processing of EJBCLient invocations in some way.
>  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)