[jboss-as7-dev] Graceful Shutdown (Revision 2)

[Scott Marlow]

On 06/27/2012 08:34 AM, David M. Lloyd wrote:
> On 6/26/12 7:13 PM, Bill Burke wrote:
>> Are you sure explicity granting "permits" through a "Shutdown Manager"
>> the way to go?
>
> Yes, we are 100% sure.
>
>> Having to interact with a central service that grants
>> permits per-request is very worrisome.  The subsystem may already have a
>> better way to do graceful shutdown, or just cannot follow a permit pattern.
>
> No, we went into great detail to work this out.  But if you have a
> specific case that you think warrants another look, feel free to raise it.
>
>> Is it not simpler and gives more flexibility to a subsystem to drive
>> this via events rather than explicit callbacks to a Shutdown Manager?
>
> No, because the "suspended" state isn't really a state, and it is
> certainly not detectable from each subsystem.
>
> For example, there may be no remaining EJB sessions.  However, that does
> not mean that EJB can refuse new session creation.  In particular, if
> you are in a suspended state and deploy a singleton EJB which has a
> @PostConstruct on it, and that interceptor causes a session to be
> created, this is allowed as deployment must always gain a permit (else
> it'd fail).

Does deploying an application during the "suspended" state, cancel the 
shutdown (bringing the server back to a "running" state?)  You said 
below that no permits are allowed in the "suspended" state, so a state 
transition might be needed for deployment (I think).  Or is deployment 
only allowed in "Suspending/Running" states?

>
> Note, people, that permits are a *conceptual* entity.  There is no need
> to have permit objects, nor even a semaphore, nor even a non-blocking
> counter.  I do know a thing or two about eliminating concurrency
> bottlenecks.
>
>> i.e.
>>
>> class Suspending extends ShutdownEvent
>> {
>>      void iFinishedSuspending(); // the callback to the Shutdown Manager
>> }
>>
>> GIves the subsystem flexibility on how best to implement a graceful
>> shutdown.
>>
>>
>> On 6/26/12 11:00 AM, Jason T. Greene wrote:
>>> *EDITED*
>>>
>>> We've worked out the rough outline of how graceful shutdown will work in
>>> AS7.
>>>
>>> The process of graceful shutdown actually is reflected by a number of
>>> states:
>>>
>>> 1. Running - all services acting normally
>>> 2. Suspending - services refuse new "permits" (see below), existing
>>> permits are allowed to be retained (and threads running under such a
>>> permit may still acquire new permits)
>>> 3. Suspended - no permits are present and none may be issued
>>> 4. Shutting Down - our existing server stop process / reload admin mode
>>>
>>> The following transitions are allowed:
>>>
>>> 1. Running → Suspending: Transition occurs at user request (to suspend
>>> or gracefully shut down).
>>> 2. Suspending → Suspended: Transition occurs when all permits are cleared.
>>> 3. Suspending → Running: Transition occurs at user request (to exit
>>> suspend mode or cancel graceful shutdown before it completes).
>>> 4. Suspended → Running: Transition occurs at user request (to exit
>>> suspend mode).
>>> 5. Suspended → Shutting Down: Transition occurs automatically (if a
>>> graceful shutdown was requested) or at user request (if a shut down
>>> request of any kind is entered in the Suspended state).
>>> 6. Running → Shutting Down: Transition occurs at user request (to shut
>>> down the server "un-gracefully").
>>> 7. Suspending → Shutting Down (User aborts a graceful shutdown)
>>>
>>> These "permits" are issued by the "Shutdown Manager", whose job is to
>>> manage these states.  They are issued corresponding to the following events:
>>>
>>> 1. The invocation of an EJB method

This sounds like an EJB component interceptor needs to be changed to 
handle this part.

>>> 2. The creation of a web session

Would this be handled by an org.apache.catalina.Valve or some equivalent?

>>> 3. A creation of a transaction

What is the use case that requires this that is not already issuing a 
permit at a higher (top level) container?

>>> 4. MessageEndpoint and WorkManager aquire permit allowing for release()
>>> from a thirdparty to indicate connection close.

What is the hook/API for plugging into these so the shutdown manager can 
be invoked?

5.  Unmanaged Thread use

No one has mentioned naked or unmanaged threads.  Can naked threads also 
call the Shutdown Manager?  If yes, I think the Shutdown Manager API 
needs to be public so that custom threads could also participate in 
graceful shutdown.

>>>
>>> When a permit cannot be issued due to the server shutting down, a
>>> standard exception message should be produced so that the user can see a
>>> familiar error message regardless of what mechanism is used to access
>>> the server.

Sounds like a good idea.  The Shutdown Manager should probably throw the 
familiar exception.

>>>
>>
>
>