Ensuring all pending writes complete before close

[Iain McGinniss]

I have a scenario where two peers are communicating, and once this  
communication comes to the close, the following should happen:

1. Peer A sends a "quit" message to Peer B.
2. Peer A then begins the shutdown sequence (closing open channels,  
releasing thread pools and bootstrap instances, etc).
3. Peer A's process exits.
4. Peer B, upon receiving the quit message, begins the shutdown  
sequence.
5. Peer B's process exits.

The problem I have is that there appears to be a race condition  
between the quit message actually getting across to Peer B and the  
channel being closed - sometimes peer B never receives the quit  
message, and waits endlessly for the quit message. This can be cured  
at a higher level, but I'm trying to see what I can do within the  
channel pipeline to ensure this does not happen.

The strategy I'm about to attempt is to store a set of ChannelFutures  
for all writes requested in my SimpleChannelHandler, in the  
writeRequested() method, and register a listener on these writes. As  
the writes complete, they will be removed from the set of pending  
writes. When a close is requested via the closeRequested() method, I  
do not immediately forward on the close request. Instead, I will set  
an atomic boolean "closing" to true. This will ignore all subsequent  
requests to write to the channel (check inside writeRequested()). Once  
I detect via the listener callback that the pending writes set is  
empty and closing is true, I will send the close request downstream.

I'm hoping the above will have the effect of:

1. Allowing all pending writes to be completed before I close the  
channel.
2. Preventing new writes from being requested during shutdown.
3. Allow for the entity that requested the close to wait for the  
channel to close using the standard mechanism.

One issue here could be if the writes never complete, so I suppose I  
will probably need a timeout handler after which I will forward on the  
close request anyway. Does the above sound reasonable, or is there a  
much better way of doing this in Netty?

Iain