Shutting down the netty server takes long time when using SSLHandler.

[Trustin Lee]

Please try Revision 1187 before making the modification I suggested.
I've just checked in the potential fix for this issue.

— Trustin Lee, http://gleamynode.net/

On Wed, Apr 15, 2009 at 9:25 PM, Trustin Lee <tlee at redhat.com> wrote:
> Hi Virat,
>
> On Wed, Apr 15, 2009 at 8:33 PM, Virat Gohil <virat.gohil at gmail.com> wrote:
>> Hi All!
>>
>> I am facing a small problem shutting down my Netty based server with
>> ~1200 connections.
>>
>> I am using the ChannelGroup as described in Getting Started guide,
>> following is the code:
>>
>> public void stop()
>>        {
>>                if(timer!=null)
>>                {
>>                        timer.stop();
>>                }
>>                if(g!=null && factory!=null)
>>                {
>>                         ChannelGroupFuture future = g.close();
>>                         future.awaitUninterruptibly();
>>                        if(ch!=null)
>>                        {
>>                                ch.unbind();
>>                        }
>>                        try {
>>                                bossExecutor.shutdownNow();
>>                                workerExecutor.shutdownNow();
>>                                workerExecutor.awaitTermination(3600, TimeUnit.SECONDS);
>>                        bossExecutor.awaitTermination(3600, TimeUnit.SECONDS);
>>                        } catch (InterruptedException e) {
>>                                //print the exception
>>                        }
>>                        factory.releaseExternalResources();
>>                }
>>        }
>>
>> The execution gets stuck at future.awaitUninterruptibly(); I tried
>> debugging the issue and found the following:
>>
>> 1. when g.close() is called the channel group creates a hashtable and
>> creates a new DefaultChannelGroupFuture, which becomes the registered
>> listener on all these channels.
>> 2. Whenever channel.close() is called, the DefaultChannelGroupFuture
>> gets called and increments the succes/failure count.
>> 3. if the success+failure count=number of channels in the group, then
>> the operation is considered finished and the thread waiting on the
>> defaultchannelgroupfuture is released.
>>
>> I observed in SSLHandler that Channels.close() is called only if the
>> received frame was empty and the inbound was finished:
>> SSLHandler.java:406 (decode())
>>       if (frame == null && engine.isInboundDone()) {
>>            synchronized (closeFutures) {
>>                for (;;) {
>>                    ChannelFuture future = closeFutures.poll();
>>                    if (future == null) {
>>                        break;
>>                    }
>>                    Channels.close(ctx, future);
>>                }
>>            }
>>        }
>> Sometimes, either the frame is not null or inbound is not completed,
>> this causes the SSLHandler to continue decoding.  This leads in
>> DefaultChannelGroupFuture.childListener.operationComplete() being
>> called after a very long time.
>
> 1) What is the state of the actual connection?  Is it closed or still connected?
>
> 2) What happens if you replace:
>
>        if (frame == null && engine.isInboundDone()) {
> with:
>
>        if (frame == null && engine.isInboundDone() || !channel.isConnected()) {
>
> ?
>
>> What we would prefer to do, is to abandon the incomplete data in
>> SSLHandler.decode() and close the channel immediately as soon as the
>> server's shutdown method is called. Please let me know if I am missing
>> something or if there is another way of achieving a faster shutdown.
>
> You are doing correctly and the ChannelGroupFuture should return
> quickly.  Thanks for reporting the problem!
>
> Trustin
>