[JBoss JIRA] Resolved: (NETTY-373) Deadlock in ChunkedWriteHandler on channel close.

Trustin Lee (JIRA) jira-events at lists.jboss.org
Mon Jan 31 21:55:39 EST 2011 

     [ https://issues.jboss.org/browse/NETTY-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Trustin Lee resolved NETTY-373.
-------------------------------

    Resolution: Done


I think this issue can be fixed by suppressing channelInterestChanged() event in the Netty core so that ChunkedWriteHandler does not attempt to call flush() when a channel is being closed.

Could you please confirm if my fix works for you?  Please get the latest JAR here:

    http://gleamynode.net/files/netty-3.2.4.Final-SNAPSHOT-ae6a5ca.jar
    

> Deadlock in ChunkedWriteHandler on channel close.
> -------------------------------------------------
>
>                 Key: NETTY-373
>                 URL: https://issues.jboss.org/browse/NETTY-373
>             Project: Netty
>          Issue Type: Bug
>          Components: Handler
>    Affects Versions: 3.2.3.Final
>         Environment: debian lenny
> java version "1.6.0_22"
> Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
> Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode)
>            Reporter: Łukasz Osipiuk
>            Assignee: Trustin Lee
>         Attachments: ChunkedWriteHandler.java
>
>
> Hello!
> We encountered problem resulting in deadlock. Problem touches ChunkedWriteHandler.
> If I correctly understand attached stacktraces deadlock occured in following situation:
> 1. Thread A calls ChunkedWriteHandler.resumeTransfer and locked it.
>    In the meanwhile socket is closed.
>    Because highWaterMark was hit in NIOSocketChannel INTEREST_OPS event was sent upstream.
>    This resulted in calling ChunkedWriteHandler.flush() and as socket got closed ChunkedWriteHandler.discard().
>    As a result of that thread tries to acquire lock on Channel but it fails because it is already taken by thread B
> 2. Thread B handles channel close. Acquires lock on channel and holding that reaches ChunkedWriteHandler.flush method which is synchronized
>    and already locked by thread A.
> I can try to fix it myself but I would appreciate a suggestion how to do it in proper way. 
> "pool-3-thread-22" (thread A):
>         at org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:610)
>         - waiting to lock <0x00007f9f8e115c98> (a java.lang.Object)
>         at org.jboss.netty.channel.socket.nio.NioWorker.writeFromUserCode(NioWorker.java:370)
>         at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:137)
>         at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:76)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:742)
>         at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:68)
>         at org.jboss.netty.handler.codec.http.HttpServerCodec.handleDownstream(HttpServerCodec.java:66)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:747)
>         at org.jboss.netty.channel.Channels.write(Channels.java:632)
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.discard(ChunkedWriteHandler.java:165)
>         - locked <0x00007f9f8e1156c8> (a org.jboss.netty.handler.stream.ChunkedWriteHandler)
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:179)
>         - locked <0x00007f9f8e1156c8> (a org.jboss.netty.handler.stream.ChunkedWriteHandler)
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:134)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
>         at org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelInterestChanged(SimpleChannelUpstreamHandler.java:183)
>         at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:116)
>         at org.jboss.netty.handler.codec.http.HttpServerCodec.handleUpstream(HttpServerCodec.java:61)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:540)
>         at org.jboss.netty.channel.Channels.fireChannelInterestChanged(Channels.java:335)
>         at org.jboss.netty.channel.socket.nio.NioSocketChannel$WriteRequestQueue.offer(NioSocketChannel.java:221)
>         at org.jboss.netty.channel.socket.nio.NioSocketChannel$WriteRequestQueue.offer(NioSocketChannel.java:197)
>         at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:135)
>         at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:76)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:742)
>         at org.jboss.netty.channel.Channels.write(Channels.java:632)
>         at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:70)
>         at org.jboss.netty.handler.codec.http.HttpServerCodec.handleDownstream(HttpServerCodec.java:66)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:747)
>         at org.jboss.netty.channel.Channels.write(Channels.java:632)
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:241)
>         - locked <0x00007f9f8e1156c8> (a org.jboss.netty.handler.stream.ChunkedWriteHandler)
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.resumeTransfer(ChunkedWriteHandler.java:105)
>         at pl.gadugadu.cutestore.jproxy.netty.handlers.NettyProxyHandler$WriteTransferResumer$1.run(NettyProxyHandler.java:209)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> "New I/O server worker #1-1" (Thread B):
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:177)
>         - waiting to lock <0x00007f9f8e1156c8> (a org.jboss.netty.handler.stream.ChunkedWriteHandler)
>         at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:134)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
>         at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
>         at org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelInterestChanged(SimpleChannelUpstreamHandler.java:183)
>         at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:116)
>         at org.jboss.netty.handler.codec.http.HttpServerCodec.handleUpstream(HttpServerCodec.java:61)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
>         at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:540)
>         at org.jboss.netty.channel.Channels.fireChannelInterestChanged(Channels.java:335)
>         at org.jboss.netty.channel.socket.nio.NioSocketChannel$WriteRequestQueue.poll(NioSocketChannel.java:242)
>         at org.jboss.netty.channel.socket.nio.NioSocketChannel$WriteRequestQueue.poll(NioSocketChannel.java:197)
>         at org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:642)
>         - locked <0x00007f9f8e115c98> (a java.lang.Object)
>         at org.jboss.netty.channel.socket.nio.NioWorker.close(NioWorker.java:593)
>         at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:356)
>         at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:281)
>         at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:201)
>         at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>         at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

More information about the netty-dev mailing list