Deadlock after adding ExecutionHandler

[IanS]

I have a much more useful stack I believe.  I can't tell if this is related
to the previous situation or not, but both occur only when an
ExecutionHandler is in the pipeline.

However, in this case, I believe it may be a bug in ChunkedWriteHandler's
synchronization.

Netty is writing to a socket, which turns out to be closed, propagating
events upstream.

My test program is writing a message downstream.  

The closed socket synchronizes on the channel.writeLock in
writeFromTaskLoop(), and the writing channel gets a lock on
ChunkedWriteHandler via its synchronized flush() method.  Then the messages
propagate upstream and downstream, causing a deadlock.

I suspect the solution to this problem is making the locking in flush() more
fine-grained, so the object lock is not maintained during
ctx.sendDownstream(), but that will certainly complicate the code, and I'm
not completely sure that is correct.

Thanks, Ian

Stack traces (from 3.2.0Beta1, though I don't see any changes which would
fix this in 3.2.1):

Java stack information for the threads listed above:
===================================================
"New I/O client worker #164-2":
	at
org.jboss.netty.handler.stream.ChunkedWriteHandler.discard(ChunkedWriteHandler.java:149)
	- waiting to lock <10947e6b0> (a
org.jboss.netty.handler.stream.ChunkedWriteHandler)
	at
org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:139)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
	at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:344)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.channelClosed(FrameDecoder.java:232)
	at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:98)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
	at
org.jboss.netty.handler.timeout.IdleStateHandler.channelClosed(IdleStateHandler.java:264)
	at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:98)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:540)
	at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:404)
	at org.jboss.netty.channel.socket.nio.NioWorker.close(NioWorker.java:602)
	at org.jboss.netty.channel.socket.nio.NioWorker.write0(NioWorker.java:509)
	- locked <10947c2d0> (a java.lang.Object)
	at
org.jboss.netty.channel.socket.nio.NioWorker.writeFromTaskLoop(NioWorker.java:392)
	at
org.jboss.netty.channel.socket.nio.NioSocketChannel$WriteTask.run(NioSocketChannel.java:268)
	at
org.jboss.netty.channel.socket.nio.NioWorker.processWriteTaskQueue(NioWorker.java:269)
	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)
	at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at
org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:637)
"main":
	at
org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:618)
	- waiting to lock <10947c2d0> (a java.lang.Object)
	at
org.jboss.netty.channel.socket.nio.NioWorker.writeFromUserCode(NioWorker.java:369)
	at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:117)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:742)
	at org.jboss.netty.channel.Channels.write(Channels.java:632)
	at
org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:70)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:747)
	at org.jboss.netty.channel.Channels.write(Channels.java:632)
	at
org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:70)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:747)
	at
org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:251)
	- locked <10947e6b0> (a org.jboss.netty.handler.stream.ChunkedWriteHandler)
	at
org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:123)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:747)
	at
org.jboss.netty.handler.execution.ExecutionHandler.handleDownstream(ExecutionHandler.java:167)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
	at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:747)
	at
org.jboss.netty.channel.SimpleChannelHandler.writeRequested(SimpleChannelHandler.java:304)
	at
org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:266)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:568)
	at
org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:563)
	at org.jboss.netty.channel.Channels.write(Channels.java:611)
	at org.jboss.netty.channel.Channels.write(Channels.java:578)
	at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:229)

-- 
View this message in context: http://netty-forums-and-mailing-lists.685743.n2.nabble.com/Deadlock-after-adding-ExecutionHandler-tp5237295p5248505.html
Sent from the Netty User Group mailing list archive at Nabble.com.