[jboss-jira] [JBoss JIRA] (WFLY-4413) Network connection leak in asynchronous servlet
jeremy_lv lv (JIRA)
issues at jboss.org
Thu Mar 12 03:57:19 EDT 2015
[ https://issues.jboss.org/browse/WFLY-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049251#comment-13049251 ]
jeremy_lv lv edited comment on WFLY-4413 at 3/12/15 3:56 AM:
-------------------------------------------------------------
Hi, [~swd847]:
{quote}
I just tried this out and the connection hangs around for 30s until the default async timeout is hit, then it goes away as expected. I tested against both 8.2.0.Final and the most recent upstream and the behaviour was the same.
The behaviour is basically the expected behaviour, because nothing catches the IOException thrown on flush the request basically hangs around doing nothing until the async timeout is hit. I could potentially forcibly kill the underling connection if a write fails, but either way the request object will be hanging around until the timeout.
{quote}
As you said, if we haven't add ctx.complete after invoking ctx.getResponse().getOutputStream().flush(), the connection hangs around for 30s until the default async timeout is hit, then it goes away as expected.
However, if we add ctx.complete after invoking ctx.getResponse().getOutputStream().flush(), the IOException will thrown during invoking the ctx.complete.
Here's the stacktrace as follows:
{panel:title=Stacktrace|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#FFFFCE}
Exception in thread "default task-35" java.lang.RuntimeException: java.io.IOException: Broken pipe
at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:527)
at io.undertow.servlet.spec.AsyncContextImpl$3.run(AsyncContextImpl.java:295)
at io.undertow.servlet.spec.AsyncContextImpl$6.run(AsyncContextImpl.java:433)
at io.undertow.servlet.spec.AsyncContextImpl$TaskDispatchRunnable.run(AsyncContextImpl.java:518)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:94)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466)
at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:150)
at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:531)
at io.undertow.conduits.ChunkedStreamSinkConduit.flush(ChunkedStreamSinkConduit.java:256)
at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)
at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:100)
at org.xnio.channels.Channels.flushBlocking(Channels.java:63)
at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:625)
at io.undertow.servlet.spec.HttpServletResponseImpl.closeStreamAndWriter(HttpServletResponseImpl.java:451)
at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:525)
{panel}
Finally, the current socket connection in JVM will keep in alive because of the above thrown exception. We did investigate the above stacktrace and found the IOException will thrown by HttpResponseConduit.write before close the socket connection in JVM. in the other words, there's no operation in undertow to close the socket connection in JVM if the IOException were thrown by HttpResponseConduit.write in invoking the ctx.complete.
Thanks.
was (Author: jeremy_lv):
Hi, [~swd847]:
{quote}
I just tried this out and the connection hangs around for 30s until the default async timeout is hit, then it goes away as expected. I tested against both 8.2.0.Final and the most recent upstream and the behaviour was the same.
The behaviour is basically the expected behaviour, because nothing catches the IOException thrown on flush the request basically hangs around doing nothing until the async timeout is hit. I could potentially forcibly kill the underling connection if a write fails, but either way the request object will be hanging around until the timeout.
{quote}
As you said, if we haven't add ctx.complete after invoking ctx.getResponse().getOutputStream().flush(), the connection hangs around for 30s until the default async timeout is hit, then it goes away as expected.
However, if we add ctx.complete after invoking ctx.getResponse().getOutputStream().flush(), the IOException will thrown during invoking the ctx.complete.
Here's the stacktrace as follows:
{panel:title=Stacktrace|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#FFFFCE}
Exception in thread "default task-35" java.lang.RuntimeException: java.io.IOException: Broken pipe
at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:527)
at io.undertow.servlet.spec.AsyncContextImpl$3.run(AsyncContextImpl.java:295)
at io.undertow.servlet.spec.AsyncContextImpl$6.run(AsyncContextImpl.java:433)
at io.undertow.servlet.spec.AsyncContextImpl$TaskDispatchRunnable.run(AsyncContextImpl.java:518)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:94)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466)
at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:150)
at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:531)
at io.undertow.conduits.ChunkedStreamSinkConduit.flush(ChunkedStreamSinkConduit.java:256)
at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)
at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:100)
at org.xnio.channels.Channels.flushBlocking(Channels.java:63)
at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:625)
at io.undertow.servlet.spec.HttpServletResponseImpl.closeStreamAndWriter(HttpServletResponseImpl.java:451)
at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:525)
{panel}
Finally, the current socket connection in JVM will keep in alive because of the above thrown exception. We did investigate the above stacktrace and found the IOException will thrown by HttpResponseConduit.write before close the socket connection in JVM. in the other words, there's no operation in undertow to close the socket connection in JVM if the IOException were thrown by HttpResponseConduit.write.
Thanks.
> Network connection leak in asynchronous servlet
> -----------------------------------------------
>
> Key: WFLY-4413
> URL: https://issues.jboss.org/browse/WFLY-4413
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Affects Versions: 8.1.0.Final, 8.2.0.Final
> Environment: Linux
> Reporter: jeremy_lv lv
> Assignee: Stuart Douglas
> Priority: Blocker
> Labels: undertow
>
> {panel:title=Phenomenon|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#FFFFCE}
> When the connection is suddenly terminated during the period we access the asynchronous servlet application, the connection will be leaked. However, the connection won't be leak when we access the synchronous servlet application.
> {panel}
> *Some of the stacktrace are as follows:*
> {panel:title=Stacktrace|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#FFFFCE}
> 14:34:23,751 ERROR [io.undertow.request] (default task-22) Blocking request fail
> at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpSer
> at io.undertow.servlet.spec.AsyncContextImpl$3.run(AsyncContextImpl.java
> at io.undertow.servlet.spec.AsyncContextImpl$6.run(AsyncContextImpl.java
> at io.undertow.servlet.spec.AsyncContextImpl$TaskDispatchRunnable.run(As
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
> at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]
> Caused by: java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [rt.jar:1.7.0_25]
> at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [rt.jar:1
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:94) [rt.jar:1.7.0
> at sun.nio.ch.IOUtil.write(IOUtil.java:51) [rt.jar:1.7.0_25]
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466) [rt.ja
> at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:150)
> at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(Htt
> at io.undertow.server.protocol.http.HttpResponseConduit.flush(HttpRespon
> at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.flush(Abstr
> at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkCha
> at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStre
> at org.xnio.channels.Channels.flushBlocking(Channels.java:63)
> at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputS
> at io.undertow.servlet.spec.HttpServletResponseImpl.closeStreamAndWriter
> at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpSer
> ... 6 more
> {panel}
> *Here's the test war code you can used to reproduce this phenomenon:*
> {code:title=AsyncDemoServlet.java|borderStyle=solid}
> import java.io.IOException;
> import javax.servlet.AsyncContext;
> import javax.servlet.ServletException;
> import javax.servlet.annotation.WebServlet;
> import javax.servlet.http.HttpServlet;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
> @WebServlet(urlPatterns="/asyncDemoServlet",asyncSupported=true)
> public class AsyncDemoServlet extends HttpServlet {
> private static final long serialVersionUID = 1L;
> protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
> response.setContentType("text/html;charset=UTF-8");
> //Execute the business logic in sub-thread.
> AsyncContext ctx = request.startAsync();
> new Thread(new Executor(ctx)).start();
> }
> protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
> doGet(request, response);
> }
> public class Executor implements Runnable {
> private AsyncContext ctx = null;
> public Executor(AsyncContext ctx){
> this.ctx = ctx;
> }
> public void run(){
> try {
> ctx.getResponse().getOutputStream().write("aaa".getBytes());
> //wait for 5 seconds to simulate the business logic.
> Thread.sleep(5000);
> ctx.getResponse().getOutputStream().flush();
> ctx.complete();
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
More information about the jboss-jira
mailing list