[jboss-jira] [JBoss JIRA] (WFLY-4827) Network Connection leak on client abort connection
Andrea Bertolini (JIRA)
issues at jboss.org
Fri Jun 26 03:48:07 EDT 2015
[ https://issues.jboss.org/browse/WFLY-4827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084160#comment-13084160 ]
Andrea Bertolini commented on WFLY-4827:
----------------------------------------
Even with this new version the problem still persist.
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:49180 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:49291 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:49379 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:49389 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:49820 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:49828 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:50192 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:50299 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:50399 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:50819 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:50894 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:51002 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:51035 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:51189 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:51231 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:51261 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:54821 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:54929 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:55574 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58436 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58439 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58451 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58480 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58487 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58494 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58510 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.57:58516 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:60991 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:61071 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:63941 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:63968 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64170 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64560 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64581 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64710 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64757 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64903 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64949 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64958 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64994 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:64999 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:65007 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:65025 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.62:65027 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:53325 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:54055 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:57166 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:57301 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:57382 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:57411 ESTABLISHED InHost
26/06/2015 9:37:06,13 TCP 192.168.55.106:8080 172.29.100.65:57413 ESTABLISHED InHost
This list is connection established from server to clients in this moment.
Do I need to manage in a specific way input and output stream extracted from the request and the response? Do I need to close them or flush?
When connection is closed by the client with the abort method, if server tries to read or write from/to the streams, it throws an exception
2015-06-26 09:38:31,038 ERROR no-user [io.undertow.request] UT005023: Exception handling request to /intellimag/operazioniFGS.service: javax.servlet.ServletException: java.io.IOException: An existing connection was forcibly closed by the remote host
at it.infolog.sce.framework.rpc.RPCServlet.processRequest(RPCServlet.java:624)
at it.infolog.sce.framework.rpc.RPCServlet.doPost(RPCServlet.java:60)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70)
at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:248)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:77)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:167)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:761)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.xnio.nio.NioSocketConduit.read(NioSocketConduit.java:282)
at org.xnio.conduits.AbstractStreamSourceConduit.read(AbstractStreamSourceConduit.java:51)
at io.undertow.conduits.ReadDataStreamSourceConduit.read(ReadDataStreamSourceConduit.java:67)
at io.undertow.conduits.ChunkedStreamSourceConduit.read(ChunkedStreamSourceConduit.java:162)
at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127)
at io.undertow.channels.DetachableStreamSourceChannel.read(DetachableStreamSourceChannel.java:209)
at io.undertow.server.HttpServerExchange$ReadDispatchChannel.read(HttpServerExchange.java:2090)
at org.xnio.channels.Channels.readBlocking(Channels.java:294)
at io.undertow.servlet.spec.ServletInputStreamImpl.readIntoBuffer(ServletInputStreamImpl.java:170)
at io.undertow.servlet.spec.ServletInputStreamImpl.read(ServletInputStreamImpl.java:146)
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2313)
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2326)
at java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.java:3067)
at java.io.ObjectInputStream$BlockDataInputStream.readUTF(ObjectInputStream.java:2867)
at java.io.ObjectInputStream.readUTF(ObjectInputStream.java:1073)
at java.io.ObjectStreamClass.readNonProxy(ObjectStreamClass.java:704)
at java.io.ObjectInputStream.readClassDescriptor(ObjectInputStream.java:831)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1602)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
at java.util.HashMap.readObject(HashMap.java:1396)
at sun.reflect.GeneratedMethodAccessor1179.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1707)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
at it.infolog.sce.framework.rpc.RPCServlet.processRequest(RPCServlet.java:139)
... 29 more
It seems that this exception is thrown only when it recognize that client-server connection is broken. Maybe it remains stuck in write or read from/to the stream? I need to abort client connection because answer from the server is taking too long to be received.
> Network Connection leak on client abort connection
> --------------------------------------------------
>
> Key: WFLY-4827
> URL: https://issues.jboss.org/browse/WFLY-4827
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow), Web Sockets
> Affects Versions: 8.2.0.Final
> Environment: On Windows Server 2012, JDK 1.8.0_45, Wildfly 8.2.0.Final in standalone mode.
> Reporter: Andrea Bertolini
> Assignee: Stuart Douglas
>
> We have a classic client-server application, all written in Java. Each client is installed on a forklift which can move all around a large area. This area is under wi-fi coverage.
> Sometimes the clients can have a bad connection quality and the client-server communication is interrupted; in such a case it takes too many seconds to be restored.
> To fix this situation, we add a timeout client-side. After 5 seconds it aborts the call and tries again a second time.
> To achieve this call we use apache httpcomponents library (version 4.4). We use the abort method of httppost to interrupt this call.
> Server-side, we have a group of web-servlets which listen to the incoming calls, manage requests and send a response.
> It appears that sometimes a communication remains stuck in reading or writing from/to the stream. When the client aborts the communication, an exception is thrown on the server caused by the channel being closed.
> It happens that a large number of connections remains stuck in connection status 'established' (only server-side) even if the real connection is actually closed (client doesn't have that connection active anymore).
> When the number of established connections grows up to 200, server stops responding on port 8080, so it cannot accept more connections and it seems to freeze.
> We tried to add tcp-keep-alive=true and no-request-timeout=120000 on http-listener in undertow subsystem, but sometimes it removes idle connections after any incoming requests are received for 2 minutes, some other times it keep connections as established and doesn't close them.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
More information about the jboss-jira
mailing list