Sascha Dirbach created RTGOV-674: ------------------------------------ Summary: ElasticsearchRESTServer proxy blocks on Error Key: RTGOV-674 URL: https://issues.jboss.org/browse/RTGOV-674 Project: RTGov (Run Time Governance) Issue Type: Bug Components: User Interface Affects Versions: 2.0.0.Final Reporter: Sascha Dirbach Assignee: Gary Brown Hi, it looks like the InputStream of the proxyResponse is not closed correctly when there is an exception in org.overlord.rtgov.elasticsearch.rest.ElasticsearchRESTServer.copyResponseContent(HttpResponse, HttpServletResponse) This results in a blocking behaviour, as the internally used HttpClient uses a ConnectionPool with the size of 2 connections per host, which will not be released in the above case. We experienced that with a SocketClosedException like this: ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/rtgov-ui]] (http-/0.0.0.0:9181-5) ElasticsearchRESTServer: null: ClientAbortException: java.net.SocketException: Broken pipe at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:403) at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:450) at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342) at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:301) at org.apache.catalina.connector.CoyoteOutputStream.close(CoyoteOutputStream.java:104) at org.overlord.rtgov.elasticsearch.rest.ElasticsearchRESTServer.copyResponseContent(ElasticsearchRESTServer.java:137) at org.overlord.rtgov.elasticsearch.rest.ElasticsearchRESTServer.service(ElasticsearchRESTServer.java:61) After two of these Exceptions the elasticsearch proxy did not react anymore until a server restart. A stacktrace of the running server revealed, that the threads are blocked in the connection pool: ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/rtgov-ui]] (http-/0.0.0.0:9181-5) ElasticsearchRESTServer: null: ClientAbortException: java.net.SocketException: Broken pipe at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:403) at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:450) at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342) at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:301) at org.apache.catalina.connector.CoyoteOutputStream.close(CoyoteOutputStream.java:104) at org.overlord.rtgov.elasticsearch.rest.ElasticsearchRESTServer.copyResponseContent(ElasticsearchRESTServer.java:137) at org.overlord.rtgov.elasticsearch.rest.ElasticsearchRESTServer.service(ElasticsearchRESTServer.java:61) Sascha -- This message was sent by Atlassian JIRA (v6.4.11#64026)