[undertow-dev] java.io.IOException: Connection reset by peer

[Jocke Eriksson]

OK after some more digging it looks like proxy target is sending RST, ACK. Anybody got any clue? Is this normal? What can we do about it?

The communication is JBoss to JBoss and same version EAP 7.0.

________________________________
Från: Jocke Eriksson
Skickat: den 13 september 2019 09:15
Till: undertow-dev at lists.jboss.org <undertow-dev at lists.jboss.org>
Ämne: java.io.IOException: Connection reset by peer

Hi we are using jboss EAP in production as an API gateway and we have a problem with the communication to some of our servers.

We can see this in our logs.


2019-09-13 08:57:31,337 +0200 DEBUG [io.undertow.client] (default-undertow I/O-21) Connection closed with IOException: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.8.0_222]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.8.0_222]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) [rt.jar:1.8.0_222]
at sun.nio.ch.IOUtil.read(IOUtil.java:192) [rt.jar:1.8.0_222]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) [rt.jar:1.8.0_222]
at org.xnio.nio.NioSocketConduit.read(NioSocketConduit.java:289) [xnio-nio.jar:3.4.3.Final-redhat-1]
at org.xnio.conduits.PushBackStreamSourceConduit.read(PushBackStreamSourceConduit.java:52) [xnio-api.jar:3.4.3.Final-redhat-1]
at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127) [xnio-api.jar:3.4.3.Final-redhat-1]
at io.undertow.client.http.HttpClientConnection$ClientReadListener.handleEvent(HttpClientConnection.java:510)
at io.undertow.client.http.HttpClientConnection$ClientReadListener.handleEvent(HttpClientConnection.java:473)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) [xnio-api.jar:3.4.3.Final-redhat-1]
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66) [xnio-api.jar:3.4.3.Final-redhat-1]
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89) [xnio-nio.jar:3.4.3.Final-redhat-1]
at org.xnio.nio.WorkerThread.run(WorkerThread.java:567) [xnio-nio.jar:3.4.3.Final-redhat-1]

2019-09-13 08:57:31,339 +0200 ERROR [io.undertow.proxy] (default-undertow I/O-21) UT005028: Proxy request to /*****/******* failed: java.io.IOException: UT001000: Connection closed
at io.undertow.client.http.HttpClientConnection$ClientReadListener.handleEvent(HttpClientConnection.java:516)
at io.undertow.client.http.HttpClientConnection$ClientReadListener.handleEvent(HttpClientConnection.java:473)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) [xnio-api.jar:3.4.3.Final-redhat-1]
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66) [xnio-api.jar:3.4.3.Final-redhat-1]
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89) [xnio-nio.jar:3.4.3.Final-redhat-1]
at org.xnio.nio.WorkerThread.run(WorkerThread.java:567) [xnio-nio.jar:3.4.3.Final-redhat-1]

The client receives a HTTP 503 response,
any advice on how we can solve this is very welcome.

Also it looks like we are setting up a lot of new connections, I was thinking it would be kept alive a bit longer.

2019-09-13 08:57:46,696 +0200 DEBUG [io.undertow.client] (default-undertow I/O-11) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:47,019 +0200 DEBUG [io.undertow.client] (default-undertow I/O-17) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:47,117 +0200 DEBUG [io.undertow.client] (default-undertow I/O-7) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:47,427 +0200 DEBUG [io.undertow.client] (default-undertow I/O-9) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:47,552 +0200 DEBUG [io.undertow.client] (default-undertow I/O-9) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:47,552 +0200 DEBUG [io.undertow.client] (default-undertow I/O-6) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:47,634 +0200 DEBUG [io.undertow.client] (default-undertow I/O-28) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:48,200 +0200 DEBUG [io.undertow.client] (default-undertow I/O-9) Connection to ********************:8080 was closed by the target server
2019-09-13 08:57:48,205 +0200 DEBUG [io.undertow.client] (default-undertow I/O-15) Connection to ********************8080 was closed by the target server











-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20190917/d2029a8a/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2019-09-13 14-19-27.png
Type: image/png
Size: 23522 bytes
Desc: Screenshot from 2019-09-13 14-19-27.png
Url : http://lists.jboss.org/pipermail/undertow-dev/attachments/20190917/d2029a8a/attachment-0001.png