<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I will try this and get back to you after our tests.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Ragrads Joakim.</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Fillrån:</b> undertow-dev-bounces@lists.jboss.org <undertow-dev-bounces@lists.jboss.org> för Stuart Douglas <sdouglas@redhat.com><br>
<b>Skickat:</b> den 14 februari 2019 04:11<br>
<b>Till:</b> Carter Kozak<br>
<b>Kopia:</b> Undertow Developers<br>
<b>Ämne:</b> Re: [undertow-dev] ProxyHandler returns 503 under load</font>
<div> </div>
</div>
<div>
<div dir="ltr">If there is only 20 clients then in theory this should not happen, however to some extent this is inherently racey. What can happen is that there is a slight delay between the client receiving the full response and the connection being returned
to the pool (there is nothing that can be done about this, it is impossible to atomically send the last content and return the thread in one action).
<div><br>
</div>
<div>I would try calling setMaxQueueSize and add a small queue (say 20) which should resolve the issue. If not then this is definitely a bug.</div>
<div><br>
</div>
<div>Stuart</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Thu, Feb 14, 2019 at 1:53 AM Carter Kozak <<a href="mailto:ckozak@ckozak.net">ckozak@ckozak.net</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<u></u>
<div>
<div>When the connections-per-thread limit is reached, requests result in 503 responses. Is the problem resolved by increasing the connections-per-thread value to 50 or 100?<br>
</div>
<div><br>
</div>
<div>On Wed, Feb 13, 2019, at 04:42, Jocke Eriksson wrote:<br>
</div>
<blockquote type="cite" id="x_gmail-m_-1711059870579191123fastmail-quoted">
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hello,<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
We are trying to build an API gateway using undertow as a servlet extension, this has been working great until we started our performance tests.<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Our setup is two gateways balancing traffic towards 10+ servers and we are using JBoss EAP 7.0 We have observed that some of our requests fail with 503 response. When we get a 503 response from a request, we have also observed that it never reaches the proxy
target.<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I made a small test on my machine to see if I could get the same error, to my surprise this was very easy. First I tried with the following code.<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div><span>public class ApiGatewayServletExtension implements ServletExtension {<br>
</span></div>
<div><br>
</div>
<div> @Override<br>
</div>
<div> public void handleDeployment(DeploymentInfo deploymentInfo, ServletContext servletContext) {<br>
</div>
<div> LOG.info("Deploying {}...", MethodHandles.lookup().lookupClass().getSimpleName());<br>
</div>
<div> deploymentInfo.addInitialHandlerChainWrapper(handler -> {<br>
</div>
<div> try {<br>
</div>
<div> LoadBalancingProxyClient loadBalancer = new LoadBalancingProxyClient()<br>
</div>
<div> .addHost(new URI("<a href="http://localhost:8282" target="_blank">http://localhost:8282</a>"))<br>
</div>
<div> .setConnectionsPerThread(20);<br>
</div>
<div> return new ProxyHandler(loadBalancer, 30000, handler);<br>
</div>
<div> } catch (URISyntaxException ex) {<br>
</div>
<div> throw new IllegalStateException(ex);<br>
</div>
<div> }<br>
</div>
<div> });<br>
</div>
<div> }<br>
</div>
<div><span>}</span><br>
</div>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I used JMeter with 20 threads making a simple rest GET request and after 30 seconds or so the test stopped with a 503 response.<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Then I created a small java application using undertow 2.0.17.Final like this.<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div><span> public static void main(final String[] args) {<br>
</span></div>
<div><br>
</div>
<div> try {<br>
</div>
<div> LoadBalancingProxyClient loadBalancer = new LoadBalancingProxyClient()<br>
</div>
<div> .addHost(new URI("<a href="http://localhost:8282" target="_blank">http://localhost:8282</a>"))<br>
</div>
<div> .setConnectionsPerThread(20);<br>
</div>
<div> ProxyHandler proxyHandler = new ProxyHandler(loadBalancer, 30000, ResponseCodeHandler.HANDLE_404);<br>
</div>
<div><br>
</div>
<div> Undertow server = Undertow.builder()<br>
</div>
<div> .addHttpListener(8080, "localhost")<br>
</div>
<div> .setHandler(proxyHandler).build();<br>
</div>
<div> <br>
</div>
<div> server.start();<br>
</div>
<div><br>
</div>
<div> } catch (Exception ex) {<br>
</div>
<div> System.out.println(ex.getMessage());<br>
</div>
<div> }<br>
</div>
<div><span> }</span><br>
</div>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span></span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span>Getting the same result. I have also tried just loading the home page of one of our servers and still the same result.</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span></span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span>Any help would be very much appreciated. I have not created a bug because I'm pretty sure we are just missing something here.</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span></span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span>Regards Joakim Eriksson. </span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div>_______________________________________________<br>
</div>
<div>undertow-dev mailing list<br>
</div>
<div><a href="mailto:undertow-dev@lists.jboss.org" target="_blank">undertow-dev@lists.jboss.org</a><br>
</div>
<div><a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a><br>
</div>
</blockquote>
<div><br>
</div>
</div>
_______________________________________________<br>
undertow-dev mailing list<br>
<a href="mailto:undertow-dev@lists.jboss.org" target="_blank">undertow-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a></blockquote>
</div>
</div>
</div>
</body>
</html>