[undertow-dev] ProxyHandler returns 503 under load

Hello, We are trying to build an API gateway using undertow as a servlet extension, this has been working great until we started our performance tests. Our setup is two gateways balancing traffic towards 10+ servers and we are using JBoss EAP 7.0 We have observed that some of our requests fail with 503 response. When we get a 503 response from a request, we have also observed that it never reaches the proxy target. I made a small test on my machine to see if I could get the same error, to my surprise this was very easy. First I tried with the following code. public class ApiGatewayServletExtension implements ServletExtension { @Override public void handleDeployment(DeploymentInfo deploymentInfo, ServletContext servletContext) { LOG.info("Deploying {}...", MethodHandles.lookup().lookupClass().getSimpleName()); deploymentInfo.addInitialHandlerChainWrapper(handler -> { try { LoadBalancingProxyClient loadBalancer = new LoadBalancingProxyClient() .addHost(new URI("http://localhost:8282")) .setConnectionsPerThread(20); return new ProxyHandler(loadBalancer, 30000, handler); } catch (URISyntaxException ex) { throw new IllegalStateException(ex); } }); } } I used JMeter with 20 threads making a simple rest GET request and after 30 seconds or so the test stopped with a 503 response. Then I created a small java application using undertow 2.0.17.Final like this. public static void main(final String[] args) { try { LoadBalancingProxyClient loadBalancer = new LoadBalancingProxyClient() .addHost(new URI("http://localhost:8282")) .setConnectionsPerThread(20); ProxyHandler proxyHandler = new ProxyHandler(loadBalancer, 30000, ResponseCodeHandler.HANDLE_404); Undertow server = Undertow.builder() .addHttpListener(8080, "localhost") .setHandler(proxyHandler).build(); server.start(); } catch (Exception ex) { System.out.println(ex.getMessage()); } } Getting the same result. I have also tried just loading the home page of one of our servers and still the same result. Any help would be very much appreciated. I have not created a bug because I'm pretty sure we are just missing something here. Regards Joakim Eriksson.