[undertow-dev] ProxyHandler returns 503 under load

[Carter Kozak]

When the connections-per-thread limit is reached, requests result in 503 responses. Is the problem resolved by increasing the connections-per-thread value to 50 or 100?

On Wed, Feb 13, 2019, at 04:42, Jocke Eriksson wrote:
> Hello,
> 
> We are trying to build an API gateway using undertow as a servlet extension, this has been working great until we started our performance tests.
> Our setup is two gateways balancing traffic towards 10+ servers and we are using JBoss EAP 7.0 We have observed that some of our requests fail with 503 response. When we get a 503 response from a request, we have also observed that it never reaches the proxy target.
> I made a small test on my machine to see if I could get the same error, to my surprise this was very easy. First I tried with the following code.
> 
> public class ApiGatewayServletExtension implements ServletExtension {
>   
> 
>  @Override
>  public void handleDeployment(DeploymentInfo deploymentInfo, ServletContext servletContext) {
>  LOG.info("Deploying {}...", MethodHandles.lookup().lookupClass().getSimpleName());
>  deploymentInfo.addInitialHandlerChainWrapper(handler -> {
>  try {
>  LoadBalancingProxyClient loadBalancer = new LoadBalancingProxyClient()
>  .addHost(new URI("http://localhost:8282"))
>  .setConnectionsPerThread(20);
>  return new ProxyHandler(loadBalancer, 30000, handler);
>  } catch (URISyntaxException ex) {
>  throw new IllegalStateException(ex);
>  }
>  });
>  }
> }
> 
> I used JMeter with 20 threads making a simple rest GET request and after 30 seconds or so the test stopped with a 503 response.
> 
> Then I created a small java application using undertow 2.0.17.Final like this.
> 
>  public static void main(final String[] args) {
>   
> 
>  try {
>  LoadBalancingProxyClient loadBalancer = new LoadBalancingProxyClient()
>  .addHost(new URI("http://localhost:8282"))
>  .setConnectionsPerThread(20);
>  ProxyHandler proxyHandler = new ProxyHandler(loadBalancer, 30000, ResponseCodeHandler.HANDLE_404);
> 
>  Undertow server = Undertow.builder()
>  .addHttpListener(8080, "localhost")
>  .setHandler(proxyHandler).build();
>  
>  server.start();
> 
>  } catch (Exception ex) {
>  System.out.println(ex.getMessage());
>  }
>  }
> 
> Getting the same result. I have also tried just loading the home page of one of our servers and still the same result.
> 
> Any help would be very much appreciated. I have not created a bug because I'm pretty sure we are just missing something here.
> 
> Regards Joakim Eriksson. 
> 
> 
> 
>  
> 
> 
>  
> _______________________________________________
> undertow-dev mailing list
> undertow-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/undertow-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20190213/71113006/attachment-0001.html