[undertow-dev] Lost requests when responding from Akka actor system

[Michael Barton]

Hi all,

I've been using Undertow embedded in an Akka application and it is working great apart from a small issue when running load tests.

I've reduced the problem down to a small little demo application which I've attached below. It is in Scala but I can create a Java version if required.

I fire one million POST requests at Undertow and almost all succeed, however some never receive a response. The number that fail varies from 1 to 10.

Increasing the concurrency on the client and server side together and independently increases the number of failures. I've not yet seen any failures with no concurrency on the server side.

The HTTP handler callback is not invoked for the requests that fail. I've verified this by adding a counter before I call exchange.dispatch() and pass it on to the processing actor pool.

I don't think I'm doing anything that is not thread-safe since Akka guarantees that actors do not execute concurrently. However the execution of the actor does move around a number of threads in the underlying Fork Join pool and I'm wondering if that would cause issues with Undertow?

Any suggestions for how I could debug this?

Thanks,

Michael

Code to reproduce
==============

object UndertowTest {
  def start(handler: HttpHandler) = Undertow.builder()
    .addHttpListener(8888, "localhost")
    .setHandler(handler)
    .build()
    .start()
}

class Responder extends HttpHandler {
  override def handleRequest(exchange: HttpServerExchange): Unit = {
    exchange.getResponseHeaders.put(io.undertow.util.Headers.CONTENT_TYPE, "text/plain")
    exchange.getResponseSender.send("Hello world!")
  }
}

class RequestHandler extends Actor with Loggable {
  val handler = new Responder

  override def receive = {
    case exchange: HttpServerExchange =>
        handler.handeRequest(exchange)
  }
}

object UndertowAkkaTest extends App {
  val sys = ActorSystem()

  // Changing the number here changes the number of actors handling incoming requests
  val handlerPool = sys.actorOf(RoundRobinPool(4).props(Props(classOf[RequestHandler])))

  UndertowTest.start(new HttpHandler {
    override def handleRequest(exchange: HttpServerExchange): Unit = {
      exchange.dispatch()
      handlerPool ! exchange
    }
  })
}

Apache Bench command to test
========================

# -c changes number of concurrent client side requests
ab -k -c 4 -n 100000 -p payload.json -T application/json http://localhost:8888/streams/demo/infrastructure/cpu

Data in payload.json
===============

{
     "sampleTime": "2015-02-24T06:02:47",
    "contributor": "3b8da322-ef8f-4f6b-93a3-a171dd794308",
    "host": "some-host",
    "process": "some-process",
    "user": 35.7,
    "kernel": 12.3
}



______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20150306/7aee4e3a/attachment.html