[undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance

[Kim Rasmussen]

Instead of blocking on IO, read the request asynchronously with callbacks,
similar to this:

public void handleRequest(final HttpServerExchange exchange) throws
Exception {
  if (exchange.getRequestMethod().equals(Methods.POST)) {
  exchange.getRequestReceiver().receiveFullBytes((exchange, data) -> {
      // Read succeeded
      exchange.dispatch(() -> {
        // Do something with the byte array here

        // When you are done, call:
        exchange.dispatch(exchange.getIoThread(), () -> {
          exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
"text/plain");
  exchange.getResponseSender().send("SUCCESS");
        });
      }
    },
  (exchange, exception) -> {
      // Handle failure
      exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
  exchange.getResponseSender().send("FAILURE");
    }
}

.... and never EVER (unless "debugging") write to system out :)

/Kim

2017-06-22 21:13 GMT+02:00 SenthilKumar K <senthilec566 at gmail.com>:

> At last i modified the code as below and still i see ~50K requests/sec ..
>
> public class HelloWorldServer {
>
> public static void main(final String[] args) {
> Undertow server = Undertow.builder().addHttpListener(8009,
> "localhost").setHandler(new HttpHandler() {
> @Override
> public void handleRequest(final HttpServerExchange exchange) throws
> Exception {
>
> if (exchange.isInIoThread()) {
> exchange.dispatch(this);
> return;
> }
> if (exchange.getRequestMethod().equals(Methods.POST)) {
> BufferedReader reader = null;
> StringBuilder builder = new StringBuilder();
> try {
> exchange.startBlocking();
> reader = new BufferedReader(new InputStreamReader(exchange.
> getInputStream()));
> String line;
> while ((line = reader.readLine()) != null) {
> builder.append(line);
> }
> } catch (IOException e) {
> e.printStackTrace();
> } finally {
> if (reader != null) {
> try {
> reader.close();
> } catch (IOException e) {
> e.printStackTrace();
> }
> }
> }
> String body = builder.toString();
> System.out.println("Req Body ==> " + body);
> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
> exchange.getResponseSender().send("SUCCESS");
> } else {
> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
> exchange.getResponseSender().send("FAILURE");
> }
> }
> }).build();
> server.start();
> }
> }
>
> On Thu, Jun 22, 2017 at 11:57 PM, SenthilKumar K <senthilec566 at gmail.com>
> wrote:
>
>> Seems to Reading Request body is wrong , So what is the efficient way of
>> reading request body in undertow ?
>>
>> --Senthil
>>
>> On Thu, Jun 22, 2017 at 11:30 PM, SenthilKumar K <senthilec566 at gmail.com>
>> wrote:
>>
>>> Hello Undertow Dev Team ,
>>>
>>>       I have been working on the use case where i should create simple
>>> http server to serve 1.5 Million Requests per Second per Instance ..
>>>
>>>
>>> Here is the benchmark result of Undertow :
>>>
>>> Running 1m test @ http://127.0.0.1:8009/
>>>   20 threads and 40 connections
>>>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>>>     Latency     2.51ms   10.75ms 282.22ms   99.28%
>>>     Req/Sec     1.12k   316.65     1.96k    54.50%
>>>   Latency Distribution
>>>      50%    1.43ms
>>>      75%    2.38ms
>>>      90%    2.90ms
>>>      99%   10.45ms
>>>   1328133 requests in 1.00m, 167.19MB read
>>> Requests/sec:  *22127*.92
>>> Transfer/sec:      2.79MB
>>>
>>> This is less compared to other frameworks like Jetty and Netty .. But
>>> originally Undertow is high performant http server ..
>>>
>>> Hardware details:
>>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz, Capacity 4
>>> GHz) , Memory : 32 G , Available memory 31 G.
>>>
>>> I would need Undertow experts to review the server code below and advice
>>> me on tuning to achieve my goal( ~1.5 Million requests/sec ).
>>>
>>> Server :
>>>
>>> Undertow server = Undertow.builder()
>>>                .addHttpListener(8009, "localhost")
>>>                .setHandler(new Handler()).build();
>>> server.start();
>>>
>>>
>>> Handler.Java
>>>
>>>     final Pooled<ByteBuffer> pooledByteBuffer =
>>>                  exchange.getConnection().getBufferPool().allocate();
>>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource();
>>>    byteBuffer.clear();
>>>    exchange.getRequestChannel().read(byteBuffer);
>>>    int pos = byteBuffer.position();
>>>    byteBuffer.rewind();
>>>    byte[] bytes = new byte[pos];
>>>    byteBuffer.get(bytes);
>>>    String requestBody = new String(bytes, Charset.forName("UTF-8") );
>>>    byteBuffer.clear();
>>>    pooledByteBuffer.free();
>>>    final PostToKafka post2Kafka = new PostToKafka();
>>> try {
>>> *post2Kafka.write2Kafka(requestBody);  { This API can handle  ~2
>>> Millions events per sec }*
>>> } catch (Exception e) {
>>> e.printStackTrace();
>>> }
>>>     exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>> "text/plain");
>>>     exchange.getResponseSender().send("SUCCESS");
>>>
>>>
>>> --Senthil
>>>
>>
>>
>
> _______________________________________________
> undertow-dev mailing list
> undertow-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/undertow-dev
>



-- 
Med venlig hilsen / Best regards

*Kim Rasmussen*
Partner, IT Architect

*Asseco Denmark A/S*
Kronprinsessegade 54
DK-1306 Copenhagen K
Mobile: +45 26 16 40 23
Ph.: +45 33 36 46 60
Fax: +45 33 36 46 61
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20170622/a7c66095/attachment.html