[undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance
Kim Rasmussen
kr at asseco.dk
Sat Jul 8 12:07:51 EDT 2017
Have you tried playing around with the number of io and worker threads?
lør. 8. jul. 2017 kl. 17.28 skrev SenthilKumar K <senthilec566 at gmail.com>:
> Any comments on *Undertow Vs Netty* ? Am i doing wrong benchmark testing
> ?? Should i change benchmark strategy ?
>
> --Senthil
>
> On Fri, Jul 7, 2017 at 3:14 PM, SenthilKumar K <senthilec566 at gmail.com>
> wrote:
>
>> Sorry for delay in responding to this thread!
>>
>> Thanks to everyone who helped me to Optimize Undertow Server.
>>
>> Here is the comparison after benchmarking my use case against Netty:
>>
>> *Undertow Vs Netty :*
>>
>> Test Case 1 :
>> Simple Request Response ( No Kafka ):
>>
>> *Undertow:*
>> Running 10m test @ http://198.18.134.13:8009/
>> 500 threads and 5000 connections
>> Thread Stats Avg Stdev Max +/- Stdev
>> Latency *3.52m * 2.64m 8.96m 54.63%
>> Req/Sec 376.58 103.18 0.99k 80.53%
>> 111628942 requests in 10.00m, 13.72GB read
>> Socket errors: connect 0, read 28, write 0, timeout 2
>> Requests/sec: *186122.56*
>> Transfer/sec: 23.43MB
>>
>> *Netty:*
>> Running 10m test @ http://198.18.134.13:8009/
>> 500 threads and 5000 connections
>> Thread Stats Avg Stdev Max +/- Stdev
>> Latency *3.77m* 2.10m 7.51m 57.73%
>> Req/Sec 518.63 31.78 652.00 70.25%
>> 155406992 requests in 10.00m, 13.82GB read
>> Socket errors: connect 0, read 49, write 0, timeout 0
>> Requests/sec: *259107*.30
>> Transfer/sec: 24.17MB
>>
>>
>> *Test Case 2:*
>> Request --> Read --> Send it Kafka :
>>
>> *Undertow:*
>> Running 10m test @ http://198.18.134.13:8009/
>> 500 threads and 5000 connections
>> Thread Stats Avg Stdev Max +/- Stdev
>> Latency *4.37m * 2.46m 8.72m 57.83%
>> Req/Sec 267.32 5.17 287.00 74.52%
>> 80044045 requests in 10.00m, 9.84GB read
>> Socket errors: connect 0, read 121, write 0, timeout 0
>> Requests/sec: *133459.79*
>> Transfer/sec: 16.80MB
>>
>> *Netty:*
>> Running 10m test @ http://198.18.134.13:8009/
>> 500 threads and 5000 connections
>> Thread Stats Avg Stdev Max +/- Stdev
>> Latency *3.78m * 2.10m 7.55m 57.79%
>> Req/Sec 516.92 28.84 642.00 69.60%
>> 154770536 requests in 10.00m, 13.69GB read
>> Socket errors: connect 0, read 11, write 0, timeout 101
>> Requests/sec: *258049.39*
>> Transfer/sec: 23.38MB
>>
>>
>>
>> CPU Usage:
>> *Undertow:*
>> [image: Inline image 1]
>>
>> *Netty:*
>> [image: Inline image 2]
>>
>>
>> --Senthil
>>
>> On Thu, Jun 29, 2017 at 7:34 AM, Bill O'Neil <bill at dartalley.com> wrote:
>>
>>> 1. Can you run the benchmark with the kafka line commented out at first
>>> and then again with it not commented out?
>>> 2. What rates were you getting with Jetty and Netty?
>>> 3. Are you running the tests from the same machine or a different one?
>>> If its the same machine and its using 20 threads they will be contending
>>> with undertows IO threads.
>>> 4. You can probably ignore the POST check if thats all your going to
>>> accept and its not a public api.
>>>
>>> import io.undertow.server.HttpHandler;
>>> import io.undertow.server.HttpServerExchange;
>>> import io.undertow.util.Headers;
>>> import io.undertow.util.Methods;
>>>
>>> public class DLRHandler implements HttpHandler {
>>>
>>> final public static String _SUCCESS="SUCCESS";
>>> final public static String _FAILURE="FAILURE";
>>> final PostToKafka post2Kafka = new PostToKafka();
>>>
>>> @Override
>>> public void handleRequest( final HttpServerExchange exchange) throws
>>> Exception {
>>> if (exchange.getRequestMethod().equals(Methods.POST)) {
>>> exchange.getRequestReceiver().receiveFullString((
>>> exchangeReq, data) -> {
>>> //post2Kafka.write2Kafka(data); // write it to Kafka
>>>
>>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
>>> exchangeReq.getResponseSender().send(_SUCCESS);
>>> },
>>> (exchangeReq, exception) -> {
>>>
>>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
>>> exchangeReq.getResponseSender().send(_FAILURE);
>>> });
>>> }else{
>>> throw new Exception("Method GET not supported by Server ");
>>> }
>>> }
>>> }
>>>
>>> On Wed, Jun 28, 2017 at 6:59 PM, Stuart Douglas <sdouglas at redhat.com>
>>> wrote:
>>>
>>>> The multiple dispatches() are unnecessary (well the second one to the
>>>> IO thread is definitely unnecessary, the first one is only required if
>>>> post2Kafka.write2Kafka(data); is a blocking operation and needs to be
>>>> executed in a worker thread).
>>>>
>>>> Stuart
>>>>
>>>> On Wed, Jun 28, 2017 at 5:42 PM, SenthilKumar K <senthilec566 at gmail.com>
>>>> wrote:
>>>> > After modifying the code below i could see the improvement ( not much
>>>> > slightly ) in server - 65k req/sec.
>>>> >
>>>> > import io.undertow.server.HttpHandler;
>>>> > import io.undertow.server.HttpServerExchange;
>>>> > import io.undertow.util.Headers;
>>>> > import io.undertow.util.Methods;
>>>> >
>>>> > public class DLRHandler implements HttpHandler {
>>>> >
>>>> > final public static String _SUCCESS="SUCCESS";
>>>> > final public static String _FAILURE="FAILURE";
>>>> > final PostToKafka post2Kafka = new PostToKafka();
>>>> >
>>>> > @Override
>>>> > public void handleRequest( final HttpServerExchange exchange)
>>>> throws
>>>> > Exception {
>>>> > if (exchange.getRequestMethod().equals(Methods.POST)) {
>>>> > exchange.getRequestReceiver().receiveFullString((
>>>> > exchangeReq, data) -> {
>>>> > exchangeReq.dispatch(() -> {
>>>> > post2Kafka.write2Kafka(data); // write it to
>>>> Kafka
>>>> > exchangeReq.dispatch(exchangeReq.getIoThread(),
>>>> () ->
>>>> > {
>>>> >
>>>> > exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>>> "text/plain");
>>>> >
>>>> exchangeReq.getResponseSender().send(_SUCCESS);
>>>> > });
>>>> > });
>>>> > },
>>>> > (exchangeReq, exception) -> {
>>>> >
>>>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>>> > "text/plain");
>>>> > exchangeReq.getResponseSender().send(_FAILURE);
>>>> > });
>>>> > }else{
>>>> > throw new Exception("Method GET not supported by Server
>>>> ");
>>>> > }
>>>> > }
>>>> > }
>>>> >
>>>> >
>>>> > Pls review this and let me know if i'm doing anything wrong here ...
>>>> > --Senthil
>>>> >
>>>> > On Fri, Jun 23, 2017 at 1:30 PM, Antoine Girard <
>>>> antoine.girard at ymail.com>
>>>> > wrote:
>>>> >>
>>>> >> Also, to come back on the JVM warmup, this will give you enough
>>>> answers:
>>>> >>
>>>> >>
>>>> https://stackoverflow.com/questions/36198278/why-does-the-jvm-require-warmup
>>>> >>
>>>> >> For your, it means that you have to run your tests for a few minutes
>>>> >> before starting your actual measurements.
>>>> >>
>>>> >> I am also interested about how Netty / Jetty perform under the same
>>>> >> conditions, please post!
>>>> >>
>>>> >> Cheers,
>>>> >> Antoine
>>>> >>
>>>> >> On Fri, Jun 23, 2017 at 1:24 AM, Stuart Douglas <sdouglas at redhat.com
>>>> >
>>>> >> wrote:
>>>> >>>
>>>> >>> Are you actually testing with the 'System.out.println(" Received
>>>> >>> String ==> "+message);'. System.out is incredibly slow.
>>>> >>>
>>>> >>> Stuart
>>>> >>>
>>>> >>> On Fri, Jun 23, 2017 at 7:01 AM, SenthilKumar K <
>>>> senthilec566 at gmail.com>
>>>> >>> wrote:
>>>> >>> > Sorry , I'm not an expert in JVM .. How do we do Warm Up JVM ?
>>>> >>> >
>>>> >>> > Here is the JVM args to Server:
>>>> >>> >
>>>> >>> > nohup java -Xmx4g -Xms4g -XX:MetaspaceSize=96m -XX:+UseG1GC
>>>> >>> > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35
>>>> >>> > -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50
>>>> >>> > -XX:MaxMetaspaceFreeRatio=80 -cp undertow-0.0.1.jar
>>>> HelloWorldServer
>>>> >>> >
>>>> >>> >
>>>> >>> > --Senthil
>>>> >>> >
>>>> >>> >
>>>> >>> > On Fri, Jun 23, 2017 at 2:23 AM, Antoine Girard
>>>> >>> > <antoine.girard at ymail.com>
>>>> >>> > wrote:
>>>> >>> >>
>>>> >>> >> Do you warm up your jvm prior to the testing?
>>>> >>> >>
>>>> >>> >> Cheers,
>>>> >>> >> Antoine
>>>> >>> >>
>>>> >>> >> On Thu, Jun 22, 2017 at 10:42 PM, SenthilKumar K
>>>> >>> >> <senthilec566 at gmail.com>
>>>> >>> >> wrote:
>>>> >>> >>>
>>>> >>> >>> Thanks Bill n Antoine ..
>>>> >>> >>>
>>>> >>> >>>
>>>> >>> >>> Here is the updated one : ( tried without Kafka API ) .
>>>> >>> >>>
>>>> >>> >>> public class HelloWorldServer {
>>>> >>> >>>
>>>> >>> >>> public static void main(final String[] args) {
>>>> >>> >>> Undertow server = Undertow.builder().addHttpListener(8009,
>>>> >>> >>> "localhost").setHandler(new HttpHandler() {
>>>> >>> >>> @Override
>>>> >>> >>> public void handleRequest(final HttpServerExchange exchange)
>>>> throws
>>>> >>> >>> Exception {
>>>> >>> >>> if (exchange.getRequestMethod().equals(Methods.POST)) {
>>>> >>> >>> exchange.getRequestReceiver().receiveFullString(new
>>>> >>> >>> Receiver.FullStringCallback() {
>>>> >>> >>> @Override
>>>> >>> >>> public void handle(HttpServerExchange
>>>> exchange,
>>>> >>> >>> String
>>>> >>> >>> message) {
>>>> >>> >>> System.out.println(" Received String ==>
>>>> >>> >>> "+message);
>>>> >>> >>>
>>>> exchange.getResponseSender().send(message);
>>>> >>> >>> }
>>>> >>> >>> });
>>>> >>> >>> } else {
>>>> >>> >>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>>> >>> >>> "text/plain");
>>>> >>> >>> exchange.getResponseSender().send("FAILURE");
>>>> >>> >>> }
>>>> >>> >>> }
>>>> >>> >>> }).build();
>>>> >>> >>> server.start();
>>>> >>> >>> }
>>>> >>> >>> }
>>>> >>> >>>
>>>> >>> >>>
>>>> >>> >>> Oops seems to no improvement :
>>>> >>> >>>
>>>> >>> >>> Running 1m test @ http://localhost:8009/
>>>> >>> >>> 100 threads and 1000 connections
>>>> >>> >>> Thread Stats Avg Stdev Max +/- Stdev
>>>> >>> >>> Latency 25.79ms 22.18ms 289.48ms 67.66%
>>>> >>> >>> Req/Sec 437.76 61.71 2.30k 80.26%
>>>> >>> >>> Latency Distribution
>>>> >>> >>> 50% 22.60ms
>>>> >>> >>> 75% 37.83ms
>>>> >>> >>> 90% 55.32ms
>>>> >>> >>> 99% 90.47ms
>>>> >>> >>> 2625607 requests in 1.00m, 2.76GB read
>>>> >>> >>> Requests/sec: 43688.42
>>>> >>> >>> Transfer/sec: 47.08MB
>>>> >>> >>>
>>>> >>> >>>
>>>> >>> >>> :-( :-( ..
>>>> >>> >>>
>>>> >>> >>>
>>>> >>> >>> --Senthil
>>>> >>> >>>
>>>> >>> >>>
>>>> >>> >>> On Fri, Jun 23, 2017 at 1:47 AM, Antoine Girard
>>>> >>> >>> <antoine.girard at ymail.com> wrote:
>>>> >>> >>>>
>>>> >>> >>>> You can use the Receiver API, specifically for that purpose.
>>>> >>> >>>> On the exchange, call: getRequestReceiver();
>>>> >>> >>>>
>>>> >>> >>>> You will get a receiver object:
>>>> >>> >>>>
>>>> >>> >>>>
>>>> >>> >>>>
>>>> https://github.com/undertow-io/undertow/blob/master/core/src/main/java/io/undertow/io/Receiver.java
>>>> >>> >>>>
>>>> >>> >>>> On the receiver you can call: receiveFullString, you have to
>>>> pass it
>>>> >>> >>>> a
>>>> >>> >>>> callback that will be called when the whole body has been read.
>>>> >>> >>>>
>>>> >>> >>>> Please share your results when you test this further!
>>>> >>> >>>>
>>>> >>> >>>> Cheers,
>>>> >>> >>>> Antoine
>>>> >>> >>>>
>>>> >>> >>>>
>>>> >>> >>>> On Thu, Jun 22, 2017 at 8:27 PM, SenthilKumar K
>>>> >>> >>>> <senthilec566 at gmail.com>
>>>> >>> >>>> wrote:
>>>> >>> >>>>>
>>>> >>> >>>>> Seems to Reading Request body is wrong , So what is the
>>>> efficient
>>>> >>> >>>>> way
>>>> >>> >>>>> of reading request body in undertow ?
>>>> >>> >>>>>
>>>> >>> >>>>> --Senthil
>>>> >>> >>>>>
>>>> >>> >>>>> On Thu, Jun 22, 2017 at 11:30 PM, SenthilKumar K
>>>> >>> >>>>> <senthilec566 at gmail.com> wrote:
>>>> >>> >>>>>>
>>>> >>> >>>>>> Hello Undertow Dev Team ,
>>>> >>> >>>>>>
>>>> >>> >>>>>> I have been working on the use case where i should
>>>> create
>>>> >>> >>>>>> simple
>>>> >>> >>>>>> http server to serve 1.5 Million Requests per Second per
>>>> Instance
>>>> >>> >>>>>> ..
>>>> >>> >>>>>>
>>>> >>> >>>>>>
>>>> >>> >>>>>> Here is the benchmark result of Undertow :
>>>> >>> >>>>>>
>>>> >>> >>>>>> Running 1m test @ http://127.0.0.1:8009/
>>>> >>> >>>>>> 20 threads and 40 connections
>>>> >>> >>>>>> Thread Stats Avg Stdev Max +/- Stdev
>>>> >>> >>>>>> Latency 2.51ms 10.75ms 282.22ms 99.28%
>>>> >>> >>>>>> Req/Sec 1.12k 316.65 1.96k 54.50%
>>>> >>> >>>>>> Latency Distribution
>>>> >>> >>>>>> 50% 1.43ms
>>>> >>> >>>>>> 75% 2.38ms
>>>> >>> >>>>>> 90% 2.90ms
>>>> >>> >>>>>> 99% 10.45ms
>>>> >>> >>>>>> 1328133 requests in 1.00m, 167.19MB read
>>>> >>> >>>>>> Requests/sec: 22127.92
>>>> >>> >>>>>> Transfer/sec: 2.79MB
>>>> >>> >>>>>>
>>>> >>> >>>>>> This is less compared to other frameworks like Jetty and
>>>> Netty ..
>>>> >>> >>>>>> But
>>>> >>> >>>>>> originally Undertow is high performant http server ..
>>>> >>> >>>>>>
>>>> >>> >>>>>> Hardware details:
>>>> >>> >>>>>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz,
>>>> Capacity
>>>> >>> >>>>>> 4
>>>> >>> >>>>>> GHz) , Memory : 32 G , Available memory 31 G.
>>>> >>> >>>>>>
>>>> >>> >>>>>> I would need Undertow experts to review the server code
>>>> below and
>>>> >>> >>>>>> advice me on tuning to achieve my goal( ~1.5 Million
>>>> requests/sec
>>>> >>> >>>>>> ).
>>>> >>> >>>>>>
>>>> >>> >>>>>> Server :
>>>> >>> >>>>>>
>>>> >>> >>>>>> Undertow server = Undertow.builder()
>>>> >>> >>>>>> .addHttpListener(8009, "localhost")
>>>> >>> >>>>>> .setHandler(new Handler()).build();
>>>> >>> >>>>>> server.start();
>>>> >>> >>>>>>
>>>> >>> >>>>>>
>>>> >>> >>>>>> Handler.Java
>>>> >>> >>>>>>
>>>> >>> >>>>>> final Pooled<ByteBuffer> pooledByteBuffer =
>>>> >>> >>>>>>
>>>> >>> >>>>>> exchange.getConnection().getBufferPool().allocate();
>>>> >>> >>>>>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource();
>>>> >>> >>>>>> byteBuffer.clear();
>>>> >>> >>>>>> exchange.getRequestChannel().read(byteBuffer);
>>>> >>> >>>>>> int pos = byteBuffer.position();
>>>> >>> >>>>>> byteBuffer.rewind();
>>>> >>> >>>>>> byte[] bytes = new byte[pos];
>>>> >>> >>>>>> byteBuffer.get(bytes);
>>>> >>> >>>>>> String requestBody = new String(bytes,
>>>> Charset.forName("UTF-8")
>>>> >>> >>>>>> );
>>>> >>> >>>>>> byteBuffer.clear();
>>>> >>> >>>>>> pooledByteBuffer.free();
>>>> >>> >>>>>> final PostToKafka post2Kafka = new PostToKafka();
>>>> >>> >>>>>> try {
>>>> >>> >>>>>> post2Kafka.write2Kafka(requestBody); { This API can handle
>>>> ~2
>>>> >>> >>>>>> Millions events per sec }
>>>> >>> >>>>>> } catch (Exception e) {
>>>> >>> >>>>>> e.printStackTrace();
>>>> >>> >>>>>> }
>>>> >>> >>>>>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>>> >>> >>>>>> "text/plain");
>>>> >>> >>>>>> exchange.getResponseSender().send("SUCCESS");
>>>> >>> >>>>>>
>>>> >>> >>>>>>
>>>> >>> >>>>>> --Senthil
>>>> >>> >>>>>
>>>> >>> >>>>>
>>>> >>> >>>>>
>>>> >>> >>>>> _______________________________________________
>>>> >>> >>>>> undertow-dev mailing list
>>>> >>> >>>>> undertow-dev at lists.jboss.org
>>>> >>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev
>>>> >>> >>>>
>>>> >>> >>>>
>>>> >>> >>>
>>>> >>> >>
>>>> >>> >
>>>> >>> >
>>>> >>> > _______________________________________________
>>>> >>> > undertow-dev mailing list
>>>> >>> > undertow-dev at lists.jboss.org
>>>> >>> > https://lists.jboss.org/mailman/listinfo/undertow-dev
>>>> >>
>>>> >>
>>>> >
>>>> _______________________________________________
>>>> undertow-dev mailing list
>>>> undertow-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev
>>>>
>>>
>>>
>>
> _______________________________________________
> undertow-dev mailing list
> undertow-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/undertow-dev
--
Med venlig hilsen / Best regards
*Kim Rasmussen*
Partner, IT Architect
*Asseco Denmark A/S*
Kronprinsessegade 54
DK-1306 Copenhagen K
Mobile: +45 26 16 40 23
Ph.: +45 33 36 46 60
Fax: +45 33 36 46 61
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20170708/5304450b/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 25115 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/undertow-dev/attachments/20170708/5304450b/attachment-0002.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 31196 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/undertow-dev/attachments/20170708/5304450b/attachment-0003.png
More information about the undertow-dev
mailing list