Re: [undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance

Have you tried playing around with the number of io and worker threads? lør. 8. jul. 2017 kl. 17.28 skrev SenthilKumar K <senthilec566(a)gmail.com&gt;: > Any comments on *Undertow Vs Netty* ? Am i doing wrong benchmark testing > ?? Should i change benchmark strategy ? > > --Senthil > > On Fri, Jul 7, 2017 at 3:14 PM, SenthilKumar K <senthilec566(a)gmail.com&gt; > wrote: > >> Sorry for delay in responding to this thread! >> >> Thanks to everyone who helped me to Optimize Undertow Server. >> >> Here is the comparison after benchmarking my use case against Netty: >> >> *Undertow Vs Netty :* >> >> Test Case 1 : >> Simple Request Response ( No Kafka ): >> >> *Undertow:* >> Running 10m test @ http://198.18.134.13:8009/ >> 500 threads and 5000 connections >> Thread Stats Avg Stdev Max +/- Stdev >> Latency *3.52m * 2.64m 8.96m 54.63% >> Req/Sec 376.58 103.18 0.99k 80.53% >> 111628942 requests in 10.00m, 13.72GB read >> Socket errors: connect 0, read 28, write 0, timeout 2 >> Requests/sec: *186122.56* >> Transfer/sec: 23.43MB >> >> *Netty:* >> Running 10m test @ http://198.18.134.13:8009/ >> 500 threads and 5000 connections >> Thread Stats Avg Stdev Max +/- Stdev >> Latency *3.77m* 2.10m 7.51m 57.73% >> Req/Sec 518.63 31.78 652.00 70.25% >> 155406992 requests in 10.00m, 13.82GB read >> Socket errors: connect 0, read 49, write 0, timeout 0 >> Requests/sec: *259107*.30 >> Transfer/sec: 24.17MB >> >> >> *Test Case 2:* >> Request --> Read --> Send it Kafka : >> >> *Undertow:* >> Running 10m test @ http://198.18.134.13:8009/ >> 500 threads and 5000 connections >> Thread Stats Avg Stdev Max +/- Stdev >> Latency *4.37m * 2.46m 8.72m 57.83% >> Req/Sec 267.32 5.17 287.00 74.52% >> 80044045 requests in 10.00m, 9.84GB read >> Socket errors: connect 0, read 121, write 0, timeout 0 >> Requests/sec: *133459.79* >> Transfer/sec: 16.80MB >> >> *Netty:* >> Running 10m test @ http://198.18.134.13:8009/ >> 500 threads and 5000 connections >> Thread Stats Avg Stdev Max +/- Stdev >> Latency *3.78m * 2.10m 7.55m 57.79% >> Req/Sec 516.92 28.84 642.00 69.60% >> 154770536 requests in 10.00m, 13.69GB read >> Socket errors: connect 0, read 11, write 0, timeout 101 >> Requests/sec: *258049.39* >> Transfer/sec: 23.38MB >> >> >> >> CPU Usage: >> *Undertow:* >> [image: Inline image 1] >> >> *Netty:* >> [image: Inline image 2] >> >> >> --Senthil >> >> On Thu, Jun 29, 2017 at 7:34 AM, Bill O'Neil <bill(a)dartalley.com&gt; wrote: >> >>> 1. Can you run the benchmark with the kafka line commented out at first >>> and then again with it not commented out? >>> 2. What rates were you getting with Jetty and Netty? >>> 3. Are you running the tests from the same machine or a different one? >>> If its the same machine and its using 20 threads they will be contending >>> with undertows IO threads. >>> 4. You can probably ignore the POST check if thats all your going to >>> accept and its not a public api. >>> >>> import io.undertow.server.HttpHandler; >>> import io.undertow.server.HttpServerExchange; >>> import io.undertow.util.Headers; >>> import io.undertow.util.Methods; >>> >>> public class DLRHandler implements HttpHandler { >>> >>> final public static String _SUCCESS="SUCCESS"; >>> final public static String _FAILURE="FAILURE"; >>> final PostToKafka post2Kafka = new PostToKafka(); >>> >>> @Override >>> public void handleRequest( final HttpServerExchange exchange) >>> throws Exception { >>> if (exchange.getRequestMethod().equals(Methods.POST)) { >>> exchange.getRequestReceiver().receiveFullString(( >>> exchangeReq, data) -> { >>> //post2Kafka.write2Kafka(data); // write it to Kafka >>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, >>> "text/plain"); >>> exchangeReq.getResponseSender().send(_SUCCESS); >>> }, >>> (exchangeReq, exception) -> { >>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, >>> "text/plain"); >>> exchangeReq.getResponseSender().send(_FAILURE); >>> }); >>> }else{ >>> throw new Exception("Method GET not supported by Server "); >>> } >>> } >>> } >>> >>> On Wed, Jun 28, 2017 at 6:59 PM, Stuart Douglas <sdouglas(a)redhat.com&gt; >>> wrote: >>> >>>> The multiple dispatches() are unnecessary (well the second one to the >>>> IO thread is definitely unnecessary, the first one is only required if >>>> post2Kafka.write2Kafka(data); is a blocking operation and needs to be >>>> executed in a worker thread). >>>> >>>> Stuart >>>> >>>> On Wed, Jun 28, 2017 at 5:42 PM, SenthilKumar K < >>>> senthilec566(a)gmail.com&gt; wrote: >>>> > After modifying the code below i could see the improvement ( not >>>> much >>>> > slightly ) in server - 65k req/sec. >>>> > >>>> > import io.undertow.server.HttpHandler; >>>> > import io.undertow.server.HttpServerExchange; >>>> > import io.undertow.util.Headers; >>>> > import io.undertow.util.Methods; >>>> > >>>> > public class DLRHandler implements HttpHandler { >>>> > >>>> > final public static String _SUCCESS="SUCCESS"; >>>> > final public static String _FAILURE="FAILURE"; >>>> > final PostToKafka post2Kafka = new PostToKafka(); >>>> > >>>> > @Override >>>> > public void handleRequest( final HttpServerExchange exchange) >>>> throws >>>> > Exception { >>>> > if (exchange.getRequestMethod().equals(Methods.POST)) { >>>> > exchange.getRequestReceiver().receiveFullString(( >>>> > exchangeReq, data) -> { >>>> > exchangeReq.dispatch(() -> { >>>> > post2Kafka.write2Kafka(data); // write it to >>>> Kafka >>>> > exchangeReq.dispatch(exchangeReq.getIoThread(), >>>> () -> >>>> > { >>>> > >>>> > exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, >>>> "text/plain"); >>>> > exchangeReq.getResponseSender().send(_ >>>> SUCCESS); >>>> > }); >>>> > }); >>>> > }, >>>> > (exchangeReq, exception) -> { >>>> > exchangeReq.getResponseHeaders().put( >>>> Headers.CONTENT_TYPE, >>>> > "text/plain"); >>>> > exchangeReq.getResponseSender().send(_FAILURE); >>>> > }); >>>> > }else{ >>>> > throw new Exception("Method GET not supported by Server >>>> "); >>>> > } >>>> > } >>>> > } >>>> > >>>> > >>>> > Pls review this and let me know if i'm doing anything wrong here ... >>>> > --Senthil >>>> > >>>> > On Fri, Jun 23, 2017 at 1:30 PM, Antoine Girard < >>>> antoine.girard(a)ymail.com&gt; >>>> > wrote: >>>> >> >>>> >> Also, to come back on the JVM warmup, this will give you enough >>>> answers: >>>> >> >>>> >> https://stackoverflow.com/questions/36198278/why-does- >>>> the-jvm-require-warmup >>>> >> >>>> >> For your, it means that you have to run your tests for a few minutes >>>> >> before starting your actual measurements. >>>> >> >>>> >> I am also interested about how Netty / Jetty perform under the same >>>> >> conditions, please post! >>>> >> >>>> >> Cheers, >>>> >> Antoine >>>> >> >>>> >> On Fri, Jun 23, 2017 at 1:24 AM, Stuart Douglas < >>>> sdouglas(a)redhat.com&gt; >>>> >> wrote: >>>> >>> >>>> >>> Are you actually testing with the 'System.out.println(" Received >>>> >>> String ==> "+message);'. System.out is incredibly slow. >>>> >>> >>>> >>> Stuart >>>> >>> >>>> >>> On Fri, Jun 23, 2017 at 7:01 AM, SenthilKumar K < >>>> senthilec566(a)gmail.com&gt; >>>> >>> wrote: >>>> >>> > Sorry , I'm not an expert in JVM .. How do we do Warm Up JVM ? >>>> >>> > >>>> >>> > Here is the JVM args to Server: >>>> >>> > >>>> >>> > nohup java -Xmx4g -Xms4g -XX:MetaspaceSize=96m -XX:+UseG1GC >>>> >>> > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 >>>> >>> > -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50 >>>> >>> > -XX:MaxMetaspaceFreeRatio=80 -cp undertow-0.0.1.jar >>>> HelloWorldServer >>>> >>> > >>>> >>> > >>>> >>> > --Senthil >>>> >>> > >>>> >>> > >>>> >>> > On Fri, Jun 23, 2017 at 2:23 AM, Antoine Girard >>>> >>> > <antoine.girard(a)ymail.com&gt; >>>> >>> > wrote: >>>> >>> >> >>>> >>> >> Do you warm up your jvm prior to the testing? >>>> >>> >> >>>> >>> >> Cheers, >>>> >>> >> Antoine >>>> >>> >> >>>> >>> >> On Thu, Jun 22, 2017 at 10:42 PM, SenthilKumar K >>>> >>> >> <senthilec566(a)gmail.com&gt; >>>> >>> >> wrote: >>>> >>> >>> >>>> >>> >>> Thanks Bill n Antoine .. >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> Here is the updated one : ( tried without Kafka API ) . >>>> >>> >>> >>>> >>> >>> public class HelloWorldServer { >>>> >>> >>> >>>> >>> >>> public static void main(final String[] args) { >>>> >>> >>> Undertow server = Undertow.builder().addHttpListener(8009, >>>> >>> >>> "localhost").setHandler(new HttpHandler() { >>>> >>> >>> @Override >>>> >>> >>> public void handleRequest(final HttpServerExchange exchange) >>>> throws >>>> >>> >>> Exception { >>>> >>> >>> if (exchange.getRequestMethod().equals(Methods.POST)) { >>>> >>> >>> exchange.getRequestReceiver().receiveFullString(new >>>> >>> >>> Receiver.FullStringCallback() { >>>> >>> >>> @Override >>>> >>> >>> public void handle(HttpServerExchange >>>> exchange, >>>> >>> >>> String >>>> >>> >>> message) { >>>> >>> >>> System.out.println(" Received String ==> >>>> >>> >>> "+message); >>>> >>> >>> exchange.getResponseSender(). >>>> send(message); >>>> >>> >>> } >>>> >>> >>> }); >>>> >>> >>> } else { >>>> >>> >>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, >>>> >>> >>> "text/plain"); >>>> >>> >>> exchange.getResponseSender().send("FAILURE"); >>>> >>> >>> } >>>> >>> >>> } >>>> >>> >>> }).build(); >>>> >>> >>> server.start(); >>>> >>> >>> } >>>> >>> >>> } >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> Oops seems to no improvement : >>>> >>> >>> >>>> >>> >>> Running 1m test @ http://localhost:8009/ >>>> >>> >>> 100 threads and 1000 connections >>>> >>> >>> Thread Stats Avg Stdev Max +/- Stdev >>>> >>> >>> Latency 25.79ms 22.18ms 289.48ms 67.66% >>>> >>> >>> Req/Sec 437.76 61.71 2.30k 80.26% >>>> >>> >>> Latency Distribution >>>> >>> >>> 50% 22.60ms >>>> >>> >>> 75% 37.83ms >>>> >>> >>> 90% 55.32ms >>>> >>> >>> 99% 90.47ms >>>> >>> >>> 2625607 requests in 1.00m, 2.76GB read >>>> >>> >>> Requests/sec: 43688.42 >>>> >>> >>> Transfer/sec: 47.08MB >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> :-( :-( .. >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> --Senthil >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> On Fri, Jun 23, 2017 at 1:47 AM, Antoine Girard >>>> >>> >>> <antoine.girard(a)ymail.com&gt; wrote: >>>> >>> >>>> >>>> >>> >>>> You can use the Receiver API, specifically for that purpose. >>>> >>> >>>> On the exchange, call: getRequestReceiver(); >>>> >>> >>>> >>>> >>> >>>> You will get a receiver object: >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> https://github.com/undertow-io/undertow/blob/master/core/ >>>> src/main/java/io/undertow/io/Receiver.java >>>> >>> >>>> >>>> >>> >>>> On the receiver you can call: receiveFullString, you have to >>>> pass it >>>> >>> >>>> a >>>> >>> >>>> callback that will be called when the whole body has been >>>> read. >>>> >>> >>>> >>>> >>> >>>> Please share your results when you test this further! >>>> >>> >>>> >>>> >>> >>>> Cheers, >>>> >>> >>>> Antoine >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> On Thu, Jun 22, 2017 at 8:27 PM, SenthilKumar K >>>> >>> >>>> <senthilec566(a)gmail.com&gt; >>>> >>> >>>> wrote: >>>> >>> >>>>> >>>> >>> >>>>> Seems to Reading Request body is wrong , So what is the >>>> efficient >>>> >>> >>>>> way >>>> >>> >>>>> of reading request body in undertow ? >>>> >>> >>>>> >>>> >>> >>>>> --Senthil >>>> >>> >>>>> >>>> >>> >>>>> On Thu, Jun 22, 2017 at 11:30 PM, SenthilKumar K >>>> >>> >>>>> <senthilec566(a)gmail.com&gt; wrote: >>>> >>> >>>>>> >>>> >>> >>>>>> Hello Undertow Dev Team , >>>> >>> >>>>>> >>>> >>> >>>>>> I have been working on the use case where i should >>>> create >>>> >>> >>>>>> simple >>>> >>> >>>>>> http server to serve 1.5 Million Requests per Second per >>>> Instance >>>> >>> >>>>>> .. >>>> >>> >>>>>> >>>> >>> >>>>>> >>>> >>> >>>>>> Here is the benchmark result of Undertow : >>>> >>> >>>>>> >>>> >>> >>>>>> Running 1m test @ http://127.0.0.1:8009/ >>>> >>> >>>>>> 20 threads and 40 connections >>>> >>> >>>>>> Thread Stats Avg Stdev Max +/- Stdev >>>> >>> >>>>>> Latency 2.51ms 10.75ms 282.22ms 99.28% >>>> >>> >>>>>> Req/Sec 1.12k 316.65 1.96k 54.50% >>>> >>> >>>>>> Latency Distribution >>>> >>> >>>>>> 50% 1.43ms >>>> >>> >>>>>> 75% 2.38ms >>>> >>> >>>>>> 90% 2.90ms >>>> >>> >>>>>> 99% 10.45ms >>>> >>> >>>>>> 1328133 requests in 1.00m, 167.19MB read >>>> >>> >>>>>> Requests/sec: 22127.92 >>>> >>> >>>>>> Transfer/sec: 2.79MB >>>> >>> >>>>>> >>>> >>> >>>>>> This is less compared to other frameworks like Jetty and >>>> Netty .. >>>> >>> >>>>>> But >>>> >>> >>>>>> originally Undertow is high performant http server .. >>>> >>> >>>>>> >>>> >>> >>>>>> Hardware details: >>>> >>> >>>>>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz, >>>> Capacity >>>> >>> >>>>>> 4 >>>> >>> >>>>>> GHz) , Memory : 32 G , Available memory 31 G. >>>> >>> >>>>>> >>>> >>> >>>>>> I would need Undertow experts to review the server code >>>> below and >>>> >>> >>>>>> advice me on tuning to achieve my goal( ~1.5 Million >>>> requests/sec >>>> >>> >>>>>> ). >>>> >>> >>>>>> >>>> >>> >>>>>> Server : >>>> >>> >>>>>> >>>> >>> >>>>>> Undertow server = Undertow.builder() >>>> >>> >>>>>> .addHttpListener(8009, "localhost") >>>> >>> >>>>>> .setHandler(new Handler()).build(); >>>> >>> >>>>>> server.start(); >>>> >>> >>>>>> >>>> >>> >>>>>> >>>> >>> >>>>>> Handler.Java >>>> >>> >>>>>> >>>> >>> >>>>>> final Pooled<ByteBuffer> pooledByteBuffer = >>>> >>> >>>>>> >>>> >>> >>>>>> exchange.getConnection().getBufferPool().allocate(); >>>> >>> >>>>>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource() >>>> ; >>>> >>> >>>>>> byteBuffer.clear(); >>>> >>> >>>>>> exchange.getRequestChannel().read(byteBuffer); >>>> >>> >>>>>> int pos = byteBuffer.position(); >>>> >>> >>>>>> byteBuffer.rewind(); >>>> >>> >>>>>> byte[] bytes = new byte[pos]; >>>> >>> >>>>>> byteBuffer.get(bytes); >>>> >>> >>>>>> String requestBody = new String(bytes, >>>> Charset.forName("UTF-8") >>>> >>> >>>>>> ); >>>> >>> >>>>>> byteBuffer.clear(); >>>> >>> >>>>>> pooledByteBuffer.free(); >>>> >>> >>>>>> final PostToKafka post2Kafka = new PostToKafka(); >>>> >>> >>>>>> try { >>>> >>> >>>>>> post2Kafka.write2Kafka(requestBody); { This API can >>>> handle ~2 >>>> >>> >>>>>> Millions events per sec } >>>> >>> >>>>>> } catch (Exception e) { >>>> >>> >>>>>> e.printStackTrace(); >>>> >>> >>>>>> } >>>> >>> >>>>>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, >>>> >>> >>>>>> "text/plain"); >>>> >>> >>>>>> exchange.getResponseSender().send("SUCCESS"); >>>> >>> >>>>>> >>>> >>> >>>>>> >>>> >>> >>>>>> --Senthil >>>> >>> >>>>> >>>> >>> >>>>> >>>> >>> >>>>> >>>> >>> >>>>> _______________________________________________ >>>> >>> >>>>> undertow-dev mailing list >>>> >>> >>>>> undertow-dev(a)lists.jboss.org >>>> >>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>> >>>> >>> >> >>>> >>> > >>>> >>> > >>>> >>> > _______________________________________________ >>>> >>> > undertow-dev mailing list >>>> >>> > undertow-dev(a)lists.jboss.org >>>> >>> > https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >> >>>> >> >>>> > >>>> _______________________________________________ >>>> undertow-dev mailing list >>>> undertow-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >>> >>> >> > _______________________________________________ > undertow-dev mailing list > undertow-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/undertow-dev -- Med venlig hilsen / Best regards *Kim Rasmussen* Partner, IT Architect *Asseco Denmark A/S* Kronprinsessegade 54 DK-1306 Copenhagen K Mobile: +45 26 16 40 23 Ph.: +45 33 36 46 60 Fax: +45 33 36 46 61