Re: [undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance

Any comments on *Undertow Vs Netty* ? Am i doing wrong benchmark testing ?? Should i change benchmark strategy ? --Senthil On Fri, Jul 7, 2017 at 3:14 PM, SenthilKumar K <senthilec566(a)gmail.com&gt; wrote: > Sorry for delay in responding to this thread! > > Thanks to everyone who helped me to Optimize Undertow Server. > > Here is the comparison after benchmarking my use case against Netty: > > *Undertow Vs Netty :* > > Test Case 1 : > Simple Request Response ( No Kafka ): > > *Undertow:* > Running 10m test @ http://198.18.134.13:8009/ > 500 threads and 5000 connections > Thread Stats Avg Stdev Max +/- Stdev > Latency *3.52m * 2.64m 8.96m 54.63% > Req/Sec 376.58 103.18 0.99k 80.53% > 111628942 requests in 10.00m, 13.72GB read > Socket errors: connect 0, read 28, write 0, timeout 2 > Requests/sec: *186122.56* > Transfer/sec: 23.43MB > > *Netty:* > Running 10m test @ http://198.18.134.13:8009/ > 500 threads and 5000 connections > Thread Stats Avg Stdev Max +/- Stdev > Latency *3.77m* 2.10m 7.51m 57.73% > Req/Sec 518.63 31.78 652.00 70.25% > 155406992 requests in 10.00m, 13.82GB read > Socket errors: connect 0, read 49, write 0, timeout 0 > Requests/sec: *259107*.30 > Transfer/sec: 24.17MB > > > *Test Case 2:* > Request --> Read --> Send it Kafka : > > *Undertow:* > Running 10m test @ http://198.18.134.13:8009/ > 500 threads and 5000 connections > Thread Stats Avg Stdev Max +/- Stdev > Latency *4.37m * 2.46m 8.72m 57.83% > Req/Sec 267.32 5.17 287.00 74.52% > 80044045 requests in 10.00m, 9.84GB read > Socket errors: connect 0, read 121, write 0, timeout 0 > Requests/sec: *133459.79* > Transfer/sec: 16.80MB > > *Netty:* > Running 10m test @ http://198.18.134.13:8009/ > 500 threads and 5000 connections > Thread Stats Avg Stdev Max +/- Stdev > Latency *3.78m * 2.10m 7.55m 57.79% > Req/Sec 516.92 28.84 642.00 69.60% > 154770536 requests in 10.00m, 13.69GB read > Socket errors: connect 0, read 11, write 0, timeout 101 > Requests/sec: *258049.39* > Transfer/sec: 23.38MB > > > > CPU Usage: > *Undertow:* > [image: Inline image 1] > > *Netty:* > [image: Inline image 2] > > > --Senthil > > On Thu, Jun 29, 2017 at 7:34 AM, Bill O'Neil <bill(a)dartalley.com&gt; wrote: > >> 1. Can you run the benchmark with the kafka line commented out at first >> and then again with it not commented out? >> 2. What rates were you getting with Jetty and Netty? >> 3. Are you running the tests from the same machine or a different one? >> If its the same machine and its using 20 threads they will be contending >> with undertows IO threads. >> 4. You can probably ignore the POST check if thats all your going to >> accept and its not a public api. >> >> import io.undertow.server.HttpHandler; >> import io.undertow.server.HttpServerExchange; >> import io.undertow.util.Headers; >> import io.undertow.util.Methods; >> >> public class DLRHandler implements HttpHandler { >> >> final public static String _SUCCESS="SUCCESS"; >> final public static String _FAILURE="FAILURE"; >> final PostToKafka post2Kafka = new PostToKafka(); >> >> @Override >> public void handleRequest( final HttpServerExchange exchange) throws >> Exception { >> if (exchange.getRequestMethod().equals(Methods.POST)) { >> exchange.getRequestReceiver().receiveFullString(( >> exchangeReq, data) -> { >> //post2Kafka.write2Kafka(data); // write it to Kafka >> >> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain"); >> exchangeReq.getResponseSender().send(_SUCCESS); >> }, >> (exchangeReq, exception) -> { >> >> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain"); >> exchangeReq.getResponseSender().send(_FAILURE); >> }); >> }else{ >> throw new Exception("Method GET not supported by Server "); >> } >> } >> } >> >> On Wed, Jun 28, 2017 at 6:59 PM, Stuart Douglas <sdouglas(a)redhat.com&gt; >> wrote: >> >>> The multiple dispatches() are unnecessary (well the second one to the >>> IO thread is definitely unnecessary, the first one is only required if >>> post2Kafka.write2Kafka(data); is a blocking operation and needs to be >>> executed in a worker thread). >>> >>> Stuart >>> >>> On Wed, Jun 28, 2017 at 5:42 PM, SenthilKumar K <senthilec566(a)gmail.com&gt; >>> wrote: >>> > After modifying the code below i could see the improvement ( not much >>> > slightly ) in server - 65k req/sec. >>> > >>> > import io.undertow.server.HttpHandler; >>> > import io.undertow.server.HttpServerExchange; >>> > import io.undertow.util.Headers; >>> > import io.undertow.util.Methods; >>> > >>> > public class DLRHandler implements HttpHandler { >>> > >>> > final public static String _SUCCESS="SUCCESS"; >>> > final public static String _FAILURE="FAILURE"; >>> > final PostToKafka post2Kafka = new PostToKafka(); >>> > >>> > @Override >>> > public void handleRequest( final HttpServerExchange exchange) >>> throws >>> > Exception { >>> > if (exchange.getRequestMethod().equals(Methods.POST)) { >>> > exchange.getRequestReceiver().receiveFullString(( >>> > exchangeReq, data) -> { >>> > exchangeReq.dispatch(() -> { >>> > post2Kafka.write2Kafka(data); // write it to >>> Kafka >>> > exchangeReq.dispatch(exchangeReq.getIoThread(), >>> () -> >>> > { >>> > >>> > exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, >>> "text/plain"); >>> > >>> exchangeReq.getResponseSender().send(_SUCCESS); >>> > }); >>> > }); >>> > }, >>> > (exchangeReq, exception) -> { >>> > >>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, >>> > "text/plain"); >>> > exchangeReq.getResponseSender().send(_FAILURE); >>> > }); >>> > }else{ >>> > throw new Exception("Method GET not supported by Server >>> "); >>> > } >>> > } >>> > } >>> > >>> > >>> > Pls review this and let me know if i'm doing anything wrong here ... >>> > --Senthil >>> > >>> > On Fri, Jun 23, 2017 at 1:30 PM, Antoine Girard < >>> antoine.girard(a)ymail.com&gt; >>> > wrote: >>> >> >>> >> Also, to come back on the JVM warmup, this will give you enough >>> answers: >>> >> >>> >> >>> https://stackoverflow.com/questions/36198278/why-does-the-jvm-require-warmup >>> >> >>> >> For your, it means that you have to run your tests for a few minutes >>> >> before starting your actual measurements. >>> >> >>> >> I am also interested about how Netty / Jetty perform under the same >>> >> conditions, please post! >>> >> >>> >> Cheers, >>> >> Antoine >>> >> >>> >> On Fri, Jun 23, 2017 at 1:24 AM, Stuart Douglas <sdouglas(a)redhat.com >>> > >>> >> wrote: >>> >>> >>> >>> Are you actually testing with the 'System.out.println(" Received >>> >>> String ==> "+message);'. System.out is incredibly slow. >>> >>> >>> >>> Stuart >>> >>> >>> >>> On Fri, Jun 23, 2017 at 7:01 AM, SenthilKumar K < >>> senthilec566(a)gmail.com&gt; >>> >>> wrote: >>> >>> > Sorry , I'm not an expert in JVM .. How do we do Warm Up JVM ? >>> >>> > >>> >>> > Here is the JVM args to Server: >>> >>> > >>> >>> > nohup java -Xmx4g -Xms4g -XX:MetaspaceSize=96m -XX:+UseG1GC >>> >>> > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 >>> >>> > -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50 >>> >>> > -XX:MaxMetaspaceFreeRatio=80 -cp undertow-0.0.1.jar >>> HelloWorldServer >>> >>> > >>> >>> > >>> >>> > --Senthil >>> >>> > >>> >>> > >>> >>> > On Fri, Jun 23, 2017 at 2:23 AM, Antoine Girard >>> >>> > <antoine.girard(a)ymail.com&gt; >>> >>> > wrote: >>> >>> >> >>> >>> >> Do you warm up your jvm prior to the testing? >>> >>> >> >>> >>> >> Cheers, >>> >>> >> Antoine >>> >>> >> >>> >>> >> On Thu, Jun 22, 2017 at 10:42 PM, SenthilKumar K >>> >>> >> <senthilec566(a)gmail.com&gt; >>> >>> >> wrote: >>> >>> >>> >>> >>> >>> Thanks Bill n Antoine .. >>> >>> >>> >>> >>> >>> >>> >>> >>> Here is the updated one : ( tried without Kafka API ) . >>> >>> >>> >>> >>> >>> public class HelloWorldServer { >>> >>> >>> >>> >>> >>> public static void main(final String[] args) { >>> >>> >>> Undertow server = Undertow.builder().addHttpListener(8009, >>> >>> >>> "localhost").setHandler(new HttpHandler() { >>> >>> >>> @Override >>> >>> >>> public void handleRequest(final HttpServerExchange exchange) >>> throws >>> >>> >>> Exception { >>> >>> >>> if (exchange.getRequestMethod().equals(Methods.POST)) { >>> >>> >>> exchange.getRequestReceiver().receiveFullString(new >>> >>> >>> Receiver.FullStringCallback() { >>> >>> >>> @Override >>> >>> >>> public void handle(HttpServerExchange >>> exchange, >>> >>> >>> String >>> >>> >>> message) { >>> >>> >>> System.out.println(" Received String ==> >>> >>> >>> "+message); >>> >>> >>> >>> exchange.getResponseSender().send(message); >>> >>> >>> } >>> >>> >>> }); >>> >>> >>> } else { >>> >>> >>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, >>> >>> >>> "text/plain"); >>> >>> >>> exchange.getResponseSender().send("FAILURE"); >>> >>> >>> } >>> >>> >>> } >>> >>> >>> }).build(); >>> >>> >>> server.start(); >>> >>> >>> } >>> >>> >>> } >>> >>> >>> >>> >>> >>> >>> >>> >>> Oops seems to no improvement : >>> >>> >>> >>> >>> >>> Running 1m test @ http://localhost:8009/ >>> >>> >>> 100 threads and 1000 connections >>> >>> >>> Thread Stats Avg Stdev Max +/- Stdev >>> >>> >>> Latency 25.79ms 22.18ms 289.48ms 67.66% >>> >>> >>> Req/Sec 437.76 61.71 2.30k 80.26% >>> >>> >>> Latency Distribution >>> >>> >>> 50% 22.60ms >>> >>> >>> 75% 37.83ms >>> >>> >>> 90% 55.32ms >>> >>> >>> 99% 90.47ms >>> >>> >>> 2625607 requests in 1.00m, 2.76GB read >>> >>> >>> Requests/sec: 43688.42 >>> >>> >>> Transfer/sec: 47.08MB >>> >>> >>> >>> >>> >>> >>> >>> >>> :-( :-( .. >>> >>> >>> >>> >>> >>> >>> >>> >>> --Senthil >>> >>> >>> >>> >>> >>> >>> >>> >>> On Fri, Jun 23, 2017 at 1:47 AM, Antoine Girard >>> >>> >>> <antoine.girard(a)ymail.com&gt; wrote: >>> >>> >>>> >>> >>> >>>> You can use the Receiver API, specifically for that purpose. >>> >>> >>>> On the exchange, call: getRequestReceiver(); >>> >>> >>>> >>> >>> >>>> You will get a receiver object: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> https://github.com/undertow-io/undertow/blob/master/core/src/main/java/io... >>> >>> >>>> >>> >>> >>>> On the receiver you can call: receiveFullString, you have to >>> pass it >>> >>> >>>> a >>> >>> >>>> callback that will be called when the whole body has been read. >>> >>> >>>> >>> >>> >>>> Please share your results when you test this further! >>> >>> >>>> >>> >>> >>>> Cheers, >>> >>> >>>> Antoine >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> On Thu, Jun 22, 2017 at 8:27 PM, SenthilKumar K >>> >>> >>>> <senthilec566(a)gmail.com&gt; >>> >>> >>>> wrote: >>> >>> >>>>> >>> >>> >>>>> Seems to Reading Request body is wrong , So what is the >>> efficient >>> >>> >>>>> way >>> >>> >>>>> of reading request body in undertow ? >>> >>> >>>>> >>> >>> >>>>> --Senthil >>> >>> >>>>> >>> >>> >>>>> On Thu, Jun 22, 2017 at 11:30 PM, SenthilKumar K >>> >>> >>>>> <senthilec566(a)gmail.com&gt; wrote: >>> >>> >>>>>> >>> >>> >>>>>> Hello Undertow Dev Team , >>> >>> >>>>>> >>> >>> >>>>>> I have been working on the use case where i should >>> create >>> >>> >>>>>> simple >>> >>> >>>>>> http server to serve 1.5 Million Requests per Second per >>> Instance >>> >>> >>>>>> .. >>> >>> >>>>>> >>> >>> >>>>>> >>> >>> >>>>>> Here is the benchmark result of Undertow : >>> >>> >>>>>> >>> >>> >>>>>> Running 1m test @ http://127.0.0.1:8009/ >>> >>> >>>>>> 20 threads and 40 connections >>> >>> >>>>>> Thread Stats Avg Stdev Max +/- Stdev >>> >>> >>>>>> Latency 2.51ms 10.75ms 282.22ms 99.28% >>> >>> >>>>>> Req/Sec 1.12k 316.65 1.96k 54.50% >>> >>> >>>>>> Latency Distribution >>> >>> >>>>>> 50% 1.43ms >>> >>> >>>>>> 75% 2.38ms >>> >>> >>>>>> 90% 2.90ms >>> >>> >>>>>> 99% 10.45ms >>> >>> >>>>>> 1328133 requests in 1.00m, 167.19MB read >>> >>> >>>>>> Requests/sec: 22127.92 >>> >>> >>>>>> Transfer/sec: 2.79MB >>> >>> >>>>>> >>> >>> >>>>>> This is less compared to other frameworks like Jetty and >>> Netty .. >>> >>> >>>>>> But >>> >>> >>>>>> originally Undertow is high performant http server .. >>> >>> >>>>>> >>> >>> >>>>>> Hardware details: >>> >>> >>>>>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz, >>> Capacity >>> >>> >>>>>> 4 >>> >>> >>>>>> GHz) , Memory : 32 G , Available memory 31 G. >>> >>> >>>>>> >>> >>> >>>>>> I would need Undertow experts to review the server code >>> below and >>> >>> >>>>>> advice me on tuning to achieve my goal( ~1.5 Million >>> requests/sec >>> >>> >>>>>> ). >>> >>> >>>>>> >>> >>> >>>>>> Server : >>> >>> >>>>>> >>> >>> >>>>>> Undertow server = Undertow.builder() >>> >>> >>>>>> .addHttpListener(8009, "localhost") >>> >>> >>>>>> .setHandler(new Handler()).build(); >>> >>> >>>>>> server.start(); >>> >>> >>>>>> >>> >>> >>>>>> >>> >>> >>>>>> Handler.Java >>> >>> >>>>>> >>> >>> >>>>>> final Pooled<ByteBuffer> pooledByteBuffer = >>> >>> >>>>>> >>> >>> >>>>>> exchange.getConnection().getBufferPool().allocate(); >>> >>> >>>>>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource(); >>> >>> >>>>>> byteBuffer.clear(); >>> >>> >>>>>> exchange.getRequestChannel().read(byteBuffer); >>> >>> >>>>>> int pos = byteBuffer.position(); >>> >>> >>>>>> byteBuffer.rewind(); >>> >>> >>>>>> byte[] bytes = new byte[pos]; >>> >>> >>>>>> byteBuffer.get(bytes); >>> >>> >>>>>> String requestBody = new String(bytes, >>> Charset.forName("UTF-8") >>> >>> >>>>>> ); >>> >>> >>>>>> byteBuffer.clear(); >>> >>> >>>>>> pooledByteBuffer.free(); >>> >>> >>>>>> final PostToKafka post2Kafka = new PostToKafka(); >>> >>> >>>>>> try { >>> >>> >>>>>> post2Kafka.write2Kafka(requestBody); { This API can handle >>> ~2 >>> >>> >>>>>> Millions events per sec } >>> >>> >>>>>> } catch (Exception e) { >>> >>> >>>>>> e.printStackTrace(); >>> >>> >>>>>> } >>> >>> >>>>>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, >>> >>> >>>>>> "text/plain"); >>> >>> >>>>>> exchange.getResponseSender().send("SUCCESS"); >>> >>> >>>>>> >>> >>> >>>>>> >>> >>> >>>>>> --Senthil >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> _______________________________________________ >>> >>> >>>>> undertow-dev mailing list >>> >>> >>>>> undertow-dev(a)lists.jboss.org >>> >>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>> >>> >>>> >>> >>> >>>> >>> >>> >>> >>> >>> >> >>> >>> > >>> >>> > >>> >>> > _______________________________________________ >>> >>> > undertow-dev mailing list >>> >>> > undertow-dev(a)lists.jboss.org >>> >>> > https://lists.jboss.org/mailman/listinfo/undertow-dev >>> >> >>> >> >>> > >>> _______________________________________________ >>> undertow-dev mailing list >>> undertow-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>> >> >> > _______________________________________________ undertow-dev mailing list undertow-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/undertow-dev