Re: [undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance

After modifying the code below i could see the improvement ( not much slightly ) in server - 65k req/sec. import io.undertow.server.HttpHandler; import io.undertow.server.HttpServerExchange; import io.undertow.util.Headers; import io.undertow.util.Methods; public class DLRHandler implements HttpHandler { final public static String _SUCCESS="SUCCESS"; final public static String _FAILURE="FAILURE"; final PostToKafka post2Kafka = new PostToKafka(); @Override public void handleRequest( final HttpServerExchange exchange) throws Exception { if (exchange.getRequestMethod().equals(Methods.POST)) { exchange.getRequestReceiver().receiveFullString(( exchangeReq, data) -> { exchangeReq.dispatch(() -> { post2Kafka.write2Kafka(data); // write it to Kafka exchangeReq.dispatch(exchangeReq.getIoThread(), () -> { exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain"); exchangeReq.getResponseSender().send(_SUCCESS); }); }); }, (exchangeReq, exception) -> { exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain"); exchangeReq.getResponseSender().send(_FAILURE); }); }else{ throw new Exception("Method GET not supported by Server "); } } } Pls review this and let me know if i'm doing anything wrong here ... --Senthil On Fri, Jun 23, 2017 at 1:30 PM, Antoine Girard <antoine.girard(a)ymail.com&gt; wrote: > > Also, to come back on the JVM warmup, this will give you enough answers: > > https://stackoverflow.com/questions/36198278/why-does-the-jvm-require-warmup > > For your, it means that you have to run your tests for a few minutes > before starting your actual measurements. > > I am also interested about how Netty / Jetty perform under the same > conditions, please post! > > Cheers, > Antoine > > On Fri, Jun 23, 2017 at 1:24 AM, Stuart Douglas <sdouglas(a)redhat.com&gt; > wrote: >> >> Are you actually testing with the 'System.out.println(" Received >> String ==> "+message);'. System.out is incredibly slow. >> >> Stuart >> >> On Fri, Jun 23, 2017 at 7:01 AM, SenthilKumar K <senthilec566(a)gmail.com&gt; >> wrote: >> > Sorry , I'm not an expert in JVM .. How do we do Warm Up JVM ? >> > >> > Here is the JVM args to Server: >> > >> > nohup java -Xmx4g -Xms4g -XX:MetaspaceSize=96m -XX:+UseG1GC >> > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 >> > -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50 >> > -XX:MaxMetaspaceFreeRatio=80 -cp undertow-0.0.1.jar HelloWorldServer >> > >> > >> > --Senthil >> > >> > >> > On Fri, Jun 23, 2017 at 2:23 AM, Antoine Girard >> > <antoine.girard(a)ymail.com&gt; >> > wrote: >> >> >> >> Do you warm up your jvm prior to the testing? >> >> >> >> Cheers, >> >> Antoine >> >> >> >> On Thu, Jun 22, 2017 at 10:42 PM, SenthilKumar K >> >> <senthilec566(a)gmail.com&gt; >> >> wrote: >> >>> >> >>> Thanks Bill n Antoine .. >> >>> >> >>> >> >>> Here is the updated one : ( tried without Kafka API ) . >> >>> >> >>> public class HelloWorldServer { >> >>> >> >>> public static void main(final String[] args) { >> >>> Undertow server = Undertow.builder().addHttpListener(8009, >> >>> "localhost").setHandler(new HttpHandler() { >> >>> @Override >> >>> public void handleRequest(final HttpServerExchange exchange) throws >> >>> Exception { >> >>> if (exchange.getRequestMethod().equals(Methods.POST)) { >> >>> exchange.getRequestReceiver().receiveFullString(new >> >>> Receiver.FullStringCallback() { >> >>> @Override >> >>> public void handle(HttpServerExchange exchange, >> >>> String >> >>> message) { >> >>> System.out.println(" Received String ==> >> >>> "+message); >> >>> exchange.getResponseSender().send(message); >> >>> } >> >>> }); >> >>> } else { >> >>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, >> >>> "text/plain"); >> >>> exchange.getResponseSender().send("FAILURE"); >> >>> } >> >>> } >> >>> }).build(); >> >>> server.start(); >> >>> } >> >>> } >> >>> >> >>> >> >>> Oops seems to no improvement : >> >>> >> >>> Running 1m test @ http://localhost:8009/ >> >>> 100 threads and 1000 connections >> >>> Thread Stats Avg Stdev Max +/- Stdev >> >>> Latency 25.79ms 22.18ms 289.48ms 67.66% >> >>> Req/Sec 437.76 61.71 2.30k 80.26% >> >>> Latency Distribution >> >>> 50% 22.60ms >> >>> 75% 37.83ms >> >>> 90% 55.32ms >> >>> 99% 90.47ms >> >>> 2625607 requests in 1.00m, 2.76GB read >> >>> Requests/sec: 43688.42 >> >>> Transfer/sec: 47.08MB >> >>> >> >>> >> >>> :-( :-( .. >> >>> >> >>> >> >>> --Senthil >> >>> >> >>> >> >>> On Fri, Jun 23, 2017 at 1:47 AM, Antoine Girard >> >>> <antoine.girard(a)ymail.com&gt; wrote: >> >>>> >> >>>> You can use the Receiver API, specifically for that purpose. >> >>>> On the exchange, call: getRequestReceiver(); >> >>>> >> >>>> You will get a receiver object: >> >>>> >> >>>> >> >>>> https://github.com/undertow-io/undertow/blob/master/core/src/main/java/io... >> >>>> >> >>>> On the receiver you can call: receiveFullString, you have to pass it >> >>>> a >> >>>> callback that will be called when the whole body has been read. >> >>>> >> >>>> Please share your results when you test this further! >> >>>> >> >>>> Cheers, >> >>>> Antoine >> >>>> >> >>>> >> >>>> On Thu, Jun 22, 2017 at 8:27 PM, SenthilKumar K >> >>>> <senthilec566(a)gmail.com&gt; >> >>>> wrote: >> >>>>> >> >>>>> Seems to Reading Request body is wrong , So what is the efficient >> >>>>> way >> >>>>> of reading request body in undertow ? >> >>>>> >> >>>>> --Senthil >> >>>>> >> >>>>> On Thu, Jun 22, 2017 at 11:30 PM, SenthilKumar K >> >>>>> <senthilec566(a)gmail.com&gt; wrote: >> >>>>>> >> >>>>>> Hello Undertow Dev Team , >> >>>>>> >> >>>>>> I have been working on the use case where i should create >> >>>>>> simple >> >>>>>> http server to serve 1.5 Million Requests per Second per Instance >> >>>>>> .. >> >>>>>> >> >>>>>> >> >>>>>> Here is the benchmark result of Undertow : >> >>>>>> >> >>>>>> Running 1m test @ http://127.0.0.1:8009/ >> >>>>>> 20 threads and 40 connections >> >>>>>> Thread Stats Avg Stdev Max +/- Stdev >> >>>>>> Latency 2.51ms 10.75ms 282.22ms 99.28% >> >>>>>> Req/Sec 1.12k 316.65 1.96k 54.50% >> >>>>>> Latency Distribution >> >>>>>> 50% 1.43ms >> >>>>>> 75% 2.38ms >> >>>>>> 90% 2.90ms >> >>>>>> 99% 10.45ms >> >>>>>> 1328133 requests in 1.00m, 167.19MB read >> >>>>>> Requests/sec: 22127.92 >> >>>>>> Transfer/sec: 2.79MB >> >>>>>> >> >>>>>> This is less compared to other frameworks like Jetty and Netty .. >> >>>>>> But >> >>>>>> originally Undertow is high performant http server .. >> >>>>>> >> >>>>>> Hardware details: >> >>>>>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz, Capacity >> >>>>>> 4 >> >>>>>> GHz) , Memory : 32 G , Available memory 31 G. >> >>>>>> >> >>>>>> I would need Undertow experts to review the server code below and >> >>>>>> advice me on tuning to achieve my goal( ~1.5 Million requests/sec >> >>>>>> ). >> >>>>>> >> >>>>>> Server : >> >>>>>> >> >>>>>> Undertow server = Undertow.builder() >> >>>>>> .addHttpListener(8009, "localhost") >> >>>>>> .setHandler(new Handler()).build(); >> >>>>>> server.start(); >> >>>>>> >> >>>>>> >> >>>>>> Handler.Java >> >>>>>> >> >>>>>> final Pooled<ByteBuffer> pooledByteBuffer = >> >>>>>> >> >>>>>> exchange.getConnection().getBufferPool().allocate(); >> >>>>>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource(); >> >>>>>> byteBuffer.clear(); >> >>>>>> exchange.getRequestChannel().read(byteBuffer); >> >>>>>> int pos = byteBuffer.position(); >> >>>>>> byteBuffer.rewind(); >> >>>>>> byte[] bytes = new byte[pos]; >> >>>>>> byteBuffer.get(bytes); >> >>>>>> String requestBody = new String(bytes, Charset.forName("UTF-8") >> >>>>>> ); >> >>>>>> byteBuffer.clear(); >> >>>>>> pooledByteBuffer.free(); >> >>>>>> final PostToKafka post2Kafka = new PostToKafka(); >> >>>>>> try { >> >>>>>> post2Kafka.write2Kafka(requestBody); { This API can handle ~2 >> >>>>>> Millions events per sec } >> >>>>>> } catch (Exception e) { >> >>>>>> e.printStackTrace(); >> >>>>>> } >> >>>>>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, >> >>>>>> "text/plain"); >> >>>>>> exchange.getResponseSender().send("SUCCESS"); >> >>>>>> >> >>>>>> >> >>>>>> --Senthil >> >>>>> >> >>>>> >> >>>>> >> >>>>> _______________________________________________ >> >>>>> undertow-dev mailing list >> >>>>> undertow-dev(a)lists.jboss.org >> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >> >>>> >> >>>> >> >>> >> >> >> > >> > >> > _______________________________________________ >> > undertow-dev mailing list >> > undertow-dev(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/undertow-dev > >