[undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance
SenthilKumar K
senthilec566 at gmail.com
Sat Jul 8 10:56:57 EDT 2017
Any comments on *Undertow Vs Netty* ? Am i doing wrong benchmark testing
?? Should i change benchmark strategy ?
--Senthil
On Fri, Jul 7, 2017 at 3:14 PM, SenthilKumar K <senthilec566 at gmail.com>
wrote:
> Sorry for delay in responding to this thread!
>
> Thanks to everyone who helped me to Optimize Undertow Server.
>
> Here is the comparison after benchmarking my use case against Netty:
>
> *Undertow Vs Netty :*
>
> Test Case 1 :
> Simple Request Response ( No Kafka ):
>
> *Undertow:*
> Running 10m test @ http://198.18.134.13:8009/
> 500 threads and 5000 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency *3.52m * 2.64m 8.96m 54.63%
> Req/Sec 376.58 103.18 0.99k 80.53%
> 111628942 requests in 10.00m, 13.72GB read
> Socket errors: connect 0, read 28, write 0, timeout 2
> Requests/sec: *186122.56*
> Transfer/sec: 23.43MB
>
> *Netty:*
> Running 10m test @ http://198.18.134.13:8009/
> 500 threads and 5000 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency *3.77m* 2.10m 7.51m 57.73%
> Req/Sec 518.63 31.78 652.00 70.25%
> 155406992 requests in 10.00m, 13.82GB read
> Socket errors: connect 0, read 49, write 0, timeout 0
> Requests/sec: *259107*.30
> Transfer/sec: 24.17MB
>
>
> *Test Case 2:*
> Request --> Read --> Send it Kafka :
>
> *Undertow:*
> Running 10m test @ http://198.18.134.13:8009/
> 500 threads and 5000 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency *4.37m * 2.46m 8.72m 57.83%
> Req/Sec 267.32 5.17 287.00 74.52%
> 80044045 requests in 10.00m, 9.84GB read
> Socket errors: connect 0, read 121, write 0, timeout 0
> Requests/sec: *133459.79*
> Transfer/sec: 16.80MB
>
> *Netty:*
> Running 10m test @ http://198.18.134.13:8009/
> 500 threads and 5000 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency *3.78m * 2.10m 7.55m 57.79%
> Req/Sec 516.92 28.84 642.00 69.60%
> 154770536 requests in 10.00m, 13.69GB read
> Socket errors: connect 0, read 11, write 0, timeout 101
> Requests/sec: *258049.39*
> Transfer/sec: 23.38MB
>
>
>
> CPU Usage:
> *Undertow:*
> [image: Inline image 1]
>
> *Netty:*
> [image: Inline image 2]
>
>
> --Senthil
>
> On Thu, Jun 29, 2017 at 7:34 AM, Bill O'Neil <bill at dartalley.com> wrote:
>
>> 1. Can you run the benchmark with the kafka line commented out at first
>> and then again with it not commented out?
>> 2. What rates were you getting with Jetty and Netty?
>> 3. Are you running the tests from the same machine or a different one? If
>> its the same machine and its using 20 threads they will be contending with
>> undertows IO threads.
>> 4. You can probably ignore the POST check if thats all your going to
>> accept and its not a public api.
>>
>> import io.undertow.server.HttpHandler;
>> import io.undertow.server.HttpServerExchange;
>> import io.undertow.util.Headers;
>> import io.undertow.util.Methods;
>>
>> public class DLRHandler implements HttpHandler {
>>
>> final public static String _SUCCESS="SUCCESS";
>> final public static String _FAILURE="FAILURE";
>> final PostToKafka post2Kafka = new PostToKafka();
>>
>> @Override
>> public void handleRequest( final HttpServerExchange exchange) throws
>> Exception {
>> if (exchange.getRequestMethod().equals(Methods.POST)) {
>> exchange.getRequestReceiver().receiveFullString((
>> exchangeReq, data) -> {
>> //post2Kafka.write2Kafka(data); // write it to Kafka
>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE,
>> "text/plain");
>> exchangeReq.getResponseSender().send(_SUCCESS);
>> },
>> (exchangeReq, exception) -> {
>> exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE,
>> "text/plain");
>> exchangeReq.getResponseSender().send(_FAILURE);
>> });
>> }else{
>> throw new Exception("Method GET not supported by Server ");
>> }
>> }
>> }
>>
>> On Wed, Jun 28, 2017 at 6:59 PM, Stuart Douglas <sdouglas at redhat.com>
>> wrote:
>>
>>> The multiple dispatches() are unnecessary (well the second one to the
>>> IO thread is definitely unnecessary, the first one is only required if
>>> post2Kafka.write2Kafka(data); is a blocking operation and needs to be
>>> executed in a worker thread).
>>>
>>> Stuart
>>>
>>> On Wed, Jun 28, 2017 at 5:42 PM, SenthilKumar K <senthilec566 at gmail.com>
>>> wrote:
>>> > After modifying the code below i could see the improvement ( not much
>>> > slightly ) in server - 65k req/sec.
>>> >
>>> > import io.undertow.server.HttpHandler;
>>> > import io.undertow.server.HttpServerExchange;
>>> > import io.undertow.util.Headers;
>>> > import io.undertow.util.Methods;
>>> >
>>> > public class DLRHandler implements HttpHandler {
>>> >
>>> > final public static String _SUCCESS="SUCCESS";
>>> > final public static String _FAILURE="FAILURE";
>>> > final PostToKafka post2Kafka = new PostToKafka();
>>> >
>>> > @Override
>>> > public void handleRequest( final HttpServerExchange exchange)
>>> throws
>>> > Exception {
>>> > if (exchange.getRequestMethod().equals(Methods.POST)) {
>>> > exchange.getRequestReceiver().receiveFullString((
>>> > exchangeReq, data) -> {
>>> > exchangeReq.dispatch(() -> {
>>> > post2Kafka.write2Kafka(data); // write it to
>>> Kafka
>>> > exchangeReq.dispatch(exchangeReq.getIoThread(),
>>> () ->
>>> > {
>>> >
>>> > exchangeReq.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>> "text/plain");
>>> > exchangeReq.getResponseSender
>>> ().send(_SUCCESS);
>>> > });
>>> > });
>>> > },
>>> > (exchangeReq, exception) -> {
>>> > exchangeReq.getResponseHeaders
>>> ().put(Headers.CONTENT_TYPE,
>>> > "text/plain");
>>> > exchangeReq.getResponseSender().send(_FAILURE);
>>> > });
>>> > }else{
>>> > throw new Exception("Method GET not supported by Server
>>> ");
>>> > }
>>> > }
>>> > }
>>> >
>>> >
>>> > Pls review this and let me know if i'm doing anything wrong here ...
>>> > --Senthil
>>> >
>>> > On Fri, Jun 23, 2017 at 1:30 PM, Antoine Girard <
>>> antoine.girard at ymail.com>
>>> > wrote:
>>> >>
>>> >> Also, to come back on the JVM warmup, this will give you enough
>>> answers:
>>> >>
>>> >> https://stackoverflow.com/questions/36198278/why-does-the-jv
>>> m-require-warmup
>>> >>
>>> >> For your, it means that you have to run your tests for a few minutes
>>> >> before starting your actual measurements.
>>> >>
>>> >> I am also interested about how Netty / Jetty perform under the same
>>> >> conditions, please post!
>>> >>
>>> >> Cheers,
>>> >> Antoine
>>> >>
>>> >> On Fri, Jun 23, 2017 at 1:24 AM, Stuart Douglas <sdouglas at redhat.com>
>>> >> wrote:
>>> >>>
>>> >>> Are you actually testing with the 'System.out.println(" Received
>>> >>> String ==> "+message);'. System.out is incredibly slow.
>>> >>>
>>> >>> Stuart
>>> >>>
>>> >>> On Fri, Jun 23, 2017 at 7:01 AM, SenthilKumar K <
>>> senthilec566 at gmail.com>
>>> >>> wrote:
>>> >>> > Sorry , I'm not an expert in JVM .. How do we do Warm Up JVM ?
>>> >>> >
>>> >>> > Here is the JVM args to Server:
>>> >>> >
>>> >>> > nohup java -Xmx4g -Xms4g -XX:MetaspaceSize=96m -XX:+UseG1GC
>>> >>> > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35
>>> >>> > -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50
>>> >>> > -XX:MaxMetaspaceFreeRatio=80 -cp undertow-0.0.1.jar
>>> HelloWorldServer
>>> >>> >
>>> >>> >
>>> >>> > --Senthil
>>> >>> >
>>> >>> >
>>> >>> > On Fri, Jun 23, 2017 at 2:23 AM, Antoine Girard
>>> >>> > <antoine.girard at ymail.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >> Do you warm up your jvm prior to the testing?
>>> >>> >>
>>> >>> >> Cheers,
>>> >>> >> Antoine
>>> >>> >>
>>> >>> >> On Thu, Jun 22, 2017 at 10:42 PM, SenthilKumar K
>>> >>> >> <senthilec566 at gmail.com>
>>> >>> >> wrote:
>>> >>> >>>
>>> >>> >>> Thanks Bill n Antoine ..
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> Here is the updated one : ( tried without Kafka API ) .
>>> >>> >>>
>>> >>> >>> public class HelloWorldServer {
>>> >>> >>>
>>> >>> >>> public static void main(final String[] args) {
>>> >>> >>> Undertow server = Undertow.builder().addHttpListener(8009,
>>> >>> >>> "localhost").setHandler(new HttpHandler() {
>>> >>> >>> @Override
>>> >>> >>> public void handleRequest(final HttpServerExchange exchange)
>>> throws
>>> >>> >>> Exception {
>>> >>> >>> if (exchange.getRequestMethod().equals(Methods.POST)) {
>>> >>> >>> exchange.getRequestReceiver().receiveFullString(new
>>> >>> >>> Receiver.FullStringCallback() {
>>> >>> >>> @Override
>>> >>> >>> public void handle(HttpServerExchange
>>> exchange,
>>> >>> >>> String
>>> >>> >>> message) {
>>> >>> >>> System.out.println(" Received String ==>
>>> >>> >>> "+message);
>>> >>> >>> exchange.getResponseSender().s
>>> end(message);
>>> >>> >>> }
>>> >>> >>> });
>>> >>> >>> } else {
>>> >>> >>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>> >>> >>> "text/plain");
>>> >>> >>> exchange.getResponseSender().send("FAILURE");
>>> >>> >>> }
>>> >>> >>> }
>>> >>> >>> }).build();
>>> >>> >>> server.start();
>>> >>> >>> }
>>> >>> >>> }
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> Oops seems to no improvement :
>>> >>> >>>
>>> >>> >>> Running 1m test @ http://localhost:8009/
>>> >>> >>> 100 threads and 1000 connections
>>> >>> >>> Thread Stats Avg Stdev Max +/- Stdev
>>> >>> >>> Latency 25.79ms 22.18ms 289.48ms 67.66%
>>> >>> >>> Req/Sec 437.76 61.71 2.30k 80.26%
>>> >>> >>> Latency Distribution
>>> >>> >>> 50% 22.60ms
>>> >>> >>> 75% 37.83ms
>>> >>> >>> 90% 55.32ms
>>> >>> >>> 99% 90.47ms
>>> >>> >>> 2625607 requests in 1.00m, 2.76GB read
>>> >>> >>> Requests/sec: 43688.42
>>> >>> >>> Transfer/sec: 47.08MB
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> :-( :-( ..
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> --Senthil
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> On Fri, Jun 23, 2017 at 1:47 AM, Antoine Girard
>>> >>> >>> <antoine.girard at ymail.com> wrote:
>>> >>> >>>>
>>> >>> >>>> You can use the Receiver API, specifically for that purpose.
>>> >>> >>>> On the exchange, call: getRequestReceiver();
>>> >>> >>>>
>>> >>> >>>> You will get a receiver object:
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> https://github.com/undertow-io/undertow/blob/master/core/src
>>> /main/java/io/undertow/io/Receiver.java
>>> >>> >>>>
>>> >>> >>>> On the receiver you can call: receiveFullString, you have to
>>> pass it
>>> >>> >>>> a
>>> >>> >>>> callback that will be called when the whole body has been read.
>>> >>> >>>>
>>> >>> >>>> Please share your results when you test this further!
>>> >>> >>>>
>>> >>> >>>> Cheers,
>>> >>> >>>> Antoine
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> On Thu, Jun 22, 2017 at 8:27 PM, SenthilKumar K
>>> >>> >>>> <senthilec566 at gmail.com>
>>> >>> >>>> wrote:
>>> >>> >>>>>
>>> >>> >>>>> Seems to Reading Request body is wrong , So what is the
>>> efficient
>>> >>> >>>>> way
>>> >>> >>>>> of reading request body in undertow ?
>>> >>> >>>>>
>>> >>> >>>>> --Senthil
>>> >>> >>>>>
>>> >>> >>>>> On Thu, Jun 22, 2017 at 11:30 PM, SenthilKumar K
>>> >>> >>>>> <senthilec566 at gmail.com> wrote:
>>> >>> >>>>>>
>>> >>> >>>>>> Hello Undertow Dev Team ,
>>> >>> >>>>>>
>>> >>> >>>>>> I have been working on the use case where i should
>>> create
>>> >>> >>>>>> simple
>>> >>> >>>>>> http server to serve 1.5 Million Requests per Second per
>>> Instance
>>> >>> >>>>>> ..
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> Here is the benchmark result of Undertow :
>>> >>> >>>>>>
>>> >>> >>>>>> Running 1m test @ http://127.0.0.1:8009/
>>> >>> >>>>>> 20 threads and 40 connections
>>> >>> >>>>>> Thread Stats Avg Stdev Max +/- Stdev
>>> >>> >>>>>> Latency 2.51ms 10.75ms 282.22ms 99.28%
>>> >>> >>>>>> Req/Sec 1.12k 316.65 1.96k 54.50%
>>> >>> >>>>>> Latency Distribution
>>> >>> >>>>>> 50% 1.43ms
>>> >>> >>>>>> 75% 2.38ms
>>> >>> >>>>>> 90% 2.90ms
>>> >>> >>>>>> 99% 10.45ms
>>> >>> >>>>>> 1328133 requests in 1.00m, 167.19MB read
>>> >>> >>>>>> Requests/sec: 22127.92
>>> >>> >>>>>> Transfer/sec: 2.79MB
>>> >>> >>>>>>
>>> >>> >>>>>> This is less compared to other frameworks like Jetty and
>>> Netty ..
>>> >>> >>>>>> But
>>> >>> >>>>>> originally Undertow is high performant http server ..
>>> >>> >>>>>>
>>> >>> >>>>>> Hardware details:
>>> >>> >>>>>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz,
>>> Capacity
>>> >>> >>>>>> 4
>>> >>> >>>>>> GHz) , Memory : 32 G , Available memory 31 G.
>>> >>> >>>>>>
>>> >>> >>>>>> I would need Undertow experts to review the server code below
>>> and
>>> >>> >>>>>> advice me on tuning to achieve my goal( ~1.5 Million
>>> requests/sec
>>> >>> >>>>>> ).
>>> >>> >>>>>>
>>> >>> >>>>>> Server :
>>> >>> >>>>>>
>>> >>> >>>>>> Undertow server = Undertow.builder()
>>> >>> >>>>>> .addHttpListener(8009, "localhost")
>>> >>> >>>>>> .setHandler(new Handler()).build();
>>> >>> >>>>>> server.start();
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> Handler.Java
>>> >>> >>>>>>
>>> >>> >>>>>> final Pooled<ByteBuffer> pooledByteBuffer =
>>> >>> >>>>>>
>>> >>> >>>>>> exchange.getConnection().getBufferPool().allocate();
>>> >>> >>>>>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource();
>>> >>> >>>>>> byteBuffer.clear();
>>> >>> >>>>>> exchange.getRequestChannel().read(byteBuffer);
>>> >>> >>>>>> int pos = byteBuffer.position();
>>> >>> >>>>>> byteBuffer.rewind();
>>> >>> >>>>>> byte[] bytes = new byte[pos];
>>> >>> >>>>>> byteBuffer.get(bytes);
>>> >>> >>>>>> String requestBody = new String(bytes,
>>> Charset.forName("UTF-8")
>>> >>> >>>>>> );
>>> >>> >>>>>> byteBuffer.clear();
>>> >>> >>>>>> pooledByteBuffer.free();
>>> >>> >>>>>> final PostToKafka post2Kafka = new PostToKafka();
>>> >>> >>>>>> try {
>>> >>> >>>>>> post2Kafka.write2Kafka(requestBody); { This API can handle
>>> ~2
>>> >>> >>>>>> Millions events per sec }
>>> >>> >>>>>> } catch (Exception e) {
>>> >>> >>>>>> e.printStackTrace();
>>> >>> >>>>>> }
>>> >>> >>>>>> exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
>>> >>> >>>>>> "text/plain");
>>> >>> >>>>>> exchange.getResponseSender().send("SUCCESS");
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> --Senthil
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> _______________________________________________
>>> >>> >>>>> undertow-dev mailing list
>>> >>> >>>>> undertow-dev at lists.jboss.org
>>> >>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>
>>> >>> >>
>>> >>> >
>>> >>> >
>>> >>> > _______________________________________________
>>> >>> > undertow-dev mailing list
>>> >>> > undertow-dev at lists.jboss.org
>>> >>> > https://lists.jboss.org/mailman/listinfo/undertow-dev
>>> >>
>>> >>
>>> >
>>> _______________________________________________
>>> undertow-dev mailing list
>>> undertow-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/undertow-dev
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20170708/27f7d0e3/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 31196 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/undertow-dev/attachments/20170708/27f7d0e3/attachment-0002.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 25115 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/undertow-dev/attachments/20170708/27f7d0e3/attachment-0003.png
More information about the undertow-dev
mailing list