[undertow-dev] Undertow Http Server - Handling 2 Millions Requests Per Second Per Instance

[SenthilKumar K]

Thanks Bill ..

Yes Kafka is Non Blocking ( We use Async feature of Kafka Producer ).


I've changed code as below i.e Wrapped BlockingHandler .. Is this Correct
Implementation ?

public class HelloWorldServer {

public static void main(final String[] args) {
Undertow server = Undertow.builder().addHttpListener(8009,
"localhost").setHandler( new BlockingHandler( new HttpHandler() {
@Override
public void handleRequest(final HttpServerExchange exchange) throws
Exception {
if (exchange.getRequestMethod().equals(Methods.POST)) {
BufferedReader reader = null;
StringBuilder builder = new StringBuilder();
try {
exchange.startBlocking();
reader = new BufferedReader(new
InputStreamReader(exchange.getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
String body = builder.toString();
System.out.println("Req Body ==> " + body);
exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
exchange.getResponseSender().send("SUCCESS");
} else {
exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, "text/plain");
exchange.getResponseSender().send("FAILURE");
}
}
})).build();
server.start();
}
}


Without using Kafka API here , i still see only ~50K per sec..

Running 1m test @ http://localhost:8009/
  100 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    20.48ms   11.98ms 328.07ms   91.59%
    Req/Sec   510.03    104.80     4.63k    91.33%
  Latency Distribution
     50%   18.23ms
     75%   20.48ms
     90%   24.35ms
     99%   69.33ms
  3014791 requests in 1.00m, 379.52MB read
Requests/sec:  *50163*.73
Transfer/sec:      6.31MB


--Senthil

On Fri, Jun 23, 2017 at 1:04 AM, Bill O'Neil <bill at dartalley.com> wrote:

> I'm not an expert on the high performance part but heres my thoughts.
>
>
> 1. Is the Kafka API non blocking? I would assume not. If it's not you are
> basically running on only IO threads (roughly 1 per CPU) and could be
> blocking incoming requests. Try wrapping your HttpHandler in the
> BlockingHandler which will give it more threads to work with the network IO
> and shouldn't block / refuse incoming connections.
> 2. The HttpServerExchange should be handling all of the buffers under the
> hood for you. I believe you are just doing extra work here. If you use a
> blocking handler you can simply call exchange.getInputStream() to get the
> request body. I'm not sure the best way to handle the read in a non
> blocking manner.
>
> On Thu, Jun 22, 2017 at 2:00 PM, SenthilKumar K <senthilec566 at gmail.com>
> wrote:
>
>> Hello Undertow Dev Team ,
>>
>>       I have been working on the use case where i should create simple
>> http server to serve 1.5 Million Requests per Second per Instance ..
>>
>>
>> Here is the benchmark result of Undertow :
>>
>> Running 1m test @ http://127.0.0.1:8009/
>>   20 threads and 40 connections
>>   Thread Stats   Avg      Stdev     Max   +/- Stdev
>>     Latency     2.51ms   10.75ms 282.22ms   99.28%
>>     Req/Sec     1.12k   316.65     1.96k    54.50%
>>   Latency Distribution
>>      50%    1.43ms
>>      75%    2.38ms
>>      90%    2.90ms
>>      99%   10.45ms
>>   1328133 requests in 1.00m, 167.19MB read
>> Requests/sec:  *22127*.92
>> Transfer/sec:      2.79MB
>>
>> This is less compared to other frameworks like Jetty and Netty .. But
>> originally Undertow is high performant http server ..
>>
>> Hardware details:
>> Xeon CPU E3-1270 v5 machine with 4 cores ( Clock 100 MHz, Capacity 4 GHz)
>> , Memory : 32 G , Available memory 31 G.
>>
>> I would need Undertow experts to review the server code below and advice
>> me on tuning to achieve my goal( ~1.5 Million requests/sec ).
>>
>> Server :
>>
>> Undertow server = Undertow.builder()
>>                .addHttpListener(8009, "localhost")
>>                .setHandler(new Handler()).build();
>> server.start();
>>
>>
>> Handler.Java
>>
>>     final Pooled<ByteBuffer> pooledByteBuffer =
>>                  exchange.getConnection().getBufferPool().allocate();
>> final ByteBuffer byteBuffer = pooledByteBuffer.getResource();
>>    byteBuffer.clear();
>>    exchange.getRequestChannel().read(byteBuffer);
>>    int pos = byteBuffer.position();
>>    byteBuffer.rewind();
>>    byte[] bytes = new byte[pos];
>>    byteBuffer.get(bytes);
>>    String requestBody = new String(bytes, Charset.forName("UTF-8") );
>>    byteBuffer.clear();
>>    pooledByteBuffer.free();
>>    final PostToKafka post2Kafka = new PostToKafka();
>> try {
>> *post2Kafka.write2Kafka(requestBody);  { This API can handle  ~2 Millions
>> events per sec }*
>> } catch (Exception e) {
>> e.printStackTrace();
>> }
>>     exchange.getResponseHeaders().put(Headers.CONTENT_TYPE,
>> "text/plain");
>>     exchange.getResponseSender().send("SUCCESS");
>>
>>
>> --Senthil
>>
>> _______________________________________________
>> undertow-dev mailing list
>> undertow-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/undertow-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20170623/13f4f1ab/attachment-0001.html