Silly http benchmark

[Luis Neves]

Hello,

On 01/29/2010 03:39 AM, "Trustin Lee (이희승)" wrote:
> I've just looked into the source code and found a little difference
> between Netty server and others - ExecutionHandler.  It's unfair to
> Grizzly and MINA because you inserted an ExecutorFilter for MINA and
> Grizzly has an event thread pool by default.  If you don't add
> ExecutionHandler to the pipeline, the numbers will look significantly
> better than others.

I followed your suggestion and removed the ExecutorFilter from Mina,
but the results didn't improve, see below.


> I guess nginx doesn't have an event thread pool, but that's another
> story.  Perhaps you could remove the ExecutionHandler / ExecutorFilter
> from the pipeline / filterChain to compare with nginx.  (I don't know
> how to remove the event thread pool in Grizzly unfortunately.)

I also don't have any idea of how to configure the "thread pool
behavior" of grizzly.

> Therefore, I suggest:
>
> 1) Upgrade to 3.2.0.ALPHA4-SNAPSHOT
> 2) Add ExecutionHandler to Netty pipeline to be fair to MINA and Grizzly.
> 3) Remove ExecutionHandler / ExecutorFilter from Netty / MINA and
> compare with each other and nginx.


The latest netty snapshot gives a little boost in performance.
These are the average results across 5 runs.

mina (trunk) - 24K req/sec
grizzly (1.9.18-i)- 56K  req/sec
netyy (3.2-ALPHA4-SNAPSHOT) - 83K req/sec
nginx (0.8.32) - 119K req/seq

I've made a spreadsheet with a simple statistical analysis of the results:
<http://dl.dropbox.com/u/279023/netty/bench_test.ods>


I forgot to mention earlier the JVM version. I'm using the latest sun jdk.

$ java -version
java version "1.6.0_18"
Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode)


Oh ... and of course that I don't mind if you add this performance
test to the project web site.

Regards.

--
Luis Neves