Silly http benchmark

"Trustin Lee (이희승)" trustin at gmail.com
Thu Jan 28 22:39:13 EST 2010


I've just looked into the source code and found a little difference
between Netty server and others - ExecutionHandler.  It's unfair to
Grizzly and MINA because you inserted an ExecutorFilter for MINA and
Grizzly has an event thread pool by default.  If you don't add
ExecutionHandler to the pipeline, the numbers will look significantly
better than others.

I guess nginx doesn't have an event thread pool, but that's another
story.  Perhaps you could remove the ExecutionHandler / ExecutorFilter
from the pipeline / filterChain to compare with nginx.  (I don't know
how to remove the event thread pool in Grizzly unfortunately.)

Therefore, I suggest:

1) Upgrade to 3.2.0.ALPHA4-SNAPSHOT
2) Add ExecutionHandler to Netty pipeline to be fair to MINA and Grizzly.
3) Remove ExecutionHandler / ExecutorFilter from Netty / MINA and
compare with each other and nginx.

HTH,
Trustin

Trustin Lee (이희승) wrote:
> Hi Luis,
> 
> First off, thank you very much for spending significant amount of time
> for running performance tests.  This is a great news for the community
> absolutely. :)
> 
> I'd like to add your performance report to the project web site if you
> don't mind:
> 
>     http://jboss.org/netty/performance.html
> 
> I think it's fine to publish it as long as it is easily reproduceable as
> written in your instruction and each test did not fail.  Perhaps you
> could send me the full test log files then I could upload it together.
> 
> Also, could you try to run the test with the latest build and publish
> the result again?  I have done significant performance optimization
> which might yield better throughput in some situation.  Get the latest
> build here:
> 
>     http://hudson.jboss.org/hudson/view/Netty/job/netty/2531/
> 
> Cheers,
> Trustin
> 
> Luis Neves wrote:
>> Thought I would share the results of a silly http benchmark of netty
>> against the competition.
>> To keep thinks interesting I added the nginx http server to the mix to
>> get an idea of how Java server compares to a highly optimized C
>> server.
>>
>> Nginx has a module called  "empty_gif" that serves a 1x1 transparent
>> gif directly from memory which is perfect for this purpose.
>> I used the exact same gif for all the Java servers.
>>
>> Both the client and the server machines are quad cores running Linux
>> 2.6.31 that are linked via a 1GB switch.
>>
>> I used ab as the testing tool with 4 parallel processes. The testing
>> script is as follows:
>>
>> *******************************************************************
>>
>> #!/bin/bash
>>
>> set -e
>> cd $(dirname $0)
>> rm -fv *.log
>>
>> for i in $(seq 1 4); do
>> ab -c500 -n1000000 -k -S -r -d http://<host>/bench > $i.log &
>> done
>>
>> wait
>>
>> cat *.log | grep Requests | awk '{ sum+= $4 } END {printf "\n\n
>> Average requests per second: %8.2f\n\n",sum}'
>> *******************************************************************
>>
>>
>> As you can see it's not a very scientific method :-) ... anyway, each
>> of the ab processes creates 500 connections and makes 1000000
>> keep-alive requests.
>> The average results across several runs were:
>>
>> mina (trunk) - 25K req/sec
>> grizzly (1.9.18-i)- 55K  req/sec
>> netyy (3.2-ALPHA3) - 80K req/sec
>> nginx - 115K req/seq
>>
>>
>> Impressive results for Netty I think.
>> The code used is available here:
>> <http://dl.dropbox.com/u/279023/netty/http_test.tar.gz>
>>
>> If there is something wrong with it please let me know.
>>
>>
>> --
>> Luis Neves
>> _______________________________________________
>> netty-users mailing list
>> netty-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-users
> 

-- 
what we call human nature in actuality is human habit
http://gleamynode.net/


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20100129/dda88276/attachment.bin 


More information about the netty-users mailing list