Netty/HTTP dropping connections with httpperf

[Sébastien Pierre]

Hi there !

I just moved a test HTTP service from HttpCore/NIO to Netty based on the
HTTP snoop example. I experienced (despite good performance at first) that
the server is dropping connections when the load is heavy. To reproduce
this, simply start the snoop example:

java -cp ./src/main/java/:./jar/netty-3.2.0.ALPHA2.jar
org.jboss.netty.example.http.snoop.HttpServer

and then do

python -c'for r in range(500,10500,500): import os ; os.system("httperf
--hog --timeout=60 --client=0/1 --server=127.0.0.1 --port=8080 --uri=/
--rate=%s --send-buffer=4096 --recv-buffer=16384 --num-conns=10000
--num-calls=1" % (r))'

which is the same as running this command with --rate growing from 500 to
10000 with 500 increases.

httperf --hog --timeout=60 --client=0/1 --server=127.0.0.1 --port=8080
--uri=/ --rate=500 --send-buffer=4096 --recv-buffer=16384 --num-conns=10000
--num-calls=1

so in this example, Netty often drops connections between rates of 3000 and
5000, which you'll see in httpperf's log with the following lines:

httperf: connection failed with unexpected error 98
...
Errors: total 2236 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 2236

I attached a graph where you'll see that for the rate=4000 and rate=45000
the server just dropped all the connections (from httperf's perspective at
least). So do you know any way to prevent or at least detect that ?

Thanks !

 -- Sébastien

PS: I should add that I've run the test a couple of times, and quite often
have trouble in the rate=3000-5000 range
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/netty-users/attachments/20100106/dae233b5/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: netty-scalability.png
Type: image/png
Size: 30070 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20100106/dae233b5/attachment-0001.png