<html><body bgcolor="#FFFFFF"><div>I'm just guessing here but would calling setReuseAddr on both client impl and servers netty impl help? I had similar issues where I ran out of sockets until I enabled reuse addr, especially for load testing short lived sockets. May be worth a quick try. Just make sure you enable the client side in the netty socket options. <br><br><div>Nicholas Hagen</div><div>Z|NET Development, LLC</div><div><div><a href="http://www.znetdevelopment.com">www.znetdevelopment.com</a></div><div>* sent from my iPhone *</div></div></div><div><br>On Jun 5, 2010, at 8:33 AM, Marc-André Laverdière <<a href="mailto:marcandre.laverdiere@gmail.com">marcandre.laverdiere@gmail.com</a>> wrote:<br><br></div><div></div><blockquote type="cite"><div>It seems like this is something in the framework... When putting serious stress on Netty, it breaks apart...<div>This sample code talks to the sample Echo Server</div><div><br></div><div><br></div><div><div>import java.io.DataInputStream;</div>
<div>import java.io.DataOutputStream;</div><div>import java.io.IOException;</div><div>import java.io.StringWriter;</div><div>import java.net.InetSocketAddress;</div><div>import java.net.Socket;</div><div>import java.util.concurrent.Callable;</div>
<div>import java.util.concurrent.Executors;</div><div>import java.util.concurrent.ScheduledExecutorService;</div><div>import java.util.concurrent.TimeUnit;</div><div><br></div><div>import javax.net.SocketFactory;</div><div>
<br></div><div>import lombok.Data;</div><div><br></div><div><br></div><div>public class Client {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>private final static int TIMEOUT_MS = 45*1000;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>private final static int NUM_THREADS = 750;</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>private final static int INTERVAL_MS = 1* 1000;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>@Data</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>static class Worker implements Runnable{</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>protected final InetSocketAddress server;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>@Override</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>public void run(){</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>try{</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>Socket sock = SocketFactory.getDefault().createSocket();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>sock.connect(server);</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>DataOutputStream dOut = new DataOutputStream(sock.getOutputStream());</div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>dOut.writeUTF("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa");</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>dOut.flush();</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>sock.setSoTimeout(TIMEOUT_MS);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>DataInputStream dIn = new DataInputStream(sock.getInputStream());</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>String got = dIn.readUTF();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>sock.close();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>} catch (IOException e){</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>e.printStackTrace();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}<span class="Apple-tab-span" style="white-space:pre"> </span></div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>public static void main(String[] args){</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ScheduledExecutorService service = Executors.newScheduledThreadPool(NUM_THREADS);</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>InetSocketAddress serverAddr = new InetSocketAddress("localhost", 8080);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>//Set the requesters</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>for (int i = 0; i < NUM_THREADS; i++){</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Worker W = new Worker(serverAddr);</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>service.scheduleAtFixedRate(W,INTERVAL_MS, INTERVAL_MS, TimeUnit.MILLISECONDS);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div>
}</div></div><div><br></div><div>After a little bit of time, the client starts timing out, and eventually, it throws java.net.NoRouteToHostException: Cannot assign requested address</div><div><br></div><div>This shouldn't be the case: the server uses a cached thread pool, the data goes on the loopback interface, the interval is every second, which is should be more than enough time to grab a thread from the pool, fetch the data from the interface, read it and write it back...</div>
<div><br></div><div>What's going on here?</div><div><br clear="all">Marc-André LAVERDIÈRE<br>"Perseverance must finish its work so that you may be mature and complete, not lacking anything." -James 1:4<br><a href="http://mlaverd.theunixplace.com/blog"><a href="http://mlaverd.theunixplace.com/blog">mlaverd.theunixplace.com/blog</a></a><br>
<br> /"\<br> \ / ASCII Ribbon Campaign<br> X against HTML e-mail<br> / \<br>
<br><br><div class="gmail_quote">2010/6/4 mlaverd <span dir="ltr"><<a href="mailto:marcandre.laverdiere@gmail.com"><a href="mailto:marcandre.laverdiere@gmail.com">marcandre.laverdiere@gmail.com</a></a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Hello list,<br>
<br>
I'm a Netty newbie and I decided to port my application's custom-made IO<br>
layer to Netty. I thought it would be a piece of cake. It was more or less<br>
the case, and things work fine when users are clicking their way to make<br>
requests.<br>
<br>
But the problem is when I'm load-testing it... the server will take all the<br>
CPU and everything is so slow that the load-testing client threads keep on<br>
timing out or get disconnect by the server (I set a timeout of 20 seconds).<br>
I don't understand what I did wrong...<br>
<br>
Here are some anonymized code snippets:<br>
<br>
//First, the code that initializes the ServerBootStrap<br>
<br>
ThreadPoolExecutor basePool = new<br>
MemoryAwareThreadPoolExecutor(NUM_THREADS/10, 0, MAX_MEMORY_PER_POOL_MB *<br>
FileUtils.ONE_MB/4);<br>
//We need twice the number of threads, since we have the timers too<br>
WORKER_THREAD_POOL = new MemoryAwareThreadPoolExecutor(NUM_THREADS*2, 0,<br>
MAX_MEMORY_PER_POOL_MB * FileUtils.ONE_MB);<br>
channelFactory = new NioServerSocketChannelFactory(basePool,<br>
WORKER_THREAD_POOL);<br>
clientServerBootStrap = new ServerBootstrap(channelFactory);<br>
<br>
SSLContext context = CryptoUtils.initTlsContext(KEYSTORE_PATH, KS_PASS,<br>
TRUSTSTORE_PATH, TS_PASS, TLS_SESSION_CACHE_SIZE);<br>
clientServerBootStrap.setPipelineFactory(new MyPipelineFactory(context));<br>
Channel clientChannel = clientServerBootStrap.bind(new<br>
InetSocketAddress(BINDING_PORT));<br>
allChannels.add(clientChannel);<br>
<br>
// What the pipeline looks like<br>
class MyPipelineFactory implements ChannelPipelineFactory{<br>
<br>
final private SSLContext context;<br>
<br>
public MyPipelineFactory(SSLContext context){<br>
this.context = context;<br>
}<br>
<br>
@Override<br>
public ChannelPipeline getPipeline() throws Exception {<br>
<br>
<br>
//Init the TLS for that channel<br>
SSLEngine engine = context.createSSLEngine();<br>
engine.setUseClientMode(false);<br>
engine.setWantClientAuth(false);<br>
engine.setEnableSessionCreation(true);<br>
<br>
ChannelPipeline pipeline = Channels.pipeline();<br>
pipeline.addLast("TLS", new SslHandler(engine));<br>
pipeline.addLast("Timeout", new Disconnector(new<br>
HashedWheelTimer(WORKER_THREAD_POOL.getThreadFactory()),<br>
SESSION_TIMEOUT,SESSION_TIMEOUT,SESSION_TIMEOUT,<br>
TimeUnit.MILLISECONDS));<br>
<br>
//Decode requests: first get the protobuf message length, then convert to<br>
protobuf, then convert to our own objects<br>
pipeline.addLast("frameDecoder",<br>
new LengthFieldBasedFrameDecoder(1048576, 0, 4, 0, 4));<br>
pipeline.addLast("protobufDecoder",<br>
new ProtobufDecoder(MyMessage.getDefaultInstance()));<br>
pipeline.addLast("requestDecoder", new MessageDecoder());<br>
<br>
//Encode responses: first convert from our objects to protobuf, then<br>
protobuf to binary, then add the length field<br>
pipeline.addLast("frameEncoder", new LengthFieldPrepender(4));<br>
pipeline.addLast("protobufEncoder", new ProtobufEncoder());<br>
pipeline.addLast("responseEncoder", new MessageEncoder());<br>
<br>
pipeline.addLast("Logging", new ChannelLogger()); //logs using our own API<br>
pipeline.addLast("handler", new RequestHandler()); //actually processes<br>
the request. Normal processing time: ~5-10 seconds<br>
pipeline.addLast("disconnectionNotifier", new DisconnectionNotifier());<br>
//Informs the observers that a channel is disconnected<br>
<br>
return pipeline;<br>
}<br>
<br>
}<br>
<br>
// How the disconnections are handled<br>
class Disconnector extends IdleStateHandler{<br>
public Disconnector(Timer timer, int readerIdleTimeSeconds,<br>
int writerIdleTimeSeconds, int allIdleTimeSeconds) {<br>
super(timer, readerIdleTimeSeconds, writerIdleTimeSeconds,<br>
allIdleTimeSeconds);<br>
}<br>
<br>
public Disconnector(Timer timer, long readerIdleTime,<br>
long writerIdleTime, long allIdleTime, TimeUnit unit) {<br>
super(timer, readerIdleTime, writerIdleTime, allIdleTime, unit);<br>
}<br>
<br>
@Override<br>
protected void channelIdle(ChannelHandlerContext ctx, IdleState state,<br>
long lastActivityTimeMillis) throws Exception {<br>
super.channelIdle(ctx, state, lastActivityTimeMillis);<br>
ctx.getChannel().close();<br>
releaseExternalResources();<br>
<br>
}<br>
<br>
@Override<br>
protected void finalize() throws Throwable {<br>
super.finalize();<br>
releaseExternalResources();<br>
}<br>
<br>
}<br>
<br>
Can anyone point out what I'm doing wrong???<br>
<br>
Thanks in advance!<br>
<font color="#888888">--<br>
View this message in context: <a href="http://netty-forums-and-mailing-lists.685743.n2.nabble.com/Netty-is-Freezing-on-Load-Testing-tp5138397p5138397.html" target="_blank"><a href="http://netty-forums-and-mailing-lists.685743.n2.nabble.com/Netty-is-Freezing-on-Load-Testing-tp5138397p5138397.html">http://netty-forums-and-mailing-lists.685743.n2.nabble.com/Netty-is-Freezing-on-Load-Testing-tp5138397p5138397.html</a></a><br>
Sent from the Netty User Group mailing list archive at <a href="http://Nabble.com">Nabble.com</a>.<br>
_______________________________________________<br>
netty-users mailing list<br>
<a href="mailto:netty-users@lists.jboss.org"><a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a></a><br>
<a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank"><a href="https://lists.jboss.org/mailman/listinfo/netty-users">https://lists.jboss.org/mailman/listinfo/netty-users</a></a><br>
</font></blockquote></div><br></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>netty-users mailing list</span><br><span><a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a></span><br><span><a href="https://lists.jboss.org/mailman/listinfo/netty-users">https://lists.jboss.org/mailman/listinfo/netty-users</a></span></div></blockquote></body></html>