OutOfMemoryError: when with large number of Worker Threads in NioClientSocketChannelFactory

[Wesly smith]

Hello Trustin, 

I got OutOfMemoryError. The profile showed me that the
NioClientSockeetChannelFactory holds the buffer too long.

The message:
=============
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at test.netty.http.HttpClient.main(HttpClient.java:78)

A sample code:
==============
I have a long-live http client case. In the code below, I demoed as a loop
to connect the server, and create a ChannelBuffer with 1M byte, and send it
as content to server. The Handler will close the channel when
messageReceived.

FYI, I had NioClientSocketChannelFactory with a big number of workerCount,
such as 160. 
  ChannelFactory factory =
    new NioClientSocketChannelFactory(
        Executors.newCachedThreadPool(),
         Executors.newCachedThreadPool(), 160);

I noticed for each request (could be only the first 160. On my try, it got
OutOfMemory less than 160 request.), the java memory use will jump 1M. It
seems those losts are the 1M byte buffers I allocated as content. For some
reason, they are always referred by those Executors, and so it can not be
released.

Any suggestions?

======== The code snip. It is modified from the http sample =======
public class HttpClient {

    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            System.err.println(
                    "Usage: " + HttpClient.class.getSimpleName() +
                    " <URL>");
            return;
        }

        URI uri = new URI(args[0]);
        String scheme = uri.getScheme() == null? "http" : uri.getScheme();
        String host = uri.getHost() == null? "localhost" : uri.getHost();
        int port = uri.getPort() == -1? 80 : uri.getPort();

        if (!scheme.equals("http")) {
            // We can actually support HTTPS fairly easily by inserting
            // an SslHandler to the pipeline - left as an exercise.
            System.err.println("Only HTTP is supported.");
            return;
        }

        // Configure the client.
        ChannelFactory factory =
            new NioClientSocketChannelFactory(
                    Executors.newCachedThreadPool(),
                    Executors.newCachedThreadPool(), 160);

        ClientBootstrap bootstrap = new ClientBootstrap(factory);
        HttpClientPipelineFactory handler = new
HttpClientPipelineFactory(new HttpResponseHandler());
        bootstrap.setPipelineFactory(handler);
        bootstrap.setOption("keepAlive", true);

        // Send 10000 HTTP requests, each with 1M content
        for ( int loop = 0; loop < 10000; loop++)
        {       
        	// Start the connection attempt.
			ChannelFuture future = bootstrap.connect(new InetSocketAddress(
					host, port));

			// Wait until the connection attempt succeeds or fails.
			Channel channel = future.awaitUninterruptibly().getChannel();
			if (!future.isSuccess()) {
				future.getCause().printStackTrace();
				factory.releaseExternalResources();
				return;
			}

			HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1,
					HttpMethod.POST, uri.toASCIIString());
			request.addHeader(HttpHeaders.HOST, host);

			ChannelBuffer content = ChannelBuffers.buffer(1024*1024);
			byte[] b = new byte[1024*1024];
			for (int i = 0; i < 1024*1024; i++) {
				b[i] = 100;
			}
			content.writeBytes(b, 0, 1024*1024);
			request.setContent(content);
            // set headers
			request.addHeader("Connection", "Keep-Alive");
			request.addHeader(HttpHeaders.CONTENT_LENGTH,
String.valueOf(content.readableBytes()));
            
			channel.write(request);
			Thread.sleep(1000);
        }
        
        // Wait for the server to close the connection.
        //channel.getCloseFuture().awaitUninterruptibly();

        // Shut down executor threads to exit.
        factory.releaseExternalResources();
    }
}
-- 
View this message in context: http://n2.nabble.com/OutOfMemoryError%3A-when-with-large-number-of-Worker-Threads-in-NioClientSocketChannelFactory-tp2140700p2140700.html
Sent from the Netty User Group mailing list archive at Nabble.com.