NIO OutOfMemory Errors with HexDumpProxy example

[Erick Lichtas]

Hi everyone,

 

I am experiencing out of memory issues with Netty when it is used as a proxy
server with NIO.  

 

Jul 29, 2010 2:13:54 PM org.jboss.netty.channel.socket.nio.NioWorker

WARNING: Unexpected exception in the selector loop.

java.lang.OutOfMemoryError: Java heap space

      at
org.jboss.netty.buffer.HeapChannelBuffer.<init>(HeapChannelBuffer.java:49)

      at
org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(BigEndianHeapChanne
lBuffer.java:39)

      at
org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:131)

      at
org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferF
actory.java:72)

      at
org.jboss.netty.buffer.AbstractChannelBufferFactory.getBuffer(AbstractChanne
lBufferFactory.java:53)

      at
org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:298)

      at
org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.j
ava:275)

      at
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196)

      at
org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)

      at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)

      at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)

      at java.lang.Thread.run(Thread.java:619)

 

I have an example program that I would like to send, but I'm not sure if I
can send it directly to the user mailing list, or if I must open a case in
order post the code.  Please advise.

 

In my example, I have a simple TestServer and TestClient.  The server is a
simple server based on plain old Java sockets.  When the TestClient
downloads directly (no proxy) from the TestServer with 5 or so concurrent
clients, the files are downloaded with minimum amount of memory and seems is
able to handle any amount of data without any issues.

 

When I use Netty's HexDumpProxy example as my proxy server (slightly
modified to toggle throttling) to connect to the TestServer, I experience
the OOM rather quickly.  I  and using the same TestClient app, simply
adjusted to connect through the proxy instead of direct.  A timer has been
set up in the HexDumpProxy to display the memory usage information to the
console every 5 seconds, which displays the memory usage growing out of
control.  I'm guessing this is due to the messages queuing up faster than
the client can read them.  However, even after adding throttling to the
HexDumpProxyInboundHandler to suspend the reading when the inboundChannel is
not writeable, the proxy still runs out of memory. 

 

Netty Versions:                tested both netty-3.1.5.GA.jar and
netty-3.2.1.Final.jar

Java Version:                     JDK 1.6.0_13

JRE Memory:                     128MB max (-Xmx128M)

 

Because of this, I am currently stuck with using OIO, but need to use NIO
for scalability reasons.  I am wondering if this is a bug in Netty, and if
so, should I open an issue?  Please let me know where I can send the example
code.

 

Kind regards,

 

E R I C K   L I C H T A S

Linoma Software

p. 402.944.4242 x714

f. 402.944.4243

 <http://www.linomasoftware.com/> www.LinomaSoftware.com

 <http://www.goanywheremft.com/> www.GoAnywhereMFT.com

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/netty-users/attachments/20100729/93de6107/attachment.html