Too many open files error with Netty

[Albert Strasheim]

Hello all

I'm a new Netty user and I'm having some problems with a "Too many
open files" error.

I'm doing some tests with a server based on the Discard example:

http://www.jboss.org/file-access/default/members/netty/freezone/xref/3.1/org/jboss/netty/example/discard/package-summary.html

The attached code starts a server.

Then it starts connecting clients, each of which writes some data and
then closes the connection.

The main thread runs a loop that keeps 15000 client connections alive.

Everything runs fine for a while, and then I get:

179027 [main] INFO frags.TooManyFiles - 50101 connections, 0/15000 idle
179489 [main] INFO frags.TooManyFiles - 50201 connections, 0/15000 idle
179995 [main] INFO frags.TooManyFiles - 50301 connections, 0/15000 idle
180445 [main] INFO frags.TooManyFiles - 50401 connections, 0/15000 idle
Exception in thread "main" org.jboss.netty.channel.ChannelException:
Failed to open a socket.
        at org.jboss.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:49)
        at org.jboss.netty.channel.socket.nio.NioClientSocketChannel.<init>(NioClientSocketChannel.java:83)
        at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:139)
        at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:86)
        at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:228)
        at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:192)
        at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:163)
        at frags.TooManyFiles.main(TooManyFiles.java:124)
Caused by: java.net.SocketException: Too many open files
        at sun.nio.ch.Net.socket0(Native Method)
        at sun.nio.ch.Net.socket(Net.java:94)
        at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:84)
        at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:37)
        at java.nio.channels.SocketChannel.open(SocketChannel.java:105)
        at org.jboss.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:47)
        ... 7 more

I don't think this error is related to the "standard" problem of too
few file handles.

I have the following in /etc/system:

set rlim_fd_max=65536
set rlim_fd_cur=65536

and ulimit says:

open files (-n) 65536

so that should be more than enough for the 15000 client sockets and
15000 server sockets.

While the program runs, the number of connections reported by netstat
seems stable. I see the 30000 sockets in the ESTABLISHED state, and
the number of sockets in the TIME_WAIT state stabilises at about
12000.

I'm using Netty 3.1.5 on Solaris 10 with JDK 1.6.0_16.

I'd appreciate any thoughts.

Regards,

Albert
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TooManyFiles.java
Type: application/octet-stream
Size: 5390 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20091014/7694b8b5/attachment-0001.obj