spurious problem with Client NIOSockets

["Trustin Lee (이희승)"]

Hi Neil,

That doesn't sound right.  It would be great if you try a couple things
for me to track down the problem:

1) After calling Channel.close() for the leaked connection, what do
isOpen(), isBound(), and isConnected() return?

2) Could you increase the connect future timeout and dump the heap while
it waits for the connect future using jmap?  Once you send me the heap
dump, I can understand Netty's internal status more easily.

3) Could you let me know if specifying "connectTimeoutMillis" option
instead of using future timeout fixes the problem?

    bootstrap.setOption("connectTimeoutMillis", 10000);
    ...
    bootstrap.connect().awaitUninterruptibly(); // note no argument

I believe (3) will fix your problem, but I need (1) and (2) to figure
out the exact steps to reproduce the problem and fix it fundamentally.

If possible, it would be very nice if I can figure out if this is a bug
or not and fix it as soon as possible so that more stable CR (or Final)
version is released tomorrow.

Thanks,
Trustin

Neil Avery wrote:
> All,
> Trustin - firstly - thanks again for such an awesome network framework -
> it has provided to be amazingly scalable for some of the loads we have
> been putting through it; 4000 servers streaming GBs of data and its been
> rock solid!
> 
> Platform Netty 3.2RC1, RedHat Linux, Java 1.6.0_18
> 
> Unfortunately we have finally hit a weird issue which only occurs on
> this particular platform. Using the NioClientSocketChannelFactory to
> create a client, after a period of use we start to experience a timeout
> when trying to bind. The code is standard in that it follows the
> tutorials and we have never seen this problem before. In this case there
> is little load on the system and its only sending messages to 6 peers.
> 
> Now for the weird bit - there is no cause excepton() sent to the handler
> (our nio listener) and - when printing
> the channelFuture.getChannel().getLocalAddress() it prints out a valid
> client address. Using lsof & netstat we can see the connection is
> actually bound. As part of the failure, we attempt to free resources for
> the channel and they also fail to release the bound connection...so we
> leak connections. Cleanup is attempted using; closeFuture =
> channelFuture.getChannel().close() and then wait 10seconds for the
> closeFuture to complete.
> 
> So I guess I have 3 questions:
> - are we handling the connection bind correctly? - it doesnt seem for
> this case as it does connect the code doesnt think it has
> - are we handling closeDown correctly? -
> channelFuture.getChannel().close() doesnt seem to release these resources...
> - is there a brute force way to access the underlying socketChannel and
> attempt to release it?
> 
> Any help appreciated,
> Regard Neil.
> 
>         ClientBootstrap bootstrap = new ClientBootstrap(nettyClientfactory);
>         handler = new NettyClientHandler(uri);
>         bootstrap.getPipeline().addLast(HANDLER, handler);
>         bootstrap.setOption(TCP_NO_DELAY, true);
>         bootstrap.setOption(KEEP_ALIVE, true);
> 
> channelFuture = bootstrap.connect(new InetSocketAddress(uri.getHost(),
> uri.getPort()));
>         
>         try {
>          boolean success = channelFuture.await(10, TimeUnit.SECONDS);
> if (!success) {
>     if (handler.getException() != null) {
> throw new RuntimeException(String.format("%s\n Failed to establish
> Connection within %d secs:%s", this.toString(),
> connectionEstablishTimeout, uri), handler.getException()); } else {
> throw new RuntimeException(String.format("%s\n Failed to establish
> Connection within %d secs:%s", this.toString(),
> connectionEstablishTimeout, uri)); }
> 
> 
> -- 
> 
> www.liquidlabs.co.uk <http://www.liquidlabs.co.uk>
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users

-- 
what we call human nature in actuality is human habit
http://gleamynode.net/


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20100527/166d2a38/attachment.bin