Setting SO_TIMEOUT on client socket channel

[Kevin Burton]

My company does a TON of HTTP connections... We do about 2TB of data per
month via HTTP ... so I can speak from a bit of experience here :)

It's not just bad networks but in practice you see LOTS of unusual TCP
activity including being firewalled, blacklisted, etc.

So it's not just broken machines it's an adversary incorrectly trying to
throttle you ...

We have a java.net.URL API that is about 4 years old that for the most part
is pretty rock solid but it took years of finding odd use cases to get it
to really handle every thing you can throw at it.

Handling connect timeout is really important...

I actually have a bug in peregrine to test the timeout code ...

Another item on this thread.

I'm going to write a HTTP pipeline handler to test peregrine with various
failure conditions.  I will probably just OSS this and put it wherever you
guys think is appropriate.

The idea is that you add the pipeline handler to the normal Netty pipeline
only it does crazy things like every 5th HTTP connection it will randomly
corrupt a header..... or the chunk data. or drop packets.. etc.

The idea is that you can deploy one machine in a faulty configuration and
then put a correctly working client against it to test failure.

I'm building Peregrine to handle clusters of 1-10k machines so crazy
network packet corruption and machine behavior is the norm in that setup.

Building failure INTO the tests means that you can test all sorts of crazy
conditions in your code and since they happen RARELY in production you can
know that your tests are working and that you have implementations of the
failure cases that actually work.

On another issue... I wrote an pipeline handler that prints all packet data
in Hex to the logger.  Normally its disabled but if you want to debug your
application without using Ethereal it can be pretty handy.

Should this be part of Netty?

I also extended the Hex encoder to print the hex better than the standard
netty hex encoder.  It prints both hex + ascii as well as the offset (in
hex) similar to hexdump.

On Fri, Oct 28, 2011 at 10:56 AM, chudak <meadandale at gmail.com> wrote:

> We aren't talking about the success rate of the deliveries...we are talking
> about the client connection hanging indefinitely when it is not getting tcp
> acks back from the server. We've found that in the case that we can setup
> the tcp connection but the data packets are not getting through that tcp
> will keep trying to retransmit every 2 minutes...and then the OS will
> finally time the connection out after 15 minutes. This is much to long to
> hang around with a failed connection.
>
> We've found that if we set the SO_TIMEOUT on the underlying socket that it
> correctly times out the connection if it can't read any data after the
> timeout period. We are only able to do this because we had to extend the
> OIO
> code to be able to inject a socket factory so we can handle SSL. To solve
> the above we've also had to add another wrapper around our socket factories
> that injects the SO_TIMEOUT, which is just another hack. It seems like this
> should be supported by the framework.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/netty-users/attachments/20111030/29302c6d/attachment.html