Strange Problem

[Alan Wolff]

Hi

I experienced a strange problem out of the blue last night with my
netty-based server running on Debian Linux. The problem is not
necessarily with netty, but any ideas would be appreciated:

Most connections were not being accepted, even from localhost. Tcpdump
 immediately showed SYN packet being received, but no ACK sent.
LoggingHandler's logging showed nothing about the connection. If and
when a connection was successfully accepted, the usual
OPEN,BOUND,CONNECTED was shown in logs immediately.
Here are the debugging steps I took:

1) Check if other TCP connections could be made to other app servers
running on the host (yes)
2) Observe exception logs and stdout log for thing like max file open
limits reached (nothing and setting is very high)
3) Observe /var/log/messages (nothing in log)
4) Observe TCP backlog queue (64 connections waiting in queue, sysctl
settings were at about 1000)
5) Flush iptable rules (didn't help)
6) Observe netty's BOSS thread in Yourkit/jstack (nothing out of ordinary)
7) Try latest JDK (6u17) (didn't help)
8) Stop the server, try start it. (didn't help)
9) OS reboot (didn't help)
10) Block all new connections to server but keep current connections
in place. (Problem gone when trying to open a connection to server
from localhost. When allowing internet traffic again, problem appeared
again)

I would like to mention that there has commonly been 2x the amount of
established connections before and load was normal and nothing out of
the ordinary.

>From my understanding, the chain of events for a connection to be accepted are:

1 SYN arrives at NIC-> 2 kernel checks if it can allow it-> 3 iptables
checks if it can allow it-> 4 netty and JVM call accept()

So I need to figure at which point the SYN packet is being dropped or ignored.

The next time it happens (if it happens) I was thinking of unloading
iptables's modules from kernel and trying IBM JVM.

Does anyone have ideas on further debugging such as TCP tunables or
other OS limitations?

Thanks a lot!