What might be causing increasing latency?

[justkevin]

After noticing my game app was occasionally running sluggish, I spent quite a
bit of time trying to figure out why without success. If anyone has
suggestions on what to try I'd appreciate it. The details are long, but I
don't know what might be important:

Details:

The client is written in AS3.

The server is written in Java using Netty 3.2 (I've updated to the most
recent alpha).

I'm testing locally, with client and server on the same box.

When the first client connects, latency as measured with a round-trip ping
pong is low, < 10 ms. It can remain low for a while, but eventually begins
increasing in spurts at varying rates.  For example, it might be < 10 ms for
a while, then 50-100 ms for a while, then 100-200 ms, and so forth. There's
no upper limit, I've seen delays in multiple seconds. The larger the
latency, the larger the variance. The increases may be correlated to large
bursts of data.

If I restart the client, the latency will usually drop, but not always.
Sometimes it restarts high. 

If I restart the server, the next client to connect will get a low latency.

Using system times, I've established that the latency is entirely on the
downstream-- the ping takes 0ms to reach the server, and all the increase is
between the time the pong is created and when its received by the client.
>From this I'm guessing it's something with how I'm sending messages.

The only ChannelPipeline handler on outbound is a LengthFieldPrepender.

Here is how I'm sending the message:

		// A Netty channel expects a message in the format of a ChannelBuffer:
		ChannelBuffer buffer = ChannelBuffers.dynamicBuffer();
		// Wrap the buffer with a ChannelBufferOutputStream, which implements
DataOutput
		// (our messages expect to write to a DataOutput)
		ChannelBufferOutputStream output = new ChannelBufferOutputStream(buffer);
		// First write the message id:
		output.writeByte(message.getId());
		// Then have the message write itself, checking to make sure the message
		// obeys its length contract:
		int beforeLength = output.writtenBytes();
		int written = message.write(output);
		if(beforeLength + written != output.writtenBytes()){
			logger.severe("Outbound message violated length contract:" + message);
			throw new RuntimeException("Message violated length contract:" +
message);
		}
		// Finally, put into the channel. The channel pipeline should already have
a length prepender:
		this.channel.write(buffer);

Thanks in advance for any suggestions,
-- 
View this message in context: http://n2.nabble.com/What-might-be-causing-increasing-latency-tp4269271p4269271.html
Sent from the Netty User Group mailing list archive at Nabble.com.