direct buffer memory issues

[Adam Fisk]

I'm experiencing an issue where LittleProxy eventually runs out of
memory, but in direct memory allocated through allocateDirect, not in
the object heap. Here's a link to the issue with the stack trace:

http://dev.littleshoot.org:8081/browse/LP-16

I tried using -XX:MaxDirectMemorySize=256M, but that just seems to
delay the issue. If I look at the actual OS-level memory allocated,
once it gets up to around 300MB, I can't allocate more (basically the
256 + 40MB or so for the object heap and whatever else).

It fails in SocketSendBufferPool when creating the "Preallocation"
with DEFAULT_PREALLOCATION_SIZE = 65536. In LittleProxy's case, that's
when we're allocating new HTTP clients to go out to servers to fetch
data. Many of those connections don't end up passing much data
upstream (although some POST requests clearly will). Any way to set
that buffer size on a per-client basis?

I'm not exactly sure what to do other than boosting
XX:MaxDirectMemorySize even further.

The other odd thing is the memory *never seems to go down*, even after
all the threads are done. Is it possible I'm not cleaning up those
client connections properly? It seems like any buffer allocated with
allocateDirect never gets freed.

Any suggestions you may have would be much appreciated.

Outside of this issue, things are looking great!!

Thanks Trustin.

-Adam

-- 
Adam Fisk
http://www.littleshoot.org | http://adamfisk.wordpress.com |
http://twitter.com/adamfisk