Too many open files error with Netty

[Trustin Lee (이희승)]

Sure.  Please refer to the Javadoc of ChannelConfig and its subtypes.
For example, writeBufferHighWaterMark and writeBufferLowWaterMark are
documented here: http://is.gd/4F15l

The documentation is far from perfection in my opinion.  Please feel
free to give some feed back to improve the documentation quality.

-- Trustin Lee, http://gleamynode.net/

On Tue, Oct 27, 2009 at 11:50 PM, Utkarsh Srivastava <utkarsh at gmail.com> wrote:
> Is there a list of all these options somewhere?
>
> Utkarsh
>
> 2009/10/27 Trustin Lee (이희승) <trustin at gmail.com>:
>> 2009/10/27 Trustin Lee (이희승) <trustin at gmail.com>:
>>> On Tue, Oct 27, 2009 at 7:42 PM, Trustin Lee (이희승) <trustin at gmail.com> wrote:
>>>> On Tue, Oct 27, 2009 at 7:37 PM, Albert Strasheim <fullung at gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> 2009/10/27 Trustin Lee (이희승) <trustin at gmail.com>:
>>>>>> Sounds like you are not getting the 'too many open files' error
>>>>>> anymore.  Could you confirm it?
>>>>>
>>>>> That is correct.
>>>>
>>>> Thanks for confirmation. :)
>>>>
>>>>>> Do you get OutOfMemoryError in the end or is it just working fine?  I
>>>>>> do see saw-toothed patterns in the memory graph, but I never got an
>>>>>> exception so far.  If the connection rate is high, GC will be busy,
>>>>>> but I'm not sure it will slow down the application seriously.
>>>>>> Additional information is appreciated.
>>>>>
>>>>> When the maximum number of 10000 connections is active, about a
>>>>> hundred sessions are completing per second. But when it reaches about
>>>>> 25000 connections in total, I don't get OutOfMemoryError, but the
>>>>> application basically stops and just spends all its time GCing.
>>>>
>>>> That is strange.  What happens if you increase the heap size to 512M
>>>> then?  Would you mind if you send me the heap dump file?  I will take
>>>> a look using my profiler.
>>>
>>> I ran it by myself with a large 'maxClients' value and succeeded to
>>> reproduce huge memory consumption.  According to the profiler, it's
>>> due to the pending write requests in the queue.  Simple calculation
>>> (30 * 20 * 32 * 10000) says that a lot of heap is required to handle
>>> this amount of simultaneous I/O.  Perhaps you should reduce the
>>> traffic?
>>
>> To reduce overall memory consumption, I decreased the write buffer water marks:
>>
>>        clientBootstrap.setOption("writeBufferHighWaterMark", 4096);
>>        clientBootstrap.setOption("writeBufferLowWaterMark", 1024);
>>
>> The defaults are 64K and 16K respectively, which is too much in this case.
>>
>> After reconfiguration, memory utilization looks much better - no more thrashing.
>>
>> HTH
>>
>> -- Trustin Lee, http://gleamynode.net/
>>
>> _______________________________________________
>> netty-users mailing list
>> netty-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>