On Fri, Feb 28, 2020 at 11:08 PM Nishant Kumar <nishantkumar35(a)gmail.com>
wrote:
I am using undertow-core (2.0.29.Final) with Java 11. Every once in
a
while (1-2 days) I got Too many open files error on my high load server
(30k JSON Request/sec) and then it stops responding. I have noticed that it
happens during full GC (but not during all full GC). Could you please help
me to fix this?
The "too many open files" error is invariably related to exceeding the max.
open file descriptors as enforced by the OS. Note that if you're using
systemd to start undertow, then you have to specify LimitNOFILE (it
ignores /etc/security/limits*); see [1] for more details. If you're not
running systemd, then I'd double-check the file descriptor limit for the
user running the undertow process. As to why this happens once in a while,
my best guess is either or both (a) there is a sudden, short-lived spike in
new connections, (b) many connections are kept alive for over 24 hours.
We've had similar issues while running undertow in high-throughput,
low-latency environment (in front of GLB). Even with the unreasonably high
file descriptor limit (LimitNOFILE=795588), we've seen traffic bursts that
would bring down undertow by triggering (linux) OOM killer (with that many
connections, non-heap memory is exhausted).
[1]
https://serverfault.com/questions/628610/increasing-nproc-for-processes-l...
Best,
stan