<div dir="auto">Thanks for the reply. I am running it under supervisord and i have updated open file limit in supervisord config. The problem seems to be same as what @Carter has mentioned. It happens mostly during sudden traffic spike and then sudden increase (~30k-300k) of TIME_WAIT socket. I will try to build Undertow with latest XNIO because latest Undertow release is still using old version of XNIO. Any suggestions on the build?</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 29, 2020, 11:34 PM Stan Rosenberg <<a href="mailto:stan.rosenberg@acm.org" rel="noreferrer noreferrer" target="_blank">stan.rosenberg@acm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small">On Fri, Feb 28, 2020 at 11:08 PM Nishant Kumar <<a href="mailto:nishantkumar35@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">nishantkumar35@gmail.com</a>> wrote:</div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I am using <span style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">undertow-core (</span><span style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">2.0.29.Final) </span>with Java 11. Every once in a while (1-2 days) I got <span style="color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:12px;font-variant-ligatures:none;white-space:pre-wrap">Too many open files</span> error on my high load server (30k JSON Request/sec) and then it stops responding. I have noticed that it happens during full GC (but not during all full GC). Could you please help me to fix this? </div></div></blockquote><div><br></div><div class="gmail_default" style="font-size:small">The "too many open files" error is invariably related to exceeding the max. open file descriptors as enforced by the OS. Note that if you're using systemd to start undertow, then you have to specify LimitNOFILE (it ignores /etc/security/limits*); see [1] for more details. If you're not running systemd, then I'd double-check the file descriptor limit for the user running the undertow process. As to why this happens once in a while, my best guess is either or both (a) there is a sudden, short-lived spike in new connections, (b) many connections are kept alive for over 24 hours. </div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">We've had similar issues while running undertow in high-throughput, low-latency environment (in front of GLB). Even with the unreasonably high file descriptor limit (LimitNOFILE=795588), we've seen traffic bursts that would bring down undertow by triggering (linux) OOM killer (with that many connections, non-heap memory is exhausted).</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">[1] <a href="https://serverfault.com/questions/628610/increasing-nproc-for-processes-launched-by-systemd-on-centos-7" rel="noreferrer noreferrer noreferrer" target="_blank">https://serverfault.com/questions/628610/increasing-nproc-for-processes-launched-by-systemd-on-centos-7</a></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Best,</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">stan</div></div></div>
</blockquote></div>