<div dir="ltr"><div dir="ltr"><div>Hi Nishant,</div><div><br></div><div> thanks for the update - that is good news!</div><div><br></div><div>Rio<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Mar 15, 2020 at 6:00 AM Nishant Kumar <<a href="mailto:nishantkumar35@gmail.com">nishantkumar35@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I have tried XNIO 3.7.7-Final with jboss.threads <span style="color:rgb(0,0,0);font-family:Menlo;font-size:9pt">2.3.3.Final in Undertow </span>2.0.30.Final and it looks good so far. I am running it in production for the last 2 days and no issues so far :). I am monitoring it and will report if find anything. <div><br></div><div>The reason I changed jboss thread version is that I observed a high CPU load in XNIO 3.7.7.Final comparing XNIO 3.3.8.Final which I reported in the last email. However, I have not done any extensive tests for it.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 12, 2020 at 1:37 PM Nishant Kumar <<a href="mailto:nishantkumar35@gmail.com" target="_blank">nishantkumar35@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I have also observed that the CPU load increases significantly on 3.7.7-Final (load average 50 on 48 core machine) on moderate load but it's very low (load average 20) on 3.3.8-Final on the same QPS (20k Req/sec) and it (3.7.7-Final) starts refusing connection on high load.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 12, 2020 at 7:20 AM Nishant Kumar <<a href="mailto:nishantkumar35@gmail.com" target="_blank">nishantkumar35@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">I didn't get time to write standalone program to reproduce it but with new version 3.7.7-Final, i am not able to survive for more than an hour. With the older default xnio version of Undertow (xnio 3.3.8-Final) i can run server for few days until it stops working with error i mentioned in first email.<div dir="auto"><br></div><div dir="auto">Just FYI, i am using OpenJDK 11 with OpenJ9 JVM.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 4, 2020, 10:54 AM Flavia Rainone <<a href="mailto:frainone@redhat.com" target="_blank">frainone@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Can someone provide a reproducer for this error?<div><br></div><div>As for the old version of XNIO, it will be upgraded in Undertow 2.1.0.Final.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 2, 2020 at 8:47 PM Stuart Douglas <<a href="mailto:sdouglas@redhat.com" rel="noreferrer" target="_blank">sdouglas@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hmm, maybe this is a bug in the HTTP/2 close code then, and somehow the connection is not being closed if the client hangs up abruptly. I had a quick look at the code though and I think it looks ok, but maybe some more investigation is needed.<div><br></div><div>Stuart</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 3 Mar 2020 at 03:41, Nishant Kumar <<a href="mailto:nishantkumar35@gmail.com" rel="noreferrer" target="_blank">nishantkumar35@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Yes, i have no control on client side. I am using HTTP2. I have tried increasing open file limit to 400k but that consumes all memory and system hangs. I will probably try to put a nginx in front of Undertow and test.<div dir="auto"><br></div><div dir="auto"><span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,monospace;font-size:12px;white-space:pre-wrap;background-color:rgb(255,255,255)">setServerOption(</span><span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,monospace;font-size:12px;white-space:pre-wrap;background-color:rgb(255,255,255)">UndertowOptions</span><span style="color:rgb(0,92,197);font-family:sfmono-regular,consolas,"liberation mono",menlo,monospace;font-size:12px;white-space:pre-wrap;background-color:rgb(255,255,255)"><span style="color:rgb(215,58,73)">.</span>ENABLE_HTTP2</span><span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,monospace;font-size:12px;white-space:pre-wrap;background-color:rgb(255,255,255)">, </span><span style="color:rgb(0,92,197);font-family:sfmono-regular,consolas,"liberation mono",menlo,monospace;font-size:12px;white-space:pre-wrap;background-color:rgb(255,255,255)">true</span><span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,monospace;font-size:12px;white-space:pre-wrap;background-color:rgb(255,255,255)">)</span><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 2, 2020, 7:48 PM David Lloyd <<a href="mailto:david.lloyd@redhat.com" rel="noreferrer noreferrer noreferrer" target="_blank">david.lloyd@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Mon, Mar 2, 2020 at 7:56 AM Stan Rosenberg <<a href="mailto:stan.rosenberg@acm.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">stan.rosenberg@acm.org</a>> wrote:<br>
><br>
> Stuck in CLOSE_WAIT is a symptom of the client-side not properly shutting down [1].<br>
<br>
I would partially disagree. In the article you linked: "It all starts<br>
with a listening application that leaks sockets and forgets to call<br>
close(). This kind of bug does happen in complex applications." This<br>
seems to be essentially what's happening here: the server isn't<br>
completing the connection (for some reason), stranding the socket in<br>
`CLOSE_WAIT`.<br>
<br>
We can't assume that the client is abandoning the connection after<br>
`FIN_WAIT2` (the titular RFC violation); if the server stays in<br>
`CLOSE_WAIT`, then even if the client dutifully stays in `FIN_WAIT2`<br>
forever, the resolving condition still needs to be that the server<br>
shuts down its side of the connection.<br>
<br>
This diagram is a useful visual aid, mapping TCP states to the XNIO<br>
API: <a href="https://www.lucidchart.com/publicSegments/view/524ec20a-5c40-4fd0-8bde-0a1c0a0046e1/image.png" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">https://www.lucidchart.com/publicSegments/view/524ec20a-5c40-4fd0-8bde-0a1c0a0046e1/image.png</a><br>
<br>
-- <br>
- DML<br>
<br>
</blockquote></div>
</blockquote></div>
_______________________________________________<br>
undertow-dev mailing list<br>
<a href="mailto:undertow-dev@lists.jboss.org" rel="noreferrer" target="_blank">undertow-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" rel="noreferrer noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>
<p style="font-weight:bold;margin:0px;padding:0px;font-size:14px;font-family:RedHatText,sans-serif">
<span>Flavia</span> <span>Rainone</span><span style="color:rgb(170,170,170);margin:0px"></span>
</p>
<p style="font-size:12px;margin:0px;font-family:RedHatText,sans-serif">
<span>Principal Software Engineer</span>
</p>
<p style="margin:0px 0px 4px;font-size:12px;font-family:RedHatText,sans-serif">
<a style="color:rgb(0,136,206);margin:0px" href="https://www.redhat.com" rel="noreferrer" target="_blank">Red Hat </a>
</p>
<div style="margin-bottom:4px">
</div>
<p style="margin:0px;font-size:12px;font-family:RedHatText,sans-serif">
<span style="margin:0px;padding:0px"><a style="color:rgb(0,0,0);margin:0px" href="mailto:frainone@redhat.com" rel="noreferrer" target="_blank">frainone@redhat.com</a> </span>
</p>
<div style="margin-top:12px">
<table border="0">
<tbody><tr>
<td width="100px"><a href="https://www.redhat.com" rel="noreferrer" target="_blank"> <img src="https://static.redhat.com/libs/redhat/brand-assets/2/corp/logo--200.png" width="90" height="auto"></a> </td>
</tr>
</tbody></table>
</div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">Nishant Kumar<br>Bangalore, India<br>Mob: +91 80088 42030<br>Email: <a href="mailto:nishantkumar35@gmail.com" target="_blank">nishantkumar35@gmail.com</a></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">Nishant Kumar<br>Bangalore, India<br>Mob: +91 80088 42030<br>Email: <a href="mailto:nishantkumar35@gmail.com" target="_blank">nishantkumar35@gmail.com</a></div></div>
_______________________________________________<br>
undertow-dev mailing list<br>
<a href="mailto:undertow-dev@lists.jboss.org" target="_blank">undertow-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a></blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">Richard Opalka<br>Principal Software Engineer<br>Red Hat JBoss Middleware<br>Mobile: +420 731 186 942<br>E-mail: <a href="mailto:ropalka@redhat.com" target="_blank">ropalka@redhat.com</a><br><br></div></div></div></div></div>