<div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default">We ended up implementing hard timeout using CAS; no noticeable drop in throughput or p99 latency.</div><div class="gmail_default">I thought I'd share in case others on the list may need it. Here is the code snippet (using Kotlin).</div><div class="gmail_default"><br></div><div class="gmail_default"><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"><span style="color:rgb(0,0,128);font-weight:bold">override fun </span>handleRequest(exchange: HttpServerExchange) {</pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"> val <span style="font-size:9pt">timeoutWhenTrue = AtomicBoolean(</span><span style="font-size:9pt;color:rgb(0,0,128);font-weight:bold">true</span><span style="font-size:9pt">)</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"><span style="font-size:9pt"> </span>exchange.getIoThread().executeAfter(<span style="font-size:9pt;font-weight:bold">{</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"><span style="font-weight:bold"> </span><span style="color:rgb(0,0,128);font-weight:bold">if </span>(timeoutWhenTrue.compareAndSet(<span style="color:rgb(0,0,128);font-weight:bold">true</span>, <span style="color:rgb(0,0,128);font-weight:bold">false</span>)) {<br> <span style="color:rgb(128,128,128);font-style:italic">// we're first, so end the exchange</span><br> exchange.<span style="color:rgb(102,14,122);font-style:italic">statusCode </span>= StatusCodes.<span style="color:rgb(102,14,122);font-style:italic">REQUEST_TIME_OUT<br></span><span style="color:rgb(102,14,122);font-style:italic"> </span>exchange.endExchange()<br> }<br><span style="font-weight:bold"> }</span>, <span style="color:rgb(102,14,122);font-weight:bold">requestTimeoutMillis</span>, TimeUnit.<span style="color:rgb(102,14,122);font-weight:bold">MILLISECONDS</span>)</pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"> exchange.dispatch(SameThreadExecutor.<span style="font-size:9pt;color:rgb(102,14,122);font-style:italic">INSTANCE</span><span style="font-size:9pt">, </span><span style="font-size:9pt;font-style:italic">Runnable </span><span style="font-size:9pt;font-weight:bold">{ </span>launch <span style="font-size:9pt;font-weight:bold">{</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"><span style="font-size:9pt;font-weight:bold"> //</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"><span style="font-size:9pt;font-weight:bold"> // request servicing logic</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"><span style="font-size:9pt;font-weight:bold"> //</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"> if <span style="font-size:9pt">(timeoutWhenTrue.compareAndSet(</span><span style="font-size:9pt;color:rgb(0,0,128);font-weight:bold">true</span><span style="font-size:9pt">, </span><span style="font-size:9pt;color:rgb(0,0,128);font-weight:bold">false</span><span style="font-size:9pt">)) {</span></pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"> response.<span style="color:rgb(102,14,122);font-weight:bold">headers</span>.<span style="font-style:italic">forEach </span><span style="font-weight:bold">{ </span>exchange.<span style="color:rgb(102,14,122);font-style:italic">responseHeaders</span>.add(<span style="font-weight:bold">it</span>.<span style="color:rgb(102,14,122);font-weight:bold">key</span>, <span style="font-weight:bold">it</span>.<span style="color:rgb(102,14,122);font-weight:bold">value</span><span style="color:rgb(102,14,122)"> </span><span style="font-weight:bold">}<br></span><span style="font-weight:bold"> </span>exchange.<span style="color:rgb(102,14,122);font-style:italic">statusCode </span>= response.<span style="color:rgb(102,14,122);font-weight:bold">statusCode<br></span><span style="color:rgb(102,14,122);font-weight:bold"> </span>response.<span style="color:rgb(102,14,122);font-weight:bold">response</span>?.<span style="font-style:italic">let </span><span style="font-weight:bold">{ </span>exchange.<span style="color:rgb(102,14,122);font-style:italic">responseSender</span>.send(response.<span style="color:rgb(102,14,122);font-weight:bold">response</span>) <span style="font-weight:bold">} </span>?: exchange.endExchange() }</pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt"> }})</pre><pre style="white-space:pre-wrap;color:rgb(0,0,0);font-family:Menlo;font-size:9pt">}</pre></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Jul 22, 2018 at 2:59 AM Stan Rosenberg <<a href="mailto:stan.rosenberg@gmail.com">stan.rosenberg@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small">On Sun, Jul 15, 2018 at 7:45 PM, Stuart Douglas <span dir="ltr"><<a href="mailto:sdouglas@redhat.com" target="_blank">sdouglas@redhat.com</a>></span> wrote:<br></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div style="font-size:small"><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>Are you using the Servlet API or the HttpServerExchange API? The best way to approach this is a bit different depending on what you are doing.</div></div></blockquote><div><br></div><div> <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">HttpServerExchange API<div style="font-size:small;display:inline">. Thanks!</div></span></div></div></div></div></blockquote><div><br></div></span><div>This is a bit harder to do in the general case. With Servlet you could just create a thread safe wrapper, where the wrapper basically disconnects from the underlying request on timeout. The Undertow native API is not designed around wrapping though, so it needs cooperation from the application to manage this.</div><div><br></div><div>If you know the application is only going to be writing data (and not setting headers) then you should be able to make this work via a ConduitFactory implementation that handles the locking, although if this is not the case then you are going to need some kind of external lock.</div><span class="m_2031543079434414749HOEnZb"><font color="#888888"><div><br></div></font></span></div></div></blockquote><div><br></div><div class="gmail_default" style="font-size:small">Could you please elaborate on the approach using <span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">ConduitFactory. Do you mean using 'exchange.addResponseWrapper' to intercept writes? That would require locking since write can be invoked from a different thread concurrently with timeout thread.</span></div></div></div></div>
</blockquote></div>