<div dir="ltr"><div>Hi, David!</div><div><br></div><div>I had exactly this problem. It is not relationed with the Apiman. The problem is with Wildfly. If your Wildfly is with default configuration, you have a limit of concurrent requests (Default "<i style="color:rgb(0,0,0);font-family:calibri,sans-serif;font-size:14.6667px"><span lang="EN-US">Task max threads" = </span></i>16 * vCPUs). So, when the Wildfly is working in the limit, it create a queue of requests.</div><div><br></div><div><br></div><div>Suggested settings for your wildfly (adjust according to your server):</div><div><br></div><div><br></div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Configuration -> Subsystem -> IO</div><div> </div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>WORKER</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Io threads: 128</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Task max threads: 2048</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Stack size: 0</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Task keepalive: 120</div><div> </div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>BUFFER POLL</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Buffer size: 16384</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Buffers per slice: 128</div><div><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>Direct buffers: true </div></div><div class="gmail_extra"><br><div class="gmail_quote">2017-01-05 17:54 GMT-02:00 David Rush <span dir="ltr"><<a href="mailto:David.Rush@ihsmarkit.com" target="_blank">David.Rush@ihsmarkit.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div class="m_-937777452272892485WordSection1">
<p class="MsoNormal">Good afternoon,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I’m having issues with on a production API which has started to receive high volumes of traffic. At peak volume times I see many exceptions being thrown in the logs. The stack is indicating that an error occurred in the policy chain,
but when it tries to write the error to the response the connection has been closed. There seems to be a couple of flavors of IOException (broken pipe or stream closed from undertow). My plugin is calling chain.doApply(request) when it succeeds but the stack
trace is indicating there is then an exception being caught in doApply(Chain.java:153). I am on version 1.2.2-Final.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) java.io.IOException: UT010029: Stream is closed<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.undertow.servlet.spec.<wbr>ServletOutputStreamImpl.write(<wbr>ServletOutputStreamImpl.java:<wbr>136)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.undertow.servlet.spec.<wbr>ServletOutputStreamImpl.write(<wbr>ServletOutputStreamImpl.java:<wbr>128)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.platforms.<wbr>servlet.GatewayServlet$4.<wbr>write(GatewayServlet.java:406)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.platforms.<wbr>servlet.GatewayServlet$4.<wbr>write(GatewayServlet.java:395)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.impl.<wbr>DefaultPolicyErrorWriter.<wbr>write(<wbr>DefaultPolicyErrorWriter.java:<wbr>87)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.platforms.<wbr>servlet.GatewayServlet.<wbr>writeError(GatewayServlet.<wbr>java:392)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.platforms.<wbr>servlet.GatewayServlet$1.<wbr>handle(GatewayServlet.java:<wbr>210)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.platforms.<wbr>servlet.GatewayServlet$1.<wbr>handle(GatewayServlet.java:<wbr>157)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.impl.<wbr>ApiRequestExecutorImpl.lambda$<wbr>wrapResultHandler$0(<wbr>ApiRequestExecutorImpl.java:<wbr>159)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.impl.<wbr>ApiRequestExecutorImpl.lambda$<wbr>createPolicyErrorHandler$17(<wbr>ApiRequestExecutorImpl.java:<wbr>614)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;background:yellow">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.<wbr>policy.Chain.throwError(Chain.<wbr>java:249)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;background:yellow">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.<wbr>policy.Chain.doApply(Chain.<wbr>java:153)</span><span style="font-size:10.0pt;font-family:Consolas"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy.<wbr>doSuccess(PingFedOauthPolicy.<wbr>java:114)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy.<wbr>access$100(PingFedOauthPolicy.<wbr>java:38)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy$2.<wbr>handle(PingFedOauthPolicy.<wbr>java:193)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy$2.<wbr>handle(PingFedOauthPolicy.<wbr>java:174)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at io.apiman.gateway.platforms.<wbr>servlet.components.<wbr>HttpClientRequestImpl.end(<wbr>HttpClientRequestImpl.java:<wbr>140)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy.<wbr>retrieveAccessTokenFromPing(<wbr>PingFedOauthPolicy.java:232)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy.<wbr>retrieveAccessTokenFromPing(<wbr>PingFedOauthPolicy.java:174)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy.<wbr>doApply(PingFedOauthPolicy.<wbr>java:83)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.<wbr>apiman.plugins.pingfed_oauth_<wbr>policy.PingFedOauthPolicy.<wbr>doApply(PingFedOauthPolicy.<wbr>java:38)<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">My plugin is making either an external request to ElasticSearch or an HTTP request using IHttpClientComponent and then calling chain.doApply or chain.doFailure from within the IAsyncResultHandler handle method. Is there a problem with
that pattern?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Any help you can provide would be great.<u></u><u></u></p>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<table class="m_-937777452272892485MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="630" style="width:472.5pt;background:white">
<tbody>
<tr>
<td style="padding:0in 0in 0in 0in">
<p class="MsoNormal"><a href="https://ihsmarkit.com/" target="_blank"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";text-decoration:none"><img border="0" width="145" height="39" id="m_-937777452272892485logo" src="cid:image001.png@01D26752.C7702380" alt="IHS Markit"></span></a><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><u></u><u></u></span></p>
</td>
</tr>
<tr>
<td valign="bottom" style="padding:0in 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:8.5pt;font-family:"Verdana","sans-serif""> <span style="color:#404040">David Rush</span></span></b><span style="font-size:12.0pt;font-family:"Verdana","sans-serif";color:#404040">
</span><span style="font-size:12.0pt;font-family:"Verdana","sans-serif""><br>
</span><span style="font-size:8.5pt;font-family:"Verdana","sans-serif""> <span style="color:#595959">Director, API Development | Markit Digital
<br>
5775 Flatiron Parkway | Boulder, CO 80301</span><br>
<b><span style="color:#00b140">P:</span></b> <span style="color:#595959">+1 303 583 4244</span><span style="color:#4b4b4b"> </span><b><span style="color:#00b140">Main:</span></b><span style="color:#4b4b4b"> +1 303 417 9999</span><span style="color:#595959">
</span><br>
</span><a href="mailto:david.rush@ihsmarkit.com" target="_blank"><span style="font-size:8.5pt;font-family:"Verdana","sans-serif";color:#00b050;text-decoration:none">david.rush@ihsmarkit.com</span></a><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<br>
<hr>
<font face="Arial" color="Black" size="1"><br>
This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other
terms at the following link: <a href="http://www.markit.com/en/about/legal/email-disclaimer.page" target="_blank">http://www.markit.com/en/<wbr>about/legal/email-disclaimer.<wbr>page</a><br>
<br>
Please visit <a href="http://www.markit.com/en/about/contact/contact-us.page" target="_blank">http://www.markit.com/en/<wbr>about/contact/contact-us.page</a> for contact information on our offices worldwide.<br>
</font>
</div>
<br>______________________________<wbr>_________________<br>
Apiman-user mailing list<br>
<a href="mailto:Apiman-user@lists.jboss.org">Apiman-user@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/apiman-user" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/apiman-user</a><br>
<br></blockquote></div><br></div>