8:16 a.m.
Hi Harry & David,
Very interesting and glad to see a workaround/solution.
I wonder if it's a manifestation of this bug:
https://issues.jboss.org/browse/UNDERTOW-849
In the servlet implementation we use Undertow. I'll do some more research
to see whether we can handle this more gracefully and in a more informative
manner.
May I ask what sort of load levels you were seeing that made the errors
evident?
Regards,
Marc
On 6 January 2017 at 03:51, Harry Trinta <harrytpc(a)gmail.com> wrote:
Hi, David!
I had exactly this problem. It is not relationed with the Apiman. The
problem is with Wildfly. If your Wildfly is with default configuration, you
have a limit of concurrent requests (Default "*Task max threads" = *16 *
vCPUs). So, when the Wildfly is working in the limit, it create a queue of
requests.
Suggested settings for your wildfly (adjust according to your server):
Configuration -> Subsystem -> IO
WORKER
Io threads: 128
Task max threads: 2048
Stack size: 0
Task keepalive: 120
BUFFER POLL
Buffer size: 16384
Buffers per slice: 128
Direct buffers: true
2017-01-05 17:54 GMT-02:00 David Rush <David.Rush(a)ihsmarkit.com>:
> Good afternoon,
>
>
>
> I’m having issues with on a production API which has started to receive
> high volumes of traffic. At peak volume times I see many exceptions being
> thrown in the logs. The stack is indicating that an error occurred in the
> policy chain, but when it tries to write the error to the response the
> connection has been closed. There seems to be a couple of flavors of
> IOException (broken pipe or stream closed from undertow). My plugin is
> calling chain.doApply(request) when it succeeds but the stack trace is
> indicating there is then an exception being caught in
> doApply(Chain.java:153). I am on version 1.2.2-Final.
>
>
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)
> java.io.IOException: UT010029: Stream is closed
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.undertow.servlet.spec.ServletOutputStreamImpl.write(Servl
> etOutputStreamImpl.java:136)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.undertow.servlet.spec.ServletOutputStreamImpl.write(Servl
> etOutputStreamImpl.java:128)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.platforms.servlet.GatewayServlet$4.write(
> GatewayServlet.java:406)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.platforms.servlet.GatewayServlet$4.write(
> GatewayServlet.java:395)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.engine.impl.DefaultPolicyErrorWriter.write
> (DefaultPolicyErrorWriter.java:87)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.platforms.servlet.GatewayServlet.writeErro
> r(GatewayServlet.java:392)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.platforms.servlet.GatewayServlet$1.handle(
> GatewayServlet.java:210)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.platforms.servlet.GatewayServlet$1.handle(
> GatewayServlet.java:157)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$
> wrapResultHandler$0(ApiRequestExecutorImpl.java:159)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$
> createPolicyErrorHandler$17(ApiRequestExecutorImpl.java:614)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.engine.policy.Chain.throwError(Chain.java:249)
>
> 2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
> io.apiman.gateway.engine.policy.Chain.doApply(Chain.java:153)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy.doSuccess(PingFedOauthPolicy.java:114)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy.access$100(PingFedOauthPolicy.java:38)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy$2.handle(PingFedOauthPolicy.java:193)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy$2.handle(PingFedOauthPolicy.java:174)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> io.apiman.gateway.platforms.servlet.components.HttpClientReq
> uestImpl.end(HttpClientRequestImpl.java:140)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy.retrieveAccessTokenFromPing(PingFedOauth
> Policy.java:232)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy.retrieveAccessTokenFromPing(PingFedOauth
> Policy.java:174)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy.doApply(PingFedOauthPolicy.java:83)
>
> 2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
> com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy
> .PingFedOauthPolicy.doApply(PingFedOauthPolicy.java:38)
>
>
>
> My plugin is making either an external request to ElasticSearch or an
> HTTP request using IHttpClientComponent and then calling chain.doApply or
> chain.doFailure from within the IAsyncResultHandler handle method. Is
> there a problem with that pattern?
>
>
>
> Any help you can provide would be great.
>
> Thanks,
>
>
>
> [image: IHS Markit] <https://ihsmarkit.com/>
>
> * David Rush*
> Director, API Development | Markit Digital
> 5775 Flatiron Parkway | Boulder, CO 80301
> *P:* +1 303 583 4244 <(303)%20583-4244> *Main:* +1 303 417 9999
> <(303)%20417-9999>
> david.rush(a)ihsmarkit.com
>
>
>
>
>
>
>
> ------------------------------
>
> This e-mail, including accompanying communications and attachments, is
> strictly confidential and only for the intended recipient. Any retention,
> use or disclosure not expressly authorised by Markit is prohibited. This
> email is subject to all waivers and other terms at the following link:
> http://www.markit.com/en/about/legal/email-disclaimer.page
>
> Please visit http://www.markit.com/en/about/contact/contact-us.page for
> contact information on our offices worldwide.
>
> _______________________________________________
> Apiman-user mailing list
> Apiman-user(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/apiman-user
>
>
_______________________________________________
Apiman-user mailing list
Apiman-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/apiman-user
