<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.m8025628769556365585gmail-apple-tab-span
{mso-style-name:m_8025628769556365585gmail-apple-tab-span;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The particular API having the problem is doing about a million hits an hour at peak, spread across a pool of 6 hosts. I think the majority of our problem stemmed
from the backend app we were calling from our custom plugin, which was itself queuing up requests. Increasing thread pool sizes in that app (and in addition Wildfly – thanks Harry!) seems to have resolved the issue for now. I think the slow requests are
causing connections to be timed out by either our front end load balancers or the client apps, thereby causing the stream errors. I am still curious as to how the exception manifests in the Chain.doApply method though. It might help to have that logged to
file in addition to being sent to the response, which is failing because the connection is dead.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">David<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Marc Savy [mailto:marc.savy@redhat.com]
<br>
<b>Sent:</b> Friday, January 06, 2017 7:17 AM<br>
<b>To:</b> Harry Trinta<br>
<b>Cc:</b> David Rush; apiman-user@lists.jboss.org<br>
<b>Subject:</b> Re: [Apiman-user] Stream closed / Broken Pipe issues with custom plugin<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi Harry & David,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Very interesting and glad to see a workaround/solution.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I wonder if it's a manifestation of this bug: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://issues.jboss.org/browse/UNDERTOW-849">https://issues.jboss.org/browse/UNDERTOW-849</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In the servlet implementation we use Undertow. I'll do some more research to see whether we can handle this more gracefully and in a more informative manner.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">May I ask what sort of load levels you were seeing that made the errors evident?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Marc<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 6 January 2017 at 03:51, Harry Trinta <<a href="mailto:harrytpc@gmail.com" target="_blank">harrytpc@gmail.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Hi, David!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I had exactly this problem. It is not relationed with the Apiman. The problem is with Wildfly. If your Wildfly is with default configuration, you have a limit of concurrent requests (Default "<i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Task
max threads" = </span></i>16 * vCPUs). So, when the Wildfly is working in the limit, it create a queue of requests.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Suggested settings for your wildfly (adjust according to your server):<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Configuration -> Subsystem -> IO<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">WORKER<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Io threads: 128<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Task max threads: 2048<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Stack size: 0<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Task keepalive: 120<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">BUFFER POLL<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Buffer size: 16384<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Buffers per slice: 128<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Direct buffers: true <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">2017-01-05 17:54 GMT-02:00 David Rush <<a href="mailto:David.Rush@ihsmarkit.com" target="_blank">David.Rush@ihsmarkit.com</a>>:<o:p></o:p></p>
</div>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Good afternoon,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I’m having issues with on a production API which has started to receive high volumes of traffic. At peak volume times I see many exceptions being thrown in the logs. The stack
is indicating that an error occurred in the policy chain, but when it tries to write the error to the response the connection has been closed. There seems to be a couple of flavors of IOException (broken pipe or stream closed from undertow). My plugin is
calling chain.doApply(request) when it succeeds but the stack trace is indicating there is then an exception being caught in doApply(Chain.java:153). I am on version 1.2.2-Final.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) java.io.IOException: UT010029: Stream is closed</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:136)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:128)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
<a href="http://io.apiman.gateway.platforms.se">io.apiman.gateway.platforms.se</a>rvlet.GatewayServlet$4.write(GatewayServlet.java:406)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
<a href="http://io.apiman.gateway.platforms.se">io.apiman.gateway.platforms.se</a>rvlet.GatewayServlet$4.write(GatewayServlet.java:395)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.impl.DefaultPolicyErrorWriter.write(DefaultPolicyErrorWriter.java:87)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
<a href="http://io.apiman.gateway.platforms.se">io.apiman.gateway.platforms.se</a>rvlet.GatewayServlet.writeError(GatewayServlet.java:392)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
<a href="http://io.apiman.gateway.platforms.se">io.apiman.gateway.platforms.se</a>rvlet.GatewayServlet$1.handle(GatewayServlet.java:210)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at
<a href="http://io.apiman.gateway.platforms.se">io.apiman.gateway.platforms.se</a>rvlet.GatewayServlet$1.handle(GatewayServlet.java:157)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$wrapResultHandler$0(ApiRequestExecutorImpl.java:159)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$createPolicyErrorHandler$17(ApiRequestExecutorImpl.java:614)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas;background:yellow">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.policy.Chain.throwError(Chain.java:249)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas;background:yellow">2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) at io.apiman.gateway.engine.policy.Chain.doApply(Chain.java:153)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.doSuccess(PingFedOauthPolicy.java:114)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.access$100(PingFedOauthPolicy.java:38)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy$2.handle(PingFedOauthPolicy.java:193)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy$2.handle(PingFedOauthPolicy.java:174)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at
<a href="http://io.apiman.gateway.platforms.se">io.apiman.gateway.platforms.se</a>rvlet.components.HttpClientRequestImpl.end(HttpClientRequestImpl.java:140)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.retrieveAccessTokenFromPing(PingFedOauthPolicy.java:232)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.retrieveAccessTokenFromPing(PingFedOauthPolicy.java:174)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.doApply(PingFedOauthPolicy.java:83)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:Consolas">2017-01-05 18:28:49,096 ERROR [stderr] (default task-17) at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.doApply(PingFedOauthPolicy.java:38)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">My plugin is making either an external request to ElasticSearch or an HTTP request using IHttpClientComponent and then calling chain.doApply or chain.doFailure from within the IAsyncResultHandler
handle method. Is there a problem with that pattern?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Any help you can provide would be great.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="630" style="width:472.5pt;background:white">
<tbody>
<tr>
<td style="padding:0in 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://ihsmarkit.com/" target="_blank"><span style="text-decoration:none"><img border="0" width="145" height="39" id="m_8025628769556365585m_-937777452272892485logo" src="cid:image001.png@01D26843.A24C1B80" alt="IHS Markit"></span></a><o:p></o:p></p>
</td>
</tr>
<tr>
<td valign="bottom" style="padding:0in 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:8.5pt;font-family:"Verdana","sans-serif""> <span style="color:#404040">David Rush</span></span></b><span style="font-family:"Verdana","sans-serif";color:#404040">
</span><span style="font-family:"Verdana","sans-serif""><br>
</span><span style="font-size:8.5pt;font-family:"Verdana","sans-serif""> <span style="color:#595959">Director, API Development | Markit Digital
<br>
5775 Flatiron Parkway | Boulder, CO 80301</span><br>
<b><span style="color:#00B140">P:</span></b> <span style="color:#595959"><a href="tel:(303)%20583-4244" target="_blank">+1 303 583 4244</a></span><span style="color:#4B4B4B"> </span><b><span style="color:#00B140">Main:</span></b><span style="color:#4B4B4B">
<a href="tel:(303)%20417-9999" target="_blank">+1 303 417 9999</a></span><span style="color:#595959">
</span><br>
</span><a href="mailto:david.rush@ihsmarkit.com" target="_blank"><span style="font-size:8.5pt;font-family:"Verdana","sans-serif";color:#00B050;text-decoration:none">david.rush@ihsmarkit.com</span></a><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:black"><br>
This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other
terms at the following link: <a href="http://www.markit.com/en/about/legal/email-disclaimer.page" target="_blank">
http://www.markit.com/en/about/legal/email-disclaimer.page</a><br>
<br>
Please visit <a href="http://www.markit.com/en/about/contact/contact-us.page" target="_blank">
http://www.markit.com/en/about/contact/contact-us.page</a> for contact information on our offices worldwide.</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">_______________________________________________<br>
Apiman-user mailing list<br>
<a href="mailto:Apiman-user@lists.jboss.org" target="_blank">Apiman-user@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/apiman-user" target="_blank">https://lists.jboss.org/mailman/listinfo/apiman-user</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Apiman-user mailing list<br>
<a href="mailto:Apiman-user@lists.jboss.org">Apiman-user@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/apiman-user" target="_blank">https://lists.jboss.org/mailman/listinfo/apiman-user</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<br>
<hr>
<font face="Arial" color="Black" size="1"><br>
This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other
terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page<br>
<br>
Please visit http://www.markit.com/en/about/contact/contact-us.page for contact information on our offices worldwide.<br>
</font>
</body>
</html>