<div dir="ltr">we put nginx upfront undertow and everything works as expected.<div><br></div><div>so we have: apache -> nginx -> undertow :S</div><div><br></div><div>apache can't be removed it and as I said before existing apps running on Tomcat/Jetty work without any issue</div><div><br></div><div>we need to move on with our project (wasted a lot of time on this) but I will try capture the traffic with wireshark and shared it.</div><div><br></div><div>still, if you find out something will be helpful too.</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 21, 2015 at 4:51 PM, Jason Greene <span dir="ltr"><<a href="mailto:jason.greene@redhat.com" target="_blank">jason.greene@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Would you be willing to capture the traffic with wireshark or (tcpdump using -s 65535) when the failure occurs? It would be helpful in diagnosing the exact reason the connection is lost (tcp, http protocol issue, etc).<br>
<br>
If you prefer not to post the dump on list, you can email us privately.<br>
<br>
Thanks!<br>
<div><div class="h5"><br>
<br>
> On Jan 21, 2015, at 1:40 PM, Edgar Espina <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>> wrote:<br>
><br>
> Nope, it doesn't help.<br>
><br>
> Stuart, you mentioned that:<br>
> "For a normal graceful close Undertow should be sending Connection:close headers."<br>
><br>
> Is it mean that on a req/res cycle, Undertow should always set the Connection response header with close or keep-alive?<br>
><br>
> Thanks<br>
><br>
><br>
><br>
><br>
> On Tue, Jan 20, 2015 at 5:07 PM, Stuart Douglas <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>> wrote:<br>
><br>
><br>
> ----- Original Message -----<br>
> > From: "Edgar Espina" <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > To: "Tomaž Cerar" <<a href="mailto:tomaz.cerar@gmail.com">tomaz.cerar@gmail.com</a>><br>
> > Cc: "Stuart Douglas" <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>>, <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > Sent: Wednesday, 21 January, 2015 2:03:10 AM<br>
> > Subject: Re: [undertow-dev] occasional 502 from Apache HTTP Proxy<br>
> ><br>
> > seems to be some sort of timeout, because everything work as expected, but<br>
> > then after waiting for a while and got again the 502.<br>
><br>
> That is odd, as there is no timeout configured. What does your network config between the LB and Undertow look like?<br>
><br>
> Does enabling TCP keep alive (Options.KEEP_ALIVE) help?<br>
><br>
> Stuart<br>
><br>
><br>
> ><br>
> > also, I did set the content-length for static resources and call<br>
> > .endExchange after a 304 response, set a backlog of 1000 and 10000 too. But<br>
> > it didn't help.<br>
> ><br>
> ><br>
> ><br>
> > On Mon, Jan 19, 2015 at 9:57 AM, Tomaž Cerar <<a href="mailto:tomaz.cerar@gmail.com">tomaz.cerar@gmail.com</a>> wrote:<br>
> ><br>
> > > Maybe related to<br>
> > > <a href="http://stackoverflow.com/questions/169453/bad-gateway-502-error-with-apache-mod-proxy-and-tomcat" target="_blank">http://stackoverflow.com/questions/169453/bad-gateway-502-error-with-apache-mod-proxy-and-tomcat</a><br>
> > ><br>
> > > or<br>
> > > <a href="http://qnalist.com/questions/4502641/users-httpd-mod-proxy-ignores-incompleteness-of-chunked-coding-response-from-backend" target="_blank">http://qnalist.com/questions/4502641/users-httpd-mod-proxy-ignores-incompleteness-of-chunked-coding-response-from-backend</a><br>
> > ><br>
> > > On Mon, Jan 19, 2015 at 1:33 PM, Edgar Espina <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > > wrote:<br>
> > ><br>
> > >> Sure, it is 2.2.15.<br>
> > >><br>
> > >><br>
> > >><br>
> > >> On Mon, Jan 19, 2015 at 9:29 AM, Tomaž Cerar <<a href="mailto:tomaz.cerar@gmail.com">tomaz.cerar@gmail.com</a>><br>
> > >> wrote:<br>
> > >><br>
> > >>> Edgar,<br>
> > >>><br>
> > >>> what is the version of Apache and mod_proxy module you are using.<br>
> > >>> maybe it will be easier to reproduce with exact version you have..<br>
> > >>><br>
> > >>> --<br>
> > >>> tomaz<br>
> > >>><br>
> > >>> On Mon, Jan 19, 2015 at 1:15 PM, Edgar Espina <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > >>> wrote:<br>
> > >>><br>
> > >>>> The source code can be found here:<br>
> > >>>> <a href="https://github.com/jooby-project/jooby" target="_blank">https://github.com/jooby-project/jooby</a><br>
> > >>>><br>
> > >>>> Undertow related classes are here:<br>
> > >>>> <a href="https://github.com/jooby-project/jooby/tree/master/jooby/src/main/java/org/jooby/internal/undertow" target="_blank">https://github.com/jooby-project/jooby/tree/master/jooby/src/main/java/org/jooby/internal/undertow</a><br>
> > >>>> .<br>
> > >>>> Server is built here:<br>
> > >>>> <a href="https://github.com/jooby-project/jooby/blob/master/jooby/src/main/java/org/jooby/internal/undertow/UndertowServer.java" target="_blank">https://github.com/jooby-project/jooby/blob/master/jooby/src/main/java/org/jooby/internal/undertow/UndertowServer.java</a><br>
> > >>>> Response is sent here:<br>
> > >>>> <a href="https://github.com/jooby-project/jooby/blob/master/jooby/src/main/java/org/jooby/internal/undertow/UndertowResponse.java#L336-382" target="_blank">https://github.com/jooby-project/jooby/blob/master/jooby/src/main/java/org/jooby/internal/undertow/UndertowResponse.java#L336-382</a><br>
> > >>>><br>
> > >>>> My microweb-framework was built on top of Jetty, so for now I used the<br>
> > >>>> blocking API and follow more or less what we usually do with Servlets<br>
> > >>>> (acquire an outstream).<br>
> > >>>><br>
> > >>>> I will review what I'm doing and try to figure it out what is going on<br>
> > >>>> too.<br>
> > >>>><br>
> > >>>> Thanks for your help, Stuart.<br>
> > >>>><br>
> > >>>><br>
> > >>>><br>
> > >>>><br>
> > >>>> On Mon, Jan 19, 2015 at 2:12 AM, Stuart Douglas <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>> wrote:<br>
> > >>>><br>
> > >>>>> My best guess as to what is happening is that Undertow closing a<br>
> > >>>>> connection after a request is done for some reason, and the apache<br>
> > >>>>> attempts<br>
> > >>>>> to re-use this connection without realising that it is dead.<br>
> > >>>>><br>
> > >>>>> In general this should not happen, Undertow should only forcibly close<br>
> > >>>>> a connection if it knows that it is broken (e.g. a content length is<br>
> > >>>>> set<br>
> > >>>>> and the full amount of content is not written). For a normal graceful<br>
> > >>>>> close<br>
> > >>>>> Undertow should be sending Connection:close headers.<br>
> > >>>>><br>
> > >>>>> I am going to investigate some more, and see if I can figure out what<br>
> > >>>>> is going on. It seems unlikely but is there any chance your code<br>
> > >>>>> forcibly<br>
> > >>>>> closes the ServerConnection (HttpServerExchange.getConnection())<br>
> > >>>>> because<br>
> > >>>>> that could potentially cause this issue. Setting the exchange to<br>
> > >>>>> non-persistent after headers have been sent could also cause it.<br>
> > >>>>><br>
> > >>>>> Stuart<br>
> > >>>>><br>
> > >>>>> ----- Original Message -----<br>
> > >>>>> > From: "Edgar Espina" <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > >>>>> > To: "Stuart Douglas" <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > Cc: <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > >>>>> > Sent: Monday, 19 January, 2015 2:53:06 PM<br>
> > >>>>> > Subject: Re: [undertow-dev] occasional 502 from Apache HTTP Proxy<br>
> > >>>>> ><br>
> > >>>>> > Done, but makes no difference. Still got the 502 random errors :S<br>
> > >>>>> ><br>
> > >>>>> > On Sun, Jan 18, 2015 at 11:13 PM, Stuart Douglas <<br>
> > >>>>> <a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > wrote:<br>
> > >>>>> ><br>
> > >>>>> > > I had a play around with apache locally, and I could reproduce<br>
> > >>>>> this in<br>
> > >>>>> > > some circumstances, and it looks like we are not setting a high<br>
> > >>>>> enough<br>
> > >>>>> > > backlog by default.<br>
> > >>>>> > ><br>
> > >>>>> > > Can you try adding:<br>
> > >>>>> > ><br>
> > >>>>> > > undertow.setSocketOption(Options.BACKLOG, 1000)<br>
> > >>>>> > ><br>
> > >>>>> > > To your Undertow builder? I am going to increase this in our<br>
> > >>>>> default<br>
> > >>>>> > > config upstream.<br>
> > >>>>> > ><br>
> > >>>>> > > Hopefully this is the issue that you are running into.<br>
> > >>>>> > ><br>
> > >>>>> > > Stuart<br>
> > >>>>> > ><br>
> > >>>>> > ><br>
> > >>>>> > > ----- Original Message -----<br>
> > >>>>> > > > From: "Edgar Espina" <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > >>>>> > > > To: "Stuart Douglas" <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > > > Cc: <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > >>>>> > > > Sent: Monday, 19 January, 2015 12:10:16 PM<br>
> > >>>>> > > > Subject: Re: [undertow-dev] occasional 502 from Apache HTTP Proxy<br>
> > >>>>> > > ><br>
> > >>>>> > > > It is a default undertow instance with a HTTP listener, just set<br>
> > >>>>> work<br>
> > >>>>> > > > threads to 200. Is there a default idle timeout? I can' tell<br>
> > >>>>> from javadoc<br>
> > >>>>> > > > and looking at the code.<br>
> > >>>>> > > ><br>
> > >>>>> > > ><br>
> > >>>>> > > > On Sun, Jan 18, 2015 at 9:59 PM, Stuart Douglas <<br>
> > >>>>> <a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > > wrote:<br>
> > >>>>> > > ><br>
> > >>>>> > > > ><br>
> > >>>>> > > > ><br>
> > >>>>> > > > > Edgar Espina wrote:<br>
> > >>>>> > > > ><br>
> > >>>>> > > > >> I'm not, channel does it.<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> Problem is present on both. For example, home page has a<br>
> > >>>>> > > content-length<br>
> > >>>>> > > > >> header but jquery.js use chunked.<br>
> > >>>>> > > > >><br>
> > >>>>> > > > ><br>
> > >>>>> > > > > In general if you are serving static resources you are better<br>
> > >>>>> off<br>
> > >>>>> > > setting<br>
> > >>>>> > > > > the content length (slightly more efficient, and the browser<br>
> > >>>>> can<br>
> > >>>>> > > display<br>
> > >>>>> > > > > progress on downloads).<br>
> > >>>>> > > > ><br>
> > >>>>> > > > ><br>
> > >>>>> > > > >> Do I have to call .endExchange? or closing the output stream<br>
> > >>>>> is<br>
> > >>>>> > > enough?<br>
> > >>>>> > > > >><br>
> > >>>>> > > > ><br>
> > >>>>> > > > > Closing the output stream is enough. endExchange is<br>
> > >>>>> automatically<br>
> > >>>>> > > called<br>
> > >>>>> > > > > once the call stack returns anyway, unless you have dispatched<br>
> > >>>>> the<br>
> > >>>>> > > exchange<br>
> > >>>>> > > > > or started async IO.<br>
> > >>>>> > > > ><br>
> > >>>>> > > > > The 'connection reset by peer' error in the apache log while<br>
> > >>>>> reading<br>
> > >>>>> > > the<br>
> > >>>>> > > > > status like kinda indicates that the request does not even get<br>
> > >>>>> to this<br>
> > >>>>> > > > > point anyway, and the underlying TCP connection is probably<br>
> > >>>>> being torn<br>
> > >>>>> > > down<br>
> > >>>>> > > > > somehow.<br>
> > >>>>> > > > ><br>
> > >>>>> > > > > Do you have any kind of timeouts set? If you have an idle<br>
> > >>>>> timeout set<br>
> > >>>>> > > on<br>
> > >>>>> > > > > the listener there is a race where Undertow can close the<br>
> > >>>>> channel due<br>
> > >>>>> > > to<br>
> > >>>>> > > > > inactivity just as the front end starts to send a request.<br>
> > >>>>> > > > ><br>
> > >>>>> > > > > Stuart<br>
> > >>>>> > > > ><br>
> > >>>>> > > > ><br>
> > >>>>> > > > >> Thanks<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> On Sun, Jan 18, 2015 at 9:48 PM, Stuart Douglas <<br>
> > >>>>> <a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>>> wrote:<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> Another question, are you setting a content length on the<br>
> > >>>>> > > responses?<br>
> > >>>>> > > > >> If not the channel will automatically set one if the<br>
> > >>>>> response fits<br>
> > >>>>> > > > >> inside a buffer, otherwise chunked encoding will be used.<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> It might be helpful to know if this only happens on<br>
> > >>>>> chunked, fixed<br>
> > >>>>> > > > >> length or both.<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> Stuart<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> Edgar Espina wrote:<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> latest: 1.2.0.Beta8<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> On Sun, Jan 18, 2015 at 9:42 PM, Stuart Douglas<br>
> > >>>>> > > > >> <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a> <mailto:<br>
> > >>>>> <a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>>>><br>
> > >>>>> > > wrote:<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> Also what version of Undertow are you using?<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> Stuart<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> ----- Original Message -----<br>
> > >>>>> > > > >> > From: "Edgar Espina" <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a> <mailto:<br>
> > >>>>> <a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a><br>
> > >>>>> > > > >> >__>><br>
> > >>>>> > > > >> > To: "Stuart Douglas" <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a> <mailto:<br>
> > >>>>> <a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>>>><br>
> > >>>>> > > > >> > Cc: <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.">undertow-dev@lists.</a>__<a href="http://jboss.org" target="_blank">jboss.org</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a>>><br>
> > >>>>> > > > >> > Sent: Monday, 19 January, 2015 11:13:21 AM<br>
> > >>>>> > > > >> > Subject: Re: [undertow-dev] occasional 502 from<br>
> > >>>>> Apache HTTP<br>
> > >>>>> > > > >> Proxy<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > found this in apache:<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > (104) Connection reset by peer: proxy: error<br>
> > >>>>> reading status<br>
> > >>>>> > > > >> line from<br>
> > >>>>> > > > >> > remote server<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > but nothing in undertow.<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > got 502 on HTTP GET. Sometimes while calling the<br>
> > >>>>> home page<br>
> > >>>>> > > / or<br>
> > >>>>> > > > >> when page<br>
> > >>>>> > > > >> > loads OK, got 502 on page resources (js, css,<br>
> > >>>>> images).<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > I call startBlocking and use the outputstream to<br>
> > >>>>> write the<br>
> > >>>>> > > > >> response, when<br>
> > >>>>> > > > >> > everything has been written I call the<br>
> > >>>>> "outputstream.close"<br>
> > >>>>> > > > >> method. Do I<br>
> > >>>>> > > > >> > need to call exchange.endExchange too?<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > Thanks<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > On Sun, Jan 18, 2015 at 8:52 PM, Stuart Douglas<br>
> > >>>>> > > > >> <<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a> <mailto:<br>
> > >>>>> <a href="mailto:sdouglas@redhat.com">sdouglas@redhat.com</a>>>><br>
> > >>>>> > > wrote:<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > > Is there any info in the log? Or is there any<br>
> > >>>>> specific<br>
> > >>>>> > > type<br>
> > >>>>> > > > >> of<br>
> > >>>>> > > > >> request<br>
> > >>>>> > > > >> > > that causes this?<br>
> > >>>>> > > > >> > ><br>
> > >>>>> > > > >> > > Stuart<br>
> > >>>>> > > > >> > ><br>
> > >>>>> > > > >> > ><br>
> > >>>>> > > > >> > > ----- Original Message -----<br>
> > >>>>> > > > >> > > > From: "Edgar Espina" <<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a> <mailto:<br>
> > >>>>> <a href="mailto:espina.edgar@gmail.com">espina.edgar@gmail.com</a><br>
> > >>>>> > > > >> >__>><br>
> > >>>>> > > > >> > > > To: <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.">undertow-dev@lists.</a>__<a href="http://jboss.org" target="_blank">jboss.org</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a>>><br>
> > >>>>> > > > >> > > > Sent: Monday, 19 January, 2015 9:42:19 AM<br>
> > >>>>> > > > >> > > > Subject: [undertow-dev] occasional 502 from<br>
> > >>>>> Apache HTTP<br>
> > >>>>> > > > >> Proxy<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > Hi,<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > I've an Undertow application behind apache<br>
> > >>>>> reverse<br>
> > >>>>> > > proxy,<br>
> > >>>>> > > > >> trying to load<br>
> > >>>>> > > > >> > > a<br>
> > >>>>> > > > >> > > > page displays error 502 proxy error.<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > Still couldn't find why so I wonder if any of<br>
> > >>>>> you find<br>
> > >>>>> > > a<br>
> > >>>>> > > > >> similar problem<br>
> > >>>>> > > > >> > > with<br>
> > >>>>> > > > >> > > > Undertow and Apache.<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > Please note this is our first app on top of<br>
> > >>>>> Undertow,<br>
> > >>>>> > > > >> existing apps<br>
> > >>>>> > > > >> > > running<br>
> > >>>>> > > > >> > > > on Tomcat/Jetty are OK.<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > Appreciate any help.<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > Thanks<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > > --<br>
> > >>>>> > > > >> > > > edgar<br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > > >> > > ><br>
> > >>>>> _________________________________________________<br>
> > >>>>> > > > >> > > > undertow-dev mailing list<br>
> > >>>>> > > > >> > > > <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a>><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.">undertow-dev@lists.</a>__<a href="http://jboss.org" target="_blank">jboss.org</a><br>
> > >>>>> > > > >> <mailto:<a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a>>><br>
> > >>>>> > > > >> > > ><br>
> > >>>>> > > <a href="https://lists.jboss.org/__mailman/listinfo/undertow-dev" target="_blank">https://lists.jboss.org/__mailman/listinfo/undertow-dev</a><br>
> > >>>>> > > > >> <<br>
> > >>>>> <a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a>><br>
> > >>>>> > > > >> > ><br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >> > --<br>
> > >>>>> > > > >> > edgar<br>
> > >>>>> > > > >> ><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> --<br>
> > >>>>> > > > >> edgar<br>
> > >>>>> > > > >><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >><br>
> > >>>>> > > > >> --<br>
> > >>>>> > > > >> edgar<br>
> > >>>>> > > > >><br>
> > >>>>> > > > ><br>
> > >>>>> > > ><br>
> > >>>>> > > ><br>
> > >>>>> > > > --<br>
> > >>>>> > > > edgar<br>
> > >>>>> > > ><br>
> > >>>>> > ><br>
> > >>>>> ><br>
> > >>>>> ><br>
> > >>>>> ><br>
> > >>>>> > --<br>
> > >>>>> > edgar<br>
> > >>>>> ><br>
> > >>>>><br>
> > >>>><br>
> > >>>><br>
> > >>>><br>
> > >>>> --<br>
> > >>>> edgar<br>
> > >>>><br>
> > >>>> _______________________________________________<br>
> > >>>> undertow-dev mailing list<br>
> > >>>> <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> > >>>> <a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a><br>
> > >>>><br>
> > >>><br>
> > >>><br>
> > >><br>
> > >><br>
> > >> --<br>
> > >> edgar<br>
> > >><br>
> > ><br>
> > ><br>
> ><br>
> ><br>
> > --<br>
> > edgar<br>
> ><br>
><br>
><br>
><br>
> --<br>
> edgar<br>
> _______________________________________________<br>
> undertow-dev mailing list<br>
> <a href="mailto:undertow-dev@lists.jboss.org">undertow-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/undertow-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/undertow-dev</a><br>
<br>
--<br>
</div></div>Jason T. Greene<br>
WildFly Lead / JBoss EAP Platform Architect<br>
JBoss, a division of Red Hat<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">edgar</div>
</div>