From Brian.Wallis at infomedix.com.au Fri Aug 3 05:17:31 2018 From: Brian.Wallis at infomedix.com.au (Brian R Wallis) Date: Fri, 3 Aug 2018 09:17:31 +0000 Subject: [undertow-dev] comma separators in cookie headers Message-ID: <9C76865C-19CA-4F35-AB6E-157900694160@infomedix.com.au> Are commas allowed as a separator of multiple cookies in a cookie header? I am running an application in wildfly-11.0.0.Final and another application is making a request with two cookies, JSESSIONID and JSESSIONIDSSO. It is sending these as JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID=lDA5h47Pk_jrnIwAshNsQ7Ot269XyVSTR1mwYNEL.localhost which then seems to be parsed into a single cookie JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID which of course does not work for the authentication so the request fails. This seems to be a failure in parsing the original cookie string. There is a bit of confusion in this area in the RFCs as the earlier ones allowed comma as a separator but the most recent, RFC-6265, does not. Undertow should probably allow a comma separator for backward compatibility with older implementations. Thanks brian wallis? The full dump from the undertow request dumper is 18:33:29,249 INFO [io.undertow.request.dump] (Unknown) ----------------------------REQUEST--------------------------- URI=/infoapi/user/profile characterEncoding=null contentLength=-1 contentType=[none] cookie=JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID header=Connection=Keep-Alive header=Orbeon-Token=5b4085e06896f374e8dec7a22f9e411a2b0d2105 header=Accept-Encoding=gzip,deflate header=Content-Type=none header=Cookie=JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID=lDA5h47Pk_jrnIwAshNsQ7Ot269XyVSTR1mwYNEL.localhost header=Cookie2=$Version=1 header=Host=localhost locale=[] method=GET protocol=HTTP/1.1 queryString= remoteAddr=/127.0.0.1:55984 remoteHost=localhost scheme=http host=localhost serverPort=80 --------------------------RESPONSE-------------------------- contentLength=71 contentType=text/html;charset=UTF-8 cookie=JSESSIONIDSSO=null; domain=null; path=/ header=Expires=0 header=Cache-Control=no-cache, no-store, must-revalidate header=X-Powered-By=Undertow/1 header=Set-Cookie=JSESSIONIDSSO=""; path=/; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:00 GMT header=Server=WildFly/11 header=Pragma=no-cache header=Date=Fri, 03 Aug 2018 08:33:29 GMT header=WWW-Authenticate=Basic realm="REST API authentication module" header=Content-Type=text/html;charset=UTF-8 header=Content-Length=71 status=401 From mmiura at redhat.com Fri Aug 3 06:14:03 2018 From: mmiura at redhat.com (Masafumi Miura) Date: Fri, 3 Aug 2018 19:14:03 +0900 Subject: [undertow-dev] comma separators in cookie headers In-Reply-To: <9C76865C-19CA-4F35-AB6E-157900694160@infomedix.com.au> References: <9C76865C-19CA-4F35-AB6E-157900694160@infomedix.com.au> Message-ID: Hi, This was already reported at https://issues.jboss.org/browse/UNDERTOW-1163 If you upgrade to WildFly 12 or later, you can allow a comma as cookie separator by setting the system property "io.undertow.legacy.cookie.COMMA_IS_SEPARATOR" to true: ./bin/standalone.sh ... -Dio.undertow.legacy.cookie.COMMA_IS_SEPARATOR=true Thanks, -- Masafumi Miura / ?? ?? On Fri, Aug 3, 2018 at 6:17 PM, Brian R Wallis < Brian.Wallis at infomedix.com.au> wrote: > Are commas allowed as a separator of multiple cookies in a cookie header? > > I am running an application in wildfly-11.0.0.Final and another application is making a request with two cookies, JSESSIONID and JSESSIONIDSSO. It is sending these as > > JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID=lDA5h47Pk_jrnIwAshNsQ7Ot269XyVSTR1mwYNEL.localhost > > which then seems to be parsed into a single cookie > > JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID > > which of course does not work for the authentication so the request fails. This seems to be a failure in parsing the original cookie string. There is a bit of confusion in this area in the RFCs as the earlier ones allowed comma as a separator but the most recent, RFC-6265, does not. Undertow should probably allow a comma separator for backward compatibility with older implementations. > > Thanks > brian wallis? > > > The full dump from the undertow request dumper is > > > 18:33:29,249 INFO [io.undertow.request.dump] (Unknown) > ----------------------------REQUEST--------------------------- > URI=/infoapi/user/profile > characterEncoding=null > contentLength=-1 > contentType=[none] > cookie=JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID > header=Connection=Keep-Alive > header=Orbeon-Token=5b4085e06896f374e8dec7a22f9e411a2b0d2105 > header=Accept-Encoding=gzip,deflate > header=Content-Type=none > header=Cookie=JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID=lDA5h47Pk_jrnIwAshNsQ7Ot269XyVSTR1mwYNEL.localhost > header=Cookie2=$Version=1 > header=Host=localhost > locale=[] > method=GET > protocol=HTTP/1.1 > queryString= > remoteAddr=/127.0.0.1:55984 > remoteHost=localhost > scheme=http > host=localhost > serverPort=80 > --------------------------RESPONSE-------------------------- > contentLength=71 > contentType=text/html;charset=UTF-8 > cookie=JSESSIONIDSSO=null; domain=null; path=/ > header=Expires=0 > header=Cache-Control=no-cache, no-store, must-revalidate > header=X-Powered-By=Undertow/1 > header=Set-Cookie=JSESSIONIDSSO=""; path=/; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:00 GMT > header=Server=WildFly/11 > header=Pragma=no-cache > header=Date=Fri, 03 Aug 2018 08:33:29 GMT > header=WWW-Authenticate=Basic realm="REST API authentication module" > header=Content-Type=text/html;charset=UTF-8 > header=Content-Length=71 > status=401 > > > > > _______________________________________________ > undertow-dev mailing list > undertow-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/undertow-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180803/4e07fc7e/attachment.html From Brian.Wallis at infomedix.com.au Sat Aug 4 00:03:17 2018 From: Brian.Wallis at infomedix.com.au (Brian R Wallis) Date: Sat, 4 Aug 2018 04:03:17 +0000 Subject: [undertow-dev] comma separators in cookie headers In-Reply-To: References: <9C76865C-19CA-4F35-AB6E-157900694160@infomedix.com.au> Message-ID: <6CCA0098-D958-44E0-8AF2-A8CD0CA0580D@infomedix.com.au> Thanks for that. I did do a search but didn?t manage to find that ticket, better polish up my search skills :-) brian? On 3 Aug 2018, at 8:14 PM, Masafumi Miura > wrote: Hi, This was already reported at https://issues.jboss.org/browse/UNDERTOW-1163 If you upgrade to WildFly 12 or later, you can allow a comma as cookie separator by setting the system property "io.undertow.legacy.cookie.COMMA_IS_SEPARATOR" to true: ./bin/standalone.sh ... -Dio.undertow.legacy.cookie.COMMA_IS_SEPARATOR=true Thanks, -- Masafumi Miura / ?? ?? On Fri, Aug 3, 2018 at 6:17 PM, Brian R Wallis > wrote: > Are commas allowed as a separator of multiple cookies in a cookie header? > > I am running an application in wildfly-11.0.0.Final and another application is making a request with two cookies, JSESSIONID and JSESSIONIDSSO. It is sending these as > > JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID=lDA5h47Pk_jrnIwAshNsQ7Ot269XyVSTR1mwYNEL.localhost > > which then seems to be parsed into a single cookie > > JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID > > which of course does not work for the authentication so the request fails. This seems to be a failure in parsing the original cookie string. There is a bit of confusion in this area in the RFCs as the earlier ones allowed comma as a separator but the most recent, RFC-6265, does not. Undertow should probably allow a comma separator for backward compatibility with older implementations. > > Thanks > brian wallis? > > > The full dump from the undertow request dumper is > > > 18:33:29,249 INFO [io.undertow.request.dump] (Unknown) > ----------------------------REQUEST--------------------------- > URI=/infoapi/user/profile > characterEncoding=null > contentLength=-1 > contentType=[none] > cookie=JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID > header=Connection=Keep-Alive > header=Orbeon-Token=5b4085e06896f374e8dec7a22f9e411a2b0d2105 > header=Accept-Encoding=gzip,deflate > header=Content-Type=none > header=Cookie=JSESSIONIDSSO=jIEqQ-kTedwXrvqm9CBACBg8QlCXzJKILwCftnaV, JSESSIONID=lDA5h47Pk_jrnIwAshNsQ7Ot269XyVSTR1mwYNEL.localhost > header=Cookie2=$Version=1 > header=Host=localhost > locale=[] > method=GET > protocol=HTTP/1.1 > queryString= > remoteAddr=/127.0.0.1:55984 > remoteHost=localhost > scheme=http > host=localhost > serverPort=80 > --------------------------RESPONSE-------------------------- > contentLength=71 > contentType=text/html;charset=UTF-8 > cookie=JSESSIONIDSSO=null; domain=null; path=/ > header=Expires=0 > header=Cache-Control=no-cache, no-store, must-revalidate > header=X-Powered-By=Undertow/1 > header=Set-Cookie=JSESSIONIDSSO=""; path=/; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:00 GMT > header=Server=WildFly/11 > header=Pragma=no-cache > header=Date=Fri, 03 Aug 2018 08:33:29 GMT > header=WWW-Authenticate=Basic realm="REST API authentication module" > header=Content-Type=text/html;charset=UTF-8 > header=Content-Length=71 > status=401 > > > > > _______________________________________________ > undertow-dev mailing list > undertow-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/undertow-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180804/e88684e7/attachment-0001.html From bdw429s at gmail.com Sat Aug 4 17:48:08 2018 From: bdw429s at gmail.com (Brad Wood) Date: Sat, 4 Aug 2018 16:48:08 -0500 Subject: [undertow-dev] Access control examples Message-ID: Hi, I'm looking for some examples of locking down access to certain directories, similar to how IIS has "hidden segments". For instance, I'd like all URLs starting with /CFIDE to be blocked, or perhaps only access to a certain range of IPs I swear I had looked at some examples of this about a year ago, but after quite a lot of Googling today I was coming up empty handed. I found some basic information on the access control handlers, but couldn't find a single example of using them. Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad at coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180804/dffd91f9/attachment.html From bdw429s at gmail.com Fri Aug 10 01:53:03 2018 From: bdw429s at gmail.com (Brad Wood) Date: Fri, 10 Aug 2018 00:53:03 -0500 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: Anyone? Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad at coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: > Hi, I'm looking for some examples of locking down access to certain > directories, similar to how IIS has "hidden segments". For instance, I'd > like all URLs starting with /CFIDE to be blocked, or perhaps only access to > a certain range of IPs > > I swear I had looked at some examples of this about a year ago, but after > quite a lot of Googling today I was coming up empty handed. I found some > basic information on the access control handlers, but couldn't find a > single example of using them. > > Thanks! > > ~Brad > > *Developer Advocate* > *Ortus Solutions, Corp * > > E-mail: brad at coldbox.org > ColdBox Platform: http://www.coldbox.org > Blog: http://www.codersrevolution.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180810/e237c3bc/attachment.html From sdouglas at redhat.com Fri Aug 10 02:47:40 2018 From: sdouglas at redhat.com (Stuart Douglas) Date: Fri, 10 Aug 2018 16:47:40 +1000 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: It depenends a bit on what you want to do. If you just want to block /CFIDE you can just use a PredicateHandler, with a PathPrefixPredicate, and if it matches use ResponseCodeHandler to return the desired response code. You could combine it with io.undertow.server.handlers.AccessControlListHandler or io.undertow.server.handlers.IPAddressAccessControlHandler if you want to limit the IP range. Basically you just use a predicate to decide what you want to restrict, and then map it to a handler that either rejects the request outright or performs an access control check. Stuart On Fri, Aug 10, 2018 at 3:59 PM Brad Wood wrote: > Anyone? > > Thanks! > > ~Brad > > *Developer Advocate* > *Ortus Solutions, Corp * > > E-mail: brad at coldbox.org > ColdBox Platform: http://www.coldbox.org > Blog: http://www.codersrevolution.com > > > > On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: > >> Hi, I'm looking for some examples of locking down access to certain >> directories, similar to how IIS has "hidden segments". For instance, I'd >> like all URLs starting with /CFIDE to be blocked, or perhaps only access to >> a certain range of IPs >> >> I swear I had looked at some examples of this about a year ago, but after >> quite a lot of Googling today I was coming up empty handed. I found some >> basic information on the access control handlers, but couldn't find a >> single example of using them. >> >> Thanks! >> >> ~Brad >> >> *Developer Advocate* >> *Ortus Solutions, Corp * >> >> E-mail: brad at coldbox.org >> ColdBox Platform: http://www.coldbox.org >> Blog: http://www.codersrevolution.com >> >> _______________________________________________ > undertow-dev mailing list > undertow-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/undertow-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180810/721400f9/attachment.html From bdw429s at gmail.com Fri Aug 10 11:24:56 2018 From: bdw429s at gmail.com (Brad Wood) Date: Fri, 10 Aug 2018 10:24:56 -0500 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: > > It depenends a bit on what you want to do. Thanks for the reply Stuart. Honestly, I'm just brainstorming a little here to see what's possible but I just couldn't find any docs or examples to help solidify what was out in there. My primary use for this as I explained just now in a separate reply is to be able to add some security rules to CommandBox servers to do things such as: - Block access to CF admins in the root (such as paths starting with */CFIDE*) - Block access to special files in any directory such as *box.json*, *server.json*, or *.cfconfig.json* - Block access to hidden files in any directory (starting with a period ) - Block access to custom folders defined by the user such as */tests/* or */workbench* I'm thinking a bit how the IIS "hidden segments" feature works. In addition to using this behind the scenes in CommandBox, I'd like to expose it to my users in the *server.json * so they can configure basic access control. I generally don't expose 100% of what Undertow does since CommandBox aims to be a drop-in dead-easy way to just fire up a server, but I'm interested in the IP matching since that could be a common use case. i.e., "Block access to the administrator unless the IP is in this range or localhost" So basically, yes, I'm interested in all of those things and I don't have a super specific solution in mind, but I'm rather just looking for some better examples to help me understand what's there and what I can best expose in CommandBox. Basically you just use a predicate to decide what you want to restrict, and > then map it to a handler that either rejects the request outright or > performs an access control check. This makes sense and I think the predicate part was what I was missing, but are there examples of this anywhere? It helps me way more to see some code. Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad at coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com On Fri, Aug 10, 2018 at 1:47 AM Stuart Douglas wrote: > It depenends a bit on what you want to do. > > If you just want to block /CFIDE you can just use a PredicateHandler, with > a PathPrefixPredicate, and if it matches use ResponseCodeHandler to return > the desired response code. You could combine it > with io.undertow.server.handlers.AccessControlListHandler > or io.undertow.server.handlers.IPAddressAccessControlHandler if you want to > limit the IP range. > > Basically you just use a predicate to decide what you want to restrict, > and then map it to a handler that either rejects the request outright or > performs an access control check. > > Stuart > > > On Fri, Aug 10, 2018 at 3:59 PM Brad Wood wrote: > >> Anyone? >> >> Thanks! >> >> ~Brad >> >> *Developer Advocate* >> *Ortus Solutions, Corp * >> >> E-mail: brad at coldbox.org >> ColdBox Platform: http://www.coldbox.org >> Blog: http://www.codersrevolution.com >> >> >> >> On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: >> >>> Hi, I'm looking for some examples of locking down access to certain >>> directories, similar to how IIS has "hidden segments". For instance, I'd >>> like all URLs starting with /CFIDE to be blocked, or perhaps only access to >>> a certain range of IPs >>> >>> I swear I had looked at some examples of this about a year ago, but >>> after quite a lot of Googling today I was coming up empty handed. I found >>> some basic information on the access control handlers, but couldn't find a >>> single example of using them. >>> >>> Thanks! >>> >>> ~Brad >>> >>> *Developer Advocate* >>> *Ortus Solutions, Corp * >>> >>> E-mail: brad at coldbox.org >>> ColdBox Platform: http://www.coldbox.org >>> Blog: http://www.codersrevolution.com >>> >>> _______________________________________________ >> undertow-dev mailing list >> undertow-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/undertow-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180810/423c72b6/attachment-0001.html From barnett at rice.edu Mon Aug 13 16:49:25 2018 From: barnett at rice.edu (R. Matt Barnett) Date: Mon, 13 Aug 2018 15:49:25 -0500 Subject: [undertow-dev] Unable to concurrently use all available IO Threads under load on Red Hat In-Reply-To: References: Message-ID: <397845f3-f718-fddc-fd76-49ee3fbc869c@rice.edu> Did you all ever open a ticket on this? If so could you link me so I can follow along? Thanks, Matt On 7/26/2018 9:11 PM, Jason Greene wrote: > Looks like we need to tweak the hash: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=07f4c90062f8fc7c8c26f8f95324cbe8fa3145a5 > > > > > On Jul 26, 2018, at 7:13 PM, Stuart Douglas > wrote: > >> They are all even numbers :-( >> >> This does not play well with our hash if C1 is also even: >> >> (((C1 * 23)?+ P) * 23?+ C2) % 8 >> >> If C1 is even the C1 * 23 is even. This means ((C1 * 23)?+ P) * 23 is >> even. Depending on the value of C2 this means the result is always >> even or always odd, so with an evenly divisible number of threads you >> are only ever going to allocate to half of them. >> >> The good news is this should be easily fixed by using an odd number >> of IO threads, but we probably should revisit this. >> >> Stuart >> >> On Fri, Jul 27, 2018 at 4:34 AM R. Matt Barnett > > wrote: >> >> Backlog setting is 1000. >> >> Is this what you are interested in from netstat? This was for ab >> with a -c of 50. >> >> >> [barnett at apigateway_test ~]$ java -jar >> undertow-test-0.1.0-jar-with-dependencies.jar & >> [1] 7329 >> [barnett at apigateway_test ~]$ Jul 26, 2018 1:30:22 PM >> org.xnio.Xnio >> INFO: XNIO version 3.3.8.Final >> Jul 26, 2018 1:30:23 PM org.xnio.nio.NioXnio >> INFO: XNIO NIO Implementation Version 3.3.8.Final >> >> >> Server started on port 8080 >> 1 >> 2 >> 3 >> 4 >> [barnett at apigateway_test ~]$ netstat -t | grep apigateway_loadge >> | grep ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51580 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51614 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51622 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51626 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51612 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51578 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51636 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51616 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51582 ESTABLISHED >> tcp6?????? 0????? 0 apigateway_tes:webcache >> apigateway_loadge:51556 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51588 ESTABLISHED >> tcp6?????? 0????? 0 apigateway_tes:webcache >> apigateway_loadge:51558 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51586 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51648 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51632 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51652 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51654 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51574 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51640 ESTABLISHED >> tcp6?????? 0????? 0 apigateway_tes:webcache >> apigateway_loadge:51564 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51590 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51610 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51594 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51592 ESTABLISHED >> tcp6?????? 0????? 0 apigateway_tes:webcache >> apigateway_loadge:51568 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51620 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51598 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51600 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51584 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51630 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51596 ESTABLISHED >> tcp6?????? 0????? 0 apigateway_tes:webcache >> apigateway_loadge:51566 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51650 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51656 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51624 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51662 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51642 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51604 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51608 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51634 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51658 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51628 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51660 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51572 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51606 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51602 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51638 ESTABLISHED >> tcp6?????? 0????? 0 apigateway_tes:webcache >> apigateway_loadge:51570 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51618 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51646 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51644 ESTABLISHED >> tcp6????? 97????? 0 apigateway_tes:webcache >> apigateway_loadge:51576 ESTABLISHED >> >> >> On 7/25/2018 9:23 PM, Jason Greene wrote: >>> Could you post a netstat output so we can see what port numbers >>> your host is picking? >>> >>> Also is your backlog setting low by chance? >>> >>> On Jul 25, 2018, at 6:24 PM, Stuart Douglas >> > wrote: >>> >>>> The mapping is done by a hash of the remote IP+port. It sounds >>>> like maybe this machine is allocating ports in a way that does >>>> not map well to our hash. >>>> >>>> Because the remote IP is the same it is really only the port >>>> that comes into effect. The algorithm is >>>> in?org.xnio.nio.QueuedNioTcpServer#handleReady and in this case >>>> would simplify down to: >>>> >>>> (((C1 * 23)?+ P) * 23?+ C2) % 8 >>>> >>>> Where C1 is a hash of the remote IP, and C2 is a hash of the >>>> local IP+port combo. >>>> >>>> Stuart >>>> >>>> On Thu, Jul 26, 2018 at 3:52 AM R. Matt Barnett >>>> > wrote: >>>> >>>> I did. I set the concurrency level of ab to 128. I still >>>> see only 4 overlaps: >>>> >>>> $ java -jar undertow-test-0.1.0-jar-with-dependencies.jar & >>>> >>>> Server started on port 8080 >>>> 1 >>>> 2 >>>> 3 >>>> 4 >>>> >>>> $ netstat -t | grep apigateway_loadge | grep -c ESTABLISHED >>>> 126 >>>> >>>> >>>> What is the algorithm for mapping connections to IO >>>> threads?? As a new Undertow user I had assumed round robin, >>>> but it sounds like this is not the case. >>>> >>>> >>>> -- Matt >>>> >>>> >>>> On 7/25/2018 11:49 AM, Bill O'Neil wrote: >>>>> Did you try setting the concurrency level much higher than >>>>> 8 like I suggested earlier? You are probably having >>>>> multiple connections assigned to the same IO threads. >>>>> >>>>> On Wed, Jul 25, 2018 at 12:26 PM, R. Matt Barnett >>>>> > wrote: >>>>> >>>>> Corrected test to resolve test/set race. >>>>> >>>>> >>>>> https://gist.github.com/rmbarnett-rice/1179c4ad1d3344bb247c8b8daed3e4fa >>>>> >>>>> >>>>> I've also discovered this morning that I *can* see 1-8 >>>>> printed on Red >>>>> Hat when I generate load using ab from Windows, but >>>>> only 1-4 when >>>>> running ab on Red Hat (both locally and from a remote >>>>> server).? I'm >>>>> wondering if perhaps there is some sort of connection >>>>> reuse shenanigans >>>>> going on.? My assumption of the use of the -c 8 >>>>> parameter was "make 8 >>>>> sockets" but maybe not. I'll dig in and report back. >>>>> >>>>> >>>>> -- Matt >>>>> >>>>> >>>>> On 7/24/2018 6:56 PM, R. Matt Barnett wrote: >>>>> > Hello, >>>>> > >>>>> > I'm experiencing an Undertow performance issue I >>>>> fail to understand.? I >>>>> > am able to reproduce the issue with the code linked >>>>> bellow. The problem >>>>> > is that on Red Hat (and not Windows) I'm unable to >>>>> concurrently process >>>>> > more than 4 overlapping requests even with 8 >>>>> configured IO Threads. >>>>> > For example, if I run the following program (1 file, >>>>> 55 lines): >>>>> > >>>>> > >>>>> https://gist.github.com/rmbarnett-rice/668db6b4e9f8f8da7093a3659b6ae2b5 >>>>> > >>>>> > ... on Red Hat and then send requests to the server >>>>> using Apache >>>>> > Benchmark... >>>>> > >>>>> >? ???? > ab -n 1000 -c 8 localhost:8080/ >>>>> > >>>>> > I see the following output from the Undertow process: >>>>> > >>>>> >? ???? Server started on port 8080 >>>>> > >>>>> >? ???? 1 >>>>> >? ???? 2 >>>>> >? ???? 3 >>>>> >? ???? 4 >>>>> > >>>>> > I believe this demonstrates that only 4 requests are >>>>> ever processed in >>>>> > parallel.? I would expect 8.? In fact, when I run >>>>> the same experiment on >>>>> > Windows I see the expected output of >>>>> > >>>>> >? ???? Server started on port 8080 >>>>> >? ???? 1 >>>>> >? ???? 2 >>>>> >? ???? 3 >>>>> >? ???? 4 >>>>> >? ???? 5 >>>>> >? ???? 6 >>>>> >? ???? 7 >>>>> >? ???? 8 >>>>> > >>>>> > Any thoughts as to what might explain this behavior? >>>>> > >>>>> > Best, >>>>> > >>>>> > Matt >>>>> > >>>>> > _______________________________________________ >>>>> > undertow-dev mailing list >>>>> > undertow-dev at lists.jboss.org >>>>> >>>>> > https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>> >>>>> _______________________________________________ >>>>> undertow-dev mailing list >>>>> undertow-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> undertow-dev mailing list >>>> undertow-dev at lists.jboss.org >>>> >>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >>>> _______________________________________________ >>>> undertow-dev mailing list >>>> undertow-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180813/5a9bacf1/attachment-0001.html From sdouglas at redhat.com Wed Aug 15 00:27:43 2018 From: sdouglas at redhat.com (Stuart Douglas) Date: Wed, 15 Aug 2018 14:27:43 +1000 Subject: [undertow-dev] Unable to concurrently use all available IO Threads under load on Red Hat In-Reply-To: <397845f3-f718-fddc-fd76-49ee3fbc869c@rice.edu> References: <397845f3-f718-fddc-fd76-49ee3fbc869c@rice.edu> Message-ID: I have created https://issues.jboss.org/browse/XNIO-328. Stuart On Tue, Aug 14, 2018 at 6:49 AM R. Matt Barnett wrote: > Did you all ever open a ticket on this? If so could you link me so I can > follow along? > > > Thanks, > > Matt > > On 7/26/2018 9:11 PM, Jason Greene wrote: > > Looks like we need to tweak the hash: > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=07f4c90062f8fc7c8c26f8f95324cbe8fa3145a5 > > > > On Jul 26, 2018, at 7:13 PM, Stuart Douglas wrote: > > They are all even numbers :-( > > This does not play well with our hash if C1 is also even: > > (((C1 * 23) + P) * 23 + C2) % 8 > > If C1 is even the C1 * 23 is even. This means ((C1 * 23) + P) * 23 is > even. Depending on the value of C2 this means the result is always even or > always odd, so with an evenly divisible number of threads you are only ever > going to allocate to half of them. > > The good news is this should be easily fixed by using an odd number of IO > threads, but we probably should revisit this. > > Stuart > > On Fri, Jul 27, 2018 at 4:34 AM R. Matt Barnett wrote: > >> Backlog setting is 1000. >> >> Is this what you are interested in from netstat? This was for ab with a >> -c of 50. >> >> >> [barnett at apigateway_test ~]$ java -jar >> undertow-test-0.1.0-jar-with-dependencies.jar & >> [1] 7329 >> [barnett at apigateway_test ~]$ Jul 26, 2018 1:30:22 PM org.xnio.Xnio >> >> INFO: XNIO version 3.3.8.Final >> Jul 26, 2018 1:30:23 PM org.xnio.nio.NioXnio >> INFO: XNIO NIO Implementation Version 3.3.8.Final >> >> >> Server started on port 8080 >> 1 >> 2 >> 3 >> 4 >> [barnett at apigateway_test ~]$ netstat -t | grep apigateway_loadge | grep >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51580 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51614 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51622 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51626 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51612 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51578 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51636 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51616 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51582 >> ESTABLISHED >> tcp6 0 0 apigateway_tes:webcache apigateway_loadge:51556 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51588 >> ESTABLISHED >> tcp6 0 0 apigateway_tes:webcache apigateway_loadge:51558 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51586 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51648 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51632 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51652 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51654 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51574 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51640 >> ESTABLISHED >> tcp6 0 0 apigateway_tes:webcache apigateway_loadge:51564 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51590 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51610 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51594 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51592 >> ESTABLISHED >> tcp6 0 0 apigateway_tes:webcache apigateway_loadge:51568 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51620 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51598 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51600 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51584 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51630 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51596 >> ESTABLISHED >> tcp6 0 0 apigateway_tes:webcache apigateway_loadge:51566 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51650 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51656 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51624 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51662 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51642 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51604 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51608 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51634 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51658 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51628 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51660 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51572 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51606 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51602 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51638 >> ESTABLISHED >> tcp6 0 0 apigateway_tes:webcache apigateway_loadge:51570 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51618 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51646 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51644 >> ESTABLISHED >> tcp6 97 0 apigateway_tes:webcache apigateway_loadge:51576 >> ESTABLISHED >> >> On 7/25/2018 9:23 PM, Jason Greene wrote: >> >> Could you post a netstat output so we can see what port numbers your host >> is picking? >> >> Also is your backlog setting low by chance? >> >> On Jul 25, 2018, at 6:24 PM, Stuart Douglas wrote: >> >> The mapping is done by a hash of the remote IP+port. It sounds like maybe >> this machine is allocating ports in a way that does not map well to our >> hash. >> >> Because the remote IP is the same it is really only the port that comes >> into effect. The algorithm is >> in org.xnio.nio.QueuedNioTcpServer#handleReady and in this case would >> simplify down to: >> >> (((C1 * 23) + P) * 23 + C2) % 8 >> >> Where C1 is a hash of the remote IP, and C2 is a hash of the local >> IP+port combo. >> >> Stuart >> >> On Thu, Jul 26, 2018 at 3:52 AM R. Matt Barnett wrote: >> >>> I did. I set the concurrency level of ab to 128. I still see only 4 >>> overlaps: >>> >>> $ java -jar undertow-test-0.1.0-jar-with-dependencies.jar & >>> >>> Server started on port 8080 >>> 1 >>> 2 >>> 3 >>> 4 >>> >>> $ netstat -t | grep apigateway_loadge | grep -c ESTABLISHED >>> 126 >>> >>> >>> What is the algorithm for mapping connections to IO threads? As a new >>> Undertow user I had assumed round robin, but it sounds like this is not the >>> case. >>> >>> >>> -- Matt >>> >>> On 7/25/2018 11:49 AM, Bill O'Neil wrote: >>> >>> Did you try setting the concurrency level much higher than 8 like I >>> suggested earlier? You are probably having multiple connections assigned to >>> the same IO threads. >>> >>> On Wed, Jul 25, 2018 at 12:26 PM, R. Matt Barnett >>> wrote: >>> >>>> Corrected test to resolve test/set race. >>>> >>>> >>>> https://gist.github.com/rmbarnett-rice/1179c4ad1d3344bb247c8b8daed3e4fa >>>> >>>> >>>> I've also discovered this morning that I *can* see 1-8 printed on Red >>>> Hat when I generate load using ab from Windows, but only 1-4 when >>>> running ab on Red Hat (both locally and from a remote server). I'm >>>> wondering if perhaps there is some sort of connection reuse shenanigans >>>> going on. My assumption of the use of the -c 8 parameter was "make 8 >>>> sockets" but maybe not. I'll dig in and report back. >>>> >>>> >>>> -- Matt >>>> >>>> >>>> On 7/24/2018 6:56 PM, R. Matt Barnett wrote: >>>> > Hello, >>>> > >>>> > I'm experiencing an Undertow performance issue I fail to understand. >>>> I >>>> > am able to reproduce the issue with the code linked bellow. The >>>> problem >>>> > is that on Red Hat (and not Windows) I'm unable to concurrently >>>> process >>>> > more than 4 overlapping requests even with 8 configured IO Threads. >>>> > For example, if I run the following program (1 file, 55 lines): >>>> > >>>> > >>>> https://gist.github.com/rmbarnett-rice/668db6b4e9f8f8da7093a3659b6ae2b5 >>>> > >>>> > ... on Red Hat and then send requests to the server using Apache >>>> > Benchmark... >>>> > >>>> > > ab -n 1000 -c 8 localhost:8080/ >>>> > >>>> > I see the following output from the Undertow process: >>>> > >>>> > Server started on port 8080 >>>> > >>>> > 1 >>>> > 2 >>>> > 3 >>>> > 4 >>>> > >>>> > I believe this demonstrates that only 4 requests are ever processed in >>>> > parallel. I would expect 8. In fact, when I run the same experiment >>>> on >>>> > Windows I see the expected output of >>>> > >>>> > Server started on port 8080 >>>> > 1 >>>> > 2 >>>> > 3 >>>> > 4 >>>> > 5 >>>> > 6 >>>> > 7 >>>> > 8 >>>> > >>>> > Any thoughts as to what might explain this behavior? >>>> > >>>> > Best, >>>> > >>>> > Matt >>>> > >>>> > _______________________________________________ >>>> > undertow-dev mailing list >>>> > undertow-dev at lists.jboss.org >>>> > https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >>>> _______________________________________________ >>>> undertow-dev mailing list >>>> undertow-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >>> >>> >>> _______________________________________________ >>> undertow-dev mailing list >>> undertow-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/undertow-dev >> >> _______________________________________________ >> undertow-dev mailing list >> undertow-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/undertow-dev >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180815/4613c24f/attachment-0001.html From barnett at rice.edu Wed Aug 15 12:06:44 2018 From: barnett at rice.edu (R. Matt Barnett) Date: Wed, 15 Aug 2018 11:06:44 -0500 Subject: [undertow-dev] Unable to concurrently use all available IO Threads under load on Red Hat In-Reply-To: References: <397845f3-f718-fddc-fd76-49ee3fbc869c@rice.edu> Message-ID: <44545da4-6d21-4a1e-ec67-2342f4623197@rice.edu> Cool thx. On 8/14/2018 11:27 PM, Stuart Douglas wrote: > I have created https://issues.jboss.org/browse/XNIO-328. > > Stuart > > On Tue, Aug 14, 2018 at 6:49 AM R. Matt Barnett > wrote: > > Did you all ever open a ticket on this? If so could you link me so > I can follow along? > > > Thanks, > > Matt > > > On 7/26/2018 9:11 PM, Jason Greene wrote: >> Looks like we need to tweak the hash: >> >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=07f4c90062f8fc7c8c26f8f95324cbe8fa3145a5 >> >> >> >> >> On Jul 26, 2018, at 7:13 PM, Stuart Douglas > > wrote: >> >>> They are all even numbers :-( >>> >>> This does not play well with our hash if C1 is also even: >>> >>> (((C1 * 23)?+ P) * 23?+ C2) % 8 >>> >>> If C1 is even the C1 * 23 is even. This means ((C1 * 23)?+ P) * >>> 23 is even. Depending on the value of C2 this means the result >>> is always even or always odd, so with an evenly divisible number >>> of threads you are only ever going to allocate to half of them. >>> >>> The good news is this should be easily fixed by using an odd >>> number of IO threads, but we probably should revisit this. >>> >>> Stuart >>> >>> On Fri, Jul 27, 2018 at 4:34 AM R. Matt Barnett >>> > wrote: >>> >>> Backlog setting is 1000. >>> >>> Is this what you are interested in from netstat?? This was >>> for ab with a -c of 50. >>> >>> >>> [barnett at apigateway_test ~]$ java -jar >>> undertow-test-0.1.0-jar-with-dependencies.jar & >>> [1] 7329 >>> [barnett at apigateway_test ~]$ Jul 26, 2018 1:30:22 PM >>> org.xnio.Xnio >>> INFO: XNIO version 3.3.8.Final >>> Jul 26, 2018 1:30:23 PM org.xnio.nio.NioXnio >>> INFO: XNIO NIO Implementation Version 3.3.8.Final >>> >>> >>> Server started on port 8080 >>> 1 >>> 2 >>> 3 >>> 4 >>> [barnett at apigateway_test ~]$ netstat -t | grep >>> apigateway_loadge | grep ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51580 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51614 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51622 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51626 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51612 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51578 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51636 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51616 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51582 ESTABLISHED >>> tcp6?????? 0????? 0 apigateway_tes:webcache >>> apigateway_loadge:51556 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51588 ESTABLISHED >>> tcp6?????? 0????? 0 apigateway_tes:webcache >>> apigateway_loadge:51558 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51586 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51648 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51632 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51652 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51654 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51574 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51640 ESTABLISHED >>> tcp6?????? 0????? 0 apigateway_tes:webcache >>> apigateway_loadge:51564 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51590 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51610 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51594 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51592 ESTABLISHED >>> tcp6?????? 0????? 0 apigateway_tes:webcache >>> apigateway_loadge:51568 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51620 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51598 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51600 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51584 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51630 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51596 ESTABLISHED >>> tcp6?????? 0????? 0 apigateway_tes:webcache >>> apigateway_loadge:51566 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51650 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51656 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51624 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51662 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51642 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51604 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51608 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51634 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51658 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51628 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51660 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51572 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51606 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51602 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51638 ESTABLISHED >>> tcp6?????? 0????? 0 apigateway_tes:webcache >>> apigateway_loadge:51570 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51618 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51646 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51644 ESTABLISHED >>> tcp6????? 97????? 0 apigateway_tes:webcache >>> apigateway_loadge:51576 ESTABLISHED >>> >>> >>> On 7/25/2018 9:23 PM, Jason Greene wrote: >>>> Could you post a netstat output so we can see what port >>>> numbers your host is picking? >>>> >>>> Also is your backlog setting low by chance? >>>> >>>> On Jul 25, 2018, at 6:24 PM, Stuart Douglas >>>> > wrote: >>>> >>>>> The mapping is done by a hash of the remote IP+port. It >>>>> sounds like maybe this machine is allocating ports in a >>>>> way that does not map well to our hash. >>>>> >>>>> Because the remote IP is the same it is really only the >>>>> port that comes into effect. The algorithm is >>>>> in?org.xnio.nio.QueuedNioTcpServer#handleReady and in this >>>>> case would simplify down to: >>>>> >>>>> (((C1 * 23)?+ P) * 23?+ C2) % 8 >>>>> >>>>> Where C1 is a hash of the remote IP, and C2 is a hash of >>>>> the local IP+port combo. >>>>> >>>>> Stuart >>>>> >>>>> On Thu, Jul 26, 2018 at 3:52 AM R. Matt Barnett >>>>> > wrote: >>>>> >>>>> I did. I set the concurrency level of ab to 128. I >>>>> still see only 4 overlaps: >>>>> >>>>> $ java -jar >>>>> undertow-test-0.1.0-jar-with-dependencies.jar & >>>>> >>>>> Server started on port 8080 >>>>> 1 >>>>> 2 >>>>> 3 >>>>> 4 >>>>> >>>>> $ netstat -t | grep apigateway_loadge | grep -c >>>>> ESTABLISHED >>>>> 126 >>>>> >>>>> >>>>> What is the algorithm for mapping connections to IO >>>>> threads?? As a new Undertow user I had assumed round >>>>> robin, but it sounds like this is not the case. >>>>> >>>>> >>>>> -- Matt >>>>> >>>>> >>>>> On 7/25/2018 11:49 AM, Bill O'Neil wrote: >>>>>> Did you try setting the concurrency level much higher >>>>>> than 8 like I suggested earlier? You are probably >>>>>> having multiple connections assigned to the same IO >>>>>> threads. >>>>>> >>>>>> On Wed, Jul 25, 2018 at 12:26 PM, R. Matt Barnett >>>>>> > wrote: >>>>>> >>>>>> Corrected test to resolve test/set race. >>>>>> >>>>>> >>>>>> https://gist.github.com/rmbarnett-rice/1179c4ad1d3344bb247c8b8daed3e4fa >>>>>> >>>>>> >>>>>> I've also discovered this morning that I *can* >>>>>> see 1-8 printed on Red >>>>>> Hat when I generate load using ab from Windows, >>>>>> but only 1-4 when >>>>>> running ab on Red Hat (both locally and from a >>>>>> remote server).? I'm >>>>>> wondering if perhaps there is some sort of >>>>>> connection reuse shenanigans >>>>>> going on.? My assumption of the use of the -c 8 >>>>>> parameter was "make 8 >>>>>> sockets" but maybe not.? I'll dig in and report back. >>>>>> >>>>>> >>>>>> -- Matt >>>>>> >>>>>> >>>>>> On 7/24/2018 6:56 PM, R. Matt Barnett wrote: >>>>>> > Hello, >>>>>> > >>>>>> > I'm experiencing an Undertow performance issue >>>>>> I fail to understand.? I >>>>>> > am able to reproduce the issue with the code >>>>>> linked bellow. The problem >>>>>> > is that on Red Hat (and not Windows) I'm unable >>>>>> to concurrently process >>>>>> > more than 4 overlapping requests even with 8 >>>>>> configured IO Threads. >>>>>> > For example, if I run the following program (1 >>>>>> file, 55 lines): >>>>>> > >>>>>> > >>>>>> https://gist.github.com/rmbarnett-rice/668db6b4e9f8f8da7093a3659b6ae2b5 >>>>>> > >>>>>> > ... on Red Hat and then send requests to the >>>>>> server using Apache >>>>>> > Benchmark... >>>>>> > >>>>>> >? ???? > ab -n 1000 -c 8 localhost:8080/ >>>>>> > >>>>>> > I see the following output from the Undertow >>>>>> process: >>>>>> > >>>>>> > Server started on port 8080 >>>>>> > >>>>>> >? ???? 1 >>>>>> >? ???? 2 >>>>>> >? ???? 3 >>>>>> >? ???? 4 >>>>>> > >>>>>> > I believe this demonstrates that only 4 >>>>>> requests are ever processed in >>>>>> > parallel. I would expect 8.? In fact, when I >>>>>> run the same experiment on >>>>>> > Windows I see the expected output of >>>>>> > >>>>>> > Server started on port 8080 >>>>>> >? ???? 1 >>>>>> >? ???? 2 >>>>>> >? ???? 3 >>>>>> >? ???? 4 >>>>>> >? ???? 5 >>>>>> >? ???? 6 >>>>>> >? ???? 7 >>>>>> >? ???? 8 >>>>>> > >>>>>> > Any thoughts as to what might explain this >>>>>> behavior? >>>>>> > >>>>>> > Best, >>>>>> > >>>>>> > Matt >>>>>> > >>>>>> > _______________________________________________ >>>>>> > undertow-dev mailing list >>>>>> > undertow-dev at lists.jboss.org >>>>>> >>>>>> > >>>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>>> >>>>>> _______________________________________________ >>>>>> undertow-dev mailing list >>>>>> undertow-dev at lists.jboss.org >>>>>> >>>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> undertow-dev mailing list >>>>> undertow-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>> >>>>> _______________________________________________ >>>>> undertow-dev mailing list >>>>> undertow-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180815/0f985000/attachment-0001.html From sdouglas at redhat.com Wed Aug 15 20:05:44 2018 From: sdouglas at redhat.com (Stuart Douglas) Date: Thu, 16 Aug 2018 10:05:44 +1000 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: On Sat, Aug 11, 2018 at 1:25 AM Brad Wood wrote: > It depenends a bit on what you want to do. > > > Thanks for the reply Stuart. Honestly, I'm just brainstorming a little > here to see what's possible but I just couldn't find any docs or examples > to help solidify what was out in there. My primary use for this as I > explained just now in a separate reply is to be able to add some security > rules to CommandBox servers to do things such as: > > - Block access to CF admins in the root (such as paths starting with > */CFIDE*) > - Block access to special files in any directory such as *box.json*, > *server.json*, or *.cfconfig.json* > - Block access to hidden files in any directory (starting with a > period ) > - Block access to custom folders defined by the user such as */tests/* > or */workbench* > > I'm thinking a bit how the IIS "hidden segments" feature works. In > addition to using this behind the scenes in CommandBox, I'd like to expose > it to my users in the *server.json > * so they > can configure basic access control. I generally don't expose 100% of what > Undertow does since CommandBox aims to be a drop-in dead-easy way to just > fire up a server, but I'm interested in the IP matching since that could be > a common use case. i.e., "Block access to the administrator unless the IP > is in this range or localhost" > > So basically, yes, I'm interested in all of those things and I don't have > a super specific solution in mind, but I'm rather just looking for some > better examples to help me understand what's there and what I can best > expose in CommandBox. > > Basically you just use a predicate to decide what you want to restrict, >> and then map it to a handler that either rejects the request outright or >> performs an access control check. > > > This makes sense and I think the predicate part was what I was missing, > but are there examples of this anywhere? It helps me way more to see some > code. > > Most of the examples of this are in the test suite, e.g. PredicatedHandlersTestCase. There is also a text based representation you can use to configure this. e.g. to reject all box.json files: path-suffix(/box.json) -> status(404). Stuart > Thanks! > > ~Brad > > *Developer Advocate* > *Ortus Solutions, Corp * > > E-mail: brad at coldbox.org > ColdBox Platform: http://www.coldbox.org > Blog: http://www.codersrevolution.com > > > > On Fri, Aug 10, 2018 at 1:47 AM Stuart Douglas > wrote: > >> It depenends a bit on what you want to do. >> >> If you just want to block /CFIDE you can just use a PredicateHandler, >> with a PathPrefixPredicate, and if it matches use ResponseCodeHandler to >> return the desired response code. You could combine it >> with io.undertow.server.handlers.AccessControlListHandler >> or io.undertow.server.handlers.IPAddressAccessControlHandler if you want to >> limit the IP range. >> >> Basically you just use a predicate to decide what you want to restrict, >> and then map it to a handler that either rejects the request outright or >> performs an access control check. >> >> Stuart >> >> >> On Fri, Aug 10, 2018 at 3:59 PM Brad Wood wrote: >> >>> Anyone? >>> >>> Thanks! >>> >>> ~Brad >>> >>> *Developer Advocate* >>> *Ortus Solutions, Corp * >>> >>> E-mail: brad at coldbox.org >>> ColdBox Platform: http://www.coldbox.org >>> Blog: http://www.codersrevolution.com >>> >>> >>> >>> On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: >>> >>>> Hi, I'm looking for some examples of locking down access to certain >>>> directories, similar to how IIS has "hidden segments". For instance, I'd >>>> like all URLs starting with /CFIDE to be blocked, or perhaps only access to >>>> a certain range of IPs >>>> >>>> I swear I had looked at some examples of this about a year ago, but >>>> after quite a lot of Googling today I was coming up empty handed. I found >>>> some basic information on the access control handlers, but couldn't find a >>>> single example of using them. >>>> >>>> Thanks! >>>> >>>> ~Brad >>>> >>>> *Developer Advocate* >>>> *Ortus Solutions, Corp * >>>> >>>> E-mail: brad at coldbox.org >>>> ColdBox Platform: http://www.coldbox.org >>>> Blog: http://www.codersrevolution.com >>>> >>>> _______________________________________________ >>> undertow-dev mailing list >>> undertow-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/undertow-dev >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180816/91d3051f/attachment.html From bdw429s at gmail.com Thu Aug 16 14:06:20 2018 From: bdw429s at gmail.com (Brad Wood) Date: Thu, 16 Aug 2018 13:06:20 -0500 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: Thanks for the additional information Stuart. After a bit of Googling, the most comprehensive version of the documentation for the predicate language appears to be here: https://github.com/undertow-io/undertow-docs/blob/master/src/main/asciidoc/predicates-attributes-handlers.asciidoc I'll note that Google really tends to favor the older, but less complete versions of that page such as this one: http://undertow.io/undertow-docs/undertow-docs-1.2.0/predicates-attributes-handlers.html You may want to look into some SEO tricks to get Google to index the most recent version so it's easier to find. That said, for the life of me, I can't find any docs at all that talk about the *status(404)* bit you showed. Where is that covered? Did you perhaps mean this: *response-code(302)* Also, on the note of your docs, you have a handful of old pull requests for typos and such over here: https://github.com/undertow-io/undertow-docs/pulls I added one to the list. Please review and merge those :) Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad at coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com On Wed, Aug 15, 2018 at 7:05 PM Stuart Douglas wrote: > > > On Sat, Aug 11, 2018 at 1:25 AM Brad Wood wrote: > >> It depenends a bit on what you want to do. >> >> >> Thanks for the reply Stuart. Honestly, I'm just brainstorming a little >> here to see what's possible but I just couldn't find any docs or examples >> to help solidify what was out in there. My primary use for this as I >> explained just now in a separate reply is to be able to add some security >> rules to CommandBox servers to do things such as: >> >> - Block access to CF admins in the root (such as paths starting with >> */CFIDE*) >> - Block access to special files in any directory such as *box.json*, >> *server.json*, or *.cfconfig.json* >> - Block access to hidden files in any directory (starting with a >> period ) >> - Block access to custom folders defined by the user such as */tests/* >> or */workbench* >> >> I'm thinking a bit how the IIS "hidden segments" feature works. In >> addition to using this behind the scenes in CommandBox, I'd like to expose >> it to my users in the *server.json >> * so they >> can configure basic access control. I generally don't expose 100% of what >> Undertow does since CommandBox aims to be a drop-in dead-easy way to just >> fire up a server, but I'm interested in the IP matching since that could be >> a common use case. i.e., "Block access to the administrator unless the IP >> is in this range or localhost" >> >> So basically, yes, I'm interested in all of those things and I don't have >> a super specific solution in mind, but I'm rather just looking for some >> better examples to help me understand what's there and what I can best >> expose in CommandBox. >> >> Basically you just use a predicate to decide what you want to restrict, >>> and then map it to a handler that either rejects the request outright or >>> performs an access control check. >> >> >> This makes sense and I think the predicate part was what I was missing, >> but are there examples of this anywhere? It helps me way more to see some >> code. >> >> > Most of the examples of this are in the test suite, e.g. > PredicatedHandlersTestCase. There is also a text based representation you > can use to configure this. e.g. to reject all box.json files: > path-suffix(/box.json) -> status(404). > > Stuart > > >> Thanks! >> >> ~Brad >> >> *Developer Advocate* >> *Ortus Solutions, Corp * >> >> E-mail: brad at coldbox.org >> ColdBox Platform: http://www.coldbox.org >> Blog: http://www.codersrevolution.com >> >> >> >> On Fri, Aug 10, 2018 at 1:47 AM Stuart Douglas >> wrote: >> >>> It depenends a bit on what you want to do. >>> >>> If you just want to block /CFIDE you can just use a PredicateHandler, >>> with a PathPrefixPredicate, and if it matches use ResponseCodeHandler to >>> return the desired response code. You could combine it >>> with io.undertow.server.handlers.AccessControlListHandler >>> or io.undertow.server.handlers.IPAddressAccessControlHandler if you want to >>> limit the IP range. >>> >>> Basically you just use a predicate to decide what you want to restrict, >>> and then map it to a handler that either rejects the request outright or >>> performs an access control check. >>> >>> Stuart >>> >>> >>> On Fri, Aug 10, 2018 at 3:59 PM Brad Wood wrote: >>> >>>> Anyone? >>>> >>>> Thanks! >>>> >>>> ~Brad >>>> >>>> *Developer Advocate* >>>> *Ortus Solutions, Corp * >>>> >>>> E-mail: brad at coldbox.org >>>> ColdBox Platform: http://www.coldbox.org >>>> Blog: http://www.codersrevolution.com >>>> >>>> >>>> >>>> On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: >>>> >>>>> Hi, I'm looking for some examples of locking down access to certain >>>>> directories, similar to how IIS has "hidden segments". For instance, I'd >>>>> like all URLs starting with /CFIDE to be blocked, or perhaps only access to >>>>> a certain range of IPs >>>>> >>>>> I swear I had looked at some examples of this about a year ago, but >>>>> after quite a lot of Googling today I was coming up empty handed. I found >>>>> some basic information on the access control handlers, but couldn't find a >>>>> single example of using them. >>>>> >>>>> Thanks! >>>>> >>>>> ~Brad >>>>> >>>>> *Developer Advocate* >>>>> *Ortus Solutions, Corp * >>>>> >>>>> E-mail: brad at coldbox.org >>>>> ColdBox Platform: http://www.coldbox.org >>>>> Blog: http://www.codersrevolution.com >>>>> >>>>> _______________________________________________ >>>> undertow-dev mailing list >>>> undertow-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180816/fdc813fa/attachment-0001.html From bdw429s at gmail.com Thu Aug 16 14:30:43 2018 From: bdw429s at gmail.com (Brad Wood) Date: Thu, 16 Aug 2018 13:30:43 -0500 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: Is the basic auth handler part of the predicate language? I didn't see it in the docs so I wanted to see if there was a way to have a textual representation of that. Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad at coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com On Thu, Aug 16, 2018 at 1:06 PM Brad Wood wrote: > Thanks for the additional information Stuart. After a bit of Googling, > the most comprehensive version of the documentation for the predicate > language appears to be here: > > > https://github.com/undertow-io/undertow-docs/blob/master/src/main/asciidoc/predicates-attributes-handlers.asciidoc > > I'll note that Google really tends to favor the older, but less complete > versions of that page such as this one: > > > http://undertow.io/undertow-docs/undertow-docs-1.2.0/predicates-attributes-handlers.html > > You may want to look into some SEO tricks to get Google to index the most > recent version so it's easier to find. That said, for the life of me, I > can't find any docs at all that talk about the *status(404)* bit you > showed. Where is that covered? > > Did you perhaps mean this: *response-code(302)* > > Also, on the note of your docs, you have a handful of old pull requests > for typos and such over here: > https://github.com/undertow-io/undertow-docs/pulls > I added one to the list. Please review and merge those :) > > Thanks! > > ~Brad > > *Developer Advocate* > *Ortus Solutions, Corp * > > E-mail: brad at coldbox.org > ColdBox Platform: http://www.coldbox.org > Blog: http://www.codersrevolution.com > > > > On Wed, Aug 15, 2018 at 7:05 PM Stuart Douglas > wrote: > >> >> >> On Sat, Aug 11, 2018 at 1:25 AM Brad Wood wrote: >> >>> It depenends a bit on what you want to do. >>> >>> >>> Thanks for the reply Stuart. Honestly, I'm just brainstorming a little >>> here to see what's possible but I just couldn't find any docs or examples >>> to help solidify what was out in there. My primary use for this as I >>> explained just now in a separate reply is to be able to add some security >>> rules to CommandBox servers to do things such as: >>> >>> - Block access to CF admins in the root (such as paths starting >>> with */CFIDE*) >>> - Block access to special files in any directory such as *box.json*, >>> *server.json*, or *.cfconfig.json* >>> - Block access to hidden files in any directory (starting with a >>> period ) >>> - Block access to custom folders defined by the user such as >>> */tests/* or */workbench* >>> >>> I'm thinking a bit how the IIS "hidden segments" feature works. In >>> addition to using this behind the scenes in CommandBox, I'd like to expose >>> it to my users in the *server.json >>> * so >>> they can configure basic access control. I generally don't expose 100% of >>> what Undertow does since CommandBox aims to be a drop-in dead-easy way to >>> just fire up a server, but I'm interested in the IP matching since that >>> could be a common use case. i.e., "Block access to the administrator >>> unless the IP is in this range or localhost" >>> >>> So basically, yes, I'm interested in all of those things and I don't >>> have a super specific solution in mind, but I'm rather just looking for >>> some better examples to help me understand what's there and what I can best >>> expose in CommandBox. >>> >>> Basically you just use a predicate to decide what you want to restrict, >>>> and then map it to a handler that either rejects the request outright or >>>> performs an access control check. >>> >>> >>> This makes sense and I think the predicate part was what I was missing, >>> but are there examples of this anywhere? It helps me way more to see some >>> code. >>> >>> >> Most of the examples of this are in the test suite, e.g. >> PredicatedHandlersTestCase. There is also a text based representation you >> can use to configure this. e.g. to reject all box.json files: >> path-suffix(/box.json) -> status(404). >> >> Stuart >> >> >>> Thanks! >>> >>> ~Brad >>> >>> *Developer Advocate* >>> *Ortus Solutions, Corp * >>> >>> E-mail: brad at coldbox.org >>> ColdBox Platform: http://www.coldbox.org >>> Blog: http://www.codersrevolution.com >>> >>> >>> >>> On Fri, Aug 10, 2018 at 1:47 AM Stuart Douglas >>> wrote: >>> >>>> It depenends a bit on what you want to do. >>>> >>>> If you just want to block /CFIDE you can just use a PredicateHandler, >>>> with a PathPrefixPredicate, and if it matches use ResponseCodeHandler to >>>> return the desired response code. You could combine it >>>> with io.undertow.server.handlers.AccessControlListHandler >>>> or io.undertow.server.handlers.IPAddressAccessControlHandler if you want to >>>> limit the IP range. >>>> >>>> Basically you just use a predicate to decide what you want to restrict, >>>> and then map it to a handler that either rejects the request outright or >>>> performs an access control check. >>>> >>>> Stuart >>>> >>>> >>>> On Fri, Aug 10, 2018 at 3:59 PM Brad Wood wrote: >>>> >>>>> Anyone? >>>>> >>>>> Thanks! >>>>> >>>>> ~Brad >>>>> >>>>> *Developer Advocate* >>>>> *Ortus Solutions, Corp * >>>>> >>>>> E-mail: brad at coldbox.org >>>>> ColdBox Platform: http://www.coldbox.org >>>>> Blog: http://www.codersrevolution.com >>>>> >>>>> >>>>> >>>>> On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: >>>>> >>>>>> Hi, I'm looking for some examples of locking down access to certain >>>>>> directories, similar to how IIS has "hidden segments". For instance, I'd >>>>>> like all URLs starting with /CFIDE to be blocked, or perhaps only access to >>>>>> a certain range of IPs >>>>>> >>>>>> I swear I had looked at some examples of this about a year ago, but >>>>>> after quite a lot of Googling today I was coming up empty handed. I found >>>>>> some basic information on the access control handlers, but couldn't find a >>>>>> single example of using them. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> ~Brad >>>>>> >>>>>> *Developer Advocate* >>>>>> *Ortus Solutions, Corp * >>>>>> >>>>>> E-mail: brad at coldbox.org >>>>>> ColdBox Platform: http://www.coldbox.org >>>>>> Blog: http://www.codersrevolution.com >>>>>> >>>>>> _______________________________________________ >>>>> undertow-dev mailing list >>>>> undertow-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180816/caf093ea/attachment.html From bdw429s at gmail.com Tue Aug 21 12:51:26 2018 From: bdw429s at gmail.com (Brad Wood) Date: Tue, 21 Aug 2018 11:51:26 -0500 Subject: [undertow-dev] Access control examples In-Reply-To: References: Message-ID: Hi Stuart, did you see my notes about the unmerged pulls, the status(xxx) call and the question about basic auth being part of the predicate language? Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad at coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com On Thu, Aug 16, 2018 at 1:30 PM Brad Wood wrote: > Is the basic auth handler part of the predicate language? I didn't see it > in the docs so I wanted to see if there was a way to have a textual > representation of that. > > Thanks! > > ~Brad > > *Developer Advocate* > *Ortus Solutions, Corp * > > E-mail: brad at coldbox.org > ColdBox Platform: http://www.coldbox.org > Blog: http://www.codersrevolution.com > > > > On Thu, Aug 16, 2018 at 1:06 PM Brad Wood wrote: > >> Thanks for the additional information Stuart. After a bit of Googling, >> the most comprehensive version of the documentation for the predicate >> language appears to be here: >> >> >> https://github.com/undertow-io/undertow-docs/blob/master/src/main/asciidoc/predicates-attributes-handlers.asciidoc >> >> I'll note that Google really tends to favor the older, but less complete >> versions of that page such as this one: >> >> >> http://undertow.io/undertow-docs/undertow-docs-1.2.0/predicates-attributes-handlers.html >> >> You may want to look into some SEO tricks to get Google to index the most >> recent version so it's easier to find. That said, for the life of me, I >> can't find any docs at all that talk about the *status(404)* bit you >> showed. Where is that covered? >> >> Did you perhaps mean this: *response-code(302)* >> >> Also, on the note of your docs, you have a handful of old pull requests >> for typos and such over here: >> https://github.com/undertow-io/undertow-docs/pulls >> I added one to the list. Please review and merge those :) >> >> Thanks! >> >> ~Brad >> >> *Developer Advocate* >> *Ortus Solutions, Corp * >> >> E-mail: brad at coldbox.org >> ColdBox Platform: http://www.coldbox.org >> Blog: http://www.codersrevolution.com >> >> >> >> On Wed, Aug 15, 2018 at 7:05 PM Stuart Douglas >> wrote: >> >>> >>> >>> On Sat, Aug 11, 2018 at 1:25 AM Brad Wood wrote: >>> >>>> It depenends a bit on what you want to do. >>>> >>>> >>>> Thanks for the reply Stuart. Honestly, I'm just brainstorming a little >>>> here to see what's possible but I just couldn't find any docs or examples >>>> to help solidify what was out in there. My primary use for this as I >>>> explained just now in a separate reply is to be able to add some security >>>> rules to CommandBox servers to do things such as: >>>> >>>> - Block access to CF admins in the root (such as paths starting >>>> with */CFIDE*) >>>> - Block access to special files in any directory such as *box.json*, >>>> *server.json*, or *.cfconfig.json* >>>> - Block access to hidden files in any directory (starting with a >>>> period ) >>>> - Block access to custom folders defined by the user such as >>>> */tests/* or */workbench* >>>> >>>> I'm thinking a bit how the IIS "hidden segments" feature works. In >>>> addition to using this behind the scenes in CommandBox, I'd like to expose >>>> it to my users in the *server.json >>>> * so >>>> they can configure basic access control. I generally don't expose 100% of >>>> what Undertow does since CommandBox aims to be a drop-in dead-easy way to >>>> just fire up a server, but I'm interested in the IP matching since that >>>> could be a common use case. i.e., "Block access to the administrator >>>> unless the IP is in this range or localhost" >>>> >>>> So basically, yes, I'm interested in all of those things and I don't >>>> have a super specific solution in mind, but I'm rather just looking for >>>> some better examples to help me understand what's there and what I can best >>>> expose in CommandBox. >>>> >>>> Basically you just use a predicate to decide what you want to restrict, >>>>> and then map it to a handler that either rejects the request outright or >>>>> performs an access control check. >>>> >>>> >>>> This makes sense and I think the predicate part was what I was missing, >>>> but are there examples of this anywhere? It helps me way more to see some >>>> code. >>>> >>>> >>> Most of the examples of this are in the test suite, e.g. >>> PredicatedHandlersTestCase. There is also a text based representation you >>> can use to configure this. e.g. to reject all box.json files: >>> path-suffix(/box.json) -> status(404). >>> >>> Stuart >>> >>> >>>> Thanks! >>>> >>>> ~Brad >>>> >>>> *Developer Advocate* >>>> *Ortus Solutions, Corp * >>>> >>>> E-mail: brad at coldbox.org >>>> ColdBox Platform: http://www.coldbox.org >>>> Blog: http://www.codersrevolution.com >>>> >>>> >>>> >>>> On Fri, Aug 10, 2018 at 1:47 AM Stuart Douglas >>>> wrote: >>>> >>>>> It depenends a bit on what you want to do. >>>>> >>>>> If you just want to block /CFIDE you can just use a PredicateHandler, >>>>> with a PathPrefixPredicate, and if it matches use ResponseCodeHandler to >>>>> return the desired response code. You could combine it >>>>> with io.undertow.server.handlers.AccessControlListHandler >>>>> or io.undertow.server.handlers.IPAddressAccessControlHandler if you want to >>>>> limit the IP range. >>>>> >>>>> Basically you just use a predicate to decide what you want to >>>>> restrict, and then map it to a handler that either rejects the request >>>>> outright or performs an access control check. >>>>> >>>>> Stuart >>>>> >>>>> >>>>> On Fri, Aug 10, 2018 at 3:59 PM Brad Wood wrote: >>>>> >>>>>> Anyone? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> ~Brad >>>>>> >>>>>> *Developer Advocate* >>>>>> *Ortus Solutions, Corp * >>>>>> >>>>>> E-mail: brad at coldbox.org >>>>>> ColdBox Platform: http://www.coldbox.org >>>>>> Blog: http://www.codersrevolution.com >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Aug 4, 2018 at 4:48 PM Brad Wood wrote: >>>>>> >>>>>>> Hi, I'm looking for some examples of locking down access to certain >>>>>>> directories, similar to how IIS has "hidden segments". For instance, I'd >>>>>>> like all URLs starting with /CFIDE to be blocked, or perhaps only access to >>>>>>> a certain range of IPs >>>>>>> >>>>>>> I swear I had looked at some examples of this about a year ago, but >>>>>>> after quite a lot of Googling today I was coming up empty handed. I found >>>>>>> some basic information on the access control handlers, but couldn't find a >>>>>>> single example of using them. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> ~Brad >>>>>>> >>>>>>> *Developer Advocate* >>>>>>> *Ortus Solutions, Corp * >>>>>>> >>>>>>> E-mail: brad at coldbox.org >>>>>>> ColdBox Platform: http://www.coldbox.org >>>>>>> Blog: http://www.codersrevolution.com >>>>>>> >>>>>>> _______________________________________________ >>>>>> undertow-dev mailing list >>>>>> undertow-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180821/62978fa2/attachment-0001.html From ilpo.ruotsalainen at oath.com Fri Aug 24 04:48:38 2018 From: ilpo.ruotsalainen at oath.com (Ilpo Ruotsalainen) Date: Fri, 24 Aug 2018 10:48:38 +0200 Subject: [undertow-dev] Request header order Message-ID: Hi, A project we're working on has recently ran into a need to access the HTTP request headers in order they were received. This does not seem currently possible as HeaderMap is just a hash table. One way I see to get this functionality would be to make HeaderMap non-final and provide a mechanism for configuring the class used for request headers, but I don't particularly like this as the HeaderMap API is fairly wide and some of the methods are semantically tricky for this particular purpose. Obviously we'd prefer a solution where the modifications to Undertow itself are acceptable for upstreaming so we don't have to maintain a custom fork of it. Any suggestions would be appreciated. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180824/766ed245/attachment.html From sdouglas at redhat.com Sun Aug 26 18:52:01 2018 From: sdouglas at redhat.com (Stuart Douglas) Date: Mon, 27 Aug 2018 08:52:01 +1000 Subject: [undertow-dev] Request header order In-Reply-To: References: Message-ID: There is not really any way to do this at the moment. The order should not matter for HTTP headers, so we don't provide any way to access this. If you want to do this I think you will have to maintain a custom fork. Stuart On Fri, Aug 24, 2018 at 6:49 PM Ilpo Ruotsalainen < ilpo.ruotsalainen at oath.com> wrote: > Hi, > > A project we're working on has recently ran into a need to access the HTTP > request headers in order they were received. This does not seem currently > possible as HeaderMap is just a hash table. > > One way I see to get this functionality would be to make HeaderMap > non-final and provide a mechanism for configuring the class used for > request headers, but I don't particularly like this as the HeaderMap API is > fairly wide and some of the methods are semantically tricky for this > particular purpose. > > Obviously we'd prefer a solution where the modifications to Undertow > itself are acceptable for upstreaming so we don't have to maintain a custom > fork of it. Any suggestions would be appreciated. > _______________________________________________ > undertow-dev mailing list > undertow-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/undertow-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/undertow-dev/attachments/20180827/b4f36cfb/attachment.html