Re: [undertow-dev] response-code handler bypasses configured error pages

*Can you give a bigger picture of exactly what you are trying to solve? * > Hi Bill. I know I've talked with Stuart about my use of Undertow in the past and he's even helped review our code. It's understandable you probably don't know where I'm coming from :) I typically don't get into the weeds of exactly how I'm using Undertow since it's a long story, but since you asked I'll fill you in. I'm the lead developer on a tool called CommandBox which is a - CLI - REPL - package manager - scaffold - server for CFML applications (powered under the hood using pre-packaged WARs and Undertow https://commandbox.ortusbooks.com/getting-started-guide Undertow is packaged quite nicely inside of a black box and frankly, my users don't know or care that it's there. We use a custom resource manager that allows us to serve JIT'd CFML templates from a separate web root outside the WAR while powering the servlet from a pre-packaged WAR in a manner that is the same as Adobe ColdFusion or Lucee Server's Tomcat-based installations. Overall, that works great and has been in use for 5-6 years now, powering roughly 50% of the docker usage in the CF community. All a CommandBox user needs to do to start up a CF server is (box has a built in interactive shell based on JLine) CommandBox> echo "Hello world" > index.cfm CommandBox> server start port=8080 And boom, a browser window opens up serving their hello world message. They can even use CommandBox/Undertow to serve up a static HTML server just like the "npm serve" command, but with no Node :) CommandBox also has a singular JSON file that allows users to control every aspect of their server - JRE - auto installed from the AdoptOpenJDK API - CF engine/version - auto installed from Forgebox.io - Ports (HTTP/HTTPS/AJP) - SSL Certs - virtual directories - Lib directories to class load jars from for the app - welcome pages - Basic auth - heap size - URL rewrites (Currently powered by Tuckey) - *Error pages* - And coming soon-- rules defined in Undertow's predicate language placed right into an array in the JSON file. Doc Reference for all configuration options: https://commandbox.ortusbooks.com/embedded-server/server.json So to review, my users spin up servers externally controlled by CLI parameters, local JSON files shipped with the app, or even global default settings they've baked into their CLI. My users don't know or care about Undertow, it "just works" for them. The addition of the predicate language will give my users an option other than Tuckey to accomplish the following - Add rewrites to their app - Block common administrator paths from prying eyes - Secure sensitive configuration files they don't want served from the web root - Configure reverse proxies and load balanced proxies on the fly I even plan on baking some of these security rules into the CommandBox core so their servers are better secured by default. So here is a totally realistic example of how I envision a server being started (The "server set" command simply modifies the server.json files for you) CommandBox> server set web.errorPages.404=notfound.html CommandBox> server set web.rules = '[ "path(/box.json)->response-code(404)" ]' CommandBox> server start So, in that user's mind, they just 1. Configured all *404 *requests to use a custom *notfound.html* file 2. Placed a predicate rule that responds with a *404* to any request to the secret file *box.json* 3. It would be the completely reasonable expectation of that user that if they slap */box.json* in their browser, they will see the contents of their *notFound.html* error page However, number 3 won't/doesn't happen for various behind-the-scenes technical reasons. I get those reasons, but at the end of the day I need this behavior to be consistent in my usage of undertow. I can't just tell people, "Oh yeah, your error pages don't work half the time, but don't worry, there's a really good reason for it!", lol. The error pages in the JSON configuration for CommandBox are fed directly to the servlet's error pages deployment info and the web.rules are parsed into a PredicatesHandler. But again, this is a black box so it needs to "just work" like people will expect. That's what I'm after. So, now that the background stuff is out of the way, allow me also address the rest of your questions categorically: Where will your actual routing logic live? I'm not quite sure if you're asking where the actual text-based predicate language will be stored or if you're asking where in the handler chain the PredicatesHandler will be processed. If you're asking about the rules, they will be provided by the users of my tool and can literally come from anywhere based on their needs. If you're asking where in the handler chain I'll process them, I've covered previously in this thread having them "inside" the servlet initial handler and "outside" the servlet and both approaches have had issues. Perhaps the short answer is, I'm willing to put it wherever the heck it needs to be in order to make this work :) What types of things are you expecting the predicate language to > accomplish for you? Everything. Literally everything. I am exposing Undertow's predicate language directly to the end users (developers, sysops guys, etc) for them to tap into and add whatever rules they deem necessary for rewrites, security, or even reverse proxies in their app. I'm not even sure why this matters to the conversation. But suffice it to say, I need Undertow's predicate language to be capable of doing everything it does, but I need consistency in how error status codes are handled in a manner that matches the rest of the server. Why do you need to be tied to servlets? I don't. Quite frankly, I couldn't care less what it's tied to. That said, servlets already have the ability to control error pages. CommandBox will parse the *web.xml* in the WAR to look for any configured error pages (since user's can supply a custom WAR) as well as combine the configuration in the user's server.json file. Right now we tap into the error pages functionality of the servlet because it's simple, it's built in, and it just makes sense. I need an error page functionality that is consistent across all uses of Undertow. Not just firing in some scenarios on Tuesdays after 3pm depending on a complex internal routing of your handler chain. Again, my users don't know or care how this works, if they configure a custom 404 page, they expect it to be used by default for all 404s that don't otherwise return content. > path(/box.json)->response-code(404) > This example seems pretty impractical because if you just left the line > out entirely then wouldn't your router decide this route doesn't exist and > handle the 404 in your servlet code? Sorry, but I'm not quite following you. That line is a prime example of what I expect people to use the predicate language for in securing parts of their app they don't want served. The way CFML apps work is you place all files in your web root (images, jss, css, html, and CFMs). There is a default static handler that just serves up whatever files you request, and a servlet mapping for *.cfm files that route them through the CF servlet (which JIT compiles and processes them). If I don't have the predicate rule above, a user would be able to directly hit that configuration file in the web root-- and being just a normal JSON file, it would be served up no problem. The request would not be a 404 in the servlet because that file actually exists on disk and would be served. Which, of course, is the entire purpose of this feature! With attributes and custom HttpHandlers you should be able to accomplish > pretty much anything you want Yes, I know. Undertow is very powerful. My questions here are two fold - How in the heck do we accomplish it?? - Can Undertow be made better? The default behavior of the response-code handler in this case seems ... *not useful. *I mean, in what universe is it ok to respond to a request with a 404 status code but no response body. Surely we can agree, this stands to be improved. But the predicate language doesn't seem to provide a consistent way to tap into the error pages in the deployment. is there a specific need to have some of this logic live in servlets? Like I said above, not really -- other than the fact that this is a *servlet deployment* making use of *servlet features* such as error pages. Thanks! ~Brad *Developer Advocate* *Ortus Solutions, Corp * E-mail: brad(a)coldbox.org ColdBox Platform: http://www.coldbox.org Blog: http://www.codersrevolution.com On Fri, Jul 3, 2020 at 7:22 AM Bill O'Neil <bill(a)dartalley.com&gt; wrote: > Can you give a bigger picture of exactly what you are trying to solve? > Where will your actual routing logic live? > What types of things are you expecting the predicate language to > accomplish for you? > Why do you need to be tied to servlets? > > path(/box.json)->response-code(404) >> > This example seems pretty impractical because if you just left the line > out entirely then wouldn't your router decide this route doesn't exist and > handle the 404 in your servlet code? > > With attributes and custom HttpHandlers you should be able to accomplish > pretty much anything you want is there a specific need to have some of this > logic live in servlets? > > On Fri, Jul 3, 2020 at 3:20 AM Brad Wood <bdw429s(a)gmail.com&gt; wrote: > >> So what is our solution here? It seems I'm darned if I do and darned if >> I don't. >> >> If I move the predicates until after the servlet then I can call >> sendError() on the servlet response to trigger the error pages, however I >> lose the ability to rewrite the request (a deal breaker) >> >> But if I leave the predicates before the servlet, I can perform >> rewrites, but I can no longer call sendError() since, as you said, there is >> no servlet response yet! >> >> I understand the separation between the servlet and core, but it's >> killing me here. This seems like such a simple concept. >> >> - I have a servlet deployment with error page configured >> - I want to have a predicated handler that uses the configured error >> pages >> >> Can I accomplish this with a default response listener that accesses the >> error page configuration and dispatches the appropriate location? Even >> that feels dirty. It seems there should be a handler invoked directly from >> the predicates that can dispatch the error pages configured in the >> servlet. Even if all it did was look up the corresponding location for the >> given status code and forward the request there. But I can't even figure >> out how to access the deployment from the exchange. >> >> Thanks! >> >> ~Brad >> >> *Developer Advocate* >> *Ortus Solutions, Corp * >> >> E-mail: brad(a)coldbox.org >> ColdBox Platform: http://www.coldbox.org >> Blog: http://www.codersrevolution.com >> >> >> >> On Thu, Jul 2, 2020 at 9:22 PM Stuart Douglas <sdouglas(a)redhat.com&gt; >> wrote: >> >>> >>> >>> On Fri, 3 Jul 2020 at 10:34, Brad Wood <bdw429s(a)gmail.com&gt; wrote: >>> >>>> Before I give that workaround a try, a couple existential questions... >>>> >>>> - Should the *response-code* handler also set the *errorCode* as >>>> well in the servlet request context so it triggers the doErrorDispatch() >>>> action? >>>> - Or should there be another built-in handler called "send-error" >>>> or similar that does this. >>>> >>>> Ex: >>>> path( /secret.json ) -> send-error ( 404 ) >>>> path( /admin/ ) -> send-error( 503 ) >>>> >>>> I'm sort of questioning what the usefulness is of the *response-code* >>>> handler however (at least in the context of the predicate language) if it's >>>> default behavior is NOT to trigger the error code and therefore the error >>>> page. >>>> >>> >>> These error pages are part of Servlet, while the predicate language is >>> part of core. The response-code handler does not really know anything about >>> the Servlet deployment. >>> >>> Generally the predicate language is run before Servlet (e.g. in >>> WildFly), so the HttpServletRequest/Response has not been created yet. This >>> gives a lot of flexibility in that you can make changes to route the >>> request to different contexts and potentially skip servlet altogether, but >>> the down side is that it is not tightly integrated with Servlet. >>> >>> Stuart >>> >>> >>>> >>>> Thanks! >>>> >>>> ~Brad >>>> >>>> *Developer Advocate* >>>> *Ortus Solutions, Corp * >>>> >>>> E-mail: brad(a)coldbox.org >>>> ColdBox Platform: http://www.coldbox.org >>>> Blog: http://www.codersrevolution.com >>>> >>>> >>>> >>>> On Thu, Jul 2, 2020 at 7:01 PM Stuart Douglas <sdouglas(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Hmm, this is because the servlet status code is only triggered by a >>>>> sendError method, not by just setting the code. >>>>> >>>>> You probably need to write a custom one that looks like this (plus >>>>> the relevant predicate languge bits): >>>>> >>>>> public class SendErrorHandler implements HttpHandler { >>>>> >>>>> private final int code; >>>>> >>>>> public SendErrorHandler(int code) { >>>>> this.code = code; >>>>> } >>>>> >>>>> @Override >>>>> public void handleRequest(HttpServerExchange exchange) throws >>>>> Exception { >>>>> ServletRequestContext src = >>>>> exchange.getAttachment(ServletRequestContext.ATTACHMENT_KEY); >>>>> >>>>> ((HttpServletResponse)src.getServletResponse()).sendError(code); >>>>> } >>>>> } >>>>> >>>>> Stuart >>>>> >>>>> >>>>> >>>>> On Fri, 3 Jul 2020 at 09:40, Brad Wood <bdw429s(a)gmail.com&gt; wrote: >>>>> >>>>>> Thanks for the reply Stuart. I've tried this with no success, but >>>>>> perhaps I'm doing it wrong. >>>>>> >>>>>> List<PredicatedHandler> ph = >>>>>> PredicatedHandlersParser.parse(predicatesLines, _classLoader); >>>>>> servletBuilder.addOuterHandlerChainWrapper(next -> >>>>>> Handlers.predicates(ph,next)); >>>>>> >>>>>> When the response-code handler fires, I still get no response body. >>>>>> >>>>>> On a related note, when I move the predicates into an outer handler >>>>>> chain wrapper, my default response listener also doesn't fire at all. >>>>>> >>>>>> On an unrelated train of thought, I've been trying to see if I can >>>>>> get the default response listener to automatically dispatch the correct >>>>>> error page, but that hasn't been going well either. If I don't use the >>>>>> outer handler chain idea, but try to capture the empty response in a >>>>>> default response listener, I can return a static message using the Sender >>>>>> class >>>>>> >>>>>> Sender sender = exchange.getResponseSender(); >>>>>> sender.send(errorHTMLString); >>>>>> >>>>>> But if I try to run something this in my default response listener >>>>>> to invoke my error pages >>>>>> >>>>>> ServletRequestContext src = >>>>>> exchange.getAttachment(ServletRequestContext.ATTACHMENT_KEY); >>>>>> HttpServletResponseImpl response = src.getOriginalResponse(); >>>>>> response.doErrorDispatch( exchange.getStatusCode(), >>>>>> exchange.getReasonPhrase() ); >>>>>> >>>>>> Then it's as though nothing happens and I still get an empty >>>>>> response. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> ~Brad >>>>>> >>>>>> *Developer Advocate* >>>>>> *Ortus Solutions, Corp * >>>>>> >>>>>> E-mail: brad(a)coldbox.org >>>>>> ColdBox Platform: http://www.coldbox.org >>>>>> Blog: http://www.codersrevolution.com >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jul 2, 2020 at 6:17 PM Stuart Douglas <sdouglas(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> The predicate languages are executed before the Servler handlers, >>>>>>> so they won't be handled by Servlet error pages. >>>>>>> >>>>>>> If you are setting this all up programmatically you could use >>>>>>> io.undertow.servlet.api.DeploymentInfo#addOuterHandlerChainWrapper to setup >>>>>>> the predicate handler after the initial servlet one, which should mean that >>>>>>> the servlet error handling will handle the response code. >>>>>>> >>>>>>> Stuart >>>>>>> >>>>>>> On Fri, 3 Jul 2020 at 08:25, Brad Wood <bdw429s(a)gmail.com&gt; wrote: >>>>>>> >>>>>>>> When I configure an error page similar to this: >>>>>>>> >>>>>>>> servletBuilder.addErrorPage( new ErrorPage( "404.html", 404)); >>>>>>>> >>>>>>>> This works great when I hit a path in my browser that doesn't >>>>>>>> exist. The contents of the *404.html* file is served with a >>>>>>>> response code of *404*. >>>>>>>> >>>>>>>> However, if I also use the predicate language to define something >>>>>>>> like: >>>>>>>> >>>>>>>> path(/box.json)->response-code(404) >>>>>>>> >>>>>>>> and then I hit *localhost/box.json* in my browser, I get a *404* >>>>>>>> status code but with no response body. >>>>>>>> >>>>>>>> - The docs say the response-code handler ends the exchange, >>>>>>>> but should it still respect the error pages? >>>>>>>> - How can I modify my use of Undertow to respect the error >>>>>>>> pages when using the response-code handler? >>>>>>>> - I've seen in the docs the ability to have a >>>>>>>> *addDefaultResponseListener()* but I'm not sure if it is the >>>>>>>> correct solution for this, nor how I would access the error page >>>>>>>> configuration dynamically as to not need to duplicate my work. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> ~Brad >>>>>>>> >>>>>>>> *Developer Advocate* >>>>>>>> *Ortus Solutions, Corp * >>>>>>>> >>>>>>>> E-mail: brad(a)coldbox.org >>>>>>>> ColdBox Platform: http://www.coldbox.org >>>>>>>> Blog: http://www.codersrevolution.com >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> undertow-dev mailing list >>>>>>>> undertow-dev(a)lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/undertow-dev >>>>>>> >>>>>>> _______________________________________________ >> undertow-dev mailing list >> undertow-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/undertow-dev > >