Re: [wildfly-dev] Embedding a WF instance in the CLI

I had some time over the holidays and on planes to progress quite a bit on this. A standalone server protype is at [1]. A fairly in depth description is on the WildFly Development wiki at [2]. I tried pretty hard to have clean commits on that branch, one per issue I faced. So looking at the commits is worthwhile to get a better understanding of particular aspects. Some notes on what I did / issues to discuss: 1) I semi-ported the "embedded" module from WildFly full to WildFly Core. "Semi" in the sense that the code is now in both places, under different maven GAVs and ending up in differently named modules in full. We need to regularize this; decide if there's any point in a "full" version of embedded, decide what to do about any APIs we don't want in the core version. (There are some deprecated methods, and one method "Context getContext()" that doesn't mean much in core, which has no JNDI.) 2) The CLI's use of stdio needed to be tweaked a bit to make it possible to control what the embedded server does with stdout. That's in the "Remove direct use of System.out by most CLI code" commit at [1]. 3) I needed to deal with some general embedding issues in the server; i.e. things that would probably pop up in any embedded use case: a) controlling the LogContext so the embedded server logging can be managed independent of the embedding app. See "Let apps that embed WildFly control the LogContext" commit at [1]. I don't think this is real solid though. For example, I expect CLI-side loggers that happen to get created after the embedded server starts will end up using the server LogContext. b) the server was calling System.exit in some places. See "[WFCORE-528] Use SystemExiter, not System.exit" commit at [1]. c) the embedded server code didn't deal with reload, leaving behind a broken ModelControllerClient. See "[WFCORE-511] Support reload in the embedded server" commit at [1]. 4) The modular vs non-modular environment aspects discussed at "Modular vs Non-Modular Classloading and JBOSS_HOME" in [2] are not ideal. I'm not sure how far we can/should go in improving this though. 5) This is painfully lacking in tests! Comments and suggestions are welcome! Cheers, Brian [1] https://github.com/bstansberry/wildfly-core/commits/cli-embed-server [2] https://developer.jboss.org/docs/DOC-53050 On 5/14/14 9:53 AM, John Mazzitelli wrote: > How topical :) The RHQ installer could use this - just this very second I'm debugging and trying to figure out why the RHQ installer can't connect to the running app server instance to do its initial config setup - having to try to figure out what port its running on and why I can't connect is a pain. > > ----- Original Message ----- >> Moving a thread to the dev list. >> >> This is about some prototyping I've been doing on weekends 'cause I'm >> bored with my regular tasks. I've been playing with direct local >> administration of a WF installation via the CLI without requiring a >> socket-based connection. The general use case is initial setup type >> activities where the user doesn't want to have to launch a WF server or >> HC process and potentially have it be visible on the network. >> https://issues.jboss.org/browse/WFLY-3288 is one use case; another is a >> desire some folks have expressed in being able to do configuration >> without first having to edit any xml to avoid port conflicts on 9990 or >> 9999. >> >> This isn't a major initiative or big priority or anything at this point. >> Just something I find interesting and perhaps you will too. >> >> On 5/14/14, 8:54 AM, Alexey Loubyansky wrote: >>> Neat :) Yes, figuring out the module path is biting everywhere. >>> For file system path command line arguments there is a specialized >>> FileSystemPathArgument. >>> >> Thanks; I'll switch to that. >> >>> On 05/13/2014 10:54 PM, Brian Stansberry wrote: >>>> Copying Heiko Braun as he expressed some interest in the topic. >>>> >>>> BTW, I played with this a bit more last weekend and was able to start an >>>> embedded server inside the CLI easily enough. See [1] for very raw >>>> prototype stuff. You can run bin/jboss-cli.sh (no -c) and then >>>> >>>> [disconnected/] embed-server >>>> >>>> There are a couple issues I see, besides the HC stuff I mentioned in my >>>> last message. >>>> >>>> 1) If the CLI is started in a non-modular environment via java -jar >>>> bin/client/jboss-cli-client.jar, we'd have to shade jboss-modules into >>>> the jar. And then the embed-server command would need params specifying >>>> the location of JBOSS_HOME, possibly module path etc. But it could embed >>>> a server installed in any accessible filesystem location. >>>> >>>> But what I did at [1] is based on bin/jboss-cli.sh, where the CLI is >>>> running from a WF dist in a modular environment and the embedded server >>>> modules are coming from the CLI's own module path. It would be more >>>> effort to support embedding a server based on some other module path. >>>> Maybe it's no big deal; maybe it's really hard. :) >>>> >>>> 2) The console logging from the embedded server goes to stdout mixed in >>>> with the CLI output. Maybe that's good, maybe it's bad. >>>> >>>> [1] https://github.com/bstansberry/wildfly/tree/cli-embed >>>> >>>> On 4/28/14, 10:04 AM, Brian Stansberry wrote: >>>>> I was poking around at this for an hour or so over the weekend. >>>>> >>>>> The standalone case seems pretty straightforward. Seems the existing >>>>> embedded server API could work readily enough. The >>>>> org.jboss.as.embedded.StandaloneServer interface already provides a >>>>> ModelControllerClient. >>>>> >>>>> The domain case is much harder, as the CLI wants a HostController, not a >>>>> ProcessController. I'd really like this to use an in-VM client, not a >>>>> remote one, so I don't like having the CLI embed a PC and then the HC is >>>>> an external process. My thoughts of the morning are to allow inverting >>>>> the HC/PC relationship for this kind of usage. That is, remove >>>>> controlling the HC lifecycle from the charge of the PC component. CLI >>>>> launches HC, and then the HC creates an in-process PC-ish component (not >>>>> a separate process) to manage the server lifecycles. There could be all >>>>> sorts of problems with that; it's just the thought for the morning. >>>>> >>>>> On 4/25/14, 11:49 AM, Alexey Loubyansky wrote: >>>>>> Embedding the AS is the best starting point to achieve that! And more >>>>>> fun, I agree :) >>>>>> >>>>>> On 04/25/2014 06:28 PM, Darran Lofthouse wrote: >>>>>>> And to think my reason for opening the Jira was just for a common >>>>>>> way to >>>>>>> mask password inputs where java.io.Console is not available ;-) >>>>>>> >>>>>>> On 25/04/14 17:09, Brian Stansberry wrote: >>>>>>>> On 4/25/14, 10:40 AM, Alexey Loubyansky wrote: >>>>>>>>> Wow! Indeed :) >>>>>>>>> >>>>>>>>> There could be an embedded scope - true, i.e. commands available >>>>>>>>> only >>>>>>>>> this mode, like add-user, module mgmt related stuff, etc. >>>>>>>> Those commands wouldn't need to be only in that mode though. The >>>>>>>> implementation of all of them would be based in the server; the >>>>>>>> "client" >>>>>>>> aspect of the CLI would just use the management interface. The >>>>>>>> difference between an embedded mode and what we have now would >>>>>>>> just be >>>>>>>> in how the "client" side gets its ModelControllerClient -- what we >>>>>>>> have >>>>>>>> now vs starting an embedded server and getting some sort of in-vm >>>>>>>> client. >>>>>>>> >>>>>>>>> But it would still mean the server/controller would have to actually >>>>>>>>> provide implementations of that functionality and expose it to the >>>>>>>>> management tools like the CLI in the embedded mode. >>>>>>>> Yep. >>>>>>>> >>>>>>>>> I like this idea as a concept - direct local management. W/o any >>>>>>>>> remote >>>>>>>>> connect/re-connect/disconnect burden. >>>>>>>>> >>>>>>>>> Extending the CLI with custom modules is on the list too. It's >>>>>>>>> probably >>>>>>>>> easier to implement at this point. >>>>>>>>> >>>>>>>> Likely so, but maybe less fun. ;) I copied you on a PRD-related >>>>>>>> thread >>>>>>>> where I briefly get into this general area too. >>>>>>>> >>>>>>>>> Alexey >>>>>>>>> >>>>>>>>> On 04/25/2014 05:00 PM, Brian Stansberry wrote: >>>>>>>>>> Hi Alexey, >>>>>>>>>> >>>>>>>>>> Wanted to point the discussion on this JIRA out to you as it gets >>>>>>>>>> into >>>>>>>>>> some fairly fundamental brainstorming that you may find >>>>>>>>>> interesting. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -------- Original Message -------- >>>>>>>>>> Subject: [JBoss JIRA] (WFLY-3288) Update add-user to use AESH or >>>>>>>>>> move it >>>>>>>>>> into the CLI >>>>>>>>>> Date: Fri, 25 Apr 2014 09:44:35 -0400 (EDT) >>>>>>>>>> From: Darran Lofthouse (JIRA) <issues(a)jboss.org&gt; >>>>>>>>>> To: brian.stansberry(a)redhat.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [ >>>>>>>>>> https://issues.jboss.org/browse/WFLY-3288?page=com.atlassian.jira.plugin.... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ] >>>>>>>>>> >>>>>>>>>> Darran Lofthouse commented on WFLY-3288: >>>>>>>>>> ---------------------------------------- >>>>>>>>>> >>>>>>>>>> That could be very interested, won't go into too much detail in >>>>>>>>>> this >>>>>>>>>> Jira as it is not directly related shortly I am switching to the >>>>>>>>>> SSL >>>>>>>>>> related tasks we have outstanding including the out of the box >>>>>>>>>> enablement we talked about in Brno - managing an embedded instance >>>>>>>>>> could >>>>>>>>>> be useful there as well to get it all op based. >>>>>>>>>> >>>>>>>>>> I can see this task may end up coming back my way combined with the >>>>>>>>>> other stuff ;-) >>>>>>>>>> >>>>>>>>>>> Update add-user to use AESH or move it into the CLI >>>>>>>>>>> --------------------------------------------------- >>>>>>>>>>> >>>>>>>>>>> Key: WFLY-3288 >>>>>>>>>>> URL: https://issues.jboss.org/browse/WFLY-3288 >>>>>>>>>>> Project: WildFly >>>>>>>>>>> Issue Type: Feature Request >>>>>>>>>>> Security Level: Public(Everyone can see) >>>>>>>>>>> Components: Domain Management, Scripts >>>>>>>>>>> Reporter: Darran Lofthouse >>>>>>>>>>> Fix For: Awaiting Volunteers >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Within the add-user utility it is difficult to handle situations >>>>>>>>>>> where >>>>>>>>>>> we do not have access to a java.io.Console which is the easiest >>>>>>>>>>> way to >>>>>>>>>>> handle password reading without an echo to the user e.g. in Cygwin >>>>>>>>>>> Switching to AESH would allow us to use the implementation >>>>>>>>>>> there to >>>>>>>>>>> handle this. >>>>>>>>>>> Alternatively it may actually make sense to make add-user a >>>>>>>>>>> special >>>>>>>>>>> mode of the CLI, we may at some point want to switch to runtime >>>>>>>>>>> operations being executed on the server so porting to the CLI >>>>>>>>>>> could be >>>>>>>>>>> the first step to make this possible. >>>>>>>>>>> Overall this is going to require further discussion so the >>>>>>>>>>> comments >>>>>>>>>>> here are just a starting point. >>>>>>>>>> -- >>>>>>>>>> This message is automatically generated by JIRA. >>>>>>>>>> If you think it was sent incorrectly, please contact your JIRA >>>>>>>>>> administrators >>>>>>>>>> For more information on JIRA, see: >>>>>>>>>> http://www.atlassian.com/software/jira >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>> >>>> >> >> -- >> Brian Stansberry >> Senior Principal Software Engineer >> JBoss by Red Hat >> _______________________________________________ >> wildfly-dev mailing list >> wildfly-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/wildfly-dev >>