Embedding a WF instance in the CLI
by Brian Stansberry
Moving a thread to the dev list.
This is about some prototyping I've been doing on weekends 'cause I'm
bored with my regular tasks. I've been playing with direct local
administration of a WF installation via the CLI without requiring a
socket-based connection. The general use case is initial setup type
activities where the user doesn't want to have to launch a WF server or
HC process and potentially have it be visible on the network.
https://issues.jboss.org/browse/WFLY-3288 is one use case; another is a
desire some folks have expressed in being able to do configuration
without first having to edit any xml to avoid port conflicts on 9990 or
9999.
This isn't a major initiative or big priority or anything at this point.
Just something I find interesting and perhaps you will too.
On 5/14/14, 8:54 AM, Alexey Loubyansky wrote:
> Neat :) Yes, figuring out the module path is biting everywhere.
> For file system path command line arguments there is a specialized
> FileSystemPathArgument.
>
Thanks; I'll switch to that.
>
> On 05/13/2014 10:54 PM, Brian Stansberry wrote:
>> Copying Heiko Braun as he expressed some interest in the topic.
>>
>> BTW, I played with this a bit more last weekend and was able to start an
>> embedded server inside the CLI easily enough. See [1] for very raw
>> prototype stuff. You can run bin/jboss-cli.sh (no -c) and then
>>
>> [disconnected/] embed-server
>>
>> There are a couple issues I see, besides the HC stuff I mentioned in my
>> last message.
>>
>> 1) If the CLI is started in a non-modular environment via java -jar
>> bin/client/jboss-cli-client.jar, we'd have to shade jboss-modules into
>> the jar. And then the embed-server command would need params specifying
>> the location of JBOSS_HOME, possibly module path etc. But it could embed
>> a server installed in any accessible filesystem location.
>>
>> But what I did at [1] is based on bin/jboss-cli.sh, where the CLI is
>> running from a WF dist in a modular environment and the embedded server
>> modules are coming from the CLI's own module path. It would be more
>> effort to support embedding a server based on some other module path.
>> Maybe it's no big deal; maybe it's really hard. :)
>>
>> 2) The console logging from the embedded server goes to stdout mixed in
>> with the CLI output. Maybe that's good, maybe it's bad.
>>
>> [1] https://github.com/bstansberry/wildfly/tree/cli-embed
>>
>> On 4/28/14, 10:04 AM, Brian Stansberry wrote:
>>> I was poking around at this for an hour or so over the weekend.
>>>
>>> The standalone case seems pretty straightforward. Seems the existing
>>> embedded server API could work readily enough. The
>>> org.jboss.as.embedded.StandaloneServer interface already provides a
>>> ModelControllerClient.
>>>
>>> The domain case is much harder, as the CLI wants a HostController, not a
>>> ProcessController. I'd really like this to use an in-VM client, not a
>>> remote one, so I don't like having the CLI embed a PC and then the HC is
>>> an external process. My thoughts of the morning are to allow inverting
>>> the HC/PC relationship for this kind of usage. That is, remove
>>> controlling the HC lifecycle from the charge of the PC component. CLI
>>> launches HC, and then the HC creates an in-process PC-ish component (not
>>> a separate process) to manage the server lifecycles. There could be all
>>> sorts of problems with that; it's just the thought for the morning.
>>>
>>> On 4/25/14, 11:49 AM, Alexey Loubyansky wrote:
>>>> Embedding the AS is the best starting point to achieve that! And more
>>>> fun, I agree :)
>>>>
>>>> On 04/25/2014 06:28 PM, Darran Lofthouse wrote:
>>>>> And to think my reason for opening the Jira was just for a common
>>>>> way to
>>>>> mask password inputs where java.io.Console is not available ;-)
>>>>>
>>>>> On 25/04/14 17:09, Brian Stansberry wrote:
>>>>>> On 4/25/14, 10:40 AM, Alexey Loubyansky wrote:
>>>>>>> Wow! Indeed :)
>>>>>>>
>>>>>>> There could be an embedded scope - true, i.e. commands available
>>>>>>> only
>>>>>>> this mode, like add-user, module mgmt related stuff, etc.
>>>>>>
>>>>>> Those commands wouldn't need to be only in that mode though. The
>>>>>> implementation of all of them would be based in the server; the
>>>>>> "client"
>>>>>> aspect of the CLI would just use the management interface. The
>>>>>> difference between an embedded mode and what we have now would
>>>>>> just be
>>>>>> in how the "client" side gets its ModelControllerClient -- what we
>>>>>> have
>>>>>> now vs starting an embedded server and getting some sort of in-vm
>>>>>> client.
>>>>>>
>>>>>>> But it would still mean the server/controller would have to actually
>>>>>>> provide implementations of that functionality and expose it to the
>>>>>>> management tools like the CLI in the embedded mode.
>>>>>>
>>>>>> Yep.
>>>>>>
>>>>>>> I like this idea as a concept - direct local management. W/o any
>>>>>>> remote
>>>>>>> connect/re-connect/disconnect burden.
>>>>>>>
>>>>>>> Extending the CLI with custom modules is on the list too. It's
>>>>>>> probably
>>>>>>> easier to implement at this point.
>>>>>>>
>>>>>>
>>>>>> Likely so, but maybe less fun. ;) I copied you on a PRD-related
>>>>>> thread
>>>>>> where I briefly get into this general area too.
>>>>>>
>>>>>>> Alexey
>>>>>>>
>>>>>>> On 04/25/2014 05:00 PM, Brian Stansberry wrote:
>>>>>>>> Hi Alexey,
>>>>>>>>
>>>>>>>> Wanted to point the discussion on this JIRA out to you as it gets
>>>>>>>> into
>>>>>>>> some fairly fundamental brainstorming that you may find
>>>>>>>> interesting.
>>>>>>>>
>>>>>>>>
>>>>>>>> -------- Original Message --------
>>>>>>>> Subject: [JBoss JIRA] (WFLY-3288) Update add-user to use AESH or
>>>>>>>> move it
>>>>>>>> into the CLI
>>>>>>>> Date: Fri, 25 Apr 2014 09:44:35 -0400 (EDT)
>>>>>>>> From: Darran Lofthouse (JIRA) <issues(a)jboss.org>
>>>>>>>> To: brian.stansberry(a)redhat.com
>>>>>>>>
>>>>>>>>
>>>>>>>> [
>>>>>>>> https://issues.jboss.org/browse/WFLY-3288?page=com.atlassian.jira.plugin....
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ]
>>>>>>>>
>>>>>>>> Darran Lofthouse commented on WFLY-3288:
>>>>>>>> ----------------------------------------
>>>>>>>>
>>>>>>>> That could be very interested, won't go into too much detail in
>>>>>>>> this
>>>>>>>> Jira as it is not directly related shortly I am switching to the
>>>>>>>> SSL
>>>>>>>> related tasks we have outstanding including the out of the box
>>>>>>>> enablement we talked about in Brno - managing an embedded instance
>>>>>>>> could
>>>>>>>> be useful there as well to get it all op based.
>>>>>>>>
>>>>>>>> I can see this task may end up coming back my way combined with the
>>>>>>>> other stuff ;-)
>>>>>>>>
>>>>>>>>> Update add-user to use AESH or move it into the CLI
>>>>>>>>> ---------------------------------------------------
>>>>>>>>>
>>>>>>>>> Key: WFLY-3288
>>>>>>>>> URL: https://issues.jboss.org/browse/WFLY-3288
>>>>>>>>> Project: WildFly
>>>>>>>>> Issue Type: Feature Request
>>>>>>>>> Security Level: Public(Everyone can see)
>>>>>>>>> Components: Domain Management, Scripts
>>>>>>>>> Reporter: Darran Lofthouse
>>>>>>>>> Fix For: Awaiting Volunteers
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Within the add-user utility it is difficult to handle situations
>>>>>>>>> where
>>>>>>>>> we do not have access to a java.io.Console which is the easiest
>>>>>>>>> way to
>>>>>>>>> handle password reading without an echo to the user e.g. in Cygwin
>>>>>>>>> Switching to AESH would allow us to use the implementation
>>>>>>>>> there to
>>>>>>>>> handle this.
>>>>>>>>> Alternatively it may actually make sense to make add-user a
>>>>>>>>> special
>>>>>>>>> mode of the CLI, we may at some point want to switch to runtime
>>>>>>>>> operations being executed on the server so porting to the CLI
>>>>>>>>> could be
>>>>>>>>> the first step to make this possible.
>>>>>>>>> Overall this is going to require further discussion so the
>>>>>>>>> comments
>>>>>>>>> here are just a starting point.
>>>>>>>>
>>>>>>>> --
>>>>>>>> This message is automatically generated by JIRA.
>>>>>>>> If you think it was sent incorrectly, please contact your JIRA
>>>>>>>> administrators
>>>>>>>> For more information on JIRA, see:
>>>>>>>> http://www.atlassian.com/software/jira
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>
>>
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
11 years, 3 months
Automatically resolving expressions in the CLI
by Edward Wertz
I'm looking into whether it's possible to automatically resolve expressions when executing operations and commands in the CLI.
>From my understanding, there are two variations of the problem.
* Operations are server-side processes that are accessed via ':' in the CLI and, currently, the CLI presents the results returned as-is to the users. ex: ':read-resource'
* Commands are processes that get manipulated by the CLI before getting presented to users. ex: 'ls'
I've been experimenting with adding arguments to the CLI commands, like 'ls --resolve-expressions', and gotten it working for the standalone and domain side of things. However, I can't control the scope of the argument, so it's available in situations that cannot accurately resolve expressions like the 'profile=full' section of the domain tree. The results wouldn't be reliable.
The same problem would apply to adding parameters to the server-side operations. The scope of the operations themselves can be controlled, but not their parameters. An execution like ':read-resource(recursive=true resolve-expressions=true)' can't resolve expressions unless it's used against an actual server or host, but the operation is available almost everywhere. Again, the results wouldn't be reliable.
I'm wondering if anyone can suggest a way to attack this problem? There is already a ':resolve-expression(expression=___)' operation, so users can somewhat laboriously get the runtime values they want, but I can't figure out a way to integrate the values into the existing framework successfully. Other than creating entirely new operations and commands, like 'ls-resolve' and ':read-resource-resolve', which seems like an unsustainable way to solve the problem.
Thanks,
Joe Wertz
11 years, 7 months
Design Proposal: Server suspend/resume (AKA Graceful Shutdown)
by Stuart Douglas
Server suspend and resume is a feature that allows a running server to
gracefully finish of all running requests. The most common use case for
this is graceful shutdown, where you would like a server to complete all
running requests, reject any new ones, and then shut down, however there
are also plenty of other valid use cases (e.g. suspend the server,
modify a data source or some other config, then resume).
User View:
For the users point of view two new operations will be added to the server:
suspend(timeout)
resume()
A runtime only attribute suspend-state (is this a good name?) will also
be added, that can take one of three possible values, RUNNING,
SUSPENDING, SUSPENDED.
A timeout attribute will also be added to the shutdown operation. If
this is present then the server will first be suspended, and the server
will not shut down until either the suspend is successful or the timeout
occurs. If no timeout parameter is passed to the operation then a normal
non-graceful shutdown will take place.
In domain mode these operations will be added to both individual server
and a complete server group.
Implementation Details
Suspend/resume operates on entry points to the server. Any request that
is currently running must not be affected by the suspend state, however
any new request should be rejected. In general subsystems will track the
number of outstanding requests, and when this hits zero they are
considered suspended.
We will introduce the notion of a global SuspendController, that manages
the servers suspend state. All subsystems that wish to do a graceful
shutdown register callback handlers with this controller.
When the suspend() operation is invoked the controller will invoke all
these callbacks, letting the subsystem know that the server is suspend,
and providing the subsystem with a SuspendContext object that the
subsystem can then use to notify the controller that the suspend is
complete.
What the subsystem does when it receives a suspend command, and when it
considers itself suspended will vary, but in the common case it will
immediatly start rejecting external requests (e.g. Undertow will start
responding with a 503 to all new requests). The subsystem will also
track the number of outstanding requests, and when this hits zero then
the subsystem will notify the controller that is has successfully
suspended.
Some subsystems will obviously want to do other actions on suspend, e.g.
clustering will likely want to fail over, mod_cluster will notify the
load balancer that the node is no longer available etc. In some cases we
may want to make this configurable to an extent (e.g. Undertow could be
configured to allow requests with an existing session, and not consider
itself timed out until all sessions have either timed out or been
invalidated, although this will obviously take a while).
If anyone has any feedback let me know. In terms of implementation my
basic plan is to get the core functionality and the Undertow
implementation into Wildfly, and then work with subsystem authors to
implement subsystem specific functionality once the core is in place.
Stuart
The
A timeout attribute will also be added to the shutdown command,
11 years, 10 months
Core and subsystem capabilities and requirements
by Brian Stansberry
As we continue with our work splitting the WildFly code base into a core
repo and then separate repos related to sets of features that we need to
solidify the contracts between the various features and between features
and the core.
I've taken a crack at an initial design document on this: see [1]. We
also need to do the practical work of identifying the various
dependencies between our existing subsystems, see [2] for a start on that.
I'd love to get feedback on this thread regarding the proposed design,
as well as get direct edits on the [2] doc to flesh out the existing
relationships.
Short version:
A capability is a set of functionality that becomes available when a
user triggers it by including some configuration resource or attribute
in the management model.
We'll identify capabilities via two strings: the name of the providing
subsystem and then a capability name. A null subsystem name means a core
capability; a null capability name means the base capability of the
subsystem.
Capabilities will also declare the identifiers of any other capabilities
they require.
There are two use cases for this capability/requirement data:
provisioning (hopefully) and runtime.
Hopefully this information can be used by provisioning tooling when
building up a configuration document for the server/domain it is
provisioning. So instead of always including a stock configuration for a
subsystem, allow the user to tailor it a bit by declaring what
capabilities are required.
At runtime, when the configuration model is updated in such a way that a
capability is now required, the OSH that handles that update will
register the capability with the management layer in the MODEL stage. At
the end of the MODEL stage the management layer will resolve all
provided and required capabilities, failing the op if anything required
is unavailable.
Thereafter, in the RUNTIME stage an OSH that needs a capability from
another subsystem or the core can request an object implementing the API
for that capability from the OperationContext.
I've thought a lot more about the runtime use case than I have about the
provisioning use case.
[1] https://community.jboss.org/docs/DOC-52712
[2] https://community.jboss.org/docs/DOC-52700
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
11 years, 10 months
Pending core split
by Stuart Douglas
Hi all,
So I am moderately confident that we will be ready to split out Wildfly
core into a separate repository early next week (I'm not saying that it
will definitely happen in this time frame, just that it should be possible).
Once this is ready to go I think the basic process will be:
- Code freeze on Master
- Create the core repo, push new rewritten core history
- Release core 1.0.0.Beta1
- Create PR against core WF repo that deletes everything in core, and
uses the core 1.0.0.Beta1 release
- End of code freeze
Stuart
11 years, 10 months
CDI overhead
by Andrig Miller
I should have posted this some time ago, but just forgot.
In my early testing of Wildfly 8, CDI adds quite a bit of overhead (12% reduction in throughput) for even an application that only uses servlets. The only way I could get that back was to remove the subsystem. In talking with Stuart at the time, he was looking at ways to make the overhead less.
Is there anything on the docket for making this overhead go away for deployments that don't require CDI? If not, can we get something going in that direction. It would be great to not have to remove the CDI subsystem, but not have it impact performance for deployments that don't use it.
Thanks.
--
Andrig (Andy) Miller
Global Platform Director for JBoss Middle-ware
Red Hat, Inc.
11 years, 10 months
build on Solaris broken now
by Frank Langelage
for those who might want to build current 9.0.0-SNAPSHOT from github
sources on Solaris:
after commit of checking / downloading maven if needed this currently
fails with a syntax error in tools/maven/bin/mvn after downloading the
latest version 3.2.2.
Maven 3.2.2 's bin/mvn contains incompatible syntax for Solaris /bin/sh
and perhaps others:
if [[ -z "$JAVA_HOME" && -x /usr/libexec/java_home ]] ; then
#
# Apple JDKs
#
export JAVA_HOME=$(/usr/libexec/java_home) // here it fails
fi
;;
I created a JIRA fpr Apache Maven to change this line to
export JAVA_HOME=/usr/libexec/java_home
Doing this change after download / unzip in my local repo make the build
working again for me.
Regards, Frank
11 years, 10 months
Strange enforcer error
by Stan Silvert
I just rebased from master and I get the enforcer error below. Any idea
where the reference to org.wildfly:wildfly-server:8.1.0.Final comes from?
mvn dependency:tree doesn't show it. I've grepped everything for
8.1.0.Final and don't see anything either.
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building WildFly: Server 9.0.0.Alpha1-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (ban-bad-dependencies) @
wildfly-server ---
[WARNING]
Dependency convergence error for
org.wildfly:wildfly-server:9.0.0.Alpha1-SNAPSHOT paths to dependency are:
+-org.wildfly:wildfly-server:9.0.0.Alpha1-SNAPSHOT
and
+-org.wildfly:wildfly-server:9.0.0.Alpha1-SNAPSHOT
+-org.wildfly:wildfly-domain-http-interface:9.0.0.Alpha1-SNAPSHOT
+-org.wildfly:wildfly-server:8.1.0.Final
[WARNING] Rule 1:
org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
Failed while enforcing releasability the error(s) are [
Dependency convergence error for
org.wildfly:wildfly-server:9.0.0.Alpha1-SNAPSHOT paths to dependency are:
+-org.wildfly:wildfly-server:9.0.0.Alpha1-SNAPSHOT
and
+-org.wildfly:wildfly-server:9.0.0.Alpha1-SNAPSHOT
+-org.wildfly:wildfly-domain-http-interface:9.0.0.Alpha1-SNAPSHOT
+-org.wildfly:wildfly-server:8.1.0.Final
]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 1.744 s
[INFO] Finished at: 2014-06-23T20:56:18-05:00
[INFO] Final Memory: 19M/429M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
(ban-bad-dependencies) on project wildfly-server: Some Enforcer rules
have fai
led. Look above for specific messages explaining why the rule failed. ->
[Help 1]
[ERROR]
11 years, 10 months
