Proposal for improving handling complex types in CLI
by Tomaž Cerar
Hi guys,
there ware some discussions on how we should improve handling complex types
of attribute bit better in CLI.
For most part that was about Map & List types.
After some discussions with few of you I came up with plan / ideas what all
options are there for us to improve on.
you can see current state of proposed enhancements at
https://gist.github.com/ctomc/91055a6f4e7502dcd130
In short, I propose to add set of map-* and list-* global operations and
improve :read-attribute & :write-attribute
with EL like syntax for reading / updating map, list and generic complex
attributes.
Let me know what you think about it, especially Console & CLI folks.
--
tomaz
9 years
Customizing a provisioned server
by Stuart Douglas
Hi everyone,
Work on the provisioning tool is now well underway, so I would like to
revisit something I mentioned in my original email, which is allowing
the provisioning tool to customize a provisioned server.
I think there are a few options here, some more palatable than others.
In no particular order:
1) Customize the XML directly
Using this approach we would just directly customize the XML
configuration files. This would basically require the use of XSLT
(yuck), or require us to basically invent our own version of XSLT (even
more yuck). Even though this approach will work, and will be fairly easy
to implement, I think it would really suck from an end-user point of
view, and I think we should discount it.
2) Allow the user to provide CLI commands to customise the server
This is by far my favorite approach. The provisioning file would just
contain a list of CLI commands, and would execute them in order. I think
this is by far the most intuitive, and the CLI is well documented.
3) Allow the user to provide DMR operations to customize the server
Similar to 2, but allow the user to provide DMR or JSON operations to
customize the server. I think this is not nearly as nice as 2, as users
are much more likely to be familiar with the CLI rather than DMR.
I think 2 is by far the best approach, however it does open up the
question of how and when to execute the operations. I think the easiest
way to do this would be to just start the server in admin only mode on a
custom port (so it will not interfere with any existing running Wildfly
instances), and just execute the CLI commands in admin only mode.
Does this all sound reasonable?
Stuart
9 years, 2 months
Embedding a WF instance in the CLI
by Brian Stansberry
Moving a thread to the dev list.
This is about some prototyping I've been doing on weekends 'cause I'm
bored with my regular tasks. I've been playing with direct local
administration of a WF installation via the CLI without requiring a
socket-based connection. The general use case is initial setup type
activities where the user doesn't want to have to launch a WF server or
HC process and potentially have it be visible on the network.
https://issues.jboss.org/browse/WFLY-3288 is one use case; another is a
desire some folks have expressed in being able to do configuration
without first having to edit any xml to avoid port conflicts on 9990 or
9999.
This isn't a major initiative or big priority or anything at this point.
Just something I find interesting and perhaps you will too.
On 5/14/14, 8:54 AM, Alexey Loubyansky wrote:
> Neat :) Yes, figuring out the module path is biting everywhere.
> For file system path command line arguments there is a specialized
> FileSystemPathArgument.
>
Thanks; I'll switch to that.
>
> On 05/13/2014 10:54 PM, Brian Stansberry wrote:
>> Copying Heiko Braun as he expressed some interest in the topic.
>>
>> BTW, I played with this a bit more last weekend and was able to start an
>> embedded server inside the CLI easily enough. See [1] for very raw
>> prototype stuff. You can run bin/jboss-cli.sh (no -c) and then
>>
>> [disconnected/] embed-server
>>
>> There are a couple issues I see, besides the HC stuff I mentioned in my
>> last message.
>>
>> 1) If the CLI is started in a non-modular environment via java -jar
>> bin/client/jboss-cli-client.jar, we'd have to shade jboss-modules into
>> the jar. And then the embed-server command would need params specifying
>> the location of JBOSS_HOME, possibly module path etc. But it could embed
>> a server installed in any accessible filesystem location.
>>
>> But what I did at [1] is based on bin/jboss-cli.sh, where the CLI is
>> running from a WF dist in a modular environment and the embedded server
>> modules are coming from the CLI's own module path. It would be more
>> effort to support embedding a server based on some other module path.
>> Maybe it's no big deal; maybe it's really hard. :)
>>
>> 2) The console logging from the embedded server goes to stdout mixed in
>> with the CLI output. Maybe that's good, maybe it's bad.
>>
>> [1] https://github.com/bstansberry/wildfly/tree/cli-embed
>>
>> On 4/28/14, 10:04 AM, Brian Stansberry wrote:
>>> I was poking around at this for an hour or so over the weekend.
>>>
>>> The standalone case seems pretty straightforward. Seems the existing
>>> embedded server API could work readily enough. The
>>> org.jboss.as.embedded.StandaloneServer interface already provides a
>>> ModelControllerClient.
>>>
>>> The domain case is much harder, as the CLI wants a HostController, not a
>>> ProcessController. I'd really like this to use an in-VM client, not a
>>> remote one, so I don't like having the CLI embed a PC and then the HC is
>>> an external process. My thoughts of the morning are to allow inverting
>>> the HC/PC relationship for this kind of usage. That is, remove
>>> controlling the HC lifecycle from the charge of the PC component. CLI
>>> launches HC, and then the HC creates an in-process PC-ish component (not
>>> a separate process) to manage the server lifecycles. There could be all
>>> sorts of problems with that; it's just the thought for the morning.
>>>
>>> On 4/25/14, 11:49 AM, Alexey Loubyansky wrote:
>>>> Embedding the AS is the best starting point to achieve that! And more
>>>> fun, I agree :)
>>>>
>>>> On 04/25/2014 06:28 PM, Darran Lofthouse wrote:
>>>>> And to think my reason for opening the Jira was just for a common
>>>>> way to
>>>>> mask password inputs where java.io.Console is not available ;-)
>>>>>
>>>>> On 25/04/14 17:09, Brian Stansberry wrote:
>>>>>> On 4/25/14, 10:40 AM, Alexey Loubyansky wrote:
>>>>>>> Wow! Indeed :)
>>>>>>>
>>>>>>> There could be an embedded scope - true, i.e. commands available
>>>>>>> only
>>>>>>> this mode, like add-user, module mgmt related stuff, etc.
>>>>>>
>>>>>> Those commands wouldn't need to be only in that mode though. The
>>>>>> implementation of all of them would be based in the server; the
>>>>>> "client"
>>>>>> aspect of the CLI would just use the management interface. The
>>>>>> difference between an embedded mode and what we have now would
>>>>>> just be
>>>>>> in how the "client" side gets its ModelControllerClient -- what we
>>>>>> have
>>>>>> now vs starting an embedded server and getting some sort of in-vm
>>>>>> client.
>>>>>>
>>>>>>> But it would still mean the server/controller would have to actually
>>>>>>> provide implementations of that functionality and expose it to the
>>>>>>> management tools like the CLI in the embedded mode.
>>>>>>
>>>>>> Yep.
>>>>>>
>>>>>>> I like this idea as a concept - direct local management. W/o any
>>>>>>> remote
>>>>>>> connect/re-connect/disconnect burden.
>>>>>>>
>>>>>>> Extending the CLI with custom modules is on the list too. It's
>>>>>>> probably
>>>>>>> easier to implement at this point.
>>>>>>>
>>>>>>
>>>>>> Likely so, but maybe less fun. ;) I copied you on a PRD-related
>>>>>> thread
>>>>>> where I briefly get into this general area too.
>>>>>>
>>>>>>> Alexey
>>>>>>>
>>>>>>> On 04/25/2014 05:00 PM, Brian Stansberry wrote:
>>>>>>>> Hi Alexey,
>>>>>>>>
>>>>>>>> Wanted to point the discussion on this JIRA out to you as it gets
>>>>>>>> into
>>>>>>>> some fairly fundamental brainstorming that you may find
>>>>>>>> interesting.
>>>>>>>>
>>>>>>>>
>>>>>>>> -------- Original Message --------
>>>>>>>> Subject: [JBoss JIRA] (WFLY-3288) Update add-user to use AESH or
>>>>>>>> move it
>>>>>>>> into the CLI
>>>>>>>> Date: Fri, 25 Apr 2014 09:44:35 -0400 (EDT)
>>>>>>>> From: Darran Lofthouse (JIRA) <issues(a)jboss.org>
>>>>>>>> To: brian.stansberry(a)redhat.com
>>>>>>>>
>>>>>>>>
>>>>>>>> [
>>>>>>>> https://issues.jboss.org/browse/WFLY-3288?page=com.atlassian.jira.plugin....
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ]
>>>>>>>>
>>>>>>>> Darran Lofthouse commented on WFLY-3288:
>>>>>>>> ----------------------------------------
>>>>>>>>
>>>>>>>> That could be very interested, won't go into too much detail in
>>>>>>>> this
>>>>>>>> Jira as it is not directly related shortly I am switching to the
>>>>>>>> SSL
>>>>>>>> related tasks we have outstanding including the out of the box
>>>>>>>> enablement we talked about in Brno - managing an embedded instance
>>>>>>>> could
>>>>>>>> be useful there as well to get it all op based.
>>>>>>>>
>>>>>>>> I can see this task may end up coming back my way combined with the
>>>>>>>> other stuff ;-)
>>>>>>>>
>>>>>>>>> Update add-user to use AESH or move it into the CLI
>>>>>>>>> ---------------------------------------------------
>>>>>>>>>
>>>>>>>>> Key: WFLY-3288
>>>>>>>>> URL: https://issues.jboss.org/browse/WFLY-3288
>>>>>>>>> Project: WildFly
>>>>>>>>> Issue Type: Feature Request
>>>>>>>>> Security Level: Public(Everyone can see)
>>>>>>>>> Components: Domain Management, Scripts
>>>>>>>>> Reporter: Darran Lofthouse
>>>>>>>>> Fix For: Awaiting Volunteers
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Within the add-user utility it is difficult to handle situations
>>>>>>>>> where
>>>>>>>>> we do not have access to a java.io.Console which is the easiest
>>>>>>>>> way to
>>>>>>>>> handle password reading without an echo to the user e.g. in Cygwin
>>>>>>>>> Switching to AESH would allow us to use the implementation
>>>>>>>>> there to
>>>>>>>>> handle this.
>>>>>>>>> Alternatively it may actually make sense to make add-user a
>>>>>>>>> special
>>>>>>>>> mode of the CLI, we may at some point want to switch to runtime
>>>>>>>>> operations being executed on the server so porting to the CLI
>>>>>>>>> could be
>>>>>>>>> the first step to make this possible.
>>>>>>>>> Overall this is going to require further discussion so the
>>>>>>>>> comments
>>>>>>>>> here are just a starting point.
>>>>>>>>
>>>>>>>> --
>>>>>>>> This message is automatically generated by JIRA.
>>>>>>>> If you think it was sent incorrectly, please contact your JIRA
>>>>>>>> administrators
>>>>>>>> For more information on JIRA, see:
>>>>>>>> http://www.atlassian.com/software/jira
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>
>>
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
9 years, 2 months
Management Model: Squatter Resources
by Heiko Braun
TL;DR: A proposal for improving parts of the management API that deal with static resource definitions
Background:
For future Wildfly versions we plan to re-architect the management console to make better use the existing meta data. The goal is to provide a data binding layer that automates much of the retrieval and update of the configuration and runtime data. To achieve this goal, we need to remove some of the roadblocks that prevent further automation. This is the first of a series of posts that explains some of the challenges we facing and a proposal to improve the situation.
Problem:
Typically we deal with resources that addressable through a key value pair, where the key of the tuple depicts the type of the resource and the value the name or identify of a specific resource instance, i.e.:
/subsystem=datasources/data-source=ExampleDS
In this case 'ExampleDS' is the name of a resource of type 'data-source'. The type is associated with a specific resource definition, that's typically retrieved through the :read-resource-description operation. In the following sections I am going to refer to these resource as 'regular' resources.
In some situations, it seems more feasible (and valid) to construct a model representation without instance names, i.e.:
/subsystem=ejb3/service=[async | remote | timer-service]
In this case each resource under /subsystem=ejb3/service has a different type and only a single instance can exist:
/service=async
/service=remote
/service=timer-service
Lacking a better alternative, I coined the term 'squatter' resources, because these resources squat a specific address slots. Squatter resources can be pre-registered or non-exisiting when the management layer is started.
In order to provide a data binding layer for the management console, we need to access the definition of a resource (:read-resource-description). With squatter resources this is basically impossible, because the existence of these types is hidden in the general API. I.e. sticking to the previous example, when querying the resource definition for the ejb3 subsystem, it pretends only single type 'service' exists, although it's actually three different 'service' types
./subsystem=ejb3:read-children-types
{
"outcome" => "success",
"result" => [
[...],
"service",
[...]
]
}
Proposal:
After a chat with Brian yesterday, we are proposing a new request parameter to the :read-children-types operations, that augments the result to reveal the existence of squatter resources:
./subsystem=ejb3:read-children-types(include-squatters=true)
{
"outcome" => "success",
"result" => [
[...],
"service=async",
"service=remote",
"service=timer-service"
[...]
]
}
This works in a backwards compatible way and allows us to identify squatter resources and get to their resource definition.
Thoughts and comments welcome.
If you can think of better term than 'squatter' resources, please let me know as well.
Regards, Heiko
9 years, 6 months
Proposal to add notifications to WildFly management model and API
by Jeff Mesnil
# Add Notification support to WildFly Management
Tracked by https://issues.jboss.org/browse/WFLY-266
Use Cases
---------
Notifications are an useful mechanism to observe management changes on WildFly servers.
It allows an administrator to be informed of changes outside of his own actions (e.g. a server has been killed, a new application is deployed, etc.)
Currently WildFly lacks notifications and users that were depending on JMX notifications in previous versions have no similar feature to use.
The most expected use cases for WildFly notifications are:
- enhance UX for Web console. Using notifications, the Web console could notify the users of changes outside its own actions.
- replacement for JMX notifications. Users that were listening for JMX notifications to observe management changes would have a similar feature using WildFly own notifications
- integration with JMX. Notifications emitted by WildFly could be converted and made available using JMX notifications (including notifications for mbean registered/unregistered)
Part 1: Notification Definition
-------------------------------
A resource will define the notifications it emits. These definitions will be added to the attributes and operations definitions on a resource.
{
"description" => "A manageable resource",
"attributes" => {
...
},
"operations" => {
...
},
"notifications" => {
"resource-added" => {
...
}
},
"children" => {
...
}
}
The description of a notification will be composed of:
* type - String - the type of notification (resource-added, server-stopped, etc.)
* description - String - i18ned description of the notification
* access-constraints - the RBAC access constraints that controls who can receive the notifications
* data-type - ModelType or complex structure - optional - only present if the notification will have a data value. data-type will detail the structure of the data value, enumerating the value's fields and the type of their value
The read-resource-description will be enhanced with a notifications parameter (boolean) to include the notifications descriptions (default value is false, same as the operations parameter).
The ManagementResourceRegistration interface will be enhanced to register a notification definition with registerNotification(NotificationDefinition notification). The NotificationDefinition interface corresponds to the detyped representation of a notifications and comes with a builder API.
Part 2: Emitting a notification
-------------------------------
A notification can be emitted in any OperationStepHandler using the OperationContext.emit(Notification method)
public void execute(OperationContext context, ModelNode operation) throws OperationFailedException {
// perform some actions
...
context.emit(new Notification(SERVER_RESTARTED_NOTIFICATION, address, ROOT_LOGGER.serverHasBeenRestarted()));
context.stepCompleted();
}
The notification is *not* emitted (i.e. delivered to interested parties) when OperationContext.emit() is called. It is emitted at the end of the operation step only if it is successful. A call to OperationContext.emit() will have no effect if the operation is rolled back.
Notification emission is done asynchronously using the server thread pool and does not block the execution of the operation that triggered the notification: having zero or any notification handlers must have no impact of the execution of the operation.
A Notification is a simple Java class that represents the notification. It is composed of:
* type - String - the notification type
* address - PathAddress - the address of the resource that emits the notification
* message - String - the i18ned description of the message
* timestamp - long - the timestamp of the notification. It is set when the Notification object is created.
* data - ModelNode - optional - a detyped representation of data associated to the notification. If a notification includes a data field, its definition must describe it (in its data-type parameter).
If RBAC is enabled, the notification access-constraints will be checked to ensure that the handler have the required privileges to receive the notification. Notification will potentially contain critical information (e.g. if a security-credential attribute is updated, the notification will contain its old and new values) and must be constrained accordingly.
Part 3: Global Resource Notifications
——————————————————
In the same way that some operations are available for any resource (e.g. add, remove, read-resource-description), some notifications will be added to any resource of WildFly management model:
* resource-added - when a resource is added, it emits a resource-added notification
* resource-removed - when a resource is removed, it emits a resource-removed notification
* attribute-value-written - when a write-attribute operation is performed successfully on a resource, it emits a attribute-value-written notification. The notification's data field contains the following information:
* name - String - the name of the attribute
* old-value - the detyped representation of the previous value of the attribute
* new-value - the detyped representation of the new value
Part 4: Notification Handlers
——————————————
Any interested parties can receive notifications by registering a NotificationHandler using the ModelController.getNotificationSupport().registerNotificationHandler(source, handler, filter) method.
The source is a path address to handle notifications emitted by resources at this address.
The NotificationHandler is an interface with a single handleNotification(Notification notification) method.
The isNotificationEnabled(Notification notification) is an interface with a single isNotificationEnabled(Notification notification) method to filter out uninteresting notifications.
There is a similar unregister method to unregister a (handler, filter)
To be useful, the source path address will have to accept wildcards for the address' values:
* /subsystem=messaging/hornetq-server=* to receive notifications emitted by any hornetq-server resources
* /subsystem=messaging/hornetq-server=*/jms-queue=* to receive notifications emitted by any jms-queue on any hornetq-server resources
Wildcards for address' keys or key/value paris are not allowed (/subsystem=messaging/*=*/jms-queue=* and /subsystem=messaging/*/jms-queue=* are not valid).
This notion of wildcard for the resource addresses should be made to match current usage (e.g. in the CLI).
The main reason for the wildcard is for the resource-added/resource-removed notifications. I find more intuitive to have the notifications at the same resource-level than their corresponding add/remove operations. However until the resource is created, there is no way to register a notification listener on it without using a wildcard.
If that proves problematic, we could change this approach with two alternatives:
* have a single well-known resource emit the notifications for all resource (that's the JMX approach). A likely candidate would be /core-service=management
* the resource-added/-removed notifications can be emitted by the resource parents (but it only fixes the issue for the last leaf of the address tree…)
I still have questions about RBAC enforcements and it is possible that the registration of a handler will have to be done with additional metadata identifying the user roles wrt RBAC...
Part 5: Domain Notifications
——————————————
Notifications are also intended to work in domain mode. In particular, they will be used to observe server state.
The following notifications will be emitted by resources at /host=XXX/server-config=YYY (i.e. the resource to start/stop/etc. a server):
* server-started
* server-stopped
* server-restarted
* server-destroyed
* server-killed
Part 6: Integration with local JMX
—————————————————
The jmx subsystem will be updated to leverage the WildFly notifications and expose them as MBean notifications in our jmx facade for the management model:
* the WildFly notification description will be converted to MBeanNotificationInfo and added to the MBeanInfo
* when a JMX notification listener is added to an ObjectName, a WildFly NotificationHandler will be added to the path address corresponding to the ObjectName.
* depending on the user feedback, we may provide a hack to convert some WildFly notifications to their well-known JMX equivalent notifications (e.g. resource-added => jmx.mbean.registered).
In a first step, integration will be limited to use of JMX locally. Remoting will not be supported.
Part 7: Integration with Remote Management API
———————————————————————
We will enhance the remote management native API to register/unregister a notification handler from the ModelControllerClient
void registerNotificationHandler(ModelNode resourceAddress, NotificationClientHandler handler, NotificationClientFilter filter);
The client contract will have to taken into account reconnection when server is reloaded (possibly by caching the handler & filter and register them again after reconnection to the server...)
The Management HTTP API will also be enhance to support notifications with its REST API.
A neat addition will be to provide a browser-specific way to push notifications to the browser (e.g. using Server-Sent Events or Web Sockets).
=> the Web Console is the recipient for this feature and will have their say in how they prefer to consume notifications
Part 8: Integration with Remote JMX
—————————————————
Once the WildFly Management API will support notifications (for both native and HTTP), we can add support to JMX remotely (if there is any user interest for it).
Part 9: Web Console UX improvement
—————————————————
Once the Management HTTP API supports notifications, the Web console can leverage it to improve its UX.
This is a task that touch different parts of the app server (mainly in wildfly-core though) and I intend to split it in different JIRA issues (approx. one for each part) that can be merged one after the other instead of a big huge commit.
What do you think?
jeff
--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/
9 years, 6 months
Deliver 3rd party subsystem to customers?
by Heiko W.Rupp
Hey,
suppose I am writing a subsystem for WildFly, that I can or want not add to the generic WildFly codebase (e.g. because it is not open source or because WildFly team would consider it too special for general consumption).
Is there a way to package that up in a .zip or any other format (in the good 'ol days we used .shar files :-)
to deliver such a subsystem with its module but also the xsl to modify standalone.xml (or similar)?
If not (yet), would it make sense for WildFly to reconsider allowing to provide e.g.
* ext.xml
* subsystem.xml
* ports.xml
that will get merged / added to standalone.xml when the module is loaded for the very first time
(like when the module.jar is also indexed for the first time).
Thanks
Heiko
9 years, 6 months
Undertow statistics
by Heiko Braun
I am looking for some undertow metrics, like HTTP request/error rates, etc. I was expecting them to be exposed alongside with the listeners, but cannot find any. Is this missing or am I looking at the wrong resources?
/Heiko
9 years, 6 months
About the security manager lifecycle
by David M. Lloyd
At present in WildFly upstream, the security manager is only installed
when the security manager subsystem installation commences, leading to
PRs like this one [1] being rejected. However, feedback from various
quarters indicates that this relatively late installation may not be
acceptable for a couple different reasons. The current EAP version
supports using the -secmgr flag to the start scripts to tell the
bootstrap to install the security manager via jboss-modules' discovery
process, which happens at the very beginning of process start.
I'm thinking maybe we should bring this functionality forward, resurrect
#175, and modify the security manager subsystem to attach to the
currently installed security manager. This is also more friendly to
embedded processes; we should support (for example) permission
specification in deployments even if we don't directly control the
security manager. This also allows the security manager subsystem to
run even if no security manager is installed, so validation of
permissions.xml (for example) will still take place.
Thoughts?
[1] https://github.com/wildfly/wildfly-core/pull/175
--
- DML
9 years, 6 months
Adding jar to WAR from DeploymentProcessor
by Stan Silvert
The Keycloak auth server allows third-parties to add providers using the
ServiceLoader. To do this, you need to open Keycloak's auth-server.war
and drop your jar into the WEB-INF/lib directory.
I'd like to eliminate opening the WAR and have a DeploymentProcessor
look in a particular directory and add the jar at deployment time. What
is the cleanest way to do that?
Bonus points if you tell me that I can treat the service provider jar as
a deployment and let the user upload it via CLI. :-)
Thanks,
Stan
9 years, 7 months
