Thanks Brian,

reply inlined.


On 04/12/2018 23:15, Brian Stansberry wrote:
This sounds pretty reasonable. There are a lot of layers but the aggregation layers let people deal with things with a reasonable granularity.

In the future in the tooling I don't know if there's a way to annotate layers somehow so some get some sort of prominence in the UI (a la what the WF CLI does with required attributes and tab completion.)  That's a very vague thought though, and something for the future.

More inline...


On Fri, Nov 30, 2018 at 11:54 AM Jean-Francois Denise <jdenise@redhat.com> wrote:
Hi,

here is an attempt to describe the galleon layers we could see exposed
in order to build custom configurations. The way layers are provisioned
by galleon make the provisioned server to be much smaller (in term of
XML config and num of provisioned modules) than the default
installation/configurations. We are here covering standalone kind of
configuration.

Feedbacks would be really welcome.

I took the approach to not have layers to depend on legacy
security-realms, we want elytron to be the way to go, so depending on
these first place would create a maintainability issue. This does create
some complexity when defining default configs, default config have to
associate realms with management and subsystems to produce the default
standalone configs we know. This complexity is not exposed.

Each layer is "runnable", you can provision it and run it. Only common.*
and standalone.* scripts are provisioned. Doesn't mean that running it
alone makes sense.

Any combination of layers should be valid.

A layer brings its own dependencies and its XML configuration. Each
subsystem (even server) is in charge to advertise to galleon what are
the additional modules it is injecting or loading that should be
provisioned but are not referenced from extension module. All the
subsystems involved in the following layers have been updated to express
these dependencies.

Because Full layers are far from being defined, we could expect some
adjustment.

For core:

Layers not bound to subsystems

- base-server: root resource, pubic interface and socket binding. All
layers depend at least on it.
- management: unsecured management interface, management sockets,
http-interface
- secure-management: depends on management, configures elytron sasl
authentication factory for management http interface
- tools: all modules and tools script files (cli, add-user,...)
- patching: patching modules.

AFAIK legacy patching won't work with a custom Galleon provisioned server. Is this layer just here as necessary bit to let us produce our standard dists, or do you see it being used for custom servers?
It is a layer introduced to be able to provision a complete default standalone configuration without inheriting default packages (that contains all modules). Your question makes me realize that it could be hidden by having the default config to simply includes the package (what the layer does).


Layers bound to subsystems

- core-management
- deployment-scanner
- discovery
- elytron (brings openssl artifacts, openssl can't be used if elytron
openssl provider is not configured.).
- io
- jmx
- jmx-remoting: depends on jmx and configure remoting-connector. Depend
on management (requires jboss.remoting.endpoint.management service)
- logging
- remoting: depends on io
- request-controller
- security-manager

Aggregation Layer

This layer defines a server with elytron secured management and all
layers one would expect when running a core-server

- core-server: depends on secure-management (brings elytron),
jmx-remoting (brings jmx), logging, core-management, request-controller,
security-manager.

For servlet

Layers not bound to subsystems

- picketbox: A layer to brings picketbox module (and optional deps of
picketbox present in servlet FP). Picketbox could be combined with some
core layers without requiring the legacy security subsystem to be present.

Layers bound to subsystems

- ee: depends on naming
- legacy-security (security sub-system): depends on naming and picketbox.
- naming
- undertow: depends on io and picketbox (to bring optional deps of
picketbox, we still have hard dep on picketbox from
security-negociation). This is a base undertow server, no security, no
https-listener, empty servlet container (no support for servlet, jsp
deployments). NB: welcome-content is not part of the layer, it is bring
by the default configs. legacy default-security-domain "other" is also
not configured, that is done in the default config too.
- undertow-load-balancer: depends on io and picketbox. Undertow
configured to act as a load-balancer.

Web server layer

This layer defines a server containing all layers one would expect when
running an unsecure web-server. legacy-security is not included. In
addition, this layer evolves undertow to be a full servlet-container
(jsp, servlet, web-socket) and http-invoker.

- web-server: ee (brings naming), deployment-scanner, undertow (brings
picketbox and io)

A note on combining layers.

In order to evolve a web-server to become fully manageable, one would
combine core-server (the management features) with web-server (web
support). 

Because core layers are accessible from servlet FP, one could pick and
choose from servlet FP to assemble his server. For example, a basic
manageable unsecured (but ready to be configured for security) undertow
server: undertow+management+elytron

What, roughly, is the disk footprint of an installation like this? I mean with known tweaks and optimizations you have in your WIP.
The size is 38MB. The provisioned standalone servlet config is around 55MB.

What, roughly do the Galleon CLI commands look like to provision a server with this, plus, say core tools and jaxrs from the full feature pack?
 
For now the tools layer embeds all tools. I used to have layers for each kind of tools (tools, ejb-appclient, wstools, jdr, jboss-client, jboss-cli-client) but decided to group them all for the first step. It seems that, according to the cloud context requirements, only the core-tools should be layerized (cli, add-user and elytron). Other tools being provisioned with default dist.

We have jaxrs that depends on web-server, we would provision: jaxrs,elytron,management,tools The command would look like:

install wildfly:current#16.0.0.Beta1-SNAPSHOT --dir=server --layers=jaxrs,management,elytron,tools

The size with all tools is 155MB, with core tools: 64MB


For full:

We have identified an initial set of layers that would be the building
blocks to help define cloud oriented configurations.

Layers bound to subsystem:

These layers depend (at least) on web-server (from servlet)

- cdi: brings in weld and bean-validation
- jaxrs
- jpa: brings in hibernate default providers. In addition brings-in
transactions, jca and infinispan (hibernate infinispan-cache).

Does that bring in JGroups? Or is this a local-only cache?
At the configuration level it only configures a local cache hibernate-cache but in terms of modules, it brings jgroups.
- activemq-jms: naked subsystem, no server configured. In addition
brings-in transactions and jca
- microprofile: opentracing, config, healths and metrics (last 2 depend
on management).

Layers that are simple extension of existing layers (would not appear as
new layers but would extend content of servlet/core ones).

- ee: add optional dependencies injected by dup present in full FP.
- undertow: add undertow.js package.

I'm curious why this is only in full. Not really a question for you as I assume you put it in full because that's where it is now.
Yes.
- tools: vault, jdr, wstools, appclient

This might need some tweaking. AIUI appclient would bring in nearly the entire current installation, so perhaps should be independent. I don't know about wstools.
Yes, splitting tools give us much better footprint. We should only incorporate core tools in a core-tools layer for now.
 

Aggregation layer

- cloud-profile: contains cdi,jaxrs,jpa,activemq-jms,microprofile . All
optionally included, allowing for exclusion (eg: exclude activemq-jms).

What the rough footprint of this, with core tools added?

The provisioning of cloud-profile alone is 121MB. This is the size of the layers with all the optional dependencies injected from DUP (minus passives that are not incorporated eg: weld.ejb). Some of them are quite big and would be not required in all cases. For example, by excluding:
org.jboss.weld.probe (cdi debug)
org.hibernate.search.orm (hibernate search)
org.jboss.resteasy.resteasy-crypto (jaxrs crypto, brings bouncycastle)
org.jboss.resteasy.resteasy-multipart-provider (brings email, multipart support).

the cloud-profile size is down to 108MB.

By not bringing any such optional deps we are at 104MB, but that is to my point of view safer to guarantee that these optional are provisioned and exclude the un-needed.

The size of cloud-profile and all its deps + core tools is 126MB. As a comparison, full provisioned standalone config is 212MB


Open questions

SSL undertow

As you noticed we don't have a layer for an SSL undertow. elytron
doesn't offer an out of the box generation of self-signed certificate.
Only legacy security realms offer this. So the way to go would be to
provision undertow+elytron+management+tools (management+tools to use
CLI), create the keystore, do the elytron configuration, add an
https-listener, set the ssl-context, remove all the management related
stuff from CLI. Then exclude the management+tools layers to get a
smaller foot-print (tools bring a lot).

Having tools take a lot is something to attack. CLI in particular. The CLI is useful on OpenShift as the OS console gives you a terminal on each pod from which you can start a local CLI session. That lets you do diagnostic stuff things like tweak log levels etc.  The CLI also gets used for things like health/readiness probes and shutdown.
core tools (add-user, cli, elytron-tool) on top of base-server brings 7MB. We could see if we can reduce that.
NB: we could imagine that the
CLI configuration is done from a provisioned server that contains tools
so no need to provision tools with undertow.

I am wandering if we should create an elytron SSL context that points to
a "REPLACE_WITH_THE_PATH_TO_YOUR_KEYSTORE" kind of configuration to
simplify configuration steps? I don't really like this idea to generate
config that points to invalid artifacts but would simplify the
configuration steps.

Picketbox layer

Because we want at some point to get rid-off this dependency, I am
wandering if we should really define a layer for it. Could be that any
dependency on it would imply the use of legacy-security layer. Picketbox
is implicit when legacy-security layer is provisioned.

The independent use is the management vault. Using that shouldn't require everything in the current p-b module though. And arguably you could say that use case means you need the vault tool and thus whatever layer has it. (You don't need the vault tool though, not on the server.)
elytron has a dependency on picketbox for jacc.

Is there any metadata type concept for marking a layer as deprecated etc?
no description, no metadata, yes that would help.

Full layers granularity

For now, we have included some major features (transactions, jca,
infinispan) without making them layers. The layers that are depending on
them are for the only consumers. When we progress with new layers, this
would be revisited.

Thank-you for having reached the end. ;-)

Phew!

JF

_______________________________________________
wildfly-dev mailing list
wildfly-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/wildfly-dev


--
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat