Brian Stansberry [
https://community.jboss.org/people/brian.stansberry] modified the
document:
"Layered Distributions and Module Path Organization"
To view the document, visit:
https://community.jboss.org/docs/DOC-47927
--------------------------------------------------------------
h1. +*This is an article outlining some thoughts by the author*+ +(Brian Stansberry)++*.
It is not an official design of the JBoss AS 7 project.*+
h1. Overview
This document describes notions for how different software distributions based on the
JBoss AS 7 project can integrate with the base AS and with each other in a controlled
manner.
+*
*+
h1. Fundamental Concepts
h2. Distribution
A coherent set of software produced by a "project" or a "product".
JBoss AS 7 is a project. JBoss EAP is such a product. Both play similar technical roles in
the JBoss ecosystem, but there are significant differences in how the two are made
available to end users.
This article uses "distribution" as a replacement for constantly saying
"project/product" everywhere. It's not a great term; if someone has a better
one, please suggest it.
h2. Distribution Base
The "distribution base" is the core set of modules, bundles and miscellaneous
files that is common to all layered distributions. It may itself be usable standalone
project or product.
Currently the JBoss AS 7 project and the JBoss EAP product are the distribution bases
relevant to this discussion. In the future a more limited subset of functionality may
become the distribution base, with the full set of functionality currently provided in
community AS and EAP coming as layered distributions or add-ons.
h2. Layered Distribution
A distribution that builds on top of the distribution base, potentially superceding some
of the distribution base's functionality. A layered distribution is allowed to
supercede any aspect of the distribution base's functionality. The less the layered
distribution does this, the easier it will be for the layered distribution to manage
things like patching, but, if necessary, a layered distribution can supercede anything in
the distribution base.
+TODO can a layered distribution build on top of another layered distribution?+
h2. Identity
An identity is a named set of distribution base + layered distribution(s) that is
certified as a valid combination. The distribution base by itself may or may not be a
valid identity. Currently it is; in the future it may not be.
A given layered distribution may be associated with more than one identity.
h3. Patching and Identities
Patches are applicable to an identity. A given patch can apply to more than one identity,
if there are no conflicts between the identities in terms of how they use the items
modified by a patch. The metadata associated with a patch will indicate the identities to
which it is applicable.
For modules and bundles within a patch, the patch metadata will provide the name of the
provider of the module or bundle -- that is, either the distribution base or the name of a
layered distribution that is part of the identity. Once the patch is applied, that
association is retained.
Miscellaneous files within a patch are not associated with the distribution base or any
particular layered distribution once they are installed.
Each identity produces a patch stream. Because a given patch can apply to more than one
identity, the same patch can appear in multiple patch streams.
A patch for a given identity may incorporate as a separately identifiable component a
patch produced by another identity or the distribution base. For example, let's an
identity named A consists of the distribution base and a layered distribution named A.
Version 1.0 of identity A is based on version 2.0 of the distribution base and version 1.0
of layered distribution A. The distribution base produces a patch that will move it to
version 2.1. The producers of the A identity could then produce a patch that will result
in version 1.1 of identity A. This would include an upgrade of layered distribution A to
its version 1.1, as well as the a separate component that provides the distribution base
upgrade to its 2.1. There are two main benefits of this sharing approach:
* The modules and bundles provided by the distribution base upgrade are associated with
the original 2.1 patch provided by the distribution base, not just with the enclosing
patch provided by A. This will make possible the sharing of modules/bundles provided by
patches across installations.
* Incorporating one patch within another as a simple discrete unit helps avoid errors
during patch production.
h2. Installation
An installation is a single filesystem location containing the distribution base plus zero
or more layered distributions. It includes the bin/ dir from which processes are launched.
The root of the installation is the path indicated by the $JBOSS_HOME environment variable
referred to in the scripts and indicated by the jboss.home.dir system property used in the
java code.
h3. Installation Identity
Each installation has a single identity. Because an installation only has a single
identity, there is no confusion as to which layered distribution "owns" a
particular filesystem location in that installation. Further, because the identity of an
installation can be easily determined (see "Establishing Identity" below),
external tools can quickly understand the functionality provided by a given installation,
even if no process is currently running from the installation.
+TODO further spell out the importance of single identity.+
h3. Sharing Files Across Installations+
+
Modules and bundles are potentially sharable across installations. Making it easy for
users to do this should be a key goal.
Miscellaneous files are not sharable across installations. (An end user could potentially
share them using symlinks, but doing so is not a supported feature nor is it a recommended
practice.)
Patches are not sharable across installations. The +modules and bundles that the patch
provides+ are potentially sharable, but the miscellaneous files provided by a patch are
not sharable. The patching history metadata that the patching tool records in the
installation is also not sharable -- the metadata is associated with the given
installation. However, making it easy to share the the modules and bundles provide by a
patch across installations should be a key goal.
h2. Add-On Distribution
An add-on distribution is a set of software that extends the capabilities of a given
"identity". An add-on can only extend capabilities. It is not allowed to
supercede capabilities of the identity it extends. For modules and OSGi bundles, the
module and bundle loading logic provided by the distribution base will enforce this
behavior. For miscellaneous files, it is up to the people providing an add-on to ensure
that the add-on does not overwrite any file provided by the distribution base, any layered
distribution associated with the identity or any other add-ons associated with the
identity. Conversely, once the providers of an identity have accepted an add-on into their
ecosystem, it is their responsibility to ensure that they do not add any files to their
distributions that will conflict with files already provided by the add-on.
In practice, the most likely sources of conflict will be the xml files in the
configuration directories and the .conf files in the bin directory. Conflicts over xml
files are solvable by having add-on providers ship their own xml file variants rather than
attempting to modify the standard ones that ship with the identity.
A given add-on is allowed to extend the capabilities of more than one identity.
An add-on may require the presence of other add-ons.
h3. Patching and Add-ons
Add-on distributions produce their own patch stream. The patches produced by an add-on
indicate the versions of the identity that must be present for the patch to apply. More
than one identity can be listed, since add-ons may be applicable to multiple identities.
The patches produced by an add-on also indicate the existing version of the add-on that
must be present, as well as the version of any other required add-ons.
Add-ons cannot use patches in their own stream to fix problems in the identity on which
they depend. Add-ons depend on the patch stream provided by their dependent identity to
provide such fixes. Modules and bundles provided by an add-on or its patches will have
lower precedence in the module loading order than those that come from the distribution
base or layered distributions, so any fix to such a module/bundle provided by an add-on
patch will have no effect.
h1. Installers
+TODO discussion of RPM+
h2. Distribution Base Installers
The distribution base can be installed via a simple unzip. If the distribution base is
itself an identity, see also "Identity Installers"
h2. Identity Installers
An identity can be installed via:
* An unzip onto an empty filesystem location of a zip file that includes the distribution
base and all layered distributions.
* An unzip onto a filesystem location that contains any depended upon identity or the
installation base of a zip that only includes
* An installer program that does one or the other or both of the above and also provides a
user interface that allows the user to customize the installation.
It is a goal but not an initial requirement to allow installation to be performed via the
patching tool. The benefit of this is the patching tool incorporates a number of
compatibility checks that can help a user avoid invalid combinations.
A further goal is to make the logic contained in the patching tool available to installer
programs via a library.
*+Once an identity is installed into a given filesystem location, the identity of that
installation is irrevocably changed.+* When an identity is installed it has the right to
modify any miscellaneous files in the installation filesystem, so once an identity is
installed, the identity of the installation is changed. In many (hopefully nearly all)
cases it may be possible for a user to remove files added by an identity install and get
the installation back to its previous identity, but doing this is not a supported feature
of the software and is not a recommended practice.
*+However, once support for installing patches via the patching tool is available,
reverting an identity installation will be possible.+* The patching tool maintains backup
copies of any miscellaneous files modified by a patch, making it possible to restore those
files.
h2. Add-on Installer
An identity can be installed via:
* An unzip onto an empty filesystem location of a zip file that includes the distribution
base and all layered distributions.
* An unzip onto a filesystem location that contains any depended upon identity or the
installation base of a zip that only includes
* An installer program that does one or the other or both of the above and also provides a
user interface that allows the user to customize the installation.
It is a goal but not an initial requirement to allow installation to be performed via the
patching tool. The benefit of this is the patching tool incorporates a number of
compatibility checks that can help a user avoid invalid combinations.
A further goal is to make the logic contained in the patching tool available to installer
programs via a library.
*+Once an add-on is installed into a given filesystem location, the identity of that
installation is not changed, and it must be possible to reverse the installation.+* An
add-on is merely additive, so removing it is a matter of deleting what was added. This
could be done via a script provided by the add-on, via an installer program, or manually
by a user following a set of instructions.
h1. Establishing Identity
It +must+ be possible for external tools to determine the identity of a given installation
without requiring the software based on that installation to be running. It's also
necessary for the software based on an installation to determine its own identity. This
will be done via the bin/product.conf file located in the distribution. This file will
include:
* The name of the identity
* The name of any layered distributions used by the identity, in order of the precedence
to be used when loading their modules.
If no bin/product.conf file exists, the installation is assumed to be the community
version of the distribution base. If no layered distributions are named in the file, only
the base and any add-ons will be used.
h1. Module Loading
JBoss AS currently loads modules based on a "module path" which is a list of
filesystem locations which JBoss Modules searches for modules. In our standard launch
scripts, users are able to manipulate this module path by setting the $JBOSS_MODULEPATH
environment variable. If not set, $JBOSS_MODULEPATH is set to $JBOSS_HOME/modules.
This mechanism creates a single default location under which modules can be located. This
is problematic when layered distributions and add-ons wish to install their own modules,
since there are no clear rules as to how those modules should be installed or how they
relate to one another.
h2. Goals
* To make it easy for other layered distributions or add-ons to add their modules to the
set of available modules or remove their modules from the set of available modules,
without fear of physically overwriting modules provided with the distribution base or by
other layered distributions.
* To make it easy for distributions that add modules to declare that a set of modules is
sharable (as defined below) with the modules provided by any distribution associated with
a given identity (including the modules provided by the distribution base)
* To establish clear rules for how distributions that add modules that must take
precedence over sharable modules with the same name and slot provided by other
distributions should:* locate those modules
* indicate at launch time to the modular classloading system which set of distribution
specific non-sharable modules should be used
* To make it easy for end users to alter the module path for their own purposes by setting
$JBOSS_MODULEPATH without needing to concern themselves with details of how the modules
provided by the distribtuion base, layered distributions, or add-ons relate to one
another. Reserve the use of $JBOSS_MODULEPATH and the value of the -mp command line
argument passed to jboss-modules for end use uses.
h2. Sharability Rules
All modules provided as part of the distribution base are shareable. When the distribution
base includes a module it accepts the following:
* The version of the module it provides will take precedence over the +shareable+ version
provided by any layered distribution or any module provide by an add-on.
* As a consequence of this, any modification of that module in a patch or version upgrade
will need to be tested for compatibility with any layered distribution or add-on based
that version of the distribution base.
* As a subset of the previous point, within a major version family the distribution base
cannot remove a module that is depended upon by another layered distribution or add-on,
even if the distribution base itself no longer needs that module.
A layered distribution can provide both shareable and non-shareable modules.
If a layered distribution declares that a given module is "shareable", it
accepts the following with respect to any identity with which the layered distribution is
associated:
* The distribution base may ship a module with the same name and slot, and that module
+*will*+ take precedence over the sharable module provided by the layered distribution.
* Another layered distribution or add-on may ship a module with the same name and slot,
and that module +*may*+ take precedence over the module provided by this layered
distribution or add-on.
* Another layered distribution or add-on may ship a module with the same name and slot,
and the version provided by this layered distribution or add-on +*may*+ take precedence
over the module provided by the other layered distribution or add-on.
* As a consequence of this, any modification of that module in a patch or version upgrade
will need to be tested for compatibility with any layered distribution or add-on based on
the same major version of the identity.
* The distribution base cannot depend upon a sharable module provided by a layered
distribution.
* An add-on associated with an identity can depend upon shareable modules provided by any
layered distribution associated with the identity. Further, other layered distributions
within the same identity can also can depend up a shareable module provided by the layered
distribution. As a result, if a layered distribution provides shareable shareable modules,
it takes on the following responsibilities:* Any modification of that module in a patch or
version upgrade will need to be tested for compatibility with any dependent add-ons or
layered distribution.
* As a subset of the previous point, within a major version family the layered
distribution cannot remove a shareable module that is depended upon by an add-on or
another layered-distribution, even if the add-on itself no longer needs that module.
If a layered distribution declares that a given module is non-shareable, it accepts the
following with respect to any identity with which the layered distribution is associated:
* The version of the module it provides will take precedence over:* The module provided by
the distribution base.
* The module provided by any other layered distribution that this layered distribution
depends upon.
* The shareable version of any add-on or layered distribution.
* As a consequence of this, any modification of that module in a patch or version upgrade
will need to be tested for compatibility with any add-on or other depended-upon layered
distribution associated with the identity.
* As a subset of the previous point, within a major version family the layered
distribution cannot remove a module that is depended upon by an add-on, even if the
layered distribution itself no longer needs that module.
* The layered distribution may allow add-ons or other layered distributions to depend upon
it. If it does so, +with respect to those other layered distributions or add-ons, even
non-shareable modules are treated as shareable, and the rules give above for layered
distribution shareable modules apply+. The only exception is if the distribution base
ships a module with the same name and slot, the distribution base version does not take
precedence.
All modules provided as part of an add-on are "shareable". When an add-on
includes a module, it accepts the following with respect to any identity with which the
add-on is associated:
* The distribution base or a layered distribution may ship a non-sharable module with the
same name and slot, and that module +*will*+ take precedence over the sharable module
provided by the add-on.
* A layered distribution or another add-on may ship a module with the same name and slot,
and that module +*may*+ take precedence over the module provided by this layered
distribution or add-on.
* A layered distribution or add-on may ship a module with the same name and slot, and the
version provided by this layered distribution or add-on +*may*+ take precedence over the
module provided by the other layered distribution or add-on.
* As a consequence of this, any modification of that module in a patch or version upgrade
will need to be tested for compatibility with any layered distribution or add-on based on
the same major version of the identity.
* The distribution base or a layered distribution cannot depend upon a sharable module
provided by an add-on.
* As a consequence of the previous point, an add-on can remove a sharable module in a
patch, without fear of breaking a core identity.
* An add-on can depend up a shareable module provided by another add-on upon which it
depends, but only with the express consent of that add-on. Such dependencies are strongly
discouraged.
* If an add-on allows another add-on to depend upon its shareable modules, it takes on the
following responsibilities:* Any modification of that module in a patch or version upgrade
will need to be tested for compatibility with any dependent add-ons based that version of
the add-on.
* As a subset of the previous point, within a major version family the add-on cannot
remove a module that is depended upon by another add-on, even if the add-on itself no
longer needs that module.+
+
* Testing procedures must be created to ensure that any patches produced by the
depended-upon add-on do not break the dependent add-on.
h2. Module Organization on the Filesystem
This section will describe how the above points are reflected in the actual organization
of modules on the filesystem.
h4. Requirement for Standardized Module Loading
JBoss Modules can support other mechanisms for finding module content and loading it
besides the filesystem-based module path mechanism. Theoretically a custom boot module
loader implementation could be specified at process launch, with that module loader using
some other means for locating module content. This article proposes that this capability
be declared as unsupported for JBoss AS and products based upon it. Rather, the AS will
have a specified boot module loader implementation.
Users of the AS community project are free to use a non-standard boot module loader
implementation, but if they do so and certain features provided by the AS don't work,
that is their problem.
h3. Filesystem Layout
Modules would be organized on the filesystem as follows:
modules
+ internal
++ add-ons
+++ abc
+++ def
++ layers
+++ base
+++ vuw
+++ xyz
The internal level exists to separate modules associated with the distribution base,
layered distribution or add-ons from those the end user may store in the modules
directory. +TODO Some better name for internal would be nice.+
h3. Translation of Filesystem Layout into Module Loading Precedence
For any location provided by the user via the -mp switch, the module loader will look for
the internal structure. If it is found in more than one location, an exception will be
thrown. The module loader will understand the structure above and will apply module
loading precedence rules as follows.
Assume the above structure exists under $JBOSS_HOME and that the user sets
$JBOSS_MODULEPATH as follows
JBOSS_MODULESPATH=/user/me/my-modules;$JBOSS_HOME/modules
In addition, the bin/product.conf file declares that the identity of the install is xyz,
the xyz layered distribution should be loaded and the xyz layered distribution depends on
the vuw layered distribution.
Assume also that patches named base-1, abc-1, def-1, vuw-1, xyz-1 and xyz-2 have been
applied.
The module loader would look for modules on the filesystem in following locations,
starting with first and terminating as soon as the module is found:
/user/me/my-modules
/modules (in case the user placed modules directly in under this directory)
/modules/layers/xyz/patches/xyz-2
/modules/layers/xyz/patches/xyz-1
/modules/layers/xyz
/modules/layers/vuw/patches/vuw-1
/modules/layers/vuw
/modules/layers/base/patches/base-1
/modules/layers/base
/modules/add-ons/abc/patches/abc-1
/modules/add-ons/abc
/modules/add-ons/def/patches/def-1
/modules/add-ons/def
The precedence order between /modules/add-ons/abc and /modules/add-ons/def is arbitrary.
An add-on should never assume its modules will take precedence over those of any other
add-on's modules.
The fact that the various patches locations are included in the path is entirely due to
+metadata+ that the patching tool will store in the $JBOSS_HOME/patches directory. The
mere presence of a directory like /modules/layers/xyz/patches/xyz-2 is insufficient to
trigger inclusion of that directory in the module search path. However, when the patching
tool installs the patch, the actual location of the +module contents+ will be in
/modules/layers/xyz/patches/xyz-2 -- only patching related metadata will be stored in
$JBOSS_HOME/patches. This behavior is important for allowing sharing of modules across
installations.
h2. Sharing Modules Across Installations
The above behavior makes it relatively straightforward for a user to share modules across
installations. For example, the user could set up a shared module repo in /user/me with
the following structure
jboss-module-repo
+ add-ons
++ abc
++ def
+ layers
++ base
++ vuw
++ xyz
The $JBOSS_HOME/modules/internal location in each installation on the filesystem could
then be replaced with a symlink to /user/me/jboss-module-repo.
The downside to this very simple sharing is the abc and def add-ons will be visible and
used by all installations, whether or not they are relevant to the installation's
identity. A solution to this problem would be to do more symlinks:
$JBOSS_HOME/modules/internal/layers -> /user/me/jboss-module-repo/layers
$JBOSS_HOME/modules/internal/add-ons/abc -> /user/me/jboss-module-repo/add-ons/abc
$JBOSS_HOME/modules/internal/add-ons/def -> /user/me/jboss-module-repo/add-ons/def
As an improvement at some point we could define some sort of file format (say xml) and
allow a file with that format to be placed in $JBOSS_HOME/modules/internal in lieu of the
subdirectories. The file would include all of the mapping information for the external
repo. The patching tool or any programmatic installers would know how to read and update
that file. The file would only include information for the add-ons relevant to the
installation.
h3. Implications for the Patching Tool
The possibility of module sharing doesn't remove the requirement to apply a patch to
each installation on a given filesystem. Applying the patch to each installation is
required in order to:
* update any miscellaneous files
* record the patching metadata for the particular installation
* allow the patching tool to understand what combinations of layered distributions and
add-ons are relevant to the given installation
However, the patching tool should recognize the possibility of this kind of module
sharing. When it applies a patch it should recognize that the modules may already exist on
the filesystem and should not apply them again if not necessary. It should however record
some information in, e.g. /modules/layers/xyz/patches/xyz-2 indicating that the
installation being patched is using that set of patch modules. This information will
prevent deletion of patch modules used by multiple installations if one of the
installations uses the patching feature's cleanup functions.
h1. OSGi Bundle Loading
This needs discussion, but the basic concept is the precedence rules for OSGi bundle
loading would following the same rules as for module loading.
--------------------------------------------------------------
Comment by going to Community
[
https://community.jboss.org/docs/DOC-47927]
Create a new document in JBoss AS 7 Development at Community
[
https://community.jboss.org/choose-container!input.jspa?contentType=102&a...]