Brian Stansberry [https://community.jboss.org/people/brian.stansberry] modified the document: "Single Installation Patching" To view the document, visit: https://community.jboss.org/docs/DOC-47500 -------------------------------------------------------------- This article outlines design notes for the single instance patching feature being developed for AS 7.2. The 7.2 version of the feature will be apply to apply a patch to a single installation of the AS (i.e. a single unzip of the AS distribution.) That single installation may be being used for a standalone server or for a managed domain Host Controller and servers. However, the 7.2 version will not support coordinated patching by the patching tool of multiple installation in a managed domain. The user can of course independently patch multiple installation in a domain, coordinating the application of those patches themselves. The basic patching process will consist of: 1. The user obtains a patch file. 2. The user uses the CLI tool to connect to a standalone server or Host Controller that is running on the installation that is to be patched.1. The server or Host Controller may be running in admin-only mode or running normally. However, some patches may not be able to be applied if the target process isn't running in admin-only mode 3. The user invokes a command on the CLI providing the location of the patch file. The CLi invokes an operation(s) on the target process telling it to stage the patch. 4. The target process stages the patch by applying the patch contents to the filesystem.1. The changes made to the filesystem during staging should not affect the running operation of the server. If this is not possible, the server should reject the patch with an error message indicating that it needs to be placed in admin-only mode before applying the patch. 5. The user uses the CLI tool to restart the target process. Upon restart the staged patch is visible in the runtime. Patches can also be rolled back: 1. The user can use the CLI tool to instruct the target process to revert a patch. 2. Patch reversion results in changes to the filesystem that are staged in the same manner as what is done with a patch installation.1. The changes made to the filesystem during patch rollback staging should not affect the running operation of the server. If this is not possible, the server should reject the patch rollback with an error message indicating that it needs to be placed in admin-only mode before rolling back the patch. 3. The user uses the CLI tool to restart the target process. Upon restart the rolled back patch is no longer visible in the runtime. h1. Patch File A patch file is a zip archive that contains a patch.xml file that is used to describe the patch along with the updated modules, OSGi bundles and miscellaneous files that comprise the patch. It's structure is as follows: + patch.xml + modules ++ patch modules in the same structure as they appear the modules dir in the normal AS dist + bundles ++ patch bundles in the same structure as they appear the bundles dir in the normal AS dist + misc ++ misc files that need to be updated, organized in a directory structure that matches the directory structure of the AS dist h3. The patch.xml File The patch.xml file describes the patch. It includes basic information about the patch along with metadata about the modules, bundles and misc files in the patch. h5. Basic patch metadata * The name of the patch * The type of the patch (one-off vs cumulative, etc -- TODO list all) * The version to which the patch applies * A text description of the patch * TODO others h5. Module metadata * The name of the module * The slot of the module * The hash of the module.xml file of the previous version of the module. Not present if the relevant module did not exist in the version being patched. h5. Removed module metadata Identifies modules that were available in the version being patched but which should no longer be accessible once the patch is applied * The name of the removed module * The slot of the removed module h5. Bundle metadata * TODO anything? h5. Removed bundle metadata Identifies bundles that were available in the version being patched but which should no longer be accessible once the patch is applied * The name of the removed bundle h5. Misc file metadata * The path of the misc file, relative to the root of the AS distribution * The hash of the version of the file that was in the version being patched. Not present if the relevant file did not exist in the version being patched, or if the relevant file is a directory. * A boolean indicator as to whether the relevant file is a directory. * A boolean indicator as to whether the file is in active use by a non-admin-mode Host Controller or server h5. Removed misc file metadata Identifies misc files that were available in the version being patched but which should no longer be accessible once the patch is applied * The path of the misc file, relative to the root of the AS distribution * The hash of the version of the file that was in the version being patched. Not present if the relevant file is a directory. * A boolean indicator as to whether the relevant file is a directory. * A boolean indicator as to whether the file is in active use by a non-admin-mode Host Controller or server h2. Patch Staging The target process when it applies a patch will create a patches directory directly underneath the root of the AS distribution. As with other directories, the location of this directory can be controlled via a system property passed to the JVM at process launch. Before patch staging begins, a check is made as to whether the target process is in admin-only mode. If it is not, and an unresolvable conflict is detected between a module, bundle or miscellaneous file contained in the patch and a user-modification of the corresponding item in the installation being patched, the patch staging process will be aborted with an error message.  See "Patch Conflict Detection" below. h4. Modules Patch modules will be staged by copying them to a patch-specific subdirectory of the patches directory. Information about the location of this directory will be stored in a location visible to the module loader used by JBoss Modules at boot.  This patch-specific subdirectory is not visible to the module loader currently in use by the target process. For modules that the patch "removes", the patch-specific subdirectory of the patches directory will include that module, but with a special module.xml file and no other contents. The module.xml file will include the tag <missing/>. The JBoss Modules release used with this will interpret that <missing/> tag and if seen will throw a ModuleNotFoundException (which is what it would throw if it were unable to locate a module.  The nice thing about this approach is if a later patch adds back the removed module, that patch will take precedence in the module path, and the "missing" module.xml will be irrelevant. h4. Bundles Patch bundes will be staged by copying them to a patch-specific subdirectory of the patches directory. This patch-specific subdirectory is not visible to the OSGi subsystem currently in use by the target process. Information about removed bundles will also be stored in a location visible to the OSGi subsystem at next boot. TODO see if an approach similar to the <missing/> approach can be used. h4. Misc files Misc files are staged by copying them directly to their normal location on the filesystem. A copy of the existing file at that location will be stored in the patch-specific subdirectory of the patches directory. TODO describe check for user-modified misc files. h4. Configuration files Patches will not modify the configuration files of the target process. Configuration patching will not be supported in this version of the feature (and may never be supported.)  Configuration files belong to the user. However, a copy of all existing configuration files will be made in a patch-specific subdirectory of the patches directory. During patch rollback, these configuration files will be restored. This backup is necessary because once the patch is applied to the runtime, any management operation that results in persistence of the configuration will use the xml schemas associated with the patched version of the core AS and any subsystems. If the patch is reverted, those schemas may not be intelligible to the version of the code to which the runtime has been reverted. h2. Patch Application A patch is "applied" by stopping the target process and then restarting it in a normal manner; e.g. from the command line or by using the :shutdown(restart=true) command from the CLI. The patch contents that have been staged are then used by the restarted process. h4. Modules In 7.2 the AS will use a custom implementation of the JBoss Modules ModuleLoader interface, instead of the default implementation used in previous AS 7 releases. The selection of this custom implementation will be part of the launch VM command created by the startup scripts (i.e. standalone.sh/standalone.bat/domain.sh/domain.bat). The module loader works using a notion of a "module path", which is an ordered list of directories in which modules are located. When the module loader needs to find a module, it searches the directories in that path in order, and once it finds the desired module the search ends. Thus, directories earlier in the path listing take precedence over those found later. This is all very similar to how an OS uses the $PATH environment variable. Previous AS7 releases allowed the user to specify a $JBOSS_MODULES_PATH environment variable to declare the "module path", with a default value of $JBOSS_HOME/modules used if not specified. The startup scripts used to the AS pass that module path into JBoss Modules (via the -mp <the_path> argument to java.)  In AS 7.2, the custom module loader we will use will support this behavior, but in addition the custom module loader we provide will be able to examine the patching metadata that will be persisted in the "staging" phase and from that will determine which of the patch-specific subdirectories of the patches directory need to be prepended to the module path. In addition, for modules that have been removed in patches, the custom module loader will know to not load those modules, even though the modules will still remain on the filesystem. h4. Bundles TODO -- basic idea is to do something similar to how modules work, but the details need to be sorted out. h4. Misc Files Application of miscellaneous files to the runtime is straightforward. The staging step places the patch's miscellaneous files in the normal location that the runtime services will expect, so when the target process is restarted, the runtime services see them. h2. Rollback Staging To do a patch rollback, the target process must be placed in admin-only mode. TODO h4. Modules TODO h4. Bundles TODO h4. Misc Files The backup copies of the miscellaneous files stored during the "Patch Staging" process will be restored to their normal locations. Any misc files that were added as part of the "Patch Staging" process (i.e. those that were new in the patch) will be removed. h4. Configuration The backup copies of the configuration files stored during the "Patch Staging" process will be restored. *+Any configuration changes made while the patch was in place will be lost.+* h2. Rollback Application A patch rollback is "applied" by stopping the target process and then restarting it in a normal manner; e.g. from the command line or by using the :shutdown(restart=true) command from the CLI. The modules, bundles and miscellaneous files that have been staged are then used by the restarted process. Issue: the client tool (i.e. CLI) may need to be able to roll back a patch itself, not relying on the target process to do it. This is because a failed patch may leave the target process in a state such that it is unable to respond to commands from the CLI. This capability will not work remotely -- the CLI process will need to be running on the same machine as the installation being patched, and running under a user account with the necessary filesystem permissions to update the filesystem. h4. Modules See the discussion in "Patch Application" above of how the module path works. When a patch is rolled back, the metadata the custom module loader uses to construct the module path will no longer indicate the patch's modules should be on the path, so they will not be available for loading. h4. Bundles TODO -- basic idea is to do something similar to how modules work, but the details need to be sorted out. h4. Misc Files As was the case in the discussion in "Patch Application" above, the patch rollback staging step places the correct miscellaneous files in the normal location that the runtime services will expect, so when the target process is restarted, the runtime services see them. h2. Patch Conflict Detection At the beginning of the patch staging process, a check will be made for conflicts between the patch and any user modifications to the same item. The user when executing the patch command can provide information to guide the tool in resolving conflicts. Any conflicts that cannot be resolved will result in the patch staging being aborted with no changes to the filesystem. Conflict types are: * Modules. If the patch is updating or removing an existing module, the patch.xml file will include the hash of the existing module's contents. The patching process will hash the current module's contents to check for modifications. If any affected module is found to have been modified, the patch command will have to have been provided with a parameter indicating that "overriding" modified modules is ok. No option will be given to optionally override some modules and leave others unmodified. If the user wishes to retain some modification they made to a module, they will need to make an equivalent change to the patch module. The sole purpose of this "module conflict detection" is to provide information to users. * Bundles: If the patch is updating or removing an existing bundle, the patch.xml file will include the hash of the existing bundle's contents. The patching process will hash the current bundle's contents to check for modifications. If any affected bundle is found to have been modified, the patch command will have to have been provided with a parameter indicating that "overriding" modified bundles is ok. No option will be given to optionally override some bundles and leave others unmodified. If the user wishes to retain some modification they made to a bundle, they will need to make an equivalent change to the patch bundle. The sole purpose of this "bundle conflict detection" is to provide information to users. * Misc files: If the patch is updating an existing misc file, the patch.xml file will include the hash of the existing file. If the patch is removing an existing directory, the patch.xml file will include the hash of the existing directory's contents. The patching process will hash the current file or directory to check for modifications. If any affected file/directory is found to have been modified, the patch command will have to have been provided with a permission indicating that "overriding" modified modules is ok. If no permission is available for a particular file or directory, the conflict will be treated as unresolved. These permissions can come in four forms:* A global permission to update all conflicting misc files, provided as a param to the CLI tool's patching command * A global permission to not update any conflicting misc files, provided as a param to the CLI tool's patching command * A file containing* a list of paths (relative to $JBOSS_HOME) for which update permission is granted. * a list of paths (relative to $JBOSS_HOME) for which update permission is denied, meaning the existing file should be retained. The patching feature will also include a command that can apply this conflict detection process and provide a report of conflicts. Users can use this command to check for conflicts in advance. TODO describe how this will work for patch rollback. h2. Patch Generation Tool * Accepts basic metadata along with the location of a distribution of the new version along with a distribution of the old version* Also includes details on any misc files that are expected to be in active use * Generates a patch.xml file * Can generate a full patch file * Optional: Can be run from inside the bin/ dir of the full distribution of the new version and if provided with the patch.xml file can generate a patch file from the contents of the full dist* Idea here is the patch.xml file could be distributed as part of the new version and the user could use it to generate patch h4. Directory structure h4. ${JBOSS_HOME} | |-- bin |-- bundles |   `-- org/jboss/as/osgi |-- docs |-- modules |   |-- org/jboss/as/... |   `-- org/jboss/as/server/main/module.xml |-- patches (overlay directory) |   |-- patch01 |   |   |-- bundles |   |   |   `-- org/jboss/as/osgi/configadmin/main |   |   `-- modules |   |   |   `-- org/jboss/as/server/main/module.xml |   |-- patch02 |   |   |-- bundles |   |   |   `-- org/jboss/as/osgi/configadmin/main |   |   `-- modules |   |   |   `-- org/jboss/as/server/main/module.xml |   `-- .metadata |       |-- cumulative (links to given patchId) |       |-- references [patch01, patch02] (list of one-off patches ) |       `-- history (rollback information for a patch) |           |-- patch01 |           |   |-- cumulative (previous cp) |           |   `-- misc |           `-- patch02 |               |-- cumulative (previous cp) |               `-- misc | |-- jboss-modules.jar `-- loader.jar (boot module-loader) -------------------------------------------------------------- Comment by going to Community [https://community.jboss.org/docs/DOC-47500] Create a new document in JBoss AS 7 Development at Community [https://community.jboss.org/choose-container!input.jspa?contentType=102&a...]