[jboss-as7-dev] Deployment Chain/Service interaction

[David M. Lloyd]

This ridiculously long email is to explain how the deployment chain 
services should be created and how the process should work, end-to-end, 
in our post-Alpha1/MSC-1.0.0.Beta5 world and to open things up to 
additional comments or critique.

Basic Theory
------------
The new architecture is hinged around the concept of modifying the way 
deployment unit processor (DUP) chains are executed.  Instead of 
executing them once at deployment time and relying on services to clean 
up (and dealing with the inherent asymmetry of having services which 
perform unrelated actions on start compared to stop), the basis of this 
idea is to do the following:

1. Break the DUP chain into segments (much like we have already laid out 
in org.jboss.as.deployment.Phase).

2. Define a new service for each segment.  On start, the "deploy" action 
of all of the DUPs are executed in sequence.  On stop, a compensating 
"undeploy" action is executed on each DUP in reverse order, exactly as 
Alexey was proposing in [1] (yes, I argued against it, but I was wrong). 
  If start fails, the DUP "deploy" action has to fully undo itself; in 
addition the DUP context will call the "undeploy" action for all 
previous DUPs in the segment, which allows start to be retried (as 
defined by the MSC service contract).  This combines the notion of 
cleanup with undeploy exactly as Alexey described.

3. The action taken on deployment is simply to create and activate (i.e. 
set its mode to ACTIVE) the service for the first DUP chain segment, 
which then causes the service for the next DUP chain segment to be 
created, etc. until the deployment is complete.

4. On undeploy, each DUP has an additional responsibility to remove all 
services added by that DUP.  However we can automate this by using a 
special ServiceTarget in the DUP context, which tracks all services 
added for the deployment (we need this anyway to correctly finish 
undeployment).  The DUP chain service can automatically trigger and 
await service removal as part of its stop() action (made convenient by 
the usage of MSC's async stop functionality).

This means that all you need to do to remove a deployment unit is remove 
its root service, which will cascade to all ancillary services 
automatically via MSC's dependency mechanism, including correct handling 
of deployment interdependencies.

This also means that it is possible for a DUP chain to be partially 
completed, rewound to an earlier point, and played forward again, which 
will become essential to correctly handling redeployment when complex 
dependencies are involved.

The DUP chain itself is represented by a single global DUP chain service 
which is started during bootstrap.  All deployment first-phase services 
(see below) have a dependency upon this service.

Deployment Phases
-----------------

A DUP chain segment is defined by a "phase", which are already more or 
less defined.  The service corresponding to each phase is based on its 
deployment, as we do today, e.g. "jboss.deployment.\"myapp-foo.war\"". 
However since we have multiple phases, we must also append the phase name.

You may notice a superficial similarity between these phases and the 
phases of an AS5/6 MC service.  I noticed that too.  :-)

1. Phase: "structure" -> "jboss.deployment.\"myapp-foo.war\".structure"

This phase is responsible for identifying the deployment type, delving 
its structure, and creating and cataloging all of the VFS mounts which 
correspond to this deployment unit.  When processing of this phase is 
complete, all mounts must be available and the deployment type must be 
identified.

2. Phase: "validate" -> "jboss.deployment.\"myapp-foo.war\".validate"

This phase is not presently used for validation tasks though there is 
currently one processor in this phase, 
org.jboss.as.web.deployment.WarStructureDeploymentProcessor, which 
probably doesn't belong there.

3. Phase: "parse" -> "jboss.deployment.\"myapp-foo.war\".parse"

In this phase, all descriptors which are available via the file system 
are processed.  This includes XML descriptors, manifest information, 
etc.  When processing of this phase is complete, all descriptors should 
be parsed and stored in the DUP context.

4. Phase "dependencies" -> "jboss.deployment.\"myapp-foo.war\".dependencies"

In this phase, the data collected is used to assemble the list of 
dependencies and class path resource roots.  Class-path dependencies on 
other deployments will be added to this deployment's "modularize" phase 
as MSC dependencies on the dependency deployments' "structure" phase. 
When processing of this phase is complete, all the information necessary 
to construct the module for this deployment is available.

5. Phase "modularize" -> "jboss.deployment.\"myapp-foo.war\".modularize"

In this phase, the module is created based on the dependency 
information, and any action which requires class loading to be enabled 
can now be performed.  This includes loading classes and reading 
annotations via reflection.

6. Phase "post-module" -> "jboss.deployment.\"myapp-foo.war\".post-module"

This phase is simply a continuation of "modularize".  See "Missing 
Stuff" for further discussion.

7. Phase "install" -> "jboss.deployment.\"myapp-foo.war\".install"

In this phase, various deployment items spec'd out by earlier phases 
(like servlets, EJBs) are converted into services and created.  See 
"Missing Stuff" for further discussion.

8. Phase "cleanup" -> "jboss.deployment.\"myapp-foo.war\".cleanup"

This phase is presently unused.

Missing Stuff
-------------

Missing from the above list:

1. Generation of Jandex annotation index.

2. Consumption of the Jandex index to identify classes to be processed 
for annotations.

3. A mechanism for deployment module-to-module dependencies.  Since 
module dependencies can be circular, two phases are needed to correctly 
make this work.  One possibility is to add module dependencies and 
relink as a prelude to "post-module", with "post-module" then depending 
on the "modularize" phase of all deployments upon which there is a 
module-level (not classpath-level) dependency.  Post-module processors 
can then be divided up based on whether external module dependencies 
should be included in processing (i.e. annotation scanning).

4. Consideration for the necessary startup sequence for EJBs with 
respect to servlets and so on.  This will be covered in more detail in 
the EJB requirements document.  Note that since these phases are built 
by the deployment system (as opposed to being ingrained in the service 
itself), we can modify them as needed to accommodate the requirements of 
all of our deployable service types.

Bootstrap
---------

The bootstrap process would necessarily be somewhat different.  I'll 
describe it by walking through a server boot:

1. Parsing/receiving the server updates.  Boot updates always come in 
order of extensions first, then subsystems, then deployments.  Because 
that's just how it is, and not by accident.

2. Run all updates in sequence.  This all happens in one loop with the 
following components:

2a. Execute extension updates.  These updates immediately load 
extensions, subsystem element handlers, and possibly add global 
services.  This all happens in the bootstrap thread.

2b. Execute subsystem add updates.  The subsystem would use the 
activation context provided to hook into any affected DUP chain(s).

2c. Execute deployments.  *Updated*.  Each deployment is executed by 
creating its first-phase service, creating an injection dependency on a 
global DUP chain service, and setting the service mode to ACTIVE.

2d. Execute the rest of the uninteresting stuff, if any.

3. Once all updates are executed, the DUP chain service is created and 
set to ACTIVE, effectively "locking in" the deployment chain and 
allowing deployments to proceed.  The deployment services then fully 
assemble themselves.

Undeployment
------------

This architecture provides a uniquely effective model for supporting 
undeployment.  With all services and dependencies correctly defined, it 
should be possible to redeploy any deployment unit in the system, no 
matter how complex the interdependencies, and all affected services are 
cleanly stopped and restarted with correctly rewired dependencies, all 
the way to the point of sending notification to load-balancers and so on.

See also
--------

[1] http://community.jboss.org/message/572259#572259

-- 
- DML