<div dir="ltr"><div><div><div><div>Hi Stuart,<br></div>Many good points.<br><br></div>I have some questions in this regard. <br>1, Could you please clarify the difference between feature packs, module, and subsystems?<br>If I understand correctly, that subsystems, and module to feature packs is like what ingredients to recipes.<br>
<br>2, To my gut feel, the feature packs you described shares the same concept of feature of Karaf. For more info, please refer to <a href="http://karaf.apache.org/manual/latest/users-guide/provisioning.html">http://karaf.apache.org/manual/latest/users-guide/provisioning.html</a>. Is it possible for Wildfly to be built on top of OSGi container in the future?<br>
<br></div>Regards,<br></div>Peter C<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jun 11, 2014 at 1:13 AM, Stuart Douglas <span dir="ltr"><<a href="mailto:sdouglas@redhat.com" target="_blank">sdouglas@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">This design proposal covers the inter related tasks of splitting up the<br>
build, and also creating a build/provisioning system that will make it<br>
easy for end users to consume Wildfly. Apologies for the length, but it<br>
is a complex topic. The first part explains what we are trying to<br>
achieve, the second part covers how we are planning to actually<br>
implement it.<br>
<br>
The Wildfly code base is over a million lines of java and has a test<br>
suite that generally takes close to two hours to run in its entirety.<br>
This makes the project very unwieldily, and the large size and slow test<br>
suite makes development painful.<br>
<br>
To deal with this issue we are going to split the Wildfly code base into<br>
smaller discrete repositories. The planned split is as follows:<br>
<br>
- Core: just the WF core<br>
- Arquillian: the arquillian adaptors<br>
- Servlet: a WF distribution with just Undertow, and some basic EE<br>
functionality such as naming<br>
- EE: All the core EE related functionality, EJB's, messaging etc<br>
- Clustering: The core clustering functionality<br>
- Console: The management console<br>
- Dist: brings all the pieces together, and allows us to run all tests<br>
against a full server<br>
<br>
Note that this list is in no way final, and is open to debate. We will<br>
most likely want to split up the EE component at some point, possibly<br>
along some kind of web profile/full profile type split.<br>
<br>
Each of these repos will build a feature pack, which will contain the<br>
following:<br>
<br>
- Feature specification / description<br>
- Core version requirements (e.g. WF10)<br>
- Dependency info on other features (e.g. RestEASY X requires CDI 1.1)<br>
- module.xml files for all required modules that are not provided by<br>
other features<br>
- References to maven GAV's for jars (possibly a level of indirection<br>
here, module.xml may just contain the group and artifact, and the<br>
version may be in a version.properties file to allow it to be easily<br>
overridden)<br>
- Default configuration snippet, subsystem snippets are packaged in the<br>
subsystem jars, templates that combine them into config files are part<br>
of the feature pack.<br>
- Misc files (e.g. xsds) with indication of where on path to place them<br>
<br>
Note that a feature pack is not a complete server, it cannot simply be<br>
extracted and run, it first needs to be assembled into a server by the<br>
provisioning tool. The feature packs also just contain references to the<br>
maven GAV of required jars, they do not have the actual jars in the pack<br>
(which should make them very lightweight).<br>
<br>
Feature packs will be assembled by the WF build tool, which is just a<br>
maven plugin that will replace our existing hacky collection of ant<br>
scripts.<br>
<br>
Actual server instances will be assembled by the provisioning tool,<br>
which will be implemented as a library with several different front<br>
ends, including a maven plugin and a CLI (possibly integrated into our<br>
existing CLI). In general the provisioning tool will be able to<br>
provision three different types of servers:<br>
<br>
- A traditional server with all jar files in the distribution<br>
- A server that uses maven coordinates in module.xml files, with all<br>
artifacts downloaded as part of the provisioning process<br>
- As above, but with artifacts being lazily loaded as needed (not<br>
recommended for production, but I think this may be useful from a<br>
developer point of view)<br>
<br>
The provisioning tool will work from an XML descriptor that describes<br>
the server that is to be built. In general this information will include:<br>
<br>
- GAV of the feature packs to use<br>
- Filtering information if not all features from a pack are required<br>
(e.g. just give me JAX-RS from the EE pack. In this case the only<br>
modules/subsystems installed from the pack will be modules and subystem<br>
that JAX-RS requires).<br>
- Version overrides (e.g. give me Reaseasy 3.0.10 instead of 3.0.8),<br>
which will allow community users to easily upgrade individual components.<br>
- Configuration changes that are required (e.g. some way to add a<br>
datasource to the assembled server). The actual form this will take<br>
still needs to be decided. Note that this need to work on both a user<br>
level (a user adding a datasource) and a feature pack level (e.g. the<br>
JON feature packing adding a required data source).<br>
- GAV of deployments to install in the server. This should allow a<br>
server complete with deployments and the necessary config to be<br>
assembled and be immediately ready to be put into service.<br>
<br>
Note that if you just want a full WF install you should be able to<br>
provision it with a single line in the provisioning file, by specifying<br>
the dist feature pack. We will still provide our traditional download,<br>
which will be build by the provisioning tool as part of our build process.<br>
<br>
The provisioning tool will also be able to upgrade servers, which<br>
basically consists of provisioning a new modules directory. Rollback is<br>
provided by provisioning from an earlier version of provisioning file.<br>
When a server is provisioned the tool will make a backup copy of the<br>
file used, so it should always be possible to examine the provisioning<br>
file that was used to build the current server config.<br>
<br>
Note that when an update is performed on an existing server config will<br>
not be updated, unless the update adds an additional config file, in<br>
which case the new config file will be generated (however existing<br>
config will not be touched).<br>
<br>
Note that as a result of this split we will need to do much more<br>
frequent releases of the individual feature packs, to allow the most<br>
recent code to be integrated into dist.<br>
<br>
Implementation Plan<br>
<br>
The above changes are obviously a big job, and will not happen<br>
overnight. They are also highly likely to conflict with other changes,<br>
so maintaining a long running branch that gets rebased is not a<br>
practical option. Instead the plan it to perform the split in<br>
incremental changes. The basic steps are listed below, some of which can<br>
be performed in parallel.<br>
<br>
1) Using the initial implementation of my build plugin (in my<br>
wildfly-build-plugin branch) we split up the server along the lines<br>
above. The code will all stay in the same repo, however the plugin will<br>
be used to build all the individual pieces, which are then assembled as<br>
part of the final build process. Note that the plugin in its current<br>
form does both the build and provision step, and the pack format is<br>
produces is far from the final pack format that we will want to use.<br>
<br>
2) Split up the test suite into modules based on the features that they<br>
test. This will result in several smaller modules in place of a single<br>
large one, which should also be a usability improvement as individual<br>
tests will be be faster to run, and run times for all tests in a module<br>
should be more manageable.<br>
<br>
3) Split the core into into own module.<br>
<br>
4) Split everything else into its own module. As part of this step we<br>
need to make sure we still have the ability to run all tests against the<br>
full server, as well as against the cut down feature pack version of the<br>
server.<br>
<br>
5) Focus on the build an provisioning tool, to implement all the<br>
features above, and to finalize the WF pack format.<br>
<br>
I think that just about covers it. There are still lots of nitty gritty<br>
details that need to be worked out, however I think this covers all the<br>
main aspects of the design. We are planning on starting work on this<br>
basically immediately, as we want to get this implemented as early in<br>
the WF9 cycle as possible.<br>
<br>
Stuart<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
wildfly-dev mailing list<br>
<a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a><br>
</blockquote></div><br></div>