<div dir="ltr">Thanks for this!<div><br></div><div>I want to make sure that what comes out of this are clear decisions about what we're willing to handle in terms of various sorts of conflicts. What kinds of things the tool is responsible for solving vs what the user has to resolve, what tooling config settings we support for tweaking what the tool will do for the user, etc.</div><div><br></div><div>When I say conflicts I don't just mean file conflicts; I also mean use case conflicts. What's the right way / wrong way to do things and what combinations are ok.</div><div><br></div><div>Part of what makes this interesting is the provisioning tool itself is meant to be used for setting up the installation the way the user wants it. So in many many cases if there is a diff it's because the user bypassed the tool. Of course that's going to be very common, particularly when it comes to the config files, where people have been using tools like the CLI to set up their configs for years. Manually adding, removing and modifying modules, hopefully we can wean people off that and get them to use the tool, but still, people won't. </div><div><br></div><div>The 3-way diffing approach you describe sounds like it should be flexible enough to handle various permutations.</div><div><br></div><div>Re: the "update feature pack" what happens if the user has produced and stored one of those, then updates their base configuration, and then applies that update feature pack. So, the user has taken an install provisioned by the tool, with standalone.xml produced according to the inputs to the tool. Then modified that via CLI/HAL and saved those mods in an update feature pack. Then taken the bulk of those mods, adjusted their inputs to the provisioning tool such that tool produces the desired config, does an installation and wishes to apply that update feature pack?</div><div><br></div><div>Detail question:</div><div><br></div><div>"<span style="font-size:12.8px">Filtering out some of the 'unimportant' files (tmp, logs)."</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">What does that mean? The tmp and logs dirs end up empty in the new installation? That might be ok for tmp but for logs it sounds wrong. I'm not sure it's necessary for tmp.</span></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Brian Stansberry<div>Manager, Senior Principal Software Engineer</div><div>Red Hat</div></div></div></div>
<br><div class="gmail_quote">On Mon, Sep 25, 2017 at 4:06 AM, Emmanuel Hugonnet <span dir="ltr"><<a href="mailto:ehugonne@redhat.com" target="_blank">ehugonne@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hey guys,<br>
<br>
TL;DR<br>
I've been experimenting with Alexey to update a customized provisioned server using the provisioning tool [1].<br>
I'm using the syncing operations [2] that I created a while back by porting the domain synchronization operations to standalone (to<br>
synchronize standalone instances in a cloud environment).<br>
I'm looking for some feedback on this approach.<br>
Cheers<br>
Emmanuel<br>
<br>
[1]: <a href="https://github.com/ehsavoie/pm/tree/upgrade-feature-pack" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/tree/upgrade-feature-pack</a><br>
[2]: <a href="https://github.com/ehsavoie/wildfly-core/tree/model_diff" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>wildfly-core/tree/model_diff</a><br>
<br>
----<br>
full version:<br>
<br>
<br>
Updating WildFly<br>
<br>
<br>
<<a href="https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#terminology" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/blob/upgrade-feature-pack/<wbr>docs/guide/wildfly/upgrade.<wbr>adoc#terminology</a>>Terminology<br>
<br>
Updating is the process of applying a fix pack that will increment the micro version. There should be no compatiblity issue. Upgrading is<br>
the transition of a minor version, compatiblity should be available but there are a lot more changes.<br>
<br>
While the mecanisms discussed here are general, they might need more refinement for an upgrade.<br>
<br>
<br>
<<a href="https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#use-case" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/blob/upgrade-feature-pack/<wbr>docs/guide/wildfly/upgrade.<wbr>adoc#use-case</a>>Use case<br>
<br>
The use case is quite simple: *I have version 1.0.0 installed and i want to update to 1.0.1 but I have locally customized my server and i’d<br>
like to keep those changes*.<br>
We have several local elements to take into account:<br>
<br>
*<br>
<br>
filesystem changes (files added, removed or deleted).<br>
<br>
*<br>
<br>
configuration changes.<br>
<br>
The basic idea is to diff the existing instance with a pure new installation of the same version then apply those changes to a new<br>
provisioned version instance for staging.<br>
We can keep it at the basic filesystem approach with some simple merge strategy (theirs, ours).<br>
We can use the plugin to go into more details. For exemple using the model diff between standalone Wildfly instances we can create a cli<br>
script to reconfigure the upgraded instance in a post-installation step.<br>
<br>
<br>
<<a href="https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#diffing-the-filesystem" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/blob/upgrade-feature-pack/<wbr>docs/guide/wildfly/upgrade.<wbr>adoc#diffing-the-filesystem</a>><wbr>Diffing the filesystem<br>
<br>
The idea is to compare the instance to be upgraded with one instance provisioned with the same feature packs as the one we want to upgrade.<br>
The plugin will provide a list of files or regexp to be excluded.<br>
Each file will be hashed and we compare the hash + the relative path to deduce deleted, modified or added files.<br>
For textual files we can provide a diff (and the means to apply it), but maybe that should be for a later version as some kind of<br>
interaction with the user might be required.<br>
<br>
<br>
<<a href="https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-standalone-plugin" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/blob/upgrade-feature-pack/<wbr>docs/guide/wildfly/upgrade.<wbr>adoc#wildfly-standalone-plugin</a><wbr>>WildFly standalone<br>
plugin<br>
<br>
This is a specialization of the upgrading algorithm:<br>
<br>
*<br>
<br>
Filtering out some of the 'unimportant' files (tmp, logs).<br>
<br>
*<br>
<br>
Creating diff of textual files (for example the realm properties) which will be applied (merging strategy à la git).<br>
<br>
*<br>
<br>
Using an embedded standalone it creates a jboss-cli script to reconfigure the server (adding/removing extensions and reconfiguring<br>
subsystems).<br>
<br>
*<br>
<br>
Deleting files that were removed.<br>
<br>
This is done on a staging upgraded instance before being copied over the old instance.<br>
I have added a diff/sync operation in standalone that is quite similar to what happens when a slave HC connects to the DC. Thus I start the<br>
current installation, and connect to it from an embedded server using the initial configuration and diff the models.<br>
This is 'experimental' but it works nicely (I was able to 'upgrade' from the standalone.xml of wildfly-core to the standalone-full.xml of<br>
wildfly).<br>
I’m talking only about the model part, I leave the files to the filesystem 'diffing' but it wiill work with managed deployments are those<br>
are added by the filesystem part and then the deployment reference is added in the model.<br>
For a future version of the tooling/plugin we might look for a way to interract more with the user (for example for applying the textual<br>
diffs to choose what to do per file instead of globally).<br>
Also currently the filters for excluding files are defined by the plugin but we could enrich them from the tooling also.<br>
<br>
<br>
<<a href="https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#producing-an-update-feature-pack" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/blob/upgrade-feature-pack/<wbr>docs/guide/wildfly/upgrade.<wbr>adoc#producing-an-update-<wbr>feature-pack</a>>Producing an<br>
update feature pack<br>
<br>
>From the initial upgrade mecanism Alexey has seen the potential to create a feature pack instead of my initial format.<br>
Currently i’m able to create and installa a feature-pack that will supersede the initial installation with its own local modifications.<br>
Thus from my customized instance I can produce a feature pack that will allow me to reinstall the same instance. Maybe this can be also use<br>
to produce upgrade feature pack for patching.<br>
<br>
<br>
<<a href="https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-domain-mode" rel="noreferrer" target="_blank">https://github.com/ehsavoie/<wbr>pm/blob/upgrade-feature-pack/<wbr>docs/guide/wildfly/upgrade.<wbr>adoc#wildfly-domain-mode</a>><wbr>WildFly domain mode<br>
<br>
Domain mode is a bit more complex, and we need to think how to manager the model changes.<br>
Those can be at the domain level or the host level and depending on the instance target we would need to get the changes from the domain.xml<br>
or/and the host.xml.<br>
I’m thinking about applying the same strategy as what was done for standalone : aka expose the sync operations to an embedded HC.<br>
<br>
<br>
<br>______________________________<wbr>_________________<br>
wildfly-dev mailing list<br>
<a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/wildfly-dev</a><br></blockquote></div><br></div>