[wildfly-dev] Re: Creating -jakarta variants of WildFly artifacts using source transformation

On Wed, Oct 20, 2021 at 4:04 PM Brian Stansberry < brian.stansberry(a)redhat.com&gt; wrote: > Last July, I started a thread[1] here about ways to get native jakarta > namespace variants of artifacts, which we need for our move to EE 10. > One of the items discussed there was "5) New maven module, transform > source and build". I'm posting here as a kind of status update and tutorial > about that approach. > > At this point there are 11 maven modules in the WildFly main source > tree[2] that are producing artifacts using the approach described there, > and there are PRs open for at least a couple more. I have filed or soon > will file JIRAs[3] for all the remaining WF main tree's modules that > produce artifacts that WildFly Preview currently is transforming when it > provisions a server[4], so there will be more coming. > > Overview > > The high level overview of how these work is that a pom-only maven module > is created that corresponds to an existing module that produces an javax > artifact. The new module's pom uses the batavia maven plugin[5] (and > Eclipse Transformer under the covers) to find the source code from the > existing maven module, transform it, and write the transformed output to > the target/ dir of the new module, specifically in the > 'generated-resources', 'generated-sources', 'generated-test-resources' and > 'generated-test-sources' dirs. That generated source is then associated > with the overall maven execution for the module, and thereafter gets > included as part of the subsequent compile, test, package, install and > deploy goals executed for the module. So, presto we have native jakarta > source available that is then compiled and tested and used to > package/install/deploy binary, source and javadoc jars.[6] > > The generated source does not get committed into the git repository. The > git repo only has the pom. > > It's a common thing for generated source to be used in an artifact build; > for example it's the technique we use to generate implementation classes > from the XXXLogger interfaces we have for each of our subsystems. What > we're doing here just takes this concept to the max by generating 100% of > the source, including test source. > > Tutorial > > I'm going to use a PR I sent up yesterday to illustrate how to create one > of these: > > https://github.com/wildfly/wildfly/pull/14822 > > Steps: > > 1) Decide whether the module you want to work on is ready. If you're not > the component lead responsible for the module, ask that lead. To see if a > module is 'ready', look at its compile time dependencies and see if it > still depends on other artifacts for which a native jakarta variant is > needed and doesn't exist yet. Ask here or in zulip if you are not sure! > Things can change quickly, and there also may be some edge cases where > you'd think you need a jakarta namespace dependency but you really don't. > > If there are dependencies that are not ready yet, stop and wait until > they are available. Or be prepared for your new module to not build, at > which point you can save your work and wait. > > 2) Create a new dir under ee-9/source-transform for your new module. If > the module you are transforming isn't directly under the root of the WF > source, then create a matching structure under ee-9/source-transform. For > example, I created ee-9/source-transform/jpa/spi to produce a jakarta.* > variant of jpa/spi. > > 3) Copy the pom.xml from an *existing source-transform module* into your > new dir. Easiest is to use one that comes from a dir the same # of levels > below source-transform as your new dir, so a couple relative paths in your > new pom are correct. For my PR I copied over > > > https://github.com/wildfly/wildfly/blob/main/ee-9/source-transform/weld/c... > > Don't start with the pom from your source input module. Unless you want > to work out how to adapt it to use the source transformation. :) Granted > it's not rocket science. But it's easier for code reviewers to look at > these if they look like the other ones. > > 4) Change the artifactId in your new pom to *exactly* match the > artifactId of the module you're transforming, but with "-jakarta" appended: > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-d9ae5e5e0388bc9e... > > 5) Change the 'name' tag to something appropriate, like > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-d9ae5e5e0388bc9e... > > 6) Change the 'transformer-input-dir' maven property to point to the root > of the module you are transforming: > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-d9ae5e5e0388bc9e... > > This property is used in the batavia maven plugin config in the parent > source-transform dir. Each child module sets the property to point to the > correct input source for that module. > > This one small tweak is the only thing you need to deal with to get the > source transformation set up. > > 7) Configure dependencies. > > a) Delete the existing 'dependency' entries in your new pom, as they are > for whatever random file you copied in. > > b) Copy over the 'dependency' entries from the pom of the module you are > transforming. > > c) For any entries where your new artifact uses a dependency with a > different groupId:artifactId from the one you're transforming, change the > GA. For example: > > EE 8 JTA > > https://github.com/wildfly/wildfly/blob/25.0.0.Final/jpa/spi/pom.xml#L58 > > became EE 9.1 JTA > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-d9ae5e5e0388bc9e... > > Or, in another example > > an EE 8 based 'ee' subsystem module dep: > > > https://github.com/wildfly/wildfly/blob/25.0.0.Final/ee-security/pom.xml#L45 > > became instead a dep on the new '-jakarta' variant: > > > https://github.com/wildfly/wildfly/pull/14652/files#diff-7ed0e6e85369889b... > > d) A nice to have is to separate from the others in the dependency > listing deps where WF Preview is using a different artifact from what > standard WF is using. For example see > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-d9ae5e5e0388bc9e... > > This separation helps code reviewers. > > 8) Tell your new module's parent to include it in the build: > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-37dfe0d6acd41e18... > > 9) The ee-9/pom.xml maintains a dependencyManagement set for all > artifacts that differ in WF Preview from what is in standard WF. These can > either be artifacts whose GA is not used at all in standard WF, or ones > where WF Preview uses a different version. Add your new artifact to this > dependencyManagement: > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-5ddee375313bf441... > > 10) Allow re-use of the module.xml that incorporates this artifact > between standard WF and WF Preview. This is done by modifying the > module.xml to add an expression that the maven-resource-plugin replaces > when preparing the module.xml resource file for use in the FP build: > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-f89017fad51e240c... > > The added @module.jakarta.suffix@ gets replaced either with an empty > string (standard WF) or "-jakarta" (WF Preview) > I forgot to say, please note the escape slash added at the beginning of the overall expression. > > 11) Add your new artifact as a dependency of ee-9/feature-pack/pom.xml. > This is needed so the wildfly-preview Galleon feature pack can utilize your > new artifact: > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-1375aa050fa94d63... > > 12) Tell the wildfly-preview feature-pack build to ignore the no-longer > relevant javax artifact you're replacing. This saves build time, compute > resources and false positives in the build log that make us think your > artifact hasn't been handled yet. > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-1375aa050fa94d63... > > (Note the precise spot to put that 'exclude' may differ, a bit. Ask if > this is unclear.) > > 13) Add an entry for your new artifact in the license.xml file for the > wildfly-preview feature back. Put it in the right alphabetical spot based > on its groupId and artifactId. > > > https://github.com/wildfly/wildfly/pull/14822/files#diff-11ae0f2f274afa8c... > > 14) Do a build and if good, commit and send up a PR! If your input module > had tests, your new module should as well and they should run. If you're > curious, have a look in the new target/generated-xxx dirs to see the > source.. > > This sounds like a lot, and I suppose it is, but other than step 7) it's > all very simple stuff, mostly things you'd do any time you add a new module > to the WF source tree. Granted I'm practiced at this, but it took me about > half an hour to work up the PR I've been using as an example. > > If you're interested in doing one of these, and the component lead > responsible for the input module is agreeable, please go for it and feel > free to ask for help. > > [1] > https://lists.jboss.org/archives/list/wildfly-dev@lists.jboss.org/thread/... > > [2] Child modules under > https://github.com/wildfly/wildfly/tree/main/ee-9/source-transform > > [3] High level tracker issues for this work are > https://issues.redhat.com/browse/WFLY-15436 and > https://issues.redhat.com/browse/WFLY-15437. I'm filing separate issues > for individual pieces and am linking those to one or the other of these two > trackers. > > [4] > https://docs.google.com/spreadsheets/d/1TekfyRb2UBCLqsPQ83WTg8XOw9qwuHVSh... > shows all artifacts (not just those from the WF source tree) that the > WildFly Galleon plugin was transforming in builds on Oct 14. The rows that > end with '-26.0.0.Beta1-SNAPSHOT.jar' are ones that are produced by > WildFly main itself. I add new tabs to that document weekly to track > progress on reducing the # of artifacts being transformed by the Galleon > plugin. > > [5] > https://github.com/wildfly/wildfly/blob/main/ee-9/source-transform/pom.xm... > > [6] > https://repository.jboss.org/org/wildfly/wildfly-mail-jakarta/25.0.0.Final/ > which was produced from a maven module with nothing in it but a pom. > > Best regards, > > -- > Brian Stansberry > Project Lead, WildFly > He/Him/His > -- Brian Stansberry Principal Architect, Red Hat JBoss EAP He/Him/His _______________________________________________ wildfly-dev mailing list -- wildfly-dev(a)lists.jboss.org To unsubscribe send an email to wildfly-dev-leave(a)lists.jboss.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s