[jboss-dev-forums] [JBoss Tools Development] - Integration Tooling Release Process

Paul Leacu [https://community.jboss.org/people/pleacu] modified the document: "Integration Tooling Release Process" To view the document, visit: https://community.jboss.org/docs/DOC-48326 -------------------------------------------------------------- h1. Overview This document details the process for building and releasing composite/aggregate update sites for the JBoss integration tooling (formerly SOA tools). h2. Goals * Provide a light-weight process with minimal overhead for component owners. * Provide an ever-current update site containing the latest, stable releases of integration tooling components. * Allow component owners to independently manage their release schedules. * Allow component owners to manage their own dependency stack. h2. Assumptions * Components use released versions of dependencies (i.e. stable builds). * Component owners manage their own interdependencies with other integration components. * Components and dependencies adhere to Eclipse API versioning rules. h2. Implications * Aggregate releases will be driven by component releases. * Aggregate releases should manage themselves. * Component nightly/milestone builds should always be compatible with other components in the aggregate. h2. Definitions * *Integration Stack (IS)* - all JBoss middleware integration tooling * *IS configuration* - the definition of specific components and versions that make up the IS * *IS build* - an update site (p2 repo) aggregating specific versions of integration tooling components. * *Release types:** *stable* - an IS build aggregating stable (i.e. final) builds of all components that has been certified/tested by QA. * *development* - an IS build aggregating stable builds of all components that has not been certified/tested by QA. * *integration* - an IS build aggregating stable, development (e.g. beta) and/or integration builds of all components * *next* - an internal IS build incorporating the latest integration builds of all components (i.e. the current "trunk" build).  This will be used for identifying potential future incompatibilities. Stable builds will align  with a JBDS release in the same way JBT core builds do today.  Development builds will reflect the current state of all stable components.  Theoretically, any development build could be QA'd and promoted to stable (assuming no test failures).  Integration builds allow for an "interim" stack which may be useful during a QA cycle (e.g. verifying a fix that requires updates to multiple components; a single component can be updated via the component update site).  Next builds are used to identify potential incompatiblilities among components before they are integrated into a release and would occur weekly/monthly via a Hudson job (e.g. targeting JBT integration builds). An IS build only needs to occur when the IS configuration changes.  Because the IS stack is composed of stable components, configuration changes will only occur when individual components are released.  For this reason, it is assumed the build would be invoked manually. h1. Roles and Responsibilities h2. Component Owners 1. *Component update site:*  Component owners must provide an update site for each release of their component that is included in an IS build. 2. *Updating IS configuration:*  Component owners must update the IS build configuration when a new version of their component is released.  This ensures the IS is up-to-date. 3. *Respond to change requests:*  Component owners must review IS configuration change requests and respond in a timely manner.  If a requested change is incompatible with the current version of your component, provide details regarding when your component will be compatible, along with a patch containing changes to the IS configuration for your compatible release. h2. Release Engineers 1. *Maintain build infrastructure:*  Release engineers will be responsible for maintianing the build infrastructure, primarily the Jenkins jobs. 2. *Processing pull requests:*  Release engineers will be responsible processing all pull requests related to IS configuration.  This is to ensure consistency in the composition of the stack (e.g. components, component versions, IS version, etc.). 3. *Initiate IS builds:*  Release engineers will be responsible for initiating IS builds. 4. *Build promotion:*  Release engineers will be responsible for the promotion of IS builds (e.g. from development to stable). h1. Release Process h2. Component Updates The following outlines the basic process for updating the IS configuration.  Typically, this would occur when a component is released, but the same process is also used for updating shared dependencies (e.g. JBT core, third-party plugins, etc.). 1. Create a branch in the user repository for your IS configuration changes. 2. Modify the Integration Stack configuration for your updates, e.g. update URL for SwitchYard from 0.5.0.Final to 0.6.0.Final; update IS version from 0.9.0 to 0.9.1 (see versioning rules)1. For changes to your component within the community aggregate:  https://github.com/jbosstools/jbosstools-integration-stack/tree/master/jb... https://github.com/jbosstools/jbosstools-integration-stack/tree/master/jb.... Modify the https://github.com/jbosstools/jbosstools-integration-stack/tree/master/jb... compositeArtifacts.xml and https://github.com/jbosstools/jbosstools-integration-stack/tree/master/jb... compositeContent.xml files to point to the new component update site. 2. Modify the https://github.com/jbosstools/jbosstools-integration-stack/tree/master/jb... category.xml file as necessary for any feature changes (e.g. adding a feature). 3. Modify the project version in https://github.com/jbosstools/jbosstools-integration-stack/blob/master/ag... pom.xml. 2. For changes to your component within the production aggregate:  https://github.com/jbosstools/jbosstools-integration-stack/tree/master/de... https://github.com/jbosstools/jbosstools-integration-stack/tree/master/de.... Modify the https://github.com/jbosstools/jbosstools-integration-stack/tree/master/de... compositeArtifacts.xml and https://github.com/jbosstools/jbosstools-integration-stack/tree/master/de... compositeContent.xml files to point to the new component update site. 2. Modify the https://github.com/jbosstools/jbosstools-integration-stack/tree/master/de... category.xml file as necessary for any feature changes (e.g. adding a feature). 3. Modify the project version in https://github.com/jbosstools/jbosstools-integration-stack/blob/master/ag... pom.xml. 3. For changes to dependencies:  https://github.com/jbosstools/jbosstools-integration-stack/tree/master/ta... https://github.com/jbosstools/jbosstools-integration-stack/tree/master/ta.... Modify https://github.com/jbosstools/jbosstools-integration-stack/blob/master/ta... integration-stack-base.target, https://github.com/jbosstools/jbosstools-integration-stack/blob/master/ta... core-base.target or https://github.com/jbosstools/jbosstools-integration-stack/blob/master/ta... community.target 2. Modify project version in https://github.com/jbosstools/jbosstools-integration-stack/blob/master/ta... pom.xml 3. Ensure plugins are mirrored on jboss.org 3. Build locally to validate stack dependencies. 4. Create a https://issues.jboss.org/browse/JBTIS JBTIS JIRA task requesting update, e.g. update SwitchYard to 0.6.0.Final* Include link to github branch in task * Include build  results (e.g. SUCCESS or specific dependency resolution errors) 5. Create subtasks against each IS component for verification/approval* For components that failed the consistency check, mark subtasks appropriately, e.g. m2e-wtp version is inconsistent 6. Merge any changes required by other components from subtasks (e.g. update Savara to 2.1.0.Final) 7. Build IS locally to validate dependency consistency 8. Issue pull request against upstream IS configuration project 9. Process pull (by release engineer) 10. Invoke build job (by release engineer) h2. Platform Updates *Goal:* release updated IS within four weeks of JBT/JBDS core release. 1. Initiate platform update during JBT beta phase 2. Follow the component update process above (e.g. update JBT core from 3.3 to 4.0) h2. JBoss Central Discovery Updates *Goal:* Modify the JBoss Central mylyn discovery install/update mechanism. JBoss Central, based on http://wiki.eclipse.org/Mylyn/Discovery Mylyn Discovery, is in part a UI for installing new features into Eclipse or JBDS. It provides a richer, yet simpler UI than what Eclipse's p2 Install Manager provides out of the box. For detailed information see https://community.jboss.org/docs/DOC-48297 https://community.jboss.org/wiki/JBossCentralsSoftwareUpdateTab-HowDoesIt... 1. For changes to the discovery update:  https://github.com/jbosstools/jbosstools-integration-stack/tree/master/ta... https://github.com/jbosstools/jbosstools-integration-stack/tree/master/di... 2. Follow the same git process as defined in the Components Updates section. 3. Modify the discovery connector descriptors and installation units as needed within the https://github.com/jbosstools/jbosstools-integration-stack/blob/master/di... discovery plugin. 4. Add any icons that may be required. h2. Dependency Updates The process for updating a shared dependency is the same as that for updating any other component, with the following additions: 1. Make sure the dependency is mirrored on jboss.org. 2. Make sure the dependency is not already included through JBT core. If a dependency is included through JBT core, you will need to coordinate with the core team on the upgrade, as the change may impact components outside the IS. h2. Versioning Rules - major.minor.update.build *Major* should be updated for: * Base platform changes, e.g. Eclipse Indigo->Juno, JBT 3.3->4.0 * Major component updates, e.g. SwitchYard 1.0->2.0 *Minor* should be updated for minor component updates, e.g. SwitchYard 0.5->0.6 *Update* should be updated for minor component updates, e.g. SwitchYard 0.5.0->0.5.1 *Build* should be updated for each build, e.g. when building against nightly/integration/development repositories ** Note, the basic assumption here is that components are adhering to basic API rules (per Eclipse; e.g. minor includes bug fixes and api additions, major may include breaking api changes). h1. Update Site Locations * *Integration builds: http://download.jboss.org/jbosstools/updates/integration/juno/integration... http://download.jboss.org/jbosstools/updates/integration/juno/integration... * *Development builds: http://http://download.jboss.org/jbosstools/updates/development/juno/inte... http://http://download.jboss.org/jbosstools/updates/development/juno/inte... * *Stable builds: http://http://download.jboss.org/jbosstools/updates/stable/juno/integrati... http://http://download.jboss.org/jbosstools/updates/stable/juno/integrati... h1. Infrastructure * *JIRA -* https://issues.jboss.org/browse/JBTIS https://issues.jboss.org/browse/JBTIS * *Source -* https://github.com/jbosstools/jbosstools-integration-stack https://github.com/jbosstools/jbosstools-integration-stack* *target-platform* - provides a target platform defining all upstream dependencies for use by IS components (e.g. JBT core, third-party plugins) * *aggregate-site* - provides category and feature definitions for the IS update site * *discovery* - provides JBoss Central discovery support * *Jenkins* - * *target platform* - https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-target-platform https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-target-platform * *aggregate* - https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate * *aggregate publish/promote* - https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate-publish https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate-publish * *discovery* - https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-discovery https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-discovery * *install tests* - https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-4.0_stable_bra... https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-4.0_stable_bra... h2. Target Platforms The provided target platforms define all upstream dependencies available to an IS component.  Two target platforms are provided: base and community.  The base target platform includes all dependencies which are also included as part of JBDS.  The community target extends the base target by including dependencies which are only available in  community JBT releases.  If your component needs to be included in a JBDS release, DO NOT use the community target platform.  For more information and an example on how to refer to the target platform definition, see: https://github.com/jbosstools/jbosstools-integration-stack/blob/master/RE... https://github.com/jbosstools/jbosstools-integration-stack/blob/master/RE... h1. Future Plans 1. Add support for executing component unit tests as part of the aggregate build.  The goal being to help ensure the stability of the aggregate. 2. Add a "next" build that will build against upcoming core platforms (e.g. Kepler, JBT 4.1).  This should serve as an early warning system to identify platform incompatibilities early in the process. -------------------------------------------------------------- Comment by going to Community [https://community.jboss.org/docs/DOC-48326] Create a new document in JBoss Tools Development at Community [https://community.jboss.org/choose-container!input.jspa?contentType=102&a...]