[jbosstools-issues] [JBoss JIRA] (JBIDE-14448) Build doesn't detect cycle dependencies between JBossTools modules

Tue Oct 15 14:15:37 EDT 2013

    [ https://issues.jboss.org/browse/JBIDE-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12814316#comment-12814316 ] 

Nick Boldt edited comment on JBIDE-14448 at 10/15/13 2:14 PM:
--------------------------------------------------------------

OK, so what I'm hearing here is that we need two things for each and every project:

* a way to easily build a given project based on THE LATEST upstream builds of dependencies for a given stream (_40, _410, _41, _master), both locally and in Jenkins [this we already do]
* a way to easily build a given project based on SPECIFIC upstream builds of dependencies (eg., Base build #456 and Server build #567), both locally and in Jenkins [this is a new requirement - see JBIDE-15482 ]

Therefore we need to:

* replace staging/JOB_NAME/all/repo/ with ci/JOB_NAME/BUILD_ID/all/repo/ when publishing [0] (JBIDE-15482)
* automatically clean out old builds [1] to avoid filling download.jboss.org and crashing it  (JBIDE-15482)
* a tool [2] to generate a composite site on dl.jb.org, fed a series of URLs for upstream projects against which you want to build (using the latest by default + some static sites) [3]
* a tool [2] to generate a composite site *locally*, fed a series of URLs for upstream projects SPECIFIC BUILDS against which you want to build (likely a subset of the full site above [3]

[0] https://github.com/jbosstools/jbosstools-build-ci/blob/master/publish/publish_new.sh
[1] https://github.com/jbosstools/jbosstools-build-ci/blob/master/util/cleanup/jbosstools-cleanup.sh
[2] https://github.com/jbosstools/jbosstools-build-ci/blob/master/util/generateCompositeStagingSiteMetadata.sh
[3] http://download.jboss.org/jbosstools/builds/staging/_composite_/core/4.1.x.kepler/

Once all the above is in place, we would then need a profile or -Dflag in parent pom (such as *-Djbosstools-nightly*) to allow a user to point at their locally-generated composite site (from the previous item) instead of using the default composite.

For example, we might want to do this, to build LiveReload against specific upstream builds:

{code}
wget https://raw.github.com/jbosstools/jbosstools-build-ci/master/util/generateCompositeStagingSiteMetadata.sh && chmod +x generateCompositeStagingSiteMetadata.sh
./generateCompositeStagingSiteMetadata.sh -NAME 'My Local Upstream Composite Site' -DESTINATION /tmp/my-local-site/ -SITES http://download.jboss.org/jbosstools/updates/requirements/xulrunner-1.9.2/,http://download.jboss.org/jbosstools/builds/staging/CI/jbosstools-base_master/2013-09-25_09-26-23-B349/all/repo/,http://download.jboss.org/jbosstools/builds/staging/CI/jbosstools-server_master/2013-09-24_04-26-25-B380/all/repo/
{code}

then this:

{code}
cd ~/path/to/jbosstools-livereload
mvn clean verify -Djbosstools-nightly=file:///tmp/my-local-site/
{code}

... and boom goes the dynamite.

      was (Author: nickboldt):
    OK, so what I'm hearing here is that we need two things for each and every project:

* a way to easily build a given project based on THE LATEST upstream builds of dependencies for a given stream (_40, _410, _41, _master), both locally and in Jenkins [this we already do]
* a way to easily build a given project based on SPECIFIC upstream builds of dependencies (eg., Base build #456 and Server build #567), both locally and in Jenkins [this is a new requirement - see JBIDE-15482]

Therefore we need to:

* replace staging/JOB_NAME/all/repo/ with ci/JOB_NAME/BUILD_ID/all/repo/ when publishing [0] (JBIDE-15482)
* automatically clean out old builds [1] to avoid filling download.jboss.org and crashing it  (JBIDE-15482)
* a tool [2] to generate a composite site on dl.jb.org, fed a series of URLs for upstream projects against which you want to build (using the latest by default + some static sites) [3]
* a tool [2] to generate a composite site *locally*, fed a series of URLs for upstream projects SPECIFIC BUILDS against which you want to build (likely a subset of the full site above [3]

[0] https://github.com/jbosstools/jbosstools-build-ci/blob/master/publish/publish_new.sh
[1] https://github.com/jbosstools/jbosstools-build-ci/blob/master/util/cleanup/jbosstools-cleanup.sh
[2] https://github.com/jbosstools/jbosstools-build-ci/blob/master/util/generateCompositeStagingSiteMetadata.sh
[3] http://download.jboss.org/jbosstools/builds/staging/_composite_/core/4.1.x.kepler/

Once all the above is in place, we would then need a profile or -Dflag in parent pom (such as *-Djbosstools-nightly*) to allow a user to point at their locally-generated composite site (from the previous item) instead of using the default composite.

For example, we might want to do this, to build LiveReload against specific upstream builds:

{code}
wget https://raw.github.com/jbosstools/jbosstools-build-ci/master/util/generateCompositeStagingSiteMetadata.sh && chmod +x generateCompositeStagingSiteMetadata.sh
./generateCompositeStagingSiteMetadata.sh -NAME 'My Local Upstream Composite Site' -DESTINATION /tmp/my-local-site/ -SITES http://download.jboss.org/jbosstools/updates/requirements/xulrunner-1.9.2/,http://download.jboss.org/jbosstools/builds/staging/CI/jbosstools-base_master/2013-09-25_09-26-23-B349/all/repo/,http://download.jboss.org/jbosstools/builds/staging/CI/jbosstools-server_master/2013-09-24_04-26-25-B380/all/repo/
{code}

then this:

{code}
cd ~/path/to/jbosstools-livereload
mvn clean verify -Djbosstools-nightly=file:///tmp/my-local-site/
{code}

... and boom goes the dynamite.

> Build doesn't detect cycle dependencies between JBossTools modules
> ------------------------------------------------------------------
>
>                 Key: JBIDE-14448
>                 URL: https://issues.jboss.org/browse/JBIDE-14448
>             Project: Tools (JBoss Tools)
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 4.1.0.Beta1
>            Reporter: Denis Golovin
>            Assignee: Nick Boldt
>            Priority: Blocker
>             Fix For: 4.2.0.Alpha1
>
>
> Currently every component references nightly composite update site during the build to get dependencies. That leads to possibility to introduce cycle dependencies between components that is not detected by build.
> Here is example of PR introduced a cycle between JST and VPE. Build works because JST have access to VPE BrowserSim through nightly composite update site.
> The solution would be to reference specific nightly update sites for modules that are dependencies for current one. 
> In this case JST should have reference only to base composite nightly update site. That would solve cycle dependency detection problem and speed up the build because it would not require to download and analyse all metadata from jbosstools nightly composite update site.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira