[jbosstools-issues] [JBoss JIRA] (JBIDE-14017) cache downloaded zips fetched as part of building aggregate site for reuse across Jenkins slaves (rather than in each slave's workspace)

Wed Apr 17 05:17:53 EDT 2013

    [ https://issues.jboss.org/browse/JBIDE-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767938#comment-12767938 ] 

Mickael Istria commented on JBIDE-14017:
----------------------------------------

I renamed zipCacheDir to zipCacheFolder for more consistency and pushed the change

[~maxandersen] As Nick explained, the use case we have make that the Mojo will not process the same files simultaneously, so it's likely there is no risk to process the same file from different aggregation jobs at the same time. The mojo uses a lot of md5 checks, so it's not possible that the output is different from what is on site.

[~nickboldt] I'm not a big fan of changing the parent pom, I'd rather put that property setting in the job config.

> cache downloaded zips fetched as part of building aggregate site for reuse across Jenkins slaves (rather than in each slave's workspace)
> ----------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JBIDE-14017
>                 URL: https://issues.jboss.org/browse/JBIDE-14017
>             Project: Tools (JBoss Tools)
>          Issue Type: Bug
>          Components: Build/Releng, updatesite
>    Affects Versions: 4.1.0.Alpha2
>            Reporter: Nick Boldt
>            Assignee: Mickael Istria
>             Fix For: 4.1.0.Beta1
>
>
> {quote}
> (2013-04-12 12:19:29) mistria: nickboldt: aggregate could compare logs
> (2013-04-12 12:19:35) nickboldt: mistria: which logs?
> (2013-04-12 12:20:58) mistria: nickboldt: the one we generate when using "fetch-zips"
> (2013-04-12 12:22:40) nickboldt: so you want the aggregate job to fetch the zips BEFORE it builds the update site to determine if it should build the update site?
> (2013-04-12 12:24:05) nickboldt: whole point of a "should I build?" check is that it happens before the build... eg., check SVN, check git, or run jbosstools-composite-install_master to check the installed IUs installed from the composite to determine if contents or versions have changed
> (2013-04-12 12:24:46) nickboldt: not adverse to the idea of splitting the "fetch zips" step out of the "build aggregate" job, but ... then we have 2 more jobs to worry about 
> (2013-04-12 12:24:53) mistria: nickboldt: actually, the new version of the fetch mojo won't re-fetch zips if it finds out the the signature of the to-be-downloaded zip is the same as the one it has in cache
> (2013-04-12 12:25:05) nickboldt: cache in workspace?
> (2013-04-12 12:25:06) mistria: nickboldt: so no change, not fetch
> (2013-04-12 12:25:23) nickboldt: or in static jenkins share folder?
> (2013-04-12 12:25:23) mistria: nickboldt: in aggregate/site/zips
> (2013-04-12 12:25:36) nickboldt: great, so it's one fetch per slave :(
> (2013-04-12 12:26:14) nickboldt: would be better to cache that outside the workspace, if building in Jenkins
> (2013-04-12 12:26:16) mistria: nickboldt: right, we should provide a way to override this directory and put in in a shared folder
> (2013-04-12 12:27:12) mistria: nickboldt: there is not system property for that yet... https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/repository-utils/src/main/java/org/jboss/tools/tycho/sitegenerator/FetchZipsFromRepo.java#L60
> (2013-04-12 12:28:21) nickboldt: Old way: https://github.com/jbosstools/jbosstools-build-sites/blob/jbosstools-4.1.0.Alpha2x/aggregate/build.xml#L510 and https://github.com/jbosstools/jbosstools-build-sites/blob/jbosstools-4.1.0.Alpha2x/aggregate/build.xml#L33 (but we still dumped to $\{output.dir\}/zips, not to \{WORKINGDIR\}/zips )
> (2013-04-12 12:28:58) nickboldt: mistria: should allow caching files in Jenkins's static file tree for use across slaves
> {quote}
> So, I guess all we need to do is expose *outputFolder* to be overridden when -Phudson is enabled.
> {code:title=https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/repository-utils/src/main/java/org/jboss/tools/tycho/sitegenerator/FetchZipsFromRepo.java#L60}
> 	/**
> * Location where to put zips
> * @parameter default-value="${basedir}/zips"
> */
> private File outputFolder;
> {code}
> We could even set the correct path in the parent pom's hudson profile so that it need not be set in all the various jobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira