[ https://issues.jboss.org/browse/JBTIS-498?page=com.atlassian.jira.plugin.... ] Nick Boldt commented on JBTIS-498: ---------------------------------- Looking through your https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate-disc/ job and I'm seeing a lot we can do to streamline and automate. 0. You're pulling the rsync script directly from github; we now prefer to fetch the zipped versions from nexus so you can see which version you're getting. I've updated your job here: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate-disc... 1. Why do you have 2 jobs for JBTIS-aggregate-disc and JBDS-aggregate-disc ? why not build them at the same time from the same job? 2. Why do you rsync the contents of sources/BUILD_KIND/site-final/target/ [1] AND its child folder sources/BUILD_KIND/site-final/target/repository [2] into the same target folder on dl.jb.org [3]? You're essentially publishing the same repo twice, plus you're publishing a lot of maven metadata that you (probably) will never need. [1] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate-disc/... [2] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/JBTIS-aggregate-disc/... [3] http://download.jboss.org/jbosstools/mars/snapshots/builds/integration-st... Do you do anything with category.xml? checkstyle*? local-artifacts? p2agent? p2artifacts? p2content? targetPlatformRepository? or the site*.zip? 3. As to the use of the --del flag... I think the problem is that you're pushing a single folder's contents into a single folder. When I use rsync.sh --del, it's to push a WHOLE folder and replace whatever is already there. So: {code:title=http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevStudio_9.0.mars/job/jbosstools-discovery_master/configure-readonly/} . ${WORKSPACE}/sources/publish/rsync.sh --del -s ${WORKSPACE}/sources/jbosstools/org.jboss.tools.central.discovery.earlyaccess/target/discovery-site \ -t mars/snapshots/updates/discovery.earlyaccess/${jbosstools_site_stream}/ {code} becomes {code:title=http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevStudio_9.0.mars/job/jbosstools-discovery_master/lastBuild/console} ++ echo '[INFO] rsync -arzq --protocol=28 --del /mnt/hudson_workspace/workspace/jbosstools-discovery_master/sources/jbosstools/org.jboss.tools.central.discovery.earlyaccess/target/discovery-site/* tools@filemgmt.jboss.org:/downloads_htdocs/tools/mars/snapshots/updates/discovery.earlyaccess/master//' {code} and results in this: http://download.jboss.org/jbosstools/mars/snapshots/updates/discovery.ear... Alternatively, if you use a versioned folder, you'll get an automated composite site and automated cleanup -- no --del flag needed! {code} . ${WORKSPACE}/sources/publish/rsync.sh -s ${WORKSPACE}/sources/jbdevstudio/com.jboss.jbds.central.discovery/target/discovery-site \ -t 9.0/snapshots/builds/jbosstools-discovery.central_${jbosstools_site_stream}/${BUILD_ID}-B${BUILD_NUMBER}/all/repo/ \ -DESTINATION devstudio@filemgmt.jboss.org:/www_htdocs/devstudio {code} results in: https://devstudio.redhat.com/9.0/snapshots/builds/jbosstools-discovery.ce... and https://devstudio.redhat.com/9.0/snapshots/builds/jbosstools-discovery.ce... and https://devstudio.redhat.com/9.0/snapshots/builds/jbosstools-discovery.ce... So... if you can change your method of publishing such that the site you produce in Jenkins is the same as the site you put on dl.b.o or ds.jb.c (without needing a dozen individual rsync/copy/move steps) not only will your local builds == your CI builds, but your release process will be simpler. 4. Why does your CI build sometimes ALSO push bits to /staging/ as well as /snapshots/ ? Surely that promotion step should ONLY ever happen by hand when it's time to do a QE staging release? 5. Why are there two copies of the same directory.xml file? https://devstudio.redhat.com/9.0/staging/updates/integration-stack/discov... (doesn't work, as it refers to a child plugins/ folder) and https://devstudio.redhat.com/9.0/staging/updates/integration-stack/discov... (works, as it refers to the child plugins/ folder) Hope that helps! -- This message was sent by Atlassian JIRA (v6.4.11#64026)