[jbosstools-issues] [JBoss JIRA] (JBIDE-19467) repository-utils :: FetchSourcesFromManifests should create its own source zip, not rely on publish.sh

[Nick Boldt (JIRA)]

    [ https://issues.jboss.org/browse/JBIDE-19467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13076433#comment-13076433 ] 

Nick Boldt commented on JBIDE-19467:
------------------------------------

PR:

https://github.com/jbosstools/jbosstools-maven-plugins/pull/46

Results when it fails:

{code}
mvn verify -Pmaximum -Pfetch-source-zips -e  -X

[ERROR] Failed to execute goal org.jboss.tools.tycho-plugins:repository-utils:0.22.1-SNAPSHOT:fetch-sources-from-manifests (fetch-sources) on project org.jboss.tools.site.core: Problem occurred checking upstream buildinfo.json files!
[ERROR] /home/nboldt/eclipse/workspace-jboss/jbosstools-github-master/jbosstools-build-sites/aggregate/site/target/buildinfo/buildinfo_jbosstools-browsersim.json
[ERROR] contains 3b22a4bbe6610e35401bfccc461cb329e8b806b0, but upstream project's MANIFEST.MF has Eclipse-SourceReferences
[ERROR] commitId 9255a5b7c04fb10768c14942e60092e860881d6b.
[ERROR] If you have locally built projects which are aggregated here,
[ERROR] ensure they are built from the latest SHA from HEAD, not a local topic branch.
[ERROR] Or, use -DskipCheckSHAs=true to bypass this check.
{code}

To bypass:

{code}mvn verify -Pmaximum -Pfetch-source-zips -e  -X -DskipCheckSHAs=true{code}

To make it fail, you just need to build something locally (like jbosstools-browsersim from https://github.com/nickboldt/jbosstools-browsersim/commit/9255a5b7c04fb10768c14942e60092e860881d6b ), then rebuilt jbosstools-build-sites/aggregate/site/pom.xml.



> repository-utils :: FetchSourcesFromManifests should create its own source zip, not rely on publish.sh
> ------------------------------------------------------------------------------------------------------
>
>                 Key: JBIDE-19467
>                 URL: https://issues.jboss.org/browse/JBIDE-19467
>             Project: Tools (JBoss Tools)
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 4.3.0.Alpha2
>            Reporter: Nick Boldt
>            Assignee: Nick Boldt
>            Priority: Blocker
>             Fix For: 4.3.0.Beta2
>
>
> In repository-utils, FetchSourcesFromManifests should create its own source zip, not rely on publish.sh. Because we no longer use publish.sh and don't want to have rsync.sh produce artifacts, only rsync them.
> Here's what publish.sh does:
> {code}
> # collect component zips from upstream aggregated build jobs
> if [[ ${JOB_NAME/.aggregate} != ${JOB_NAME} ]] && [[ -d ${WORKSPACE}/sources/aggregate/site/zips ]]; then
>   mkdir -p ${STAGINGDIR}/components
>   for z in $(find ${WORKSPACE}/sources/aggregate/site/zips -name "*updatesite-*.zip"); do
>     # generate MD5 sum for zip (file contains only the hash, not the hash + filename)
>     for m in $(md5sum ${z}); do if [[ $m != ${z} ]]; then echo $m > ${z}.MD5; fi; done
>     mv $z ${z}.MD5 ${STAGINGDIR}/components
>   done
>   # TODO :: JBIDE-9870 When we have a -Update-Sources- zip, this can be removed
>   mkdir -p ${STAGINGDIR}/all/sources  
>   # OLD: unpack component source zips like jbosstools-pi4soa-3.1_trunk-Sources-SNAPSHOT.zip or jbosstools-3.2_trunk.component--ws-Sources-SNAPSHOT.zip
>   # NEW: JBIDE-16632: unpack component source zips like jbosstools-base_Alpha2-v20140221-1555-B437_184e18cc3ac7c339ce406974b6a4917f73909cc4_sources.zip
>   for z in $(find ${WORKSPACE}/sources/aggregate/site/zips -name "*Sources*.zip" -o -name "*_sources.zip" -o -name "*-src.zip"); do
>     zn=${z%*-Sources*.zip}; zn=${zn%*_sources.zip}; zn=${zn%*-src.zip}; zn=${zn#*--}; zn=${zn##*/}; zn=${zn#jbosstools-}; 
>     # zn=${zn%_trunk}; zn=${zn%_stable_branch};
>     mkdir -p ${STAGINGDIR}/all/sources/${zn}/
>     # remove one level of folder nesting - don't want an extra jbosstools-base-184e18cc3ac7c339ce406974b6a4917f73909cc4 folder under jbosstools-base_Alpha2-v20140221-1555-B437_184e18cc3ac7c339ce406974b6a4917f73909cc4
>     unzip -qq -o -d ${tmpdir}/${zn}/ $z
>     mkdir -p ${STAGINGDIR}/all/sources/${zn}/
>     mv ${tmpdir}/${zn}/jbosstools-*/* ${STAGINGDIR}/all/sources/${zn}/
>     rm -fr ${tmpdir}/${zn}/
>   done
>   # add component sources into sources zip
>   pushd ${STAGINGDIR}/all/sources
>   zip ${STAGINGDIR}/all/${SRCSNAME} -q -r * -x hudson_workspace\* -x documentation\* -x download.jboss.org\* -x requirements\* \
>     -x workingset\* -x labs\* -x build\* -x \*test\* -x \*target\* -x \*.class -x \*classes\* -x \*bin\* -x \*.zip \
>     -x \*docs\* -x \*reference\* -x \*releng\* -x \*.git\* -x \*/lib/\*.jar -x \*getRemoteFile\*
>   popd
>   rm -fr ${STAGINGDIR}/all/sources
>   z=${STAGINGDIR}/all/${SRCSNAME}; for m in $(md5sum ${z}); do if [[ $m != ${z} ]]; then echo $m > ${z}.MD5; fi; done
>   # JBIDE-7444 get aggregate metadata xml properties file
>   if [[ -f ${WORKSPACE}/sources/aggregate/site/zips/build.properties.all.xml ]]; then
>     rsync -aq ${WORKSPACE}/sources/aggregate/site/zips/build.properties.all.xml ${STAGINGDIR}/logs/
>   fi
> fi
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)