[jbosstools-issues] [JBoss JIRA] (JBIDE-22305) automate process for fetching latest target platform mirrors

[Nick Boldt (JIRA)]

     [ https://issues.jboss.org/browse/JBIDE-22305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nick Boldt updated JBIDE-22305:
-------------------------------
    Description: 
Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.

Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace. 

Thus for webtools we might simply define:
webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/downloads/drops/R3.8.0/S-3.8.0M7-20160503010110/repository/

and we'd end up with:

http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8.0M7-20160503010110/

* Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevStudio_Master/job/jbosstools-requirements/ job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr

1. matrix job. each config is a trio of reqname/version/URL which is passed to jbosstools-requirements job to perform the mirror
* curl syntax to invoke the job
2. when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
* sed
3. when all children are done, a second downstream job runs TP update & validation against new .target files
* https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/util/verifyTarget.sh
* when done if success:
* ideally, generate a PR and attach link in email to jbosstools-dev@
* if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change



  was:
Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.

Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace. 

Thus for webtools we might simply define:
webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/downloads/drops/R3.8.0/S-3.8.0M7-20160503010110/repository/

and we'd end up with:

http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8.0M7-20160503010110/

* Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevStudio_Master/job/jbosstools-requirements/ job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr



> automate process for fetching latest target platform mirrors
> ------------------------------------------------------------
>
>                 Key: JBIDE-22305
>                 URL: https://issues.jboss.org/browse/JBIDE-22305
>             Project: Tools (JBoss Tools)
>          Issue Type: Feature Request
>          Components: target-platform, updatesite, upstream
>    Affects Versions: 4.4.0.Alpha1
>            Reporter: Nick Boldt
>            Assignee: Nick Boldt
>             Fix For: 4.4.0.Alpha3
>
>
> Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
> Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace. 
> Thus for webtools we might simply define:
> webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/downloads/drops/R3.8.0/S-3.8.0M7-20160503010110/repository/
> and we'd end up with:
> http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8.0M7-20160503010110/
> * Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevStudio_Master/job/jbosstools-requirements/ job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
> 1. matrix job. each config is a trio of reqname/version/URL which is passed to jbosstools-requirements job to perform the mirror
> * curl syntax to invoke the job
> 2. when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
> * sed
> 3. when all children are done, a second downstream job runs TP update & validation against new .target files
> * https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/util/verifyTarget.sh
> * when done if success:
> * ideally, generate a PR and attach link in email to jbosstools-dev@
> * if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)