[JBoss JIRA] (JBIDE-22305) automate process for fetching latest target platform mirrors
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-22305?page=com.atlassian.jira.plugi... ]
Nick Boldt updated JBIDE-22305:
-------------------------------
Description:
Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
Thus for webtools we might simply define:
webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
and we'd end up with:
http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
* Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
1. matrix job. each config is a trio of reqname/version/URL which is passed to a process akin to that of the jbosstools-requirements job to perform the mirror; process is now scripted here: https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/p...
2. when all children are done, a downstream job can runs TP update & validation against new .target files
* fetch http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstoolstargetplatf...
* parse that into a list of URLs
* each URL contains REQ_NAME/VERSION, which can then be matched up with similar lines in .target files
* resulting edited .target files will remain in the workspace, and we can then run
* https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
* when done if success:
* ideally, generate a PR and attach link in email to jbosstools-dev@
* if can't generate PR, then attach patch in email to nickboldt & mistria to apply locally in jbosstools-target-platforms to create a PR
* email should includes boilerplate text to send mail to list announcing new change for review
was:
Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
Thus for webtools we might simply define:
webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
and we'd end up with:
http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
* Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
1. matrix job. each config is a trio of reqname/version/URL which is passed to a process akin to that of the jbosstools-requirements job to perform the mirror; process is now scripted here: https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/p...
2. when all children are done, a downstream job can runs TP update & validation against new .target files
* fetch . when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
* sed
* https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
* when done if success:
* ideally, generate a PR and attach link in email to jbosstools-dev@
* if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change
> automate process for fetching latest target platform mirrors
> ------------------------------------------------------------
>
> Key: JBIDE-22305
> URL: https://issues.jboss.org/browse/JBIDE-22305
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: target-platform, updatesite, upstream
> Affects Versions: 4.4.0.Alpha1
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.0.Alpha3
>
>
> Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
> Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
> Thus for webtools we might simply define:
> webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
> and we'd end up with:
> http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
> * Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
> 1. matrix job. each config is a trio of reqname/version/URL which is passed to a process akin to that of the jbosstools-requirements job to perform the mirror; process is now scripted here: https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/p...
> 2. when all children are done, a downstream job can runs TP update & validation against new .target files
> * fetch http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstoolstargetplatf...
> * parse that into a list of URLs
> * each URL contains REQ_NAME/VERSION, which can then be matched up with similar lines in .target files
> * resulting edited .target files will remain in the workspace, and we can then run
> * https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
> * when done if success:
> * ideally, generate a PR and attach link in email to jbosstools-dev@
> * if can't generate PR, then attach patch in email to nickboldt & mistria to apply locally in jbosstools-target-platforms to create a PR
> * email should includes boilerplate text to send mail to list announcing new change for review
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (JBIDE-22305) automate process for fetching latest target platform mirrors
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-22305?page=com.atlassian.jira.plugi... ]
Nick Boldt updated JBIDE-22305:
-------------------------------
Description:
Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
Thus for webtools we might simply define:
webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
and we'd end up with:
http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
* Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
1. matrix job. each config is a trio of reqname/version/URL which is passed to a process akin to that of the jbosstools-requirements job to perform the mirror; process is now scripted here: https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/p...
2. when all children are done, a downstream job can runs TP update & validation against new .target files
* fetch . when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
* sed
* https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
* when done if success:
* ideally, generate a PR and attach link in email to jbosstools-dev@
* if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change
was:
Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
Thus for webtools we might simply define:
webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
and we'd end up with:
http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
* Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
1. matrix job. each config is a trio of reqname/version/URL which is passed to jbosstools-requirements job to perform the mirror
* curl syntax to invoke the job
2. when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
* sed
3. when all children are done, a second downstream job runs TP update & validation against new .target files
* https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
* when done if success:
* ideally, generate a PR and attach link in email to jbosstools-dev@
* if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change
> automate process for fetching latest target platform mirrors
> ------------------------------------------------------------
>
> Key: JBIDE-22305
> URL: https://issues.jboss.org/browse/JBIDE-22305
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: target-platform, updatesite, upstream
> Affects Versions: 4.4.0.Alpha1
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.0.Alpha3
>
>
> Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
> Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
> Thus for webtools we might simply define:
> webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
> and we'd end up with:
> http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
> * Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
> 1. matrix job. each config is a trio of reqname/version/URL which is passed to a process akin to that of the jbosstools-requirements job to perform the mirror; process is now scripted here: https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/p...
> 2. when all children are done, a downstream job can runs TP update & validation against new .target files
> * fetch . when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
> * sed
> * https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
> * when done if success:
> * ideally, generate a PR and attach link in email to jbosstools-dev@
> * if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (JBIDE-22305) automate process for fetching latest target platform mirrors
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-22305?page=com.atlassian.jira.plugi... ]
Nick Boldt commented on JBIDE-22305:
------------------------------------
I've managed to set up a matrix job [0] to create all the mirrors we need for the M7 TPs. Takes a little bit of configuration (REQ_NAME;VERSION[;optionalURLOverride]) but then it just runs all the mirrors in parallel for as long as it takes, up to 4hrs for the Neon one.
[0] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt...
Now I need to figure out how to run a downstream job to invoke verifyTarget.sh against the new mirror URLs [1] in order to produce a PR & p2diffs from those changes automatically. :D
[1] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt...
Neon.0.M7 mirror [2] is still ongoing at this time, but the rest are done.
[2] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt...
> automate process for fetching latest target platform mirrors
> ------------------------------------------------------------
>
> Key: JBIDE-22305
> URL: https://issues.jboss.org/browse/JBIDE-22305
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: target-platform, updatesite, upstream
> Affects Versions: 4.4.0.Alpha1
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.0.Alpha3
>
>
> Rather than manually pulling requirements, it would be hella sweet if there was a job config into which we could just list all the URLs to mirror and their matching project names.
> Then this job would scrape the *.target files for the current list of URLs used, grep out the /requirements/<reqname>/<version>, fetch a new mirror for each new version*, and dump an updated copy of the .target file into the job's workspace.
> Thus for webtools we might simply define:
> webtools,S-3.8.0M7-20160503010110,http://download.eclipse.org/webtools/do...
> and we'd end up with:
> http://download.jboss.org/jbosstools/updates/requirements/webtools/S-3.8....
> * Since we already have a http://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/DevSt... job we can just pass parameters to that to invoke the mirroring steps. Would be even better if we could run multiple calls in parallel as neon and wtp can take >2hr
> 1. matrix job. each config is a trio of reqname/version/URL which is passed to jbosstools-requirements job to perform the mirror
> * curl syntax to invoke the job
> 2. when job child completes, it updates the .target files in the workspace w/ the new URL that has been mirrored
> * sed
> 3. when all children are done, a second downstream job runs TP update & validation against new .target files
> * https://github.com/jbosstools/jbosstools-build-ci/blob/jbosstools-4.4.x/u...
> * when done if success:
> * ideally, generate a PR and attach link in email to jbosstools-dev@
> * if can't generate PR, then attach patch in email to nickboldt & mistria to apply in jbosstools-target-platforms, with email template attached so easier to send mail to list announcing new change
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (JBIDE-21394) Deploy docker: "Resource Name" should not error (but "*" required) if empty
by Fred Bricon (JIRA)
[ https://issues.jboss.org/browse/JBIDE-21394?page=com.atlassian.jira.plugi... ]
Fred Bricon updated JBIDE-21394:
--------------------------------
Story Points: 1
Sprint: devex #114 May 2017
> Deploy docker: "Resource Name" should not error (but "*" required) if empty
> ---------------------------------------------------------------------------
>
> Key: JBIDE-21394
> URL: https://issues.jboss.org/browse/JBIDE-21394
> Project: Tools (JBoss Tools)
> Issue Type: Enhancement
> Components: openshift
> Reporter: Andre Dietisheim
> Assignee: Viacheslav Kabanovich
> Priority: Minor
> Labels: deploy_docker_wizard, openshift_v3
> Fix For: 4.4.0.Alpha2
>
> Attachments: resource-name-empty.png
>
>
> how to reproduce:
> # EXEC: in explroer: open up context menu on an openshift 3 project and choose "Deploy Docker Image..."
> # ASSERT: Deploy Docker Image wizard pops up empty
> Result:
> The "Resource Name" field is decorated with an error marker, it should be decorated with a "*" which is used all across OpenShift when required values are missing/empty
> !resource-name-empty.png!
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (JBIDE-22177) Docker Image host is missing from Deployment Config when deploying from a private registry
by Jeff Cantrill (JIRA)
[ https://issues.jboss.org/browse/JBIDE-22177?page=com.atlassian.jira.plugi... ]
Jeff Cantrill commented on JBIDE-22177:
---------------------------------------
[~fbricon] I think '3' is out of the question for now since we dont know the ip of the registry and have no reliable way to push an image at the moment. i think for now, we just need to forgo creating an IS in this scenario
> Docker Image host is missing from Deployment Config when deploying from a private registry
> ------------------------------------------------------------------------------------------
>
> Key: JBIDE-22177
> URL: https://issues.jboss.org/browse/JBIDE-22177
> Project: Tools (JBoss Tools)
> Issue Type: Bug
> Components: openshift, upstream
> Affects Versions: 4.3.1.CR1
> Reporter: Fred Bricon
> Assignee: Jeff Cantrill
> Fix For: 4.4.0.Alpha2
>
>
> If you deploy a docker image from myregistry:5000/example/myimage, the deployment config is generated with only example/myimage, so can't find the image to deploy
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months
[JBoss JIRA] (JBIDE-22177) Docker Image host is missing from Deployment Config when deploying from a private registry
by Jeff Cantrill (JIRA)
[ https://issues.jboss.org/browse/JBIDE-22177?page=com.atlassian.jira.plugi... ]
Jeff Cantrill edited comment on JBIDE-22177 at 5/5/16 1:47 PM:
---------------------------------------------------------------
Notes from my conversation with bparees regarding what 'oc new-app NAME' generally does.
Note: openshift project matches docker username in a pullspec. User must have perms to this project
in order to use image.
‘New-app’ roughly does the following given its arg:
* Arg matches existing IS
** Ref existing IS
* Arg matches image in public registry
** Create imagestream ref to arg
* Arg matches only Image locally
** IS pointed to ref of image in project and need to push image to registry
which means for JBT, since we are starting (potentially) with a local image and that image may or may not be pulled from a public registry should:
# Check docker pull spec against project imagestreams and 'openshift' imagestreams and use that if there is a match
# Check docker pull spec and if it can be found, create an IS to it.
# Create an IS to the pull spec with the 'username' modified to equal project name. inform user they need to push the image (until we can tag and push) and how.
was (Author: jcantrill):
Notes from my conversation with bparees regarding what 'oc new-app NAME' generally does.
Note: openshift project matches docker username in a pullspec. User must have perms to this project
in order to use image.
‘New-app’ roughly does the following given its arg:
* Arg matches existing IS
** Ref existing IS
* Arg matches image in public registry
** Create imagestream ref to arg
* Arg matches only Image locally
** IS pointed to ref of image in project and need to push image to registry
which means for JBT, since we are starting (potentially) with a local image and that image may or may not be pulled from a public registry should:
# Check docker pull spec against project imagestreams and 'openshift' imagestreams and use that if there is a match
# Create an IS to the pull spec with the 'username' modified to equal project name. inform user they need to push the image (until we can tag and push) and how.
> Docker Image host is missing from Deployment Config when deploying from a private registry
> ------------------------------------------------------------------------------------------
>
> Key: JBIDE-22177
> URL: https://issues.jboss.org/browse/JBIDE-22177
> Project: Tools (JBoss Tools)
> Issue Type: Bug
> Components: openshift, upstream
> Affects Versions: 4.3.1.CR1
> Reporter: Fred Bricon
> Assignee: Jeff Cantrill
> Fix For: 4.4.0.Alpha2
>
>
> If you deploy a docker image from myregistry:5000/example/myimage, the deployment config is generated with only example/myimage, so can't find the image to deploy
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 11 months