[
https://issues.jboss.org/browse/JBIDE-20904?page=com.atlassian.jira.plugi...
]
Nick Boldt edited comment on JBIDE-20904 at 4/13/16 12:29 PM:
--------------------------------------------------------------
Agreed, but since Jenkins is slow to run (finding a slave on which to run sometimes
blocked by other jobs) & slow to config (https = painful), for now I'm just
running the script as hudson@dev01 to synthesize the effect of running in jenkins but with
more interactivity.
Once it's working for not just ci -> stage but also stage -> dev (will have to
fake this up with the master branch jobs, I suspect), I can wrap a jenkins job around it.
Here are the current staging steps:
0. run quick verification tests:
a. does o.j.t.foundation.core have the correct version in it? [Aside: this should be moved
to the Central CI job]
b. are all the branches correct?
c. are all the SHAs in github == SHAs built in Jenkins?
1. disable jobs (now scripted via curl)
2. fire discovery job w/ specific param (again, curl), then disable once the build has
started
3a. wget staging script; run with some params passed in (this is 5 + 6 steps for JBT and
for JBDS, but the 11 steps can be run in parallel processes)
3b. verify that all the new pages are created correctly (11+ URLs to review, some which
need additional time to refresh as
devstudio.redhat.com can be fickle about showing new
content) - verify content (JBIDE-22173)
4. update composite sites for JBT and JBDS [scripted w/ sed] - verify content
(JBIDE-22173)
5. merge in IS content to discovery sites (scripted but needs to know the latest IS site
URLs to use as source) - verify content (JBIDE-22173)
6. update ide-config.properties (manual for now but could eventually be scripted w/ sed?)
- verify content (JBIDE-22173)
7. re-enable jobs (scripted w/ curl)
Then we could fire the smoke tests automatically (p2director, install-grinder) and when
those were done, generate the 2 template-based email notifications that the staging was
done. Currently we run BYOE tests for JBT and JBDS.
I don't believe we have an automated test of the installer + installing all the EA
content from Central. - JBIDE-22172
Only thing I can't see a way to do is push the changes on
download.jboss.org and
devstudio.redhat.com into github. So that would need to be some sort of rsync fetch script
+ manual review and check-in.
Bonus there is that it could be done while the smoke test was underway, and should a bug
happen (eg., wrong URL for Central TP site) the fix could be done both in github and
pushed back to the server.
was (Author: nickboldt):
Agreed, but since Jenkins is slow to run (finding a slave on which to run sometimes
blocked by other jobs) & slow to config (https = painful), for now I'm just
running the script as hudson@dev01 to synthesize the effect of running in jenkins but with
more interactivity.
Once it's working for not just ci -> stage but also stage -> dev (will have to
fake this up with the master branch jobs, I suspect), I can wrap a jenkins job around it.
Here are the current staging steps:
0. run quick verification tests:
a. does o.j.t.foundation.core have the correct version in it? [Aside: this should be moved
to the Central CI job]
b. are all the branches correct?
c. are all the SHAs in github == SHAs built in Jenkins?
1. disable jobs (now scripted via curl)
2. fire discovery job w/ specific param (again, curl), then disable once the build has
started
3a. wget staging script; run with some params passed in (this is 5 + 6 steps for JBT and
for JBDS, but the 11 steps can be run in parallel processes)
3b. verify that all the new pages are created correctly (11+ URLs to review, some which
need additional time to refresh as
devstudio.redhat.com can be fickle about showing new
content)
4. update composite sites for JBT and JBDS [scripted w/ sed]
5. merge in IS content to discovery sites (scripted but needs to know the latest IS site
URLs to use as source)
6. update ide-config.properties (manual for now but could eventually be scripted w/ sed?)
7. re-enable jobs (scripted w/ curl)
Then we could fire the smoke tests automatically (p2director, install-grinder) and when
those were done, generate the 2 template-based email notifications that the staging was
done. Currently we run BYOE tests for JBT and JBDS.
I don't believe we have an automated test of the installer + installing all the EA
content from Central. - JBIDE-22172
Only thing I can't see a way to do is push the changes on
download.jboss.org and
devstudio.redhat.com into github. So that would need to be some sort of rsync script +
manual review and check-in.
Bonus there is that it could be done while the smoke test was underway, and should a bug
happen (eg., wrong URL for Central TP site) the fix could be done both in github and
pushed back to the server.
automate publishing latest CI build to staging, then from staging to
development (or stable)
--------------------------------------------------------------------------------------------
Key: JBIDE-20904
URL:
https://issues.jboss.org/browse/JBIDE-20904
Project: Tools (JBoss Tools)
Issue Type: Enhancement
Components: build
Affects Versions: 4.4.0.Alpha1
Reporter: Nick Boldt
Assignee: Nick Boldt
Fix For: 4.4.0.Alpha1
Suggestion:
rather than opening 10 bash terminals to perform the
copy-from-one-place-on-disk-to-local, copy-from-local-to-another-place steps required to
clone CI bits to Stage or from Stage to release, [~mickael_istria] and I discovered today
that we could use `wait` or `parallel` to orchestrate these steps via a bash script so
they run in parallel (as quickly as possible), but still return feedback when all parallel
steps are completed.
So, where today we run these steps sorta-by-hand (copy script into a console and wait
until it's done) [1], in future we could simply kick a job and wait for the job to
notify its completion.
[1]
https://github.com/jbdevstudio/jbdevstudio-devdoc/tree/master/release_gui...
Examples of using a series of commands in parallel w/ a wait at the end:
http://stackoverflow.com/questions/19543139/bash-script-processing-comman...
{code:title=spawns the 3 parallel steps, waits until #3 is done (2 seconds later) and
returns the PID of the last one + its return code}
echo "1" & echo 2 & (sleep 2;echo 3) & wait && echo $! $#
{code}
More discussion:
{quote}
[12:44:46 PM] Mickael Istria: I believe some parts would have to be turned into
functions
[12:54:41 PM] Mickael Istria: so, to hack the script, it could be just:
* add && after the 1st rsync in each loop
* add & after the last one
* put a wait after the last loop
* give the big piece of code to procede directly to bash
{quote}
After this job is done, releng would still have to "wire up" the new bits by
updating composite*.xml and index.html pages, but that's considerably easier to do
locally in a terminal, or even to script too. Rather than updating these files w/ sed, we
could generate them from a template.
And if we don't care about committing those changes back to github, we could even
push them to the dl.jb.o and ds.jb.c servers directly as part of the above job.
Scary, but much faster!
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)