[jbosstools-issues] [JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process
[
https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugi...
]
Nick Boldt commented on JBIDE-23017:
------------------------------------
More perf tuning and I've got the CI -> staging job down to:
{code}
devstudio.product_master INT - 13m
devstudio.product_master EXT - 35m (24 if copy from www.qa)
core + coretests 41m (or 26 min each)
bsim 30s
ds central+ea 3m45
jbt central+ea 3m18
ds tp zips 2m30
{code}
So the overall time to do a push is about 30-45min. Can't get it much faster, as when
I moved to using a local upload from www.qa -> ds.rh.c, I started getting ssh timeouts
& drops again.
--
Only thing I can still try is to implement sshfs for copying files, either still using the
tmpdir intermediate, or skipping that entirely.
use sshfs to speed up publishing process
-----------------------------------------
Key: JBIDE-23017
URL: https://issues.jboss.org/browse/JBIDE-23017
Project: Tools (JBoss Tools)
Issue Type: Feature Request
Components: build
Affects Versions: 4.4.1.AM3
Reporter: Nick Boldt
Assignee: Nick Boldt
Fix For: 4.4.2.AM2
Currently, publishing from /snapshots to /staging and from /staging to /development
requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then
push those bits to a new folder on filemgmt.jboss.org.
If we could simply copy bits over sshfs, the process might be a lot faster than these
times:
{code}
JBT coretests 1.1G: 59m20
JBT core 1.2G: 64m23
devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
devstudio (copy from local to www.qa) 3.1G: 6mins(?)
{code}
To make this work, however, we need more slaves to have sshfs installed. Currently, it
appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue
more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
[1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)