[jbosstools-issues] [JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process

[Nick Boldt (JIRA)]

    [ https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13307579#comment-13307579 ] 

Nick Boldt commented on JBIDE-23017:
------------------------------------

More perf tuning and I've got the CI -> staging job down to:

{code}
devstudio.product_master INT - 13m
devstudio.product_master EXT - 35m (24 if copy from www.qa)
core + coretests 41m (or 26 min each)
bsim 30s
ds central+ea 3m45
jbt central+ea 3m18
ds tp zips 2m30
{code}

So the overall time to do a push is about 30-45min.  Can't get it much faster, as when I moved to using a local upload from www.qa -> ds.rh.c, I started getting ssh timeouts & drops again.

--

Only thing I can still try is to implement sshfs for copying files, either still using the tmpdir intermediate, or skipping that entirely.

> use sshfs to speed up publishing process 
> -----------------------------------------
>
>                 Key: JBIDE-23017
>                 URL: https://issues.jboss.org/browse/JBIDE-23017
>             Project: Tools (JBoss Tools)
>          Issue Type: Feature Request
>          Components: build
>    Affects Versions: 4.4.1.AM3
>            Reporter: Nick Boldt
>            Assignee: Nick Boldt
>             Fix For: 4.4.2.AM2
>
>
> Currently, publishing from /snapshots to /staging and from /staging to /development requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then push those bits to a new folder on filemgmt.jboss.org.
> If we could simply copy bits over sshfs, the process might be a lot faster than these times:
> {code}
> JBT coretests 1.1G: 59m20
> JBT core 1.2G: 64m23
> devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
> devstudio (copy from local to www.qa) 3.1G: 6mins(?)
> {code}
> To make this work, however, we need more slaves to have sshfs installed. Currently, it appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)