[jbosstools-issues] [JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process

[Nick Boldt (JIRA)]

    [ https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13306392#comment-13306392 ] 

Nick Boldt edited comment on JBIDE-23017 at 10/13/16 2:02 AM:
--------------------------------------------------------------

Having fixed the jobs [1], staging.sh & checkStagingURLs.sh scripts [2], we can now copy from CI to staging in about 45 mins.

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/jbosstools-releng/job/jbosstools-releng-push-to-staging-02-copy-builds-and-update-sites_master/buildTimeTrend ~46 mins for build #22
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/jbosstools-releng/job/jbosstools-releng-push-to-staging-03-verify-builds-update-sites_master/ verification works, takes about 30 seconds

[1] https://github.com/jbdevstudio/jbdevstudio-ci/commits/master
[2] https://github.com/jbosstools/jbosstools-build-ci/commits/jbosstools-4.4.x



was (Author: nickboldt):
Having fixed the jobs [1], staging.sh & verification scripts [2], we can now copy from CI to staging in about 45 mins.

https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/jbosstools-releng/job/jbosstools-releng-push-to-staging-02-copy-builds-and-update-sites_master/buildTimeTrend ~46 mins for build #22
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/DevStudio/view/jbosstools-releng/job/jbosstools-releng-push-to-staging-03-verify-builds-update-sites_master/ verification works, takes about 30 seconds

[1] https://github.com/jbdevstudio/jbdevstudio-ci/commits/master
[2] https://github.com/jbosstools/jbosstools-build-ci/commits/jbosstools-4.4.x


> use sshfs to speed up publishing process 
> -----------------------------------------
>
>                 Key: JBIDE-23017
>                 URL: https://issues.jboss.org/browse/JBIDE-23017
>             Project: Tools (JBoss Tools)
>          Issue Type: Feature Request
>          Components: build
>    Affects Versions: 4.4.1.AM3
>            Reporter: Nick Boldt
>            Assignee: Nick Boldt
>             Fix For: 4.4.2.AM2
>
>
> Currently, publishing from /snapshots to /staging and from /staging to /development requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then push those bits to a new folder on filemgmt.jboss.org.
> If we could simply copy bits over sshfs, the process might be a lot faster than these times:
> {code}
> JBT coretests 1.1G: 59m20
> JBT core 1.2G: 64m23
> devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
> devstudio (copy from local to www.qa) 3.1G: 6mins(?)
> {code}
> To make this work, however, we need more slaves to have sshfs installed. Currently, it appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)