[JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugi... ]
Nick Boldt edited comment on JBIDE-23017 at 8/17/16 2:39 PM:
-------------------------------------------------------------
When releasing JBT 4.4.1.AM3 (without the above PR in place - wrote it while I waited for the build to be pushed), here's how long it took to copy over the big pieces:
{code}coretests: 68m55
core: 72m24{code}
Also wrote this [1] but can't run it because dev01 (and most of the RHEL slaves too!) are rammed today with a backlog of EAP jobs [2].
[1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstools-push-to-d...
[2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/
was (Author: nickboldt):
When releasing JBT 4.4.1.AM3 (without the above PR in place - wrote it while I waited for the build to be pushed), here's how long it took to copy over the big pieces:
{code}coretests: 68m55
core: 72m24{code}
> use sshfs to speed up publishing process
> -----------------------------------------
>
> Key: JBIDE-23017
> URL: https://issues.jboss.org/browse/JBIDE-23017
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: build
> Affects Versions: 4.4.1.AM3
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.1.Final
>
>
> Currently, publishing from /snapshots to /staging and from /staging to /development requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then push those bits to a new folder on filemgmt.jboss.org.
> If we could simply copy bits over sshfs, the process might be a lot faster than these times:
> {code}
> JBT coretests 1.1G: 59m20
> JBT core 1.2G: 64m23
> devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
> devstudio (copy from local to www.qa) 3.1G: 6mins(?)
> {code}
> To make this work, however, we need more slaves to have sshfs installed. Currently, it appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
[JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugi... ]
Nick Boldt commented on JBIDE-23017:
------------------------------------
When releasing JBT 4.4.1.AM3 (without the above PR in place - wrote it while I waited for the build to be pushed), here's how long it took to copy over the big pieces:
{code}coretests: 68m55
core: 72m24{code}
> use sshfs to speed up publishing process
> -----------------------------------------
>
> Key: JBIDE-23017
> URL: https://issues.jboss.org/browse/JBIDE-23017
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: build
> Affects Versions: 4.4.1.AM3
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.1.Final
>
>
> Currently, publishing from /snapshots to /staging and from /staging to /development requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then push those bits to a new folder on filemgmt.jboss.org.
> If we could simply copy bits over sshfs, the process might be a lot faster than these times:
> {code}
> JBT coretests 1.1G: 59m20
> JBT core 1.2G: 64m23
> devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
> devstudio (copy from local to www.qa) 3.1G: 6mins(?)
> {code}
> To make this work, however, we need more slaves to have sshfs installed. Currently, it appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
[JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugi... ]
Nick Boldt reassigned JBIDE-23017:
----------------------------------
Story Points: 3
Sprint: devex #119 August 2016
Assignee: Nick Boldt
> use sshfs to speed up publishing process
> -----------------------------------------
>
> Key: JBIDE-23017
> URL: https://issues.jboss.org/browse/JBIDE-23017
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: build
> Affects Versions: 4.4.1.AM3
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.1.Final
>
>
> Currently, publishing from /snapshots to /staging and from /staging to /development requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then push those bits to a new folder on filemgmt.jboss.org.
> If we could simply copy bits over sshfs, the process might be a lot faster than these times:
> {code}
> JBT coretests 1.1G: 59m20
> JBT core 1.2G: 64m23
> devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
> devstudio (copy from local to www.qa) 3.1G: 6mins(?)
> {code}
> To make this work, however, we need more slaves to have sshfs installed. Currently, it appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
[JBoss JIRA] (JBIDE-23017) use sshfs to speed up publishing process
by Nick Boldt (JIRA)
[ https://issues.jboss.org/browse/JBIDE-23017?page=com.atlassian.jira.plugi... ]
Nick Boldt updated JBIDE-23017:
-------------------------------
Fix Version/s: 4.4.1.Final
> use sshfs to speed up publishing process
> -----------------------------------------
>
> Key: JBIDE-23017
> URL: https://issues.jboss.org/browse/JBIDE-23017
> Project: Tools (JBoss Tools)
> Issue Type: Feature Request
> Components: build
> Affects Versions: 4.4.1.AM3
> Reporter: Nick Boldt
> Assignee: Nick Boldt
> Fix For: 4.4.1.Final
>
>
> Currently, publishing from /snapshots to /staging and from /staging to /development requires that we fetch several GBs of data from filemgmt.jboss.org to a temp folder, then push those bits to a new folder on filemgmt.jboss.org.
> If we could simply copy bits over sshfs, the process might be a lot faster than these times:
> {code}
> JBT coretests 1.1G: 59m20
> JBT core 1.2G: 64m23
> devstudio (copy from filemgmt to filemgmt) 2.4G: 2h13
> devstudio (copy from local to www.qa) 3.1G: 6mins(?)
> {code}
> To make this work, however, we need more slaves to have sshfs installed. Currently, it appears that only dev01 [1] is so equipped. And that slave, currently, has a build queue more than 170 jobs long, making it impractical to even TEST publishing over sshfs.
> [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/computer/dev01-rhel5-x86/
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
[JBoss JIRA] (JBDS-3862) Platform installer failed to detect existing Cygwin install with ssh and sftp
by Denis Golovin (JIRA)
[ https://issues.jboss.org/browse/JBDS-3862?page=com.atlassian.jira.plugin.... ]
Denis Golovin updated JBDS-3862:
--------------------------------
Comment: was deleted
(was: Forum post related to cygwin detection)
> Platform installer failed to detect existing Cygwin install with ssh and sftp
> -----------------------------------------------------------------------------
>
> Key: JBDS-3862
> URL: https://issues.jboss.org/browse/JBDS-3862
> Project: Red Hat JBoss Developer Studio (devstudio)
> Issue Type: Feature Request
> Components: platform-installer
> Affects Versions: 9.1.0.Beta1
> Environment: Windows 7/64 bit,
> Cygwin 2.5.1 already installed in the detault location C:\cygwin.
> Reporter: Robert Terzi
> Assignee: Denis Golovin
> Priority: Minor
> Fix For: 10.1.0.GA
>
> Attachments: screenshot-1.png
>
>
> The installer did not detect the existing Cygwin install in C:\Cygwin.
> What test is used to determine if Cygwin is already installed?
> !screenshot-1.png!
> Installer uses {code}cygcheck -c cygwin openssh rsync{code} to veify cygwin installed and ssh, sftp and rsync are installed.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months
[JBoss JIRA] (JBDS-3862) Platform installer failed to detect existing Cygwin install with ssh and sftp
by Denis Golovin (JIRA)
[ https://issues.jboss.org/browse/JBDS-3862?page=com.atlassian.jira.plugin.... ]
Denis Golovin updated JBDS-3862:
--------------------------------
Fix Version/s: 10.1.0.GA
> Platform installer failed to detect existing Cygwin install with ssh and sftp
> -----------------------------------------------------------------------------
>
> Key: JBDS-3862
> URL: https://issues.jboss.org/browse/JBDS-3862
> Project: Red Hat JBoss Developer Studio (devstudio)
> Issue Type: Feature Request
> Components: platform-installer
> Affects Versions: 9.1.0.Beta1
> Environment: Windows 7/64 bit,
> Cygwin 2.5.1 already installed in the detault location C:\cygwin.
> Reporter: Robert Terzi
> Assignee: Denis Golovin
> Priority: Minor
> Fix For: 10.1.0.GA
>
> Attachments: screenshot-1.png
>
>
> The installer did not detect the existing Cygwin install in C:\Cygwin.
> What test is used to determine if Cygwin is already installed?
> !screenshot-1.png!
> Installer uses {code}cygcheck -c cygwin openssh rsync{code} to veify cygwin installed and ssh, sftp and rsync are installed.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 7 months