[JBoss JIRA] (JBIDE-24938) server.it.weekly jenkins job runs out of disk space on Windows
by Martin Malina (JIRA)
[ https://issues.jboss.org/browse/JBIDE-24938?page=com.atlassian.jira.plugi... ]
Martin Malina resolved JBIDE-24938.
-----------------------------------
Fix Version/s: 4.5.1.AM2
Resolution: Done
Done in JBIDE-24763 by Nick. It's worth checking later that it actually works as expected.
> server.it.weekly jenkins job runs out of disk space on Windows
> --------------------------------------------------------------
>
> Key: JBIDE-24938
> URL: https://issues.jboss.org/browse/JBIDE-24938
> Project: Tools (JBoss Tools)
> Issue Type: Bug
> Components: server
> Affects Versions: 4.5.1.AM2
> Reporter: Martin Malina
> Assignee: Martin Malina
> Fix For: 4.5.1.AM2
>
>
> This job is failing on Windows:
> https://dev-platform-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/server.i...
> It's because it runs out of disk space.
> The windows instances have (I believe) 30GB of disk space. This is set up in our openstack instance by eng-ops:
> https://ci-rhos.centralci.eng.rdu2.redhat.com/dashboard/
> If we wanted more space, we would need to ask eng ops to change this. But it's probably the same for all other users, so I don't think they would change it easily. And in any case, it should be enough.
> The main thing is that we need to reduce the space the tests are using.
> They download all the runtimes via the itests pom:
> https://github.com/jbosstools/jbosstools-server/blob/master/as/itests/pom...
> The main problem is that this happens 4 times:
> 3.5G ./org.jboss.tools.as.management.itests/target/requirements
> 3.5G ./org.jboss.tools.as.itests/target/requirements
> 3.5G ./target/requirements
> 3.5G ./org.jboss.tools.as.ui.bot.itests/target/requirements
> Fortunately this doesn't mean that all the servers are really downloaded 4 times - they are downloaded once and stored in the local maven download cache. But then the zips are copied over for each of the modules and unzipped.
> So, there are several options:
> a) Get rid of the zips once they're unzipped - that would save half the space. Unfortunately the download plugin does not support this:
> https://github.com/maven-download-plugin/maven-download-plugin
> b) Delete the requirements dir for each module at the end of the maven module lifecycle. This might be doable, needs investigating.
> c) Change the overall setup so that only one download is needed for all the tests - i.e. the child modules don't inherit the download plugin call, but instead use the downloaded runtimes from the higher pom (itests)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 2 months
[JBoss JIRA] (JBIDE-24763) Some server itests are failing on Jenkins
by Martin Malina (JIRA)
[ https://issues.jboss.org/browse/JBIDE-24763?page=com.atlassian.jira.plugi... ]
Martin Malina commented on JBIDE-24763:
---------------------------------------
I was about to tackle the space problem [here|JBIDE-24938] but you were faster!
> Some server itests are failing on Jenkins
> -----------------------------------------
>
> Key: JBIDE-24763
> URL: https://issues.jboss.org/browse/JBIDE-24763
> Project: Tools (JBoss Tools)
> Issue Type: Bug
> Components: server
> Affects Versions: 4.5.0.AM2
> Reporter: Martin Malina
> Assignee: Rob Stryker
> Fix For: 4.5.1.AM2
>
>
> Here's the job:
> https://dev-platform-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Devstud...
> Currently we got around 30 failures:
> {code}
> All Failed Tests
> Test Name
> Duration
> Age
> org.jboss.tools.as.itests.server.publishing.SingleDeployableFolderTest.testSingleDeployableFolder[1] 5.3 sec 1
> org.jboss.tools.as.management.itests.AS7ManagerIntegrationTest.canGetServerState[7] 76 ms 1
> org.jboss.tools.as.ui.bot.itests.parametized.server.RuntimeDetectionDuplicatesTest.duplicateRuntimes default [0] 17 sec 1
> org.jboss.tools.as.itests.server.publishing.SingleDeployableFileTest.testSingleDeployableFullPublish[4] 1.7 sec 2
> org.jboss.tools.as.itests.server.publishing.SingleDeployableFolderTest.testSingleDeployableFolder[4] 5.3 sec 2
> org.jboss.tools.as.itests.server.publishing.SingleDeployableFileTest.testSingleDeployableFullPublish[1] 2.3 sec 3
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[0] 4.4 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[1] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[2] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[3] 2.2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[4] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[5] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[6] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[7] 2.3 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[8] 2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[9] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[10] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[11] 2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[12] 2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[13] 2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[14] 2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[15] 2.1 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[16] 2 sec 6
> org.jboss.tools.as.itests.ProjectRuntimeClasspathTest.testProjectRuntime[17] 2 sec 6
> org.jboss.tools.as.itests.server.publishing.SingleDeployableFileTest.testSingleDeployableFullPublish[0] 1.7 sec 6
> org.jboss.tools.as.ui.bot.itests.parametized.server.ServerRuntimesTest.operateDeploy default [0] 1 min 49 sec 6
> org.jboss.tools.as.ui.bot.itests.parametized.server.ServerRuntimesTest.operateDeploy default [1] 1 min 43 sec 6
> org.jboss.tools.wtp.runtimes.tomcat.itests.ServerUtilTest.testUniqueServerName 2.6 sec 6
> org.jboss.tools.wtp.runtimes.tomcat.itests.TomcatDetectionTest.testTomcatDetection 0.13 sec 6
> org.jboss.tools.as.itests.server.publishing.SingleDeployableFileTest.testSingleDeployableFullPublish[2] 1.7 sec 7
> {code}
> Nick already shared some details in JBIDE-22799
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 2 months
[JBoss JIRA] (JBIDE-24938) server.it.weekly jenkins job runs out of disk space on Windows
by Martin Malina (JIRA)
[ https://issues.jboss.org/browse/JBIDE-24938?page=com.atlassian.jira.plugi... ]
Martin Malina commented on JBIDE-24938:
---------------------------------------
OK, seems like Nick beat us to it in JBIDE-24763:
{quote}
This PR should do 3 things:
a) reduce on-disk footprint from 4 x 3.5G to 1 x 3.5G for the downloaded AS/WF/EAP runtimes, by reusing itests/target/requirements/ folder (rather than itests/target/requirements + 3 x itests/*/target/requirements/ folders)
b) prevent downloading the above runtimes when running the itests/pom.xml; instead downloads (or recycling from ~/.m2) will be done during the specific itests/*/pom.xml plugins, so we don't have to prefetch everything before starting the first plugin
c) clean the itests/target/requirements folder before fetching (or copying from local) runtimes, to ensure the folder is clean and ready for use before each test plugin run
https://github.com/jbosstools/jbosstools-server/pull/534
{quote}
> server.it.weekly jenkins job runs out of disk space on Windows
> --------------------------------------------------------------
>
> Key: JBIDE-24938
> URL: https://issues.jboss.org/browse/JBIDE-24938
> Project: Tools (JBoss Tools)
> Issue Type: Bug
> Components: server
> Affects Versions: 4.5.1.AM2
> Reporter: Martin Malina
> Assignee: Martin Malina
>
> This job is failing on Windows:
> https://dev-platform-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/server.i...
> It's because it runs out of disk space.
> The windows instances have (I believe) 30GB of disk space. This is set up in our openstack instance by eng-ops:
> https://ci-rhos.centralci.eng.rdu2.redhat.com/dashboard/
> If we wanted more space, we would need to ask eng ops to change this. But it's probably the same for all other users, so I don't think they would change it easily. And in any case, it should be enough.
> The main thing is that we need to reduce the space the tests are using.
> They download all the runtimes via the itests pom:
> https://github.com/jbosstools/jbosstools-server/blob/master/as/itests/pom...
> The main problem is that this happens 4 times:
> 3.5G ./org.jboss.tools.as.management.itests/target/requirements
> 3.5G ./org.jboss.tools.as.itests/target/requirements
> 3.5G ./target/requirements
> 3.5G ./org.jboss.tools.as.ui.bot.itests/target/requirements
> Fortunately this doesn't mean that all the servers are really downloaded 4 times - they are downloaded once and stored in the local maven download cache. But then the zips are copied over for each of the modules and unzipped.
> So, there are several options:
> a) Get rid of the zips once they're unzipped - that would save half the space. Unfortunately the download plugin does not support this:
> https://github.com/maven-download-plugin/maven-download-plugin
> b) Delete the requirements dir for each module at the end of the maven module lifecycle. This might be doable, needs investigating.
> c) Change the overall setup so that only one download is needed for all the tests - i.e. the child modules don't inherit the download plugin call, but instead use the downloaded runtimes from the higher pom (itests)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 2 months