[jboss-jira] [JBoss JIRA] (WFLY-9325) Reduce on-disk size and copying in the test suite

[Brian Stansberry (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461017#comment-13461017 ] 

Brian Stansberry commented on WFLY-9325:
----------------------------------------

The domain testsuite uses our standard feature of allowing a server to declare its jboss.server.base.dir / jboss.server.domain.dir and thus only creates those directories for custom installations, relying on a single install in testsuite/domain for everything else (with, IIRC, that single install not copying in the modules). Perhaps WFARQ could make that easy to do elsewhere. Testing a custom franken-install is ok if necessary but trying instead to utilize the footprint-slimming features we provide for our users is a form of eating our own dog food.

The CI cleanup switch I mentioned 
 is at https://github.com/wildfly/wildfly/commit/60c6a83eeaad726cb27e6f1b48c6f8091476716e and was meant to help CI (or us if we turn it on locally) to clean out no longer needed data at the end of each testsuite module. If we are still showing GBs of data accumulating in these AS copies over the course of a CI run, then something isn't working right with that.

> Reduce on-disk size and copying in the test suite
> -------------------------------------------------
>
>                 Key: WFLY-9325
>                 URL: https://issues.jboss.org/browse/WFLY-9325
>             Project: WildFly
>          Issue Type: Enhancement
>          Components: Build System, Test Suite
>            Reporter: David Lloyd
>            Assignee: Tomaz Cerar
>
> The test suite takes up large amounts of disk space (over 2.7GB) during build.  This causes overhead for copying and bogs down test runs and CI.  Find ways to mitigate this.
> First approach is to use Maven refs instead of fixed JARs in the build process:
> {code:diff}
> diff --git a/pom.xml b/pom.xml
> index fb2b68c..5805471 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -252,8 +252,8 @@
>          <linkXRef>false</linkXRef>
>  
>          <server.output.dir.prefix>wildfly</server.output.dir.prefix>
> -        <wildfly.build.output.dir>dist/target/${server.output.dir.prefix}-${jboss.as.release.version}</wildfly.build.output.dir>
> -        <wildfly.web.build.output.dir>servlet-dist/target/${server.output.dir.prefix}-servlet-${jboss.as.release.version}</wildfly.web.build.output.dir>
> +        <wildfly.build.output.dir>build/target/${server.output.dir.prefix}-${jboss.as.release.version}</wildfly.build.output.dir>
> +        <wildfly.web.build.output.dir>servlet-build/target/${server.output.dir.prefix}-servlet-${jboss.as.release.version}</wildfly.web.build.output.dir>
>  
>          <!--
>              See ChildFirstClassLoaderBuilder in model-test for the explanation of the org.jboss.model.test.cache.root and org.jboss.model.test.classpath.cache properties.
> {code}
> This saves a bit of space but copied images are still taking upwards of 55MB each, and there are very many copies; consider this example from {{testsuite/integration/iiop/target}}:
> {noformat}
> 30M	jbossas
> 55M	jbossas-iiop-client
> 55M	jbossas-iiop-server
> 55M	jbossas-iiop-ssl-client
> 55M	jbossas-iiop-ssl-legacy-server
> 55M	jbossas-iiop-ssl-server
> {noformat}
> Within a typical copy (after the above patch), about half of the data is in {{bin}} and half is in {{standalone}}.  In {{bin}}, the dominating files are:
> *  {{2.2M wildfly-elytron-tool.jar}}
> *  {{7.2M client/jboss-cli-client.jar}}
> *  {{17M  client/jboss-client.jar}}
> In {{standalone}}, the dominator is:
> * {{25M data/activemq}} (mostly journal files but also the {{bindings}} file)
> Together these four areas cover 51.4MB of the 55MB in each IIOP server copy.
> In other areas, big users include {{standalone_xml_history}}, which can be as large as 67MB.  Disabling the across the board could help.
> We're also making over 50 copies of the {{docs}} directory at 2.3M per instance.



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)