[jboss-jira] [JBoss JIRA] (WFLY-9325) Reduce on-disk size and copying in the test suite

[Tomaz Cerar (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492156#comment-13492156 ] 

Tomaz Cerar commented on WFLY-9325:
-----------------------------------

I've identified few areas where we can improve the IO / CPU cycles in testsuite
- reduce copying server around (biggest offender manual TS) partially done by "smarter" copying of resources, partially blocked by WFCORE-3409 
- copy only config files we need for current testsuite
- run XLTS transformations only when absolutely needed and only for files that we need (not catch all as we do now)
- use thin distro instead of "fat" one
- merge testsuites, web could easily be merged into basic.

> Reduce on-disk size and copying in the test suite
> -------------------------------------------------
>
>                 Key: WFLY-9325
>                 URL: https://issues.jboss.org/browse/WFLY-9325
>             Project: WildFly
>          Issue Type: Enhancement
>          Components: Build System, Test Suite
>            Reporter: David Lloyd
>            Assignee: Tomaz Cerar
>
> The test suite takes up large amounts of disk space (over 2.7GB) during build.  This causes overhead for copying and bogs down test runs and CI.  Find ways to mitigate this.
> First approach is to use Maven refs instead of fixed JARs in the build process:
> {code:diff}
> diff --git a/pom.xml b/pom.xml
> index fb2b68c..5805471 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -252,8 +252,8 @@
>          <linkXRef>false</linkXRef>
>  
>          <server.output.dir.prefix>wildfly</server.output.dir.prefix>
> -        <wildfly.build.output.dir>dist/target/${server.output.dir.prefix}-${jboss.as.release.version}</wildfly.build.output.dir>
> -        <wildfly.web.build.output.dir>servlet-dist/target/${server.output.dir.prefix}-servlet-${jboss.as.release.version}</wildfly.web.build.output.dir>
> +        <wildfly.build.output.dir>build/target/${server.output.dir.prefix}-${jboss.as.release.version}</wildfly.build.output.dir>
> +        <wildfly.web.build.output.dir>servlet-build/target/${server.output.dir.prefix}-servlet-${jboss.as.release.version}</wildfly.web.build.output.dir>
>  
>          <!--
>              See ChildFirstClassLoaderBuilder in model-test for the explanation of the org.jboss.model.test.cache.root and org.jboss.model.test.classpath.cache properties.
> {code}
> This saves a bit of space but copied images are still taking upwards of 55MB each, and there are very many copies; consider this example from {{testsuite/integration/iiop/target}}:
> {noformat}
> 30M	jbossas
> 55M	jbossas-iiop-client
> 55M	jbossas-iiop-server
> 55M	jbossas-iiop-ssl-client
> 55M	jbossas-iiop-ssl-legacy-server
> 55M	jbossas-iiop-ssl-server
> {noformat}
> Within a typical copy (after the above patch), about half of the data is in {{bin}} and half is in {{standalone}}.  In {{bin}}, the dominating files are:
> *  {{2.2M wildfly-elytron-tool.jar}}
> *  {{7.2M client/jboss-cli-client.jar}}
> *  {{17M  client/jboss-client.jar}}
> In {{standalone}}, the dominator is:
> * {{25M data/activemq}} (mostly journal files but also the {{bindings}} file)
> Together these four areas cover 51.4MB of the 55MB in each IIOP server copy.
> In other areas, big users include {{standalone_xml_history}}, which can be as large as 67MB.  Disabling the across the board could help.
> We're also making over 50 copies of the {{docs}} directory at 2.3M per instance.



--
This message was sent by Atlassian JIRA
(v7.5.0#75005)