[jboss-jira] [JBoss JIRA] (JGRP-1728) Allow testsuite parallelism to be controlled from build.xml alone

[Richard Achmatowicz (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851493#comment-12851493 ] 

Richard Achmatowicz edited comment on JGRP-1728 at 11/6/13 1:21 PM:
--------------------------------------------------------------------

So, because JGroups uses suite files to control the testsuite, there is no easy way to turn parallelism on and off. 

There was a mechanism in place which used the system properties listed in the first comment and passed those through to the runtest macro, but these had no effect on the suite definition files. We must somehow modify these files for parallel and non-parallel test execution modes. 

One way would be to have a set of suite files for non-parallel mode, and another set of suite files for parallel mode. We could then use the parallel.tests and parallel.methods build properties to control which suite definition file (parallel or non-parallel) was picked up for test execution. But if you wanted to change say the thread-count of the functional tests, you would have to go into the file and change it.

 Another way is to take the suite files in conf/testng and pre-process them before use, based on the parallel= and thread-count= attributes passed to runtest. This can be acheived by using a small XSLT transform which runs in a step just before the testNG task is invoked and configures the suite file on the fly. 

I've set up a PR and have this mechanism working. No extra jars or other mechanisms are required.

When the runtest macro gets called, it takes the parallel= and thread-count= parameters passed to runtest and uses an XSLT transform called configParallel.xsl to change the corresponding attributes in the suite file. The transformed suite file is then stored in the temp.dir where the test results are also stored in named directories. This modified suite file is then used to control the test and will be cleaned up when the clean ant task is run.

Here is the xslt file, in conf/testng/parallelConfig.xsl:
{noformat}
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output omit-xml-declaration="yes" indent="yes"/>
    
    <xsl:param name="parallel" select="'methods'"/>
    <xsl:param name="thread-count" select="'1'"/>

    <!-- copy everything -->
    <xsl:template match="@*|node()">
      <xsl:copy>
	<xsl:apply-templates select="@*|node()"/>
      </xsl:copy>
    </xsl:template>

    <!-- change value of parallel attribute -->
    <xsl:template match="@parallel[parent::suite]">
      <xsl:attribute name="parallel">
	<xsl:value-of select="$parallel"/>
      </xsl:attribute>
    </xsl:template>

    <!-- change value of thread-count attribute -->
    <xsl:template match="@thread-count[parent::suite]">
      <xsl:attribute name="thread-count">
	<xsl:value-of select="$thread-count"/>
      </xsl:attribute>
    </xsl:template>
</xsl:stylesheet>
{noformat}

and here is the invocation of the xslt task from within runtest:
{noformat}
  <!-- use XSLT to add parallelism directives to the suite definition files -->
  <xslt in="${testng.conf.dir}/@{testng.xmlfile}" out="${tmp.dir}/@{testng.xmlfile}"
        extension=".xml" style="${testng.conf.dir}/parallelConfig.xsl">
    <param name="parallel" expression="@{parallel}"/>
    <param name="thread-count" expression="@{threadcount}"/>
  </xslt>
{noformat}


                
      was (Author: rachmato):
    So, because JGroups uses suite files to control the testsuite, there is no easy way to turn parallelism on and off. 

There was a mechanism in place which used the system properties listed in the first comment and passed those through to the runtest macro, but these had no effect on the suite definition files. We must somehow modify these files for parallel and non-parallel test execution modes. 

One way would be to have a set of suite files for non-parallel mode, and another set of suite files for parallel mode. We could then use the parallel.tests and parallel.methods build properties to control which suite definition file (parallel or non-parallel) was picked up for test execution. But if you wanted to change say the thread-count of the functional tests, you would have to go into the file and change it.

 Another way is to take the suite files in conf/testng and pre-process them before use, based on the parallel= and thread-count= attributes passed to runtest. This can be acheived by using a small XSLT transform which runs in a step just before the testNG task is invoked and configures the suite file on the fly. 

When the runtest macro gets called, it takes the parallel= and thread-count= parameters passed to runtest and uses an XSLT transform called configParallel.xsl to change the corresponding attributes in the suite file. The transformed suite file is then stored in the temp.dir where the test results are also stored in named directories. This modified suite file is then used to control the test and will be cleaned up when the clean ant task is run.

Here is the xslt file, in conf/testng/parallelConfig.xsl:
{noformat}
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output omit-xml-declaration="yes" indent="yes"/>
    
    <xsl:param name="parallel" select="'methods'"/>
    <xsl:param name="thread-count" select="'1'"/>

    <!-- copy everything -->
    <xsl:template match="@*|node()">
      <xsl:copy>
	<xsl:apply-templates select="@*|node()"/>
      </xsl:copy>
    </xsl:template>

    <!-- change value of parallel attribute -->
    <xsl:template match="@parallel[parent::suite]">
      <xsl:attribute name="parallel">
	<xsl:value-of select="$parallel"/>
      </xsl:attribute>
    </xsl:template>

    <!-- change value of thread-count attribute -->
    <xsl:template match="@thread-count[parent::suite]">
      <xsl:attribute name="thread-count">
	<xsl:value-of select="$thread-count"/>
      </xsl:attribute>
    </xsl:template>
</xsl:stylesheet>
{noformat}

and here is the invocation of the xslt task from within runtest:
{noformat}
  <!-- use XSLT to add parallelism directives to the suite definition files -->
  <xslt in="${testng.conf.dir}/@{testng.xmlfile}" out="${tmp.dir}/@{testng.xmlfile}"
        extension=".xml" style="${testng.conf.dir}/parallelConfig.xsl">
    <param name="parallel" expression="@{parallel}"/>
    <param name="thread-count" expression="@{threadcount}"/>
  </xslt>
{noformat}

I've set up a PR and have this mechanism working. No extra jars or other mechanisms are required.

                  
> Allow testsuite parallelism to be controlled from build.xml alone
> -----------------------------------------------------------------
>
>                 Key: JGRP-1728
>                 URL: https://issues.jboss.org/browse/JGRP-1728
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 3.5
>            Reporter: Richard Achmatowicz
>            Assignee: Bela Ban
>            Priority: Minor
>             Fix For: 3.5
>
>
> Directives controlling parallel execution of the JGroups testsuite can be specified at two levels: as parameters to the testng ant task (in build.xml), or as attributes of the suite element in the suite definition files (found in the conf/testng directory).
> At present, parallelism directives in the various suite definition files (functional.xml, udp.xml, tcp.xml, etc.) take precedence over any directives specified in testng ant tasks. In fact, the directives passed to runtest in build.xml are (incorrectly) not passed through to the testng ant task, so that parallelism is controlled solely from the suite definition files.
> Making use of parallel execution of test cases when running in the QA lab leads to a higher proportion of test failures, leading to a proliferation of JIRA/bugzilla issues. Therefore it is important that we be able to turn on (turn off) parallel execution of test cases through some simple specification of system properties.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira