[jboss-jira] [JBoss JIRA] (JGRP-1728) Allow testsuite parallelism to be controlled from build.xml alone

[Richard Achmatowicz (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851491#comment-12851491 ] 

Richard Achmatowicz edited comment on JGRP-1728 at 11/6/13 1:19 PM:
--------------------------------------------------------------------

The first implementation of this change was incorrect. The problem lies with the way in which paralleism is configured in testNG, which I summarise here:
- you can specify attributes parallel= and thread-count= on the <testng/> ant task in order to control parallelism of test classes specified within the testng task itself ;
- you can specify attributes parallel= and thread-count= on the <suite/> elements of the XML suite definition files to control parallelism of tests defined in the suite
- the parallelism directives of the testng ant task, and the paralleism directives of the suite elements are completely independent

What this means is that if you define your tests via suite definition files (as te JGroups testsuite does), then in order to control parallelism of test execution, you have to modify the parallelism directives in the suite files themselves 
 
This situation is made worse by the fact that you cannot use variable substitution in the testNGs suite files (i.e. no chance for defining a system property and then referencing that system property in the suite file, as in 
{noformat}
<suite parallel="${parallel.methods}"/>)
{noformat}
So in order to configure parallelism, on the fly, we need to be able to modify or otherwise configure the suite files which are passed to the test cases.
                
      was (Author: rachmato):
    The first implementation of this change was incorrect. The problem lies with the way in which paralleism is configured in testNG, which I summarise here:
- you can specify attributes parallel= and thread-count= on the <testng/> ant task in order to control parallelism of test classes specified within the testng task itself ;
- you can specify attributes parallel= and thread-count= on the <suite/> elements of the XML suite definition files to control parallelism of tests defined in the suite
- the parallelism directives of the testng ant task, and the paralleism directives of the suite elements are completely independent
What this means is that if you define your tests via suite definition files (as te JGroups testsuite does), then in order to control parallelism of test execution, you have to modify the parallelism directives in the suite files themselves 
 
This situation is made worse by the fact that you cannot use variable substitution in the testNGs suite files (i.e. no chance for defining a system property and then referencing that system property in the suite file, as in 
{noformat}
<suite parallel="${parallel.methods}"/>)
{noformat}
So in order to configure parallelism, on the fly, we need to be able to modify or otherwise configure the suite files which are passed to the test cases.
                  
> Allow testsuite parallelism to be controlled from build.xml alone
> -----------------------------------------------------------------
>
>                 Key: JGRP-1728
>                 URL: https://issues.jboss.org/browse/JGRP-1728
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 3.5
>            Reporter: Richard Achmatowicz
>            Assignee: Bela Ban
>            Priority: Minor
>             Fix For: 3.5
>
>
> Directives controlling parallel execution of the JGroups testsuite can be specified at two levels: as parameters to the testng ant task (in build.xml), or as attributes of the suite element in the suite definition files (found in the conf/testng directory).
> At present, parallelism directives in the various suite definition files (functional.xml, udp.xml, tcp.xml, etc.) take precedence over any directives specified in testng ant tasks. In fact, the directives passed to runtest in build.xml are (incorrectly) not passed through to the testng ant task, so that parallelism is controlled solely from the suite definition files.
> Making use of parallel execution of test cases when running in the QA lab leads to a higher proportion of test failures, leading to a proliferation of JIRA/bugzilla issues. Therefore it is important that we be able to turn on (turn off) parallel execution of test cases through some simple specification of system properties.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira