[jboss-as7-dev] Testsuite execution

[Brian Stansberry]

That's fine by me. I don't care all that much about compiling the 
integration tests. I didn't want -P allTests because that turns off 
default profiles, but using -DallTests to activate a profile shouldn't 
have that problem.

On 5/18/11 10:13 AM, Thomas Diesler wrote:
> Using skipTests as a flag has a negative side effect such that you
> cannot do
>
>  > mvn -Dtest=SomeTestCase test
>
> in an integration module any more. Instead you have to do
>
>  > mvn -DskipTests=false -Dtest=SomeTestCase test
>
> This is non-intuitive and non-standard. Instead I propose to use an
> alternate switch like 'allTests' to turn on profiles which run the
> complete set of tests. IMHO its ok to exclude entire (testsuite) modules
> from the build (i.e. not to compile the tests) because compilation
> errors show up in the IDE and Hudson anyway.
>
> I propose
>
> 1) By default, all normal modules and testsuite/smoke will be built in
> the standard way
> (i.e. failing tests will fail the build)
>
> 2) By adding -DskipTests=true tests that would normally be executed are
> skipped
>
> 3) By adding -DallTests all modules that contain tests will be executed.
> In Hudson you could add -fae
> which fails the build at the end - so catch all failing tests in a
> single run
>
> It boils down to whether you want to compile tests that you don't want
> to execute.
> IMHO that's not necessary in day to day work - Hudson can catch this.
>
> cheers
> -thomas
>
>
>
> On 05/15/2011 04:16 AM, Brian Stansberry wrote:
>> I'd like to get feedback on changes to the testsuite found in
>> https://github.com/bstansberry/jboss-as/commit/efe2051753f6b79788d05156784665e02672613c
>>
>>
>> I'll be adding a bunch of integration tests that launch a domain, but
>> before pushing those I wanted to ensure they weren't run every time
>> someone did a build. That led to a general effort to simplify how we can
>> execute the 3 main build cases -- a standard build with some smoke
>> tests, a no-test fast build, and a full test run. The above commit
>> results in the following behavior:
>>
>> 1) By default, all normal modules and testsuite modules will be built
>> (so test compilation issues introduced by code changes will be caught)
>> but the only tests that will be executed are:
>>
>> a) the tests in non-testsuite modules
>> b) the tests in testsuite/smoke
>>
>> 2) By adding -DskipTests=true, no tests will be executed (although
>> they'll still compile).
>>
>> 3) By adding -DskipTests=false the integration tests in testsuite2,
>> testsuite/integration and the webservices module will be enabled. So
>> that results in a full testsuite run (except for the currently empty
>> benchmark and stress test modules).
>>
>> The testsuite2, testsuite/integration and the webservices module also
>> have testFailureIgnore set to true, which means failures in one of the
>> many modules will not fail the build, allowing the full set of tests to
>> run. A test failure in a non-testsuite module or in testsuite/smoke will
>> fail the build.
>>
>


-- 
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat