[jboss-as7-dev] Testsuite execution

[Jaikiran Pai]

By the way, are we still adding tests to the (outdated?) 
testsuite/integration module? I thought we were going to start using 
testsuite2/spec and testsuite2/internals going forward. Where should any 
new tests go?

-Jaikiran
On Wednesday 18 May 2011 09:35 PM, Brian Stansberry wrote:
> That's fine by me. I don't care all that much about compiling the
> integration tests. I didn't want -P allTests because that turns off
> default profiles, but using -DallTests to activate a profile shouldn't
> have that problem.
>
> On 5/18/11 10:13 AM, Thomas Diesler wrote:
>> Using skipTests as a flag has a negative side effect such that you
>> cannot do
>>
>>   >  mvn -Dtest=SomeTestCase test
>>
>> in an integration module any more. Instead you have to do
>>
>>   >  mvn -DskipTests=false -Dtest=SomeTestCase test
>>
>> This is non-intuitive and non-standard. Instead I propose to use an
>> alternate switch like 'allTests' to turn on profiles which run the
>> complete set of tests. IMHO its ok to exclude entire (testsuite) modules
>> from the build (i.e. not to compile the tests) because compilation
>> errors show up in the IDE and Hudson anyway.
>>
>> I propose
>>
>> 1) By default, all normal modules and testsuite/smoke will be built in
>> the standard way
>> (i.e. failing tests will fail the build)
>>
>> 2) By adding -DskipTests=true tests that would normally be executed are
>> skipped
>>
>> 3) By adding -DallTests all modules that contain tests will be executed.
>> In Hudson you could add -fae
>> which fails the build at the end - so catch all failing tests in a
>> single run
>>
>> It boils down to whether you want to compile tests that you don't want
>> to execute.
>> IMHO that's not necessary in day to day work - Hudson can catch this.
>>
>> cheers
>> -thomas
>>
>>
>>
>> On 05/15/2011 04:16 AM, Brian Stansberry wrote:
>>> I'd like to get feedback on changes to the testsuite found in
>>> https://github.com/bstansberry/jboss-as/commit/efe2051753f6b79788d05156784665e02672613c
>>>
>>>
>>> I'll be adding a bunch of integration tests that launch a domain, but
>>> before pushing those I wanted to ensure they weren't run every time
>>> someone did a build. That led to a general effort to simplify how we can
>>> execute the 3 main build cases -- a standard build with some smoke
>>> tests, a no-test fast build, and a full test run. The above commit
>>> results in the following behavior:
>>>
>>> 1) By default, all normal modules and testsuite modules will be built
>>> (so test compilation issues introduced by code changes will be caught)
>>> but the only tests that will be executed are:
>>>
>>> a) the tests in non-testsuite modules
>>> b) the tests in testsuite/smoke
>>>
>>> 2) By adding -DskipTests=true, no tests will be executed (although
>>> they'll still compile).
>>>
>>> 3) By adding -DskipTests=false the integration tests in testsuite2,
>>> testsuite/integration and the webservices module will be enabled. So
>>> that results in a full testsuite run (except for the currently empty
>>> benchmark and stress test modules).
>>>
>>> The testsuite2, testsuite/integration and the webservices module also
>>> have testFailureIgnore set to true, which means failures in one of the
>>> many modules will not fail the build, allowing the full set of tests to
>>> run. A test failure in a non-testsuite module or in testsuite/smoke will
>>> fail the build.
>>>
>>
>
>