[jboss-as7-dev] Testsuite execution

[Thomas Diesler]

Actually, we are going to consolidate the testsuites as part of AS7-734. 
Module testsuite/integration uses the Alpha4.SPx version of arquillian 
which has features that are not (yet) available in arq beta. For example 
it supports dynamic deployment from within a test method, injection of 
ServiceContainer, etc.

If you don't need the Alpha4.SP features, the preferred location is 
integration2. Otherwise put it were it works for you - I'll cleanup and 
consolidate later as part of that jira issue.

cheers
-thomas

On 05/18/2011 08:40 PM, Brian Stansberry wrote:
> No new tests should be going into testsuite/integration; existing ones
> should be moved out.
>
> On 5/18/11 11:58 AM, Jaikiran Pai wrote:
>> By the way, are we still adding tests to the (outdated?)
>> testsuite/integration module? I thought we were going to start using
>> testsuite2/spec and testsuite2/internals going forward. Where should any
>> new tests go?
>>
>> -Jaikiran
>> On Wednesday 18 May 2011 09:35 PM, Brian Stansberry wrote:
>>> That's fine by me. I don't care all that much about compiling the
>>> integration tests. I didn't want -P allTests because that turns off
>>> default profiles, but using -DallTests to activate a profile shouldn't
>>> have that problem.
>>>
>>> On 5/18/11 10:13 AM, Thomas Diesler wrote:
>>>> Using skipTests as a flag has a negative side effect such that you
>>>> cannot do
>>>>
>>>>     >    mvn -Dtest=SomeTestCase test
>>>>
>>>> in an integration module any more. Instead you have to do
>>>>
>>>>     >    mvn -DskipTests=false -Dtest=SomeTestCase test
>>>>
>>>> This is non-intuitive and non-standard. Instead I propose to use an
>>>> alternate switch like 'allTests' to turn on profiles which run the
>>>> complete set of tests. IMHO its ok to exclude entire (testsuite) modules
>>>> from the build (i.e. not to compile the tests) because compilation
>>>> errors show up in the IDE and Hudson anyway.
>>>>
>>>> I propose
>>>>
>>>> 1) By default, all normal modules and testsuite/smoke will be built in
>>>> the standard way
>>>> (i.e. failing tests will fail the build)
>>>>
>>>> 2) By adding -DskipTests=true tests that would normally be executed are
>>>> skipped
>>>>
>>>> 3) By adding -DallTests all modules that contain tests will be executed.
>>>> In Hudson you could add -fae
>>>> which fails the build at the end - so catch all failing tests in a
>>>> single run
>>>>
>>>> It boils down to whether you want to compile tests that you don't want
>>>> to execute.
>>>> IMHO that's not necessary in day to day work - Hudson can catch this.
>>>>
>>>> cheers
>>>> -thomas
>>>>
>>>>
>>>>
>>>> On 05/15/2011 04:16 AM, Brian Stansberry wrote:
>>>>> I'd like to get feedback on changes to the testsuite found in
>>>>> https://github.com/bstansberry/jboss-as/commit/efe2051753f6b79788d05156784665e02672613c
>>>>>
>>>>>
>>>>> I'll be adding a bunch of integration tests that launch a domain, but
>>>>> before pushing those I wanted to ensure they weren't run every time
>>>>> someone did a build. That led to a general effort to simplify how we can
>>>>> execute the 3 main build cases -- a standard build with some smoke
>>>>> tests, a no-test fast build, and a full test run. The above commit
>>>>> results in the following behavior:
>>>>>
>>>>> 1) By default, all normal modules and testsuite modules will be built
>>>>> (so test compilation issues introduced by code changes will be caught)
>>>>> but the only tests that will be executed are:
>>>>>
>>>>> a) the tests in non-testsuite modules
>>>>> b) the tests in testsuite/smoke
>>>>>
>>>>> 2) By adding -DskipTests=true, no tests will be executed (although
>>>>> they'll still compile).
>>>>>
>>>>> 3) By adding -DskipTests=false the integration tests in testsuite2,
>>>>> testsuite/integration and the webservices module will be enabled. So
>>>>> that results in a full testsuite run (except for the currently empty
>>>>> benchmark and stress test modules).
>>>>>
>>>>> The testsuite2, testsuite/integration and the webservices module also
>>>>> have testFailureIgnore set to true, which means failures in one of the
>>>>> many modules will not fail the build, allowing the full set of tests to
>>>>> run. A test failure in a non-testsuite module or in testsuite/smoke will
>>>>> fail the build.
>>>>>
>>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> jboss-as7-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx