[jboss-as7-dev] Testsuite execution

[Brian Stansberry]

No new tests should be going into testsuite/integration; existing ones 
should be moved out.

On 5/18/11 11:58 AM, Jaikiran Pai wrote:
> By the way, are we still adding tests to the (outdated?)
> testsuite/integration module? I thought we were going to start using
> testsuite2/spec and testsuite2/internals going forward. Where should any
> new tests go?
>
> -Jaikiran
> On Wednesday 18 May 2011 09:35 PM, Brian Stansberry wrote:
>> That's fine by me. I don't care all that much about compiling the
>> integration tests. I didn't want -P allTests because that turns off
>> default profiles, but using -DallTests to activate a profile shouldn't
>> have that problem.
>>
>> On 5/18/11 10:13 AM, Thomas Diesler wrote:
>>> Using skipTests as a flag has a negative side effect such that you
>>> cannot do
>>>
>>>    >   mvn -Dtest=SomeTestCase test
>>>
>>> in an integration module any more. Instead you have to do
>>>
>>>    >   mvn -DskipTests=false -Dtest=SomeTestCase test
>>>
>>> This is non-intuitive and non-standard. Instead I propose to use an
>>> alternate switch like 'allTests' to turn on profiles which run the
>>> complete set of tests. IMHO its ok to exclude entire (testsuite) modules
>>> from the build (i.e. not to compile the tests) because compilation
>>> errors show up in the IDE and Hudson anyway.
>>>
>>> I propose
>>>
>>> 1) By default, all normal modules and testsuite/smoke will be built in
>>> the standard way
>>> (i.e. failing tests will fail the build)
>>>
>>> 2) By adding -DskipTests=true tests that would normally be executed are
>>> skipped
>>>
>>> 3) By adding -DallTests all modules that contain tests will be executed.
>>> In Hudson you could add -fae
>>> which fails the build at the end - so catch all failing tests in a
>>> single run
>>>
>>> It boils down to whether you want to compile tests that you don't want
>>> to execute.
>>> IMHO that's not necessary in day to day work - Hudson can catch this.
>>>
>>> cheers
>>> -thomas
>>>
>>>
>>>
>>> On 05/15/2011 04:16 AM, Brian Stansberry wrote:
>>>> I'd like to get feedback on changes to the testsuite found in
>>>> https://github.com/bstansberry/jboss-as/commit/efe2051753f6b79788d05156784665e02672613c
>>>>
>>>>
>>>> I'll be adding a bunch of integration tests that launch a domain, but
>>>> before pushing those I wanted to ensure they weren't run every time
>>>> someone did a build. That led to a general effort to simplify how we can
>>>> execute the 3 main build cases -- a standard build with some smoke
>>>> tests, a no-test fast build, and a full test run. The above commit
>>>> results in the following behavior:
>>>>
>>>> 1) By default, all normal modules and testsuite modules will be built
>>>> (so test compilation issues introduced by code changes will be caught)
>>>> but the only tests that will be executed are:
>>>>
>>>> a) the tests in non-testsuite modules
>>>> b) the tests in testsuite/smoke
>>>>
>>>> 2) By adding -DskipTests=true, no tests will be executed (although
>>>> they'll still compile).
>>>>
>>>> 3) By adding -DskipTests=false the integration tests in testsuite2,
>>>> testsuite/integration and the webservices module will be enabled. So
>>>> that results in a full testsuite run (except for the currently empty
>>>> benchmark and stress test modules).
>>>>
>>>> The testsuite2, testsuite/integration and the webservices module also
>>>> have testFailureIgnore set to true, which means failures in one of the
>>>> many modules will not fail the build, allowing the full set of tests to
>>>> run. A test failure in a non-testsuite module or in testsuite/smoke will
>>>> fail the build.
>>>>
>>>
>>
>>
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


-- 
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat