3:45 p.m.
I still contend that it should be able to do this differently.
Simple mvn install (-DallTest=true) for developers.
Then qe should be able to use separate qe profiles/the antrun plugin/whatever with some
trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try out
if I have some spare cycles and I am sure Paul Gier could help.
Sent from my Android phone using TouchDown (www.nitrodesk.com)
-----Original Message-----
From: Shelly McGowan [smcgowan(a)redhat.com]
Received: Thursday, 03 Nov 2011, 14:35
To: Thomas Diesler [thomas.diesler(a)jboss.com]
CC: jboss-as7-dev(a)lists.jboss.org
Subject: Re: [jboss-as7-dev] Test Suite Changes
From August:
http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html
Shelly
----- Original Message -----
From: "Shelly McGowan" <smcgowan(a)redhat.com>
To: "Thomas Diesler" <thomas.diesler(a)jboss.com>
Cc: jboss-as7-dev(a)lists.jboss.org, "Darran Lofthouse"
<darran.lofthouse(a)jboss.com>
Sent: Thursday, November 3, 2011 10:24:46 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
So, the changes to the test suite were based on requirements. The issues as I see it is
that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the
initial pull request[s].
It was anticipated in the requirements discussions, though, that the integration tests
would be used for test purposes
other than during a build of AS7.
Thomas, have you looked at the requirements document links mentioned below?
As far as bugs, fair enough. They're being addressed as they are uncovered. JIRAs
should be filed.
Shelly
----- Original Message -----
From: "Thomas Diesler" <thomas.diesler(a)jboss.com>
To: "Darran Lofthouse" <darran.lofthouse(a)jboss.com>
Cc: jboss-as7-dev(a)lists.jboss.org
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.
The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.
Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.
-thomas
On 11/03/2011 12:40 PM, Darran Lofthouse wrote:
It would be nice if we could agree which parameters should actually
work
and what they mean - the one that caught me out recently was when
-DskipTests=true was broken.
Regards,
Darran Lofthouse.
On 11/03/2011 11:34 AM, Kabir Khan wrote:
> Smoke tests are running by default now which is nice. However, a './build.sh
clean install -DallTests=true' no longer runs all the tests, while they did last week.
This is starting to get very confusing, all I want to do is to process some pull requests
and run the tests.
>
> If this really needs to keep changing all the time, a
this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>
> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>
>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever
they want to run anything else - because Maven's profile activation logic really
sucks.
>> Fixing.
>>
>> Ondra
>>
>>
>>
>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>> The smoke tests must be run as part of a standard build. Please fix.
>>>
>>> If QE have other requirements, which are not possible to combine with the
above requirement maybe they should look into using alternate poms or the antrun plugin or
something.
>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>
>>>> Currently, the smoke tests don't run as part of the build. Is this
really what you want?
>>>>
>>>>
>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>
>>>> -thomas
>>>>
>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>> I just just merged some changes to the test suite that QE requires
for EAP (see below for a link to their requirements). As a result the manner in which the
tests are run has changed slightly.
>>>>>
>>>>> Due to limitations in maven it is no longer possible to activate all
tests using mvn install -DallTests, instead you should use either:
>>>>>
>>>>> ./build.sh install -DallTests - To do a full build with all tests
>>>>> ./integration-tests.sh -DallTests - To just run all integration
tests
>>>>>
>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest
should work as normal.
>>>>>
>>>>> Ondej Zizka (in CC) has been the person working on these changes, so
if you have any questions or comments he is the person to talk to.
>>>>>
>>>>> Stuart
>>>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>
>>>>> *
>>> https://github.com/jbossas/jboss-as/pull/491
>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>>>
>>> jboss-as7-dev(a)lists.jboss.org
>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>> --
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>> Thomas Diesler
>>>> JBoss OSGi Lead
>>>> JBoss, a division of Red Hat
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>> jboss-as7-dev(a)lists.jboss.org
>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>>
>>> jboss-as7-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
4:02 p.m.
New subject: Test Suite Changes
Kabir,
Communicate your ideas to Ondra (ozizka). He can help you.
Shelly
----- Original Message -----
From: "Kabir Khan" <kkhan(a)redhat.com>
To: "thomas diesler" <thomas.diesler(a)jboss.com>, smcgowan(a)redhat.com
Cc: jboss-as7-dev(a)lists.jboss.org
Sent: Thursday, November 3, 2011 10:45:36 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
I still contend that it should be able to do this differently.
Simple mvn install (-DallTest=true) for developers.
Then qe should be able to use separate qe profiles/the antrun plugin/whatever with some
trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try out
if I have some spare cycles and I am sure Paul Gier could help.
Sent from my Android phone using TouchDown (www.nitrodesk.com)
-----Original Message-----
From: Shelly McGowan [smcgowan(a)redhat.com]
Received: Thursday, 03 Nov 2011, 14:35
To: Thomas Diesler [thomas.diesler(a)jboss.com]
CC: jboss-as7-dev(a)lists.jboss.org
Subject: Re: [jboss-as7-dev] Test Suite Changes
From August:
http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html
Shelly
----- Original Message -----
From: "Shelly McGowan" <smcgowan(a)redhat.com>
To: "Thomas Diesler" <thomas.diesler(a)jboss.com>
Cc: jboss-as7-dev(a)lists.jboss.org, "Darran Lofthouse"
<darran.lofthouse(a)jboss.com>
Sent: Thursday, November 3, 2011 10:24:46 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
So, the changes to the test suite were based on requirements. The issues as I see it is
that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the
initial pull request[s].
It was anticipated in the requirements discussions, though, that the integration tests
would be used for test purposes
other than during a build of AS7.
Thomas, have you looked at the requirements document links mentioned below?
As far as bugs, fair enough. They're being addressed as they are uncovered. JIRAs
should be filed.
Shelly
----- Original Message -----
From: "Thomas Diesler" <thomas.diesler(a)jboss.com>
To: "Darran Lofthouse" <darran.lofthouse(a)jboss.com>
Cc: jboss-as7-dev(a)lists.jboss.org
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.
The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.
Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.
-thomas
On 11/03/2011 12:40 PM, Darran Lofthouse wrote:
It would be nice if we could agree which parameters should actually
work
and what they mean - the one that caught me out recently was when
-DskipTests=true was broken.
Regards,
Darran Lofthouse.
On 11/03/2011 11:34 AM, Kabir Khan wrote:
> Smoke tests are running by default now which is nice. However, a './build.sh
clean install -DallTests=true' no longer runs all the tests, while they did last week.
This is starting to get very confusing, all I want to do is to process some pull requests
and run the tests.
>
> If this really needs to keep changing all the time, a
this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>
> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>
>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever
they want to run anything else - because Maven's profile activation logic really
sucks.
>> Fixing.
>>
>> Ondra
>>
>>
>>
>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>> The smoke tests must be run as part of a standard build. Please fix.
>>>
>>> If QE have other requirements, which are not possible to combine with the
above requirement maybe they should look into using alternate poms or the antrun plugin or
something.
>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>
>>>> Currently, the smoke tests don't run as part of the build. Is this
really what you want?
>>>>
>>>>
>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>
>>>> -thomas
>>>>
>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>> I just just merged some changes to the test suite that QE requires
for EAP (see below for a link to their requirements). As a result the manner in which the
tests are run has changed slightly.
>>>>>
>>>>> Due to limitations in maven it is no longer possible to activate all
tests using mvn install -DallTests, instead you should use either:
>>>>>
>>>>> ./build.sh install -DallTests - To do a full build with all tests
>>>>> ./integration-tests.sh -DallTests - To just run all integration
tests
>>>>>
>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest
should work as normal.
>>>>>
>>>>> Ondej Zizka (in CC) has been the person working on these changes, so
if you have any questions or comments he is the person to talk to.
>>>>>
>>>>> Stuart
>>>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>
>>>>> *
>>> https://github.com/jbossas/jboss-as/pull/491
>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>>>
>>> jboss-as7-dev(a)lists.jboss.org
>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>> --
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>> Thomas Diesler
>>>> JBoss OSGi Lead
>>>> JBoss, a division of Red Hat
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>> jboss-as7-dev(a)lists.jboss.org
>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>>
>>> jboss-as7-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
5:20 p.m.
New subject: Test Suite Changes
ok, will do (tomorrow probably)
On 3 Nov 2011, at 15:02, Shelly McGowan wrote:
Kabir,
Communicate your ideas to Ondra (ozizka). He can help you.
Shelly
----- Original Message -----
From: "Kabir Khan" <kkhan(a)redhat.com>
To: "thomas diesler" <thomas.diesler(a)jboss.com>, smcgowan(a)redhat.com
Cc: jboss-as7-dev(a)lists.jboss.org
Sent: Thursday, November 3, 2011 10:45:36 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
I still contend that it should be able to do this differently.
Simple mvn install (-DallTest=true) for developers.
Then qe should be able to use separate qe profiles/the antrun plugin/whatever with some
trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try out
if I have some spare cycles and I am sure Paul Gier could help.
Sent from my Android phone using TouchDown (www.nitrodesk.com)
-----Original Message-----
From: Shelly McGowan [smcgowan(a)redhat.com]
Received: Thursday, 03 Nov 2011, 14:35
To: Thomas Diesler [thomas.diesler(a)jboss.com]
CC: jboss-as7-dev(a)lists.jboss.org
Subject: Re: [jboss-as7-dev] Test Suite Changes
> From August:
http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html
Shelly
----- Original Message -----
From: "Shelly McGowan" <smcgowan(a)redhat.com>
To: "Thomas Diesler" <thomas.diesler(a)jboss.com>
Cc: jboss-as7-dev(a)lists.jboss.org, "Darran Lofthouse"
<darran.lofthouse(a)jboss.com>
Sent: Thursday, November 3, 2011 10:24:46 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
So, the changes to the test suite were based on requirements. The issues as I see it is
that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the
initial pull request[s].
It was anticipated in the requirements discussions, though, that the integration tests
would be used for test purposes
other than during a build of AS7.
Thomas, have you looked at the requirements document links mentioned below?
As far as bugs, fair enough. They're being addressed as they are uncovered. JIRAs
should be filed.
Shelly
----- Original Message -----
From: "Thomas Diesler" <thomas.diesler(a)jboss.com>
To: "Darran Lofthouse" <darran.lofthouse(a)jboss.com>
Cc: jboss-as7-dev(a)lists.jboss.org
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes
Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.
The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.
Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.
-thomas
On 11/03/2011 12:40 PM, Darran Lofthouse wrote:
> It would be nice if we could agree which parameters should actually work
> and what they mean - the one that caught me out recently was when
> -DskipTests=true was broken.
>
> Regards,
> Darran Lofthouse.
>
>
>
> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>> Smoke tests are running by default now which is nice. However, a './build.sh
clean install -DallTests=true' no longer runs all the tests, while they did last week.
This is starting to get very confusing, all I want to do is to process some pull requests
and run the tests.
>>
>> If this really needs to keep changing all the time, a
this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>
>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>
>>> Ok, seems like QA will have to carry the burden of disabling smoke tests
whenever they want to run anything else - because Maven's profile activation logic
really sucks.
>>> Fixing.
>>>
>>> Ondra
>>>
>>>
>>>
>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>
>>>> If QE have other requirements, which are not possible to combine with the
above requirement maybe they should look into using alternate poms or the antrun plugin or
something.
>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>
>>>>> Currently, the smoke tests don't run as part of the build. Is
this really what you want?
>>>>>
>>>>>
>>>>
https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>
>>>>> -thomas
>>>>>
>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>> I just just merged some changes to the test suite that QE
requires for EAP (see below for a link to their requirements). As a result the manner in
which the tests are run has changed slightly.
>>>>>>
>>>>>> Due to limitations in maven it is no longer possible to activate
all tests using mvn install -DallTests, instead you should use either:
>>>>>>
>>>>>> ./build.sh install -DallTests - To do a full build with all
tests
>>>>>> ./integration-tests.sh -DallTests - To just run all integration
tests
>>>>>>
>>>>>> Running individual tests from an IDE or via mvn test
-Dtest=MyTest should work as normal.
>>>>>>
>>>>>> Ondej Zizka (in CC) has been the person working on these changes,
so if you have any questions or comments he is the person to talk to.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>
>>>>>> *
>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>>>
>>>> jboss-as7-dev(a)lists.jboss.org
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>> jboss-as7-dev(a)lists.jboss.org
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>>> jboss-as7-dev(a)lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> jboss-as7-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
1:24 p.m.
New subject: Test Suite Changes
I'll be talking to Ondra a bit later today, in the meantime I've created a POC for
the testsuite https://github.com/kabir/testsuite-poc. It is a very simple project with
hardly any classes/dependencies to just concentrate on the profile selection issues we
have been having. If someone has time to look at this in the next few hours and has some
feedback that would be great.
The maven modules are
* build/ - the "main codebase"
* testsuite - the testsuite aggregator for a stripped down similar structure to what we
have in AS 7
* testsuite/integration - Illustrates a project with several sets of functionality:
- smoke tests
- A fuller set of integration tests
- Some 'special' tests which must be run in a separate surefire execution from
the integration tests
* testsuite/spec - Just contains one test
* testsuite/domain - Just contains one test.
In testsuite and testsuite/integration there are some profiles for AS 7 developers and
some for QE.
**AS 7 developer profiles**
1) Smoke tests only
Run by: mvn install
The 'activeByDefault' profiles of testsuite/pom.xml and
testsuite/integration/pom.xml are triggered. These are:
- testsuite/pom.xml's integration.only profile
This just runs the integration module, and skips the spec and domain modules
- testsuite/integration/pom.xml's integration.smoke.tests profile
This just runs just the smoke tests
These profiles are 'activeByDefault' so they will always be run unless another
profile is explicitly enabled.
2) All tests
Run by: mvn install -DallTests OR mvn install -DallTests=true
Once this property is present, the 'activeByDefault' profiles are no longer
triggered, instead it triggers:
- testsuite/pom.xml's all.tests profile
This runs the integration,spec and domain modules
- testsuite/integration/pom.xml's integration.all.tests profile
This runs all the tests in two surefire executions. The first with all the smoke and
integration tests. The second with the 'special' tests.
**QE profiles**
DISCLAIMER: This might not be exactly what QE want but I'll cover that with Ondra
later.
1) testsuite/pom.xml
This contains the following profiles to control which testsuite modules are run
qe.integration - which just runs the integration module tests
qe.spec - which just runs the spec module tests
qe.domain - which just runs the domain module tests
So for example to run both the integration and spec module tests, but not the domain
tests, you could do:
mvn install -Pqe.integration,qe.spec
For the integration tests this will by default run just the smoke tests (due the the
'activeByDefault' integration.smoke.tests profile.
2) testsuite/integration/pom.xml
This contains the following profiles to control subsets of tests:
qe.integration.smoke - which just runs the smoke tests
qe.integration.integration - which just runs the integration tests
qe.integration.special - which just runs the 'special' tests
So if -Pqe.integration is specified to run testsuite/integration tests you can control
that. For example:
This runs just the testsuite/integration 'integration' tests
mvn install -Pqe.integration,qe.integration.integration
This runs just the testsuite/integration 'special' tests
mvn install -Pqe.integration,qe.integration.special
This runs the testsuite/integration smoke and 'special' tests
mvn install -Pqe.integration,qe.integration.smoke,qe.integration.special
Or you can use the integration.all.tests profile to run all the tests in the integration
module (similar to using -DallTests but ONLY running tests from testsuite/integration and
not from testsuite/spec or testsuite/domain)
mvn install -Pqe.integration,integration.all.tests
Then QE can create scripts to trigger combinations of the above as they see fit.
On 3 Nov 2011, at 16:20, Kabir Khan wrote:
ok, will do (tomorrow probably)
On 3 Nov 2011, at 15:02, Shelly McGowan wrote:
>
>
> Kabir,
>
> Communicate your ideas to Ondra (ozizka). He can help you.
>
> Shelly
>
>
> ----- Original Message -----
> From: "Kabir Khan" <kkhan(a)redhat.com>
> To: "thomas diesler" <thomas.diesler(a)jboss.com>, smcgowan(a)redhat.com
> Cc: jboss-as7-dev(a)lists.jboss.org
> Sent: Thursday, November 3, 2011 10:45:36 AM
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
> I still contend that it should be able to do this differently.
>
> Simple mvn install (-DallTest=true) for developers.
>
> Then qe should be able to use separate qe profiles/the antrun plugin/whatever with
some trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try
out if I have some spare cycles and I am sure Paul Gier could help.
>
> Sent from my Android phone using TouchDown (www.nitrodesk.com)
>
>
> -----Original Message-----
> From: Shelly McGowan [smcgowan(a)redhat.com]
> Received: Thursday, 03 Nov 2011, 14:35
> To: Thomas Diesler [thomas.diesler(a)jboss.com]
> CC: jboss-as7-dev(a)lists.jboss.org
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
>
>
>
>> From August:
>
> http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html
>
> Shelly
>
>
> ----- Original Message -----
> From: "Shelly McGowan" <smcgowan(a)redhat.com>
> To: "Thomas Diesler" <thomas.diesler(a)jboss.com>
> Cc: jboss-as7-dev(a)lists.jboss.org, "Darran Lofthouse"
<darran.lofthouse(a)jboss.com>
> Sent: Thursday, November 3, 2011 10:24:46 AM
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
>
>
> So, the changes to the test suite were based on requirements. The issues as I see it
is that there is no
> concensus (making everyone happy) which, in turn, has caused additional changes to
the initial pull request[s].
>
> It was anticipated in the requirements discussions, though, that the integration
tests would be used for test purposes
> other than during a build of AS7.
>
> Thomas, have you looked at the requirements document links mentioned below?
>
> As far as bugs, fair enough. They're being addressed as they are uncovered.
JIRAs should be filed.
>
>
>
> Shelly
>
>
>
> ----- Original Message -----
> From: "Thomas Diesler" <thomas.diesler(a)jboss.com>
> To: "Darran Lofthouse" <darran.lofthouse(a)jboss.com>
> Cc: jboss-as7-dev(a)lists.jboss.org
> Sent: Thursday, November 3, 2011 9:56:28 AM
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
> Yes, I would also like to see a document that describes how the
> testsuite is (more importantly should be) organised and how to execute
> individual test sections. The document should start with a list of
> requirements.
>
> The argument of "integration-tests.sh makes it easier" is not valid
> IMHO. It is a diversion of an established standard and quite the
> contrary from "makes it easier" IMHO. In short, start with requirements
> and show whether and how they are met and where work still needs to be
> done.
>
> Refactoring the test suite into a clean structure, such that standard
> mvn calls can be used to execute what we need would be the preferred
> solution IMHO.
>
> -thomas
>
> On 11/03/2011 12:40 PM, Darran Lofthouse wrote:
>> It would be nice if we could agree which parameters should actually work
>> and what they mean - the one that caught me out recently was when
>> -DskipTests=true was broken.
>>
>> Regards,
>> Darran Lofthouse.
>>
>>
>>
>> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>>> Smoke tests are running by default now which is nice. However, a
'./build.sh clean install -DallTests=true' no longer runs all the tests, while
they did last week. This is starting to get very confusing, all I want to do is to process
some pull requests and run the tests.
>>>
>>> If this really needs to keep changing all the time, a
this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>>
>>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>>
>>>> Ok, seems like QA will have to carry the burden of disabling smoke tests
whenever they want to run anything else - because Maven's profile activation logic
really sucks.
>>>> Fixing.
>>>>
>>>> Ondra
>>>>
>>>>
>>>>
>>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>>
>>>>> If QE have other requirements, which are not possible to combine with
the above requirement maybe they should look into using alternate poms or the antrun
plugin or something.
>>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>>
>>>>>> Currently, the smoke tests don't run as part of the build. Is
this really what you want?
>>>>>>
>>>>>>
>>>>>
https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>>
>>>>>> -thomas
>>>>>>
>>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>>> I just just merged some changes to the test suite that QE
requires for EAP (see below for a link to their requirements). As a result the manner in
which the tests are run has changed slightly.
>>>>>>>
>>>>>>> Due to limitations in maven it is no longer possible to
activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>>
>>>>>>> ./build.sh install -DallTests - To do a full build with all
tests
>>>>>>> ./integration-tests.sh -DallTests - To just run all
integration tests
>>>>>>>
>>>>>>> Running individual tests from an IDE or via mvn test
-Dtest=MyTest should work as normal.
>>>>>>>
>>>>>>> Ondej Zizka (in CC) has been the person working on these
changes, so if you have any questions or comments he is the person to talk to.
>>>>>>>
>>>>>>> Stuart
>>>>>>>
>>>>>>> *
>>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>>
>>>>>>> *
>>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>>
>>>>>>> *
>>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> jboss-as7-dev mailing list
>>>>>>>
>>>>>>>
>>>>> jboss-as7-dev(a)lists.jboss.org
>>>>>
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>>>> --
>>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>> Thomas Diesler
>>>>>> JBoss OSGi Lead
>>>>>> JBoss, a division of Red Hat
>>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>> jboss-as7-dev(a)lists.jboss.org
>>>>>
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>>> jboss-as7-dev(a)lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> jboss-as7-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>> _______________________________________________
>> jboss-as7-dev mailing list
>> jboss-as7-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
4579
days inactive
4580
days old
3 comments
3 participants
participants (3)
-
Kabir Khan -
Kabir Khan -
Shelly McGowan