Hi, I did this intentionally - the intention was to skip unit tests but run the testsuite - not to get the build failed if some module's unit tests fail.

How about having two params for that, with -DskipTests for unit tests, and, say, -DskipTestsuite for skipping the testsuite module's tests?

Ondra -------- Přeposlaná zpráva -------- > *Od*: Darran Lofthouse (Created) (JIRA) <jira-events(a)lists.jboss.org > <mailto:%22Darran%20Lofthouse%20%28Created%29%20%28JIRA%29%22%20%3cjira-events@lists.jboss.org%3e>> > *Komu* : ozizka(a)redhat.com <mailto:ozizka@redhat.com> > *Předmět* : [JBoss JIRA] (AS7-2796) -DskipTests argument no longer > preventing the execution of tests on AS7 build > *Datum* : Fri, 25 Nov 2011 10:15:40 -0500 (EST) > > -DskipTests argument no longer preventing the execution of tests on AS7 build > ----------------------------------------------------------------------------- > > Key: AS7-2796 > URL:https://issues.jboss.org/browse/AS7-2796 > Project: Application Server 7 > Issue Type: Bug > Components: Test Suite > Reporter: Darran Lofthouse > Assignee: Ondrej Zizka > Fix For: 7.1.0.CR1 > > > Since the testsuite changes have been committed the -DskipTests argument no longer skips the tests. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators:https://issues.jboss.org/secure/ContactAdministrators!defa... <https://issues.jboss.org/secure/ContactAdministrators%21default.jspa> > For more information on JIRA, see:http://www.atlassian.com/software/jira > > _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

On Monday 28 November 2011 01:48 PM, Jaikiran Pai wrote: >> How about having two params for that, with -DskipTests for unit tests, >> and, say, -DskipTestsuite for skipping the testsuite module's tests? >> > IMO, we shouldn't just be adding all these new flags which make it > complicated to understand what set of flags to use and when. The actual > question for this specific issue is what's the real usecase of skipping > unit tests (which potentially can have failures) but still want to run > the integration tests? By the way, you could still achieve the goal of running the integration testsuite and skipping the unit tests, by doing the following 2 steps: 1) From AS7 top level - mvn clean install -DskipTests This will run the entire build and skip all tests including unit and integration tests. 2) cd testsuite/integration - mvn clean install This will run the integration tests I really don't see a real need for adding another flag for this. But yes, I might be missing some usecase. -Jaikiran _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Jaikiran Pai píše v Po 28. 11. 2011 v 13:48 +0530: > There's > -Dmaven.test.failure.ignore=true if you want to ignore those test > failures. I don't know how the AS7 project handles the > -Dmaven.test.failure.ignore property due to our testsuite > profiles/customizations. But we shouldn't be adding new meaning to > already well known flags. The problem with these is that they apply to whole build.

> > How about having two params for that, with -DskipTests for unit tests, > > and, say, -DskipTestsuite for skipping the testsuite module's tests? > > > IMO, we shouldn't just be adding all these new flags which make it > complicated to understand what set of flags to use and when. I have to disappoint you but you'll see more and more flags being added in the future. Please realize that there are other groups using the testsuite e.g. for EAP testing.

But I can keep you uninformed and keep the testsuite running without changes of "API" if that helps.

> The actual > question for this specific issue is what's the real usecase of skipping > unit tests (which potentially can have failures) but still want to run > the integration tests? Often it happens that some unit tests fail and block one step AS build & testsuite execution.

And let me ask the other way around - what is the use case of -DskipTests, skipping all tests?

> To be clear - IMO, we should have the -DskipTests back without adding a > new meaning to that param. Okay, I realized I can set <skipTests> from a different property. So it will be back in a while.

Jaikiran, just have in mind that we want to keep the AS/EAP diff at the source level minimum. On 28/11/2011 11:35, Jaikiran Pai wrote: > Comments inline. > > On Monday 28 November 2011 02:52 PM, Ondřej Žižka wrote: >> Jaikiran Pai píše v Po 28. 11. 2011 v 13:48 +0530: >>> There's >>> -Dmaven.test.failure.ignore=true if you want to ignore those test >>> failures. I don't know how the AS7 project handles the >>> -Dmaven.test.failure.ignore property due to our testsuite >>> profiles/customizations. But we shouldn't be adding new meaning to >>> already well known flags. >> The problem with these is that they apply to whole build. > That's not a _problem_ since that's what it's expected to do. >> >> >>>> How about having two params for that, with -DskipTests for unit tests, >>>> and, say, -DskipTestsuite for skipping the testsuite module's tests? >>>> >>> IMO, we shouldn't just be adding all these new flags which make it >>> complicated to understand what set of flags to use and when. >> I have to disappoint you but you'll see more and more flags being >> added in the future. >> Please realize that there are other groups using the testsuite e.g. >> for EAP testing. > I understand that it's not just the developers who are using these > testsuite. My point is that most of these additional flags that we keep > adding are only relevant for the EAP testing/QA - which really means > that these shouldn't be added to the default build or in other words > should not show any impact on how the other group (i.e. the developers) > uses the current build. Since these flags apply to the EAP testing, why > not have a specific script which deals with all these flags and that > script can then be used for EAP testing? My point is not about adding > these new flags but is more about having to expect the non-QA developers > (for whom those are irrelevant) to understand and use these flags for > the builds. > >> But I can keep you uninformed and keep the testsuite running without >> changes of "API" if that helps. > I guess, you meant informed. Yes, that would help - if adding the flags > helps QA testing but doesn't have any impact on the "API" and helps keep > the testsuite running, then I have no objection to those changes/flags. > >> >>> The actual >>> question for this specific issue is what's the real usecase of skipping >>> unit tests (which potentially can have failures) but still want to run >>> the integration tests? >> Often it happens that some unit tests fail and block one step AS build >> & testsuite execution. > So far I haven't seen any unit test fail in AS7 (and if some of them are > failing then those need to be fixed). Yes, there are integration test > failures which are intermittent. But those issues need to be fixed. Like > I said in my earlier reply, building the entire project by using > -DskipTests (to skip all tests) and then cd testsuite/integration; mvn > clean install (to run the integration tests) is what you are looking for > in such cases. Ofcourse it's a 2 step process but to me that's more > logical (and can even be added to a script) than adding and integrating > some new property. > >> And let me ask the other way around - what is the use case of >> -DskipTests, skipping all tests? > Yes. >> >>> To be clear - IMO, we should have the -DskipTests back without adding a >>> new meaning to that param. >> Okay, I realized I can set<skipTests> from a different property. So >> it will be back in a while. > Thanks! > > -Jaikiran > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Comments inline.

Since these flags apply to the EAP testing, why not have a specific script which deals with all these flags and that script can then be used for EAP testing?

My point is not about adding these new flags but is more about having to expect the non-QA developers (for whom those are irrelevant) to understand and use these flags for the builds.

> But I can keep you uninformed and keep the testsuite running without > changes of "API" if that helps. I guess, you meant informed. Yes, that would help - if adding the flags helps QA testing but doesn't have any impact on the "API" and helps keep the testsuite running, then I have no objection to those changes/flags.

>> The actual >> question for this specific issue is what's the real usecase of skipping >> unit tests (which potentially can have failures) but still want to run >> the integration tests? > Often it happens that some unit tests fail and block one step AS build > & testsuite execution. So far I haven't seen any unit test fail in AS7 (and if some of them are failing then those need to be fixed). Yes, there are integration test failures which are intermittent. But those issues need to be fixed. Like I said in my earlier reply, building the entire project by using -DskipTests (to skip all tests) and then cd testsuite/integration; mvn clean install (to run the integration tests) is what you are looking for in such cases. Ofcourse it's a 2 step process but to me that's more logical (and can even be added to a script) than adding and integrating some new property.

> And let me ask the other way around - what is the use case of > -DskipTests, skipping all tests? Yes.

On 11/28/2011 08:18 AM, Jaikiran Pai wrote: > On Monday 28 November 2011 01:35 PM, Ondřej Žižka wrote: >> How about having two params for that, with -DskipTests for unit tests, >> and, say, -DskipTestsuite for skipping the testsuite module's tests? >> > IMO, we shouldn't just be adding all these new flags which make it > complicated to understand what set of flags to use and when. The actual > question for this specific issue is what's the real usecase of skipping > unit tests (which potentially can have failures) but still want to run > the integration tests? > > To be clear - IMO, we should have the -DskipTests back without adding a > new meaning to that param. +1 - To me that parameter already had an existing meaning and that was to skip ALL tests. Developing AS I can't see where I would want to be skipping unit tests to allow the rest of the testsuite to run. _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev