Jaikiran Pai píše v Po 28. 11. 2011 v 15:05 +0530:

Comments inline.

-"-

Since these flags apply to the EAP testing, why 
not have a specific script which deals with all these flags and that 
script can then be used for EAP testing?

Well, since the testsuite is maven-based, the script can only use what's really in pom.xml's.
I could move some steps (e.g. server config) to the script.
But I think that having all in one place is beneficial: If a bug in EAP is found, then it's easy for the developer to reproduce - they'll simply use same set of maven properties.

Maybe my words "more and more" scared you a bit :)   It's not that bad, especially after splitting to modules, the pom.xml's are not that hard to read anymore.
And since we have gone the modularized way, I can keep the functionality outside of scope of the "normal" build, i.e. not include some modules at all - only for EAP builds / test runs.

What I mean is:
testsuite
+ integration
     + configure-database
     + configure-ipv6
     + group-smoke
     + group-basic
     + group-iiop

And activate the submodules on demand.

My point is not about adding 
these new flags but is more about having to expect the non-QA developers 
(for whom those are irrelevant) to understand and use these flags for 
the builds.

Actually, I try not to change the behavior unless someone asks for some change - that's the case of -DskipTests btw.
I didn't assume anyone relying on that.
And, this was not in testsuite requirements, which we discussed for a while on this list.
But as I wrote - it's coming back.


> But I can keep you uninformed and keep the testsuite running without 
> changes of "API" if that helps.
I guess, you meant informed. Yes, that would help - if adding the flags 
helps QA testing but doesn't have any impact on the "API" and helps keep 
the testsuite running, then I have no objection to those changes/flags.

Okay. How about others?

>>   The actual
>> question for this specific issue is what's the real usecase of skipping
>> unit tests (which potentially can have failures) but still want to run
>> the integration tests?
> Often it happens that some unit tests fail and block one step AS build 
> & testsuite execution.
So far I haven't seen any unit test fail in AS7 (and if some of them are 
failing then those need to be fixed). Yes, there are integration test 
failures which are intermittent. But those issues need to be fixed. Like 
I said in my earlier reply, building the entire project by using 
-DskipTests (to skip all tests) and then cd testsuite/integration; mvn 
clean install (to run the integration tests) is what you are looking for 
in such cases. Ofcourse it's a 2 step process but to me that's more 
logical (and can even be added to a script) than adding and integrating 
some new property.

Okay. Someone from QA wanted to be able to run in one step due to some Jenkins issue with Maven-based build, I'll discuss that.


> And let me ask the other way around - what is the use case of 
> -DskipTests, skipping all tests?
Yes.

That was like a part of the question - what is the use case of skipping _all_ tests?
* As you said, unit tests should be rock solid and you rarely see a failure.
* So you skip tests to make the build faster, not because of unit tests failures.
* So you'd probably like to skip testsuite module to make it even faster.
Then I don't understand why you dislike the idea of having a separated flag to skip the testsuite. ?

Ondra