3:27 p.m.
Jaikiran Pai píše v Po 28. 11. 2011 v 15:05 +0530:
Comments inline.
-"-
Since these flags apply to the EAP testing, why
not have a specific script which deals with all these flags and that
script can then be used for EAP testing?
Well, since the testsuite is maven-based, the script can only use what's
really in pom.xml's.
I could move some steps (e.g. server config) to the script.
But I think that having all in one place is beneficial: If a bug in EAP
is found, then it's easy for the developer to reproduce - they'll simply
use same set of maven properties.
Maybe my words "more and more" scared you a bit :) It's not that bad,
especially after splitting to modules, the pom.xml's are not that hard
to read anymore.
And since we have gone the modularized way, I can keep the functionality
outside of scope of the "normal" build, i.e. not include some modules at
all - only for EAP builds / test runs.
What I mean is:
testsuite
+ integration
+ configure-database
+ configure-ipv6
+ group-smoke
+ group-basic
+ group-iiop
And activate the submodules on demand.
My point is not about adding
these new flags but is more about having to expect the non-QA developers
(for whom those are irrelevant) to understand and use these flags for
the builds.
Actually, I try not to change the behavior unless someone asks for some
change - that's the case of -DskipTests btw.
I didn't assume anyone relying on that.
And, this was not in testsuite requirements, which we discussed for a
while on this list.
But as I wrote - it's coming back.
> But I can keep you uninformed and keep the testsuite running without
> changes of "API" if that helps.
I guess, you meant informed. Yes, that would help - if adding the flags
helps QA testing but doesn't have any impact on the "API" and helps keep
the testsuite running, then I have no objection to those changes/flags.
Okay. How about others?
>> The actual
>> question for this specific issue is what's the real usecase of skipping
>> unit tests (which potentially can have failures) but still want to run
>> the integration tests?
> Often it happens that some unit tests fail and block one step AS build
> & testsuite execution.
So far I haven't seen any unit test fail in AS7 (and if some of them are
failing then those need to be fixed). Yes, there are integration test
failures which are intermittent. But those issues need to be fixed. Like
I said in my earlier reply, building the entire project by using
-DskipTests (to skip all tests) and then cd testsuite/integration; mvn
clean install (to run the integration tests) is what you are looking for
in such cases. Ofcourse it's a 2 step process but to me that's more
logical (and can even be added to a script) than adding and integrating
some new property.
Okay. Someone from QA wanted to be able to run in one step due to some
Jenkins issue with Maven-based build, I'll discuss that.
> And let me ask the other way around - what is the use case of
> -DskipTests, skipping all tests?
Yes.
That was like a part of the question - what is the use case of skipping
_all_ tests?
* As you said, unit tests should be rock solid and you rarely see a
failure.
* So you skip tests to make the build faster, not because of unit tests
failures.
* So you'd probably like to skip testsuite module to make it even
faster.
Then I don't understand why you dislike the idea of having a separated
flag to skip the testsuite. ?
Ondra