[jboss-as7-dev] Testsuite modularization? // Re: Testsuite refactor

[Ondřej Žižka]

Hi, 

1) Relevant resources:

Tracking jira:     https://issues.jboss.org/browse/AS7-2007
Requirements:  http://community.jboss.org/wiki/ASTestsuiteRequirements
Docs:              https://docs.jboss.org/author/display/AS71/Testsuite


2) Modularization & moving the test sources

> This is one thing I am wondering. If we all agree that many smaller
modules is the way to go, since its the standard maven approach
(supported by all existing maven tools), and it solves that various
issues introduced by doing the grouping approach then maybe we should
just do one refactor instead of doing this one and refactoring it yet
again?

I'd like to see the problems with the solution in PR 710. Please send me
instructions how to check that the IDE has problems coping with test
sources on ../src/test/java.
  (I don't know about Eclipse or IDEA, but NetBeans uses quite rational
approach, which is simply running Maven for the active module. Which
works.
   It works for smoke, for -DallTests if you add that param, and also
when running a single test. I haven't tried debugging yet but in worst
case, I'd run with -Ddebug and attach the debugger.)

> I know you put a lot of effort into this, but there is a good chance
this may net less work in the end since we wont be constantly tweaking
this version. 

It's not like switching from version A to B1 or B2,  it's like  staying
at B (pull request 710) or going further to C (which would be moving
sources to modules).
This harness is quite ready for both alternatives.
The reason I am proposing this as an intermediate state is, as I
previously stated, merely technical: Moving thousands of files is a big
operation from SCM PoV, so I want to get this tests relocation get
merged as quickly as possible - i.e. move, test, make pull request, get
it reviewed and merged, at best in a single day, otherwise I'll drown
with new tests coming.

> Or is there some set of requirements we have to meet ASAP that our
existing testsuite infrastructure doesnt cover?

There are many problems - I've mentioned them before - like:
* Smoke profile can either be default and need to be disabled
explicitely for any other group; but in that case many people will
forget to disable it which causes inference with other profiles.
  Or it would have to be defined explicitely - i.e. "no smoke tests run
by default"
* Huge unmaintainable pom.xml
* Impossible to define multiple groups
* Hard to separate web/full or even make all tests run with full
Etc etc. See the resources above, and the issues discussed on as7 list
recently.

> Although the problem stuart brought up is that we can't have different
classpaths in a single testsrun.
> As an example compatibility tests would likley not be able to be ran
in this way.

Compat tests are in different module anyway. This change is refactoring
the integration module. The testsuite/pom.xml is affected too but only
slightly - properties etc.
Do/will the integration submodules need different classpath? BTW slight
differences could be handled by tweaking deps in profiles.

Regarding moving the test sources - is there anyone seeing any advantage
in having them all in testsuite/integration/src/test/java ?  It looked
quite ellegant solution but if there's really no one liking it, let's
move it :)


3) Motivation driving the modularization

In general, I'd like to:

* Keep the number of modules at rational number - not fine-grained
(difficult pom.xml maintainance), but not keeping it all-in-one at all
costs (as it is now).
* Keep the pom.xml's easy to read and maintain - profiles interwoven
like a fancy bread isn't exactly something I'd like to add features to.
* Keep the pure maven api concise and locical. Please keep in mind that
some configs will need to be distributed across whole testsuite, and
_every_ config "axis" (databases, IPv6, security,...) will need to be
reflected in _every_ module. What's now few-lines pom.xml for each
module may become a big ball of mud, pulling in things from parent
pom.xml's.

I think we can agree on these right?


Regads,
Ondra

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-as7-dev/attachments/20111122/9a8b06a3/attachment.html