[jboss-as7-dev] Testsuite modularization? // Re: Testsuite refactor

[Ondřej Žižka]

Jason, before I write the long reply - did you read the whole post?
Maybe it got lost in sauce sentences like this, so let me emphasisze it:

Pull request 710 is the "MULTIPLE SMALL MODULES" way.

Let me know if you don't agree so I can start explaining from the right
angle.

Ondra





Jason T. Greene píše v Út 22. 11. 2011 v 14:49 -0600:

> On 11/22/11 2:31 PM, Ondřej Žižka wrote:
> > Hi,
> >
> > 1) Relevant resources:
> >
> > Tracking jira: https://issues.jboss.org/browse/AS7-2007
> > Requirements: http://community.jboss.org/wiki/ASTestsuiteRequirements
> > Docs: https://docs.jboss.org/author/display/AS71/Testsuite
> >
> >
> > 2) Modularization & moving the test sources
> >
> >  > This is one thing I am wondering. If we all agree that many smaller
> > modules is the way to go, since its the standard maven approach
> > (supported by all existing maven tools), and it solves that various
> > issues introduced by doing the grouping approach then maybe we should
> > just do one refactor instead of doing this one and refactoring it yet again?
> >
> > I'd like to see the problems with the solution in PR 710. Please send me
> > instructions how to check that the IDE has problems coping with test
> > sources on ../src/test/java.
> > (I don't know about Eclipse or IDEA, but NetBeans uses quite rational
> > approach, which is simply running Maven for the active module. Which works.
> > It works for smoke, for -DallTests if you add that param, and also when
> > running a single test. I haven't tried debugging yet but in worst case,
> > I'd run with -Ddebug and attach the debugger.
> )
> 
> I think its the same issues Stuart already brought up a few weeks ago. 
> One megaclasspath, larger build footprint, consistency with the modules 
> directory etc.
> 
> >
> >  > I know you put a lot of effort into this, but there is a good chance
> > this may net less work in the end since we wont be constantly tweaking
> > this version.
> >
> > It's not like switching from version A to B1 or B2, it's like staying at
> > B (pull request 710) or going further to C (which would be moving
> > sources to modules).
> > This harness is quite ready for both alternatives.
> > The reason I am proposing this as an intermediate state is, as I
> > previously stated, merely technical: Moving thousands of files is a big
> > operation from SCM PoV, so I want to get this tests relocation get
> > merged as quickly as possible - i.e. move, test, make pull request, get
> > it reviewed and merged, at best in a single day, otherwise I'll drown
> > with new tests coming.
> 
> I guess I don't see what if anything in this pull we would want to keep 
> if we went to a small module approach?
> 
>   Maybe you could explain that?
> 
> >
> >  > Or is there some set of requirements we have to meet ASAP that our
> > existing testsuite infrastructure doesnt cover?
> >
> > There are many problems - I've mentioned them before - like:
> > * Smoke profile can either be default and need to be disabled
> > explicitely for any other group; but in that case many people will
> > forget to disable it which causes inference with other profiles.
> > Or it would have to be defined explicitely - i.e. "no smoke tests run by
> > default"
> > * Huge unmaintainable pom.xml
> > * Impossible to define multiple groups
> 
> These all sound livable for awhile to me.
> 
> > * Hard to separate web/full or even make all tests run with full
> > Etc etc. See the resources above, and the issues discussed on as7 list
> > recently.
> 
> This works today in master. I just added multiple executions.
> 
> >
> >  > Although the problem stuart brought up is that we can't have
> > different classpaths in a single testsrun.
> >  > As an example compatibility tests would likley not be able to be ran
> > in this way.
> >
> > Compat tests are in different module anyway. This change is refactoring
> > the integration module. The testsuite/pom.xml is affected too but only
> > slightly - properties etc.
> > Do/will the integration submodules need different classpath? BTW slight
> > differences could be handled by tweaking deps in profiles.
> 
> Yeah so this is double maintenance, both of these have to be in sync right?
> 
> >
> > Regarding moving the test sources - is there anyone seeing any advantage
> > in having them all in testsuite/integration/src/test/java ? It looked
> > quite ellegant solution but if there's really no one liking it, let's
> > move it :)
> <
> >
> > 3) Motivation driving the modularization
> >
> > In general, I'd like to:
> >
> > * Keep the number of modules at rational number - not fine-grained
> > (difficult pom.xml maintainance), but not keeping it all-in-one at all
> > costs (as it is now).
> > * Keep the pom.xml's easy to read and maintain - profiles interwoven
> > like a fancy bread isn't exactly something I'd like to add features to.
> > * Keep the pure maven api concise and locical. Please keep in mind that
> > some configs will need to be distributed across whole testsuite, and
> > _every_ config "axis" (databases, IPv6, security,...) will need to be
> > reflected in _every_ module. What's now few-lines pom.xml for each
> > module may become a big ball of mud, pulling in things from parent
> > pom.xml's.
> 
> Yeah i think referencing external files is a nice solution to this problem.
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-as7-dev/attachments/20111123/fce4b3c7/attachment-0001.html