On 11/22/11 2:31 PM, Ondřej Žižka wrote: > Hi, > > 1) Relevant resources: > > Tracking jira: https://issues.jboss.org/browse/AS7-2007 > Requirements: http://community.jboss.org/wiki/ASTestsuiteRequirements > Docs: https://docs.jboss.org/author/display/AS71/Testsuite > > > 2) Modularization & moving the test sources > > > This is one thing I am wondering. If we all agree that many smaller > modules is the way to go, since its the standard maven approach > (supported by all existing maven tools), and it solves that various > issues introduced by doing the grouping approach then maybe we should > just do one refactor instead of doing this one and refactoring it yet again? > > I'd like to see the problems with the solution in PR 710. Please send me > instructions how to check that the IDE has problems coping with test > sources on ../src/test/java. > (I don't know about Eclipse or IDEA, but NetBeans uses quite rational > approach, which is simply running Maven for the active module. Which works. > It works for smoke, for -DallTests if you add that param, and also when > running a single test. I haven't tried debugging yet but in worst case, > I'd run with -Ddebug and attach the debugger. ) I think its the same issues Stuart already brought up a few weeks ago. One megaclasspath, larger build footprint, consistency with the modules directory etc. > > > I know you put a lot of effort into this, but there is a good chance > this may net less work in the end since we wont be constantly tweaking > this version. > > It's not like switching from version A to B1 or B2, it's like staying at > B (pull request 710) or going further to C (which would be moving > sources to modules). > This harness is quite ready for both alternatives. > The reason I am proposing this as an intermediate state is, as I > previously stated, merely technical: Moving thousands of files is a big > operation from SCM PoV, so I want to get this tests relocation get > merged as quickly as possible - i.e. move, test, make pull request, get > it reviewed and merged, at best in a single day, otherwise I'll drown > with new tests coming. I guess I don't see what if anything in this pull we would want to keep if we went to a small module approach? Maybe you could explain that? > > > Or is there some set of requirements we have to meet ASAP that our > existing testsuite infrastructure doesnt cover? > > There are many problems - I've mentioned them before - like: > * Smoke profile can either be default and need to be disabled > explicitely for any other group; but in that case many people will > forget to disable it which causes inference with other profiles. > Or it would have to be defined explicitely - i.e. "no smoke tests run by > default" > * Huge unmaintainable pom.xml > * Impossible to define multiple groups These all sound livable for awhile to me. > * Hard to separate web/full or even make all tests run with full > Etc etc. See the resources above, and the issues discussed on as7 list > recently. This works today in master. I just added multiple executions. > > > Although the problem stuart brought up is that we can't have > different classpaths in a single testsrun. > > As an example compatibility tests would likley not be able to be ran > in this way. > > Compat tests are in different module anyway. This change is refactoring > the integration module. The testsuite/pom.xml is affected too but only > slightly - properties etc. > Do/will the integration submodules need different classpath? BTW slight > differences could be handled by tweaking deps in profiles. Yeah so this is double maintenance, both of these have to be in sync right? > > Regarding moving the test sources - is there anyone seeing any advantage > in having them all in testsuite/integration/src/test/java ? It looked > quite ellegant solution but if there's really no one liking it, let's > move it :) < > > 3) Motivation driving the modularization > > In general, I'd like to: > > * Keep the number of modules at rational number - not fine-grained > (difficult pom.xml maintainance), but not keeping it all-in-one at all > costs (as it is now). > * Keep the pom.xml's easy to read and maintain - profiles interwoven > like a fancy bread isn't exactly something I'd like to add features to. > * Keep the pure maven api concise and locical. Please keep in mind that > some configs will need to be distributed across whole testsuite, and > _every_ config "axis" (databases, IPv6, security,...) will need to be > reflected in _every_ module. What's now few-lines pom.xml for each > module may become a big ball of mud, pulling in things from parent > pom.xml's. Yeah i think referencing external files is a nice solution to this problem.

> > There are many problems - I've mentioned them before - like: > > * Smoke profile can either be default and need to be disabled > > explicitely for any other group; but in that case many people will > > forget to disable it which causes inference with other profiles. > > Or it would have to be defined explicitely - i.e."no smoke tests run by > > default" > > * Huge unmaintainable pom.xml > > * Impossible to define multiple groups > > These all sound livable for awhile to me. AS 6 sounded livable for a while to some. > > * Hard to separate web/full or even make all tests run with full > > Etc etc. See the resources above, and the issues discussed on as7 list > > recently. > > This works today in master. I just added multiple executions. Okay, so it was ready for three days. I understand that coding is more fun than merging but what's the point of duplicating work and patching unsatisfactory harness?

> > Compat tests are in different module anyway. This change is refactoring > > the integration module. The testsuite/pom.xml is affected too but only > > slightly - properties etc. > > Do/will the integration submodules need different classpath? BTW slight > > differences could be handled by tweaking deps in profiles. > > Yeah so this is double maintenance, both of these have to be in sync right? I don't understand - what's doubled here, opposite to having two pom.xml's? It's much less code this way.

> > Regarding moving the test sources - is there anyone seeing any advantage > > in having them all in testsuite/integration/src/test/java ? It looked > > quite ellegant solution but if there's really no one liking it, let's > > move it :) Seems like you skipped this paragraph, I recommend the last sentence to your attention. Following it will take us to "MORE SMALL MODULES". > Please keep in mind that > > some configs will need to be distributed across whole testsuite, and > > _every_ config"axis" (databases, IPv6, security,...) will need to be > > reflected in _every_ module. What's now few-lines pom.xml for each > > module may become a big ball of mud, pulling in things from parent > > pom.xml's. > > Yeah i think referencing external files is a nice solution to this problem. No. You can't externalize pom.xml's code to pass tons of properties to surefire and to Arquillian (which needs them duplicated right in pom.xml).

I've tried a few different combinations and to me it seems like we're on the right track functionality-wise. mvn clean install -> runs smoke tests as expected -- mvn clean install -DallTests -> runs all tests as expected -- mvn clean install -DallTests=true -> runs all tests as expected -- mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase -> runs the one test as expected (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules) -- cd testsuite/integration/group-basic mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase -> runs the one test as expected -- cd testsuite/integration/group-basic mvn install -> Runs those tests as expected -- Disclaimer: I have not verified that the correct tests are in the correct 'groups' and that every single test that used to get run runs :-) From what I can tell, once the test classes get broken out into their individual modules , which I believe Ondrej says is the next step, we should be ok to run these from the IDE as well (I have not tried if this works already). I need to go away for a few hours, when I get back I'll look a bit more at the structure of the poms/arquillian.xml files etc. On 23 Nov 2011, at 03:50, Jason T. Greene wrote: > On 11/22/11 5:19 PM, Ondřej Žižka wrote: >>>> There are many problems - I've mentioned them before - like: >>>> * Smoke profile can either be default and need to be disabled >>>> explicitely for any other group; but in that case many people will >>>> forget to disable it which causes inference with other profiles. >>>> Or it would have to be defined explicitely - i.e."no smoke tests run by >>>> default" >>>> * Huge unmaintainable pom.xml >>>> * Impossible to define multiple groups >>> >>> These all sound livable for awhile to me. >> AS 6 sounded livable for a while to some. >> >>>> * Hard to separate web/full or even make all tests run with full >>>> Etc etc. See the resources above, and the issues discussed on as7 list >>>> recently. >>> >>> This works today in master. I just added multiple executions. >> Okay, so it was ready for three days. I understand that coding is more >> fun than merging but what's the point of duplicating work and patching >> unsatisfactory harness? > > Because its a priority for beta, and a major refactor is not, especially > one that has little consensus. > >> >>>> Compat tests are in different module anyway. This change is refactoring >>>> the integration module. The testsuite/pom.xml is affected too but only >>>> slightly - properties etc. >>>> Do/will the integration submodules need different classpath? BTW slight >>>> differences could be handled by tweaking deps in profiles. >>> >>> Yeah so this is double maintenance, both of these have to be in sync right? >> I don't understand - what's doubled here, opposite to having two >> pom.xml's? It's much less code this way. > > The resources/classpath/properties/etc must be duplicated in both the > maven module that "owns" the classes and the group module. > > >> >>>> Regarding moving the test sources - is there anyone seeing any advantage >>>> in having them all in testsuite/integration/src/test/java ? It looked >>>> quite ellegant solution but if there's really no one liking it, let's >>>> move it :) >> Seems like you skipped this paragraph, I recommend the last sentence to >> your attention. Following it will take us to "MORE SMALL MODULES". >> >>> Please keep in mind that >>>> some configs will need to be distributed across whole testsuite, and >>>> _every_ config"axis" (databases, IPv6, security,...) will need to be >>>> reflected in _every_ module. What's now few-lines pom.xml for each >>>> module may become a big ball of mud, pulling in things from parent >>>> pom.xml's. >>> >>> Yeah i think referencing external files is a nice solution to this problem. >> No. You can't externalize pom.xml's code to pass tons of properties to >> surefire and to Arquillian (which needs them duplicated right in pom.xml). > > Most of the "axis" entries you have above seem to me to be AS > configuration and arq configuratio which is a separate file. > > > -- > Jason T. Greene > JBoss AS Lead / EAP Platform Architect > JBoss, a division of Red Hat > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev