Hi everyone, as was discussed on this list, the current testsuite harness had some problems fulfilling even basic requirements for various use cases, including but not limited to: * Using from IDE through pure maven * -DallTests vs. individial modules vs. distinctive groups vs. even overlaying groups * Configuring AS using kind of hierarchical templates * Imposing certain test groups order * Invoking only some steps of the testsuite * Running same tests with different AS configuration etc etc. I've been working on an updated concept, which is currently at https://github.com/OndraZizka/jboss-as/commits/TS-modules-tmp . I would like you to review it, run it, and send me some comments. cd testsuite/integration mvn clean install -f pom2.xml -DallTests -fae mvn clean install -f pom2.xml -Dts.iiop (There are 2 pom.xml's so I can compare two harnesses easily.) The scripts are not updated yet so they do not work. CLI and OSGi tests fail. To be sorted out. The big news are: * Execution split to modules * Code and resources remained in the same location * Added few system properties which provide paths to important directories - see https://docs.jboss.org/author/display/AS71/AS+7 +Testsuite+Test+Developer+Guide * Work directory is now in target/workdir These cahnges exposed some issues in test code which need to be fixed - hardcoded relative paths. Since there were no system properties to use, there was no other way, but from now on please use those properties, and fix your existing tests. (This would happen anyway because some tests clutter workdir (which happens to be module's root dir) with various temp files.) Thanks, Ondra _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Would it be possible to create a design document (wiki) that lists the current testsuite requirements together with which of those reqs are fulfilled already? In the past we often had changes to the testsuite happen that introduced new stuff but at the same time broke working use cases. -thomas On 11/11/2011 04:50 AM, Ondřej Žižka wrote: > Hi everyone, > > as was discussed on this list, the current testsuite harness had > some problems fulfilling even basic requirements for various use > cases, including but not limited to: > * Using from IDE through pure maven > * -DallTests vs. individial modules vs. distinctive groups vs. even > overlaying groups > * Configuring AS using kind of hierarchical templates > * Imposing certain test groups order > * Invoking only some steps of the testsuite > * Running same tests with different AS configuration > etc etc. > > I've been working on an updated concept, which is currently at > https://github.com/OndraZizka/jboss-as/commits/TS-modules-tmp . > I would like you to review it, run it, and send me some comments. > > cd testsuite/integration > mvn clean install -f pom2.xml -DallTests -fae > mvn clean install -f pom2.xml -Dts.iiop > > (There are 2 pom.xml's so I can compare two harnesses easily.) > The scripts are not updated yet so they do not work. > CLI and OSGi tests fail. To be sorted out. > > > The big news are: > > * Execution split to modules > * Code and resources remained in the same location > * Added few system properties which provide paths to important > directories - see https://docs.jboss.org/author/display/AS71/AS+7 > +Testsuite+Test+Developer+Guide > * Work directory is now in target/workdir > > These cahnges exposed some issues in test code which need to be > fixed - hardcoded relative paths. > Since there were no system properties to use, there was no other > way, but from now on please use those properties, and fix your > existing tests. > (This would happen anyway because some tests clutter workdir (which > happens to be module's root dir) with various temp files.) > > > Thanks, > Ondra > > > > > > > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Thomas Diesler JBoss OSGi Lead JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Here you are: AS Testsuite Requirements page http://community.jboss.org/wiki/ASTestsuiteRequirements Consider that as a workbench, the document shall evolve into something shared between all stakeholders. Regards, Ondra Thomas Diesler píše v Út 15. 11. 2011 v 14:10 +0000: > Would it be possible to create a design document (wiki) that lists > the current testsuite requirements together with which of those reqs > are fulfilled already? In the past we often had changes to the > testsuite happen that introduced new stuff but at the same time > broke working use cases. > > -thomas > > On 11/11/2011 04:50 AM, Ondřej Žižka wrote: > > > Hi everyone, > > > > as was discussed on this list, the current testsuite harness had > > some problems fulfilling even basic requirements for various use > > cases, including but not limited to: > > * Using from IDE through pure maven > > * -DallTests vs. individial modules vs. distinctive groups vs. > > even overlaying groups > > * Configuring AS using kind of hierarchical templates > > * Imposing certain test groups order > > * Invoking only some steps of the testsuite > > * Running same tests with different AS configuration > > etc etc. > > > > I've been working on an updated concept, which is currently at > > https://github.com/OndraZizka/jboss-as/commits/TS-modules-tmp . > > I would like you to review it, run it, and send me some comments. > > > > cd testsuite/integration > > mvn clean install -f pom2.xml -DallTests -fae > > mvn clean install -f pom2.xml -Dts.iiop > > > > (There are 2 pom.xml's so I can compare two harnesses easily.) > > The scripts are not updated yet so they do not work. > > CLI and OSGi tests fail. To be sorted out. > > > > > > The big news are: > > > > * Execution split to modules > > * Code and resources remained in the same location > > * Added few system properties which provide paths to important > > directories - see https://docs.jboss.org/author/display/AS71/AS+7 > > +Testsuite+Test+Developer+Guide > > * Work directory is now in target/workdir > > > > These cahnges exposed some issues in test code which need to be > > fixed - hardcoded relative paths. > > Since there were no system properties to use, there was no other > > way, but from now on please use those properties, and fix your > > existing tests. > > (This would happen anyway because some tests clutter workdir > > (which happens to be module's root dir) with various temp files.) > > > > > > Thanks, > > Ondra > > > > > > > > > > > > > > _______________________________________________ > > jboss-as7-dev mailing list > > jboss-as7-dev(a)lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev > > > > -- > xxxxxxxxxxxxxxxxxxxxxxxxxxxx > Thomas Diesler > JBoss OSGi Lead > JBoss, a division of Red Hat > xxxxxxxxxxxxxxxxxxxxxxxxxxxx