TS: Define testsuite acceptance test (a testsuite for the testsuite) -------------------------------------------------------------------- Key: WFLY-607 URL: https://issues.jboss.org/browse/WFLY-607 Project: WildFly Issue Type: Sub-task Reporter: Ondrej Zizka Assignee: Ondrej Zizka Define a procedure to perform before committing something to the testsuite. Create scripts to help determining whether given testsuite is okay to be committed. E.g. {code} IN='-DallTests -Dinteg-tests -Dcluster-tests -Dbasic-tests -Dbenchmark-tests -Dsmoke-tests -Dstress-tests -Ddomain-tests -Dcompat-tests'; for GROUP in $(echo $IN | tr ";" "\n"); do ## Create results dir. RES_DIR=~/work/AS7/res/$GROUP mkdir -p $RES_DIR ## Run the testsuite. ./integration-tests.sh clean -DallTests; ./integration-tests.sh install $GROUP; ## Archive the reports. for REPORT in `find testsuite -name 'TEST-*.xml' -or -name '*TestCase.txt' -or -name '*TestCase-output.txt'`; do SUB_DIR=`dirname $REPORT | sed 's#testsuite/##' | sed 's#/target/surefire-reports##'` mkdir -p $RES_DIR/$SUB_DIR cp $REPORT $RES_DIR/$SUB_DIR done ## Run the testsuite for 2nd time to ensure that unclean run works too. ./integration-tests.sh install $GROUP; done ## Create a JUnitDiff report. {code} Also, check what's being passed to `mvn` on various params combinations: {code} ./build.sh -DallTests ./build.sh -DallTests -Dfoo ./build.sh install -DallTests ./build.sh clean -DallTests ./build.sh clean install -DallTests -Dfoo ./integration-tests.sh -DallTests; ./integration-tests.sh install -DallTests; ./integration-tests.sh clean -DallTests; ./integration-tests.sh clean install -DallTests -Dfoo; ./integration-tests.sh test -DallTests -Dfoo; ... {code}