[jbosscache-dev] Maven TestNG rewrite complete for JBoss Cache and POJO Cache

[Vladimir Blagojevic]

Hi,

I tried new NG testsuite and it works great. One question: can TestNG 
run individual test method in a certain test class from a command line? 
I guess TestNG plugin for eclipse has this feature?

Good stuff Jason.

Regards,
Vladimir

Jason T. Greene wrote:
> Hello Everyone,
>
> All 2516 JBoss Cache and POJO Cache tests have been ported from JUnit 
> to TestNG. Since this involved some significant changes I took the 
> opportunity to remove all compiler warnings, which notably reduces the 
> warning noise. I have introduced primary test groups (explained below) 
> and have assigned the relevant tests.
>
> I also have refactored the maven build to handle what I refer to as 
> permutations. Essentially it allows us to rerun subsections of tests 
> using different configurations. This will allow us to test things like 
> different jgroups protocol stacks using the same (but relevant) tests.
>
> There is one small outstanding issue, which is the unreliable and 
> buggy nature of system output redirection (one of the many maven 
> surefire bugs). So currently when you execute a testrun all sys out 
> will go to your terminal.
>
>
> Finally, I have written a detailed overview of the maven + testng 
> testing mechanics, and have included it in the project README:
> http://anonsvn.jboss.org/repos/jbosscache/core/trunk/README-Maven.txt
>
> I have also attached the additional overview section below.
>
> Thanks,
> -Jason
>
>
> Testing
> =======
>
> Tests are written against the TestNG testing framework. Each test 
> should belong to one or more group. The group acts as a filter, and is 
> used to select which tests are ran as part of the maven test 
> lifecycle. There are 3 groups that are currently in use, but there is 
> not formal, you can make up any group name if you like.
>
> Current Groups
> --------------
> * functional - Tests which test the general functionality of JBoss Cache
> * jgroups - Tests which need to send data on a JGroups Channel
> * transaction - Tests which use a transaction manager
>
> It should be noted that every test should at least be in the 
> functional group, since this is the default test group that is 
> executed by maven, and the one that is required to prepare a release.
>
> Executing the default test run
> ------------------------------
> The default run executes all tests in the functional group. To just 
> run the tests with txt and xml output the command is:
>
> mvn test
>
> Alternatively, you can execute the tests AND generate a report with:
>
> mvn surefire-report:report
>
> If you already have ran a test cycle, and you want to generate a 
> report off the current reports, then you use the report-only goal, ike 
> so:
>
> mvn surefire-report:report-only
>
> Executing different groups
> --------------------------
> A group can be executed (using the default configuration) by simply 
> using the groups property like so:
>
> mvn -Dgroups=jgroups surefire-report:report
>
> Mutiple groups can also be executed, although if a test is in more 
> than of the selected groups, it is executed only once:
>
> mvn -Dgroups=jgroups,transaction surefire-report:report
>
> Executing a single test
> -----------------------
> A single test can be executed using the test property. The value is 
> the short name (not the fully qualified package name) of the test.
>
> mvn -Dtest=FqnTest
>
> Skipping the test run
> ---------------------
> It is sometimes desirable to install the jboss cache package in your 
> local repository without performing a full test run. To do this, 
> simply use the maven.test.skip.exec property:
>
> mvn -Dmaven.test.skip.exec=true install
>
> Again, this is just a shortcut for local use. It SHOULD NEVER BE USED 
> when releasing. Also, make sure "exec" is included in the property, if 
> not the tests will not be built, which will prevent a test jar being 
> produced (POJO Cache needs the Core Cache test jar).
>
> Permutations
> ------------
> We use the term permutation to describe a group execution against a 
> particular config. This allows us to test a variety of environments 
> and configurations without rewriting the same basic test over and over 
> again. For example, the jgroups-tcp permutation executes the jgroups 
> group using the TCP config. Each permutation requires a maven profile 
> which defines the various options, environmental variables, etc. The 
> command to run the jgroups-tcp permutatin is:
>
> mvn -Pjgroups-tcp surefire-report:report
>
> Each permutation uses its own report directory, and its own html 
> output file name. This allows you to execute multiple permutations 
> without wiping the results from the previous run. Note that due to the 
> way maven  operates, only one permutation can be executed per mvn 
> command. So automating multiple runs requires shell scripting, or some 
> other execution framework to make multiple called to maven.
>
>
>
> _______________________________________________
> jbosscache-dev mailing list
> jbosscache-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosscache-dev