[jbosscache-dev] Maven TestNG rewrite complete for JBoss Cache and POJO Cache

[Bela Ban]

We need to do the same for JGroups some day, I created an issue some 
time ago. Channel connection/disconnection reduction would save us a ton 
of time running the entire testsuite

Vladimir Blagojevic wrote:
> Hi,
>
> I tried new NG testsuite and it works great. One question: can TestNG 
> run individual test method in a certain test class from a command 
> line? I guess TestNG plugin for eclipse has this feature?
>
> Good stuff Jason.
>
> Regards,
> Vladimir
>
> Jason T. Greene wrote:
>> Hello Everyone,
>>
>> All 2516 JBoss Cache and POJO Cache tests have been ported from JUnit 
>> to TestNG. Since this involved some significant changes I took the 
>> opportunity to remove all compiler warnings, which notably reduces 
>> the warning noise. I have introduced primary test groups (explained 
>> below) and have assigned the relevant tests.
>>
>> I also have refactored the maven build to handle what I refer to as 
>> permutations. Essentially it allows us to rerun subsections of tests 
>> using different configurations. This will allow us to test things 
>> like different jgroups protocol stacks using the same (but relevant) 
>> tests.
>>
>> There is one small outstanding issue, which is the unreliable and 
>> buggy nature of system output redirection (one of the many maven 
>> surefire bugs). So currently when you execute a testrun all sys out 
>> will go to your terminal.
>>
>>
>> Finally, I have written a detailed overview of the maven + testng 
>> testing mechanics, and have included it in the project README:
>> http://anonsvn.jboss.org/repos/jbosscache/core/trunk/README-Maven.txt
>>
>> I have also attached the additional overview section below.
>>
>> Thanks,
>> -Jason
>>
>>
>> Testing
>> =======
>>
>> Tests are written against the TestNG testing framework. Each test 
>> should belong to one or more group. The group acts as a filter, and 
>> is used to select which tests are ran as part of the maven test 
>> lifecycle. There are 3 groups that are currently in use, but there is 
>> not formal, you can make up any group name if you like.
>>
>> Current Groups
>> --------------
>> * functional - Tests which test the general functionality of JBoss Cache
>> * jgroups - Tests which need to send data on a JGroups Channel
>> * transaction - Tests which use a transaction manager
>>
>> It should be noted that every test should at least be in the 
>> functional group, since this is the default test group that is 
>> executed by maven, and the one that is required to prepare a release.
>>
>> Executing the default test run
>> ------------------------------
>> The default run executes all tests in the functional group. To just 
>> run the tests with txt and xml output the command is:
>>
>> mvn test
>>
>> Alternatively, you can execute the tests AND generate a report with:
>>
>> mvn surefire-report:report
>>
>> If you already have ran a test cycle, and you want to generate a 
>> report off the current reports, then you use the report-only goal, 
>> ike so:
>>
>> mvn surefire-report:report-only
>>
>> Executing different groups
>> --------------------------
>> A group can be executed (using the default configuration) by simply 
>> using the groups property like so:
>>
>> mvn -Dgroups=jgroups surefire-report:report
>>
>> Mutiple groups can also be executed, although if a test is in more 
>> than of the selected groups, it is executed only once:
>>
>> mvn -Dgroups=jgroups,transaction surefire-report:report
>>
>> Executing a single test
>> -----------------------
>> A single test can be executed using the test property. The value is 
>> the short name (not the fully qualified package name) of the test.
>>
>> mvn -Dtest=FqnTest
>>
>> Skipping the test run
>> ---------------------
>> It is sometimes desirable to install the jboss cache package in your 
>> local repository without performing a full test run. To do this, 
>> simply use the maven.test.skip.exec property:
>>
>> mvn -Dmaven.test.skip.exec=true install
>>
>> Again, this is just a shortcut for local use. It SHOULD NEVER BE USED 
>> when releasing. Also, make sure "exec" is included in the property, 
>> if not the tests will not be built, which will prevent a test jar 
>> being produced (POJO Cache needs the Core Cache test jar).
>>
>> Permutations
>> ------------
>> We use the term permutation to describe a group execution against a 
>> particular config. This allows us to test a variety of environments 
>> and configurations without rewriting the same basic test over and 
>> over again. For example, the jgroups-tcp permutation executes the 
>> jgroups group using the TCP config. Each permutation requires a maven 
>> profile which defines the various options, environmental variables, 
>> etc. The command to run the jgroups-tcp permutatin is:
>>
>> mvn -Pjgroups-tcp surefire-report:report
>>
>> Each permutation uses its own report directory, and its own html 
>> output file name. This allows you to execute multiple permutations 
>> without wiping the results from the previous run. Note that due to 
>> the way maven  operates, only one permutation can be executed per mvn 
>> command. So automating multiple runs requires shell scripting, or 
>> some other execution framework to make multiple called to maven.
>>
>>
>>
>> _______________________________________________
>> jbosscache-dev mailing list
>> jbosscache-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosscache-dev
>
> _______________________________________________
> jbosscache-dev mailing list
> jbosscache-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosscache-dev

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss - a division of Red Hat