[jbosscache-dev] Parallelizing TestNG tests

[Bela Ban]

In a nutshell:

    * All of our stack-dependent tests inherits a common base class with
      functionality to create 'unique' stacks
    * All tests are run by a thread pool
    * If a test class has methods who need to be run sequentially, we
      define @Test(sequential=true) (default=false)
    * Each stack dependent test creates a unique stack by creating a
      default stack, then changing the mcast_addr/mcast_port (UDP) or
      the bind_port (TCP)
    * This is only done for the first channel, subsequent channels (in
      the same test) need to use the same config, so we create
      subsequent channel by passing a ref to the first channel.
      Subsequent channels then simply copy the config of the first channel

This works very well, although there were some quirks in TestNG we had 
to work around. The test time for all-tests-cc in JGroups (functional 
tests, stack-independent and stack-dependent (4 different stacks) went 
down from 2.5 hours to 13 minutes ! This is still running the test 
suites sequentially, I expect to get under 5 minutes when switching to 
parallel


Manik Surtani wrote:
> Vladimir/Bela,
>
> Could you guys talk briefly on how you implemented parallelizing your 
> TestNG tests, making use of dynamically changing channel ports to 
> allow clustered tests to run concurrently?  Is any of this reusable?
>
> I've created a JIRA for JBC [1] to make use of this if possible.  2+ 
> hrs for our test suite is hugely annoying.  :-)
>
> Thanks
> Manik
>
> [1] http://jira.jboss.org/jira/browse/JBCACHE-1386
>

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss - a division of Red Hat