[infinispan-dev] test number variations

[Pete Muir]

TestNG/surefire in combo are dreadful at counting the number of tests. If you get errors, it will start counting failed before/after methods as well as real test methods. You basically shouldn't trust the count from any run except one where every test passed.

I would strongly advise using JUnit for this, and many other reasons:

* much better hooks into surefire, eclipse etc. etc.
* much more stable (rarely do I see bugs like this in it)
* you can actually debug the JUnit code without going blind
* much more flexible / extensible, due to use of runners

BUT it doesn't have the built in features TestNG has...

On 1 Jun 2012, at 10:59, Sanne Grinovero wrote:

> On 1 June 2012 10:54, Manik Surtani <manik at jboss.org> wrote:
>> Could be due to a bad check-in where someone annotates a test with @Test(invocationCount = xxxx) - I have seen this a few times.
> 
> Maybe, but you didn't convince me. This picture is way too unstable
> (overall testsuite) :
> 
> https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-tcp/test/?width=800&height=600
> 
> I'd rather think surefire / jenkins / testng / our listeners can't
> count correctly.
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev