[infinispan-dev] test number variations
Pete Muir
pmuir at redhat.com
Fri Jun 1 06:56:22 EDT 2012
TestNG/surefire in combo are dreadful at counting the number of tests. If you get errors, it will start counting failed before/after methods as well as real test methods. You basically shouldn't trust the count from any run except one where every test passed.
I would strongly advise using JUnit for this, and many other reasons:
* much better hooks into surefire, eclipse etc. etc.
* much more stable (rarely do I see bugs like this in it)
* you can actually debug the JUnit code without going blind
* much more flexible / extensible, due to use of runners
BUT it doesn't have the built in features TestNG has...
On 1 Jun 2012, at 10:59, Sanne Grinovero wrote:
> On 1 June 2012 10:54, Manik Surtani <manik at jboss.org> wrote:
>> Could be due to a bad check-in where someone annotates a test with @Test(invocationCount = xxxx) - I have seen this a few times.
>
> Maybe, but you didn't convince me. This picture is way too unstable
> (overall testsuite) :
>
> https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-tcp/test/?width=800&height=600
>
> I'd rather think surefire / jenkins / testng / our listeners can't
> count correctly.
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
More information about the infinispan-dev
mailing list