[infinispan-dev] test number variations

[Sanne Grinovero]

Thanks Pete!
that must be the reason; I'm glad nobody deleted 200 tests yesterday ;-)

I would prefer JUnit too, definitely more reliable and I'm loving the
latest features of it such as @Rule: makes it almost impossible to
forget proper test clean-up and fights inheritance complexity.

Cheers,
Sanne

On 1 June 2012 11:56, Pete Muir <pmuir at redhat.com> wrote:
> TestNG/surefire in combo are dreadful at counting the number of tests. If you get errors, it will start counting failed before/after methods as well as real test methods. You basically shouldn't trust the count from any run except one where every test passed.
>
> I would strongly advise using JUnit for this, and many other reasons:
>
> * much better hooks into surefire, eclipse etc. etc.
> * much more stable (rarely do I see bugs like this in it)
> * you can actually debug the JUnit code without going blind
> * much more flexible / extensible, due to use of runners
>
> BUT it doesn't have the built in features TestNG has...
>
> On 1 Jun 2012, at 10:59, Sanne Grinovero wrote:
>
>> On 1 June 2012 10:54, Manik Surtani <manik at jboss.org> wrote:
>>> Could be due to a bad check-in where someone annotates a test with @Test(invocationCount = xxxx) - I have seen this a few times.
>>
>> Maybe, but you didn't convince me. This picture is way too unstable
>> (overall testsuite) :
>>
>> https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-tcp/test/?width=800&height=600
>>
>> I'd rather think surefire / jenkins / testng / our listeners can't
>> count correctly.
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev