Sanne,
I've looked at your listener approach. I think this is the best we can
do for now but is there a potential problem with listener approach
because we sometimes create cache managers on per test method and
sometimes on per test class basis? Can listener handle cases where each
test method creates a few cache managers and does not close them properly?
Cheers
On 10-01-02 11:47 PM, Sanne Grinovero wrote:
Hi Vladimir,
thanks for the tips, please have a look into what I've already done,
suggestions on improving are welcome:
http://fisheye.jboss.org/changelog/Infinispan?cs=1335
The limitation is that it can't make tests fail, it can only print
warnings to the logs.
I didn't use @afterclass because it relies on the test itself to
verify it's correctly written: it doesn't solve the problem of
spotting an incorrectly written test.
(I could as well open them all to have to verify they all extend the
same parent test...)
I believe we can run the manual test when we spot "random behavior"
that make you think some test is misbehaving, and keep it disabled
other times.
As discussed on a testng forum thread (*1) it doesn't appear to be
possible to change the test result from a listener, I tried even with
a RuntimeException but it's ignoring them.
Anyway the results of my effort uncovered just one misbehaving test in
the query module; the tests currently failing in core are failing for
other reasons.
Hudson (the public one I can see *2) is stuck since 22 December, so I
can't really see how stable the current testing results are. Could
someone start it?
Cheers,
Sanne
*1
http://groups.google.com/group/testng-users/browse_thread/thread/c07dee91...
*2
http://hudson.jboss.org/hudson/view/Infinispan/job/Infinispan-trunk-JDK6-...
2010/1/2 Vladimir Blagojevic<vblagoje(a)redhat.com>:
> Sanne, we can use probe utility from JGroups to ping if there are any
> remaining jgroups members alive ( and therefore CacheManagers). We can
> do this verification from @afterClass in superclass of all test classes.
> We just have to somehow record which ports were used by particular
> TestNG running thread. I will look into this problem as well. We have to
> do verification somehow automatically, anything else is a futile
> nightmare....
>
> On 09-12-29 8:43 PM, Sanne Grinovero wrote:
>
>> I've found that still some tests are not properly killing the
>> CacheManager, making other random tests fail,
>> so I wrote an extension to Infinispan's TestNG listener to identify
>> the guilty implementations.
>>
>> I opened ISPN-323 to track this.
>>
>> To identify a non-cleaning test the testNG listener will create a new
>> CacheManager and verify the number of members,
>> so this is slowing down tests; to not annoy you it's only enabled in a
>> new profile "debug-tests":
>>
>> mvn -Pdebug-tests clean test
>>
>> Also keep in mind that, being a listener, it can't make the test fail:
>> to find the tests you'll have to grep the build output
>> for "CacheManagers alive after test!".
>>
>> We still need to fix the identified tests, I'll create subtasks of
>> ISPN-323 for that.
>>
>> Cheers,
>> Sanne
>>
>>
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev