[infinispan-dev] Creating cache managers in your unit tests

[Vladimir Blagojevic]

Sanne,

I've looked at your listener approach. I think this is the best we can 
do for now but is there a potential problem with listener approach 
because we sometimes create cache managers on per test method and 
sometimes on per test class basis? Can listener handle cases where each 
test method creates a few cache managers and does not close them properly?

Cheers

On 10-01-02 11:47 PM, Sanne Grinovero wrote:
> Hi Vladimir,
> thanks for the tips, please have a look into what I've already done,
> suggestions on improving are welcome:
> http://fisheye.jboss.org/changelog/Infinispan?cs=1335
>
> The limitation is that it can't make tests fail, it can only print
> warnings to the logs.
> I didn't use @afterclass because it relies on the test itself to
> verify it's correctly written: it doesn't solve the problem of
> spotting an incorrectly written test.
> (I could as well open them all to have to verify they all extend the
> same parent test...)
>
> I believe we can run the manual test when we spot "random behavior"
> that make you think some test is misbehaving, and keep it disabled
> other times.
>
> As discussed on a testng forum thread (*1) it doesn't appear to be
> possible to change the test result from a listener, I tried even with
> a RuntimeException but it's ignoring them.
>
> Anyway the results of my effort uncovered just one misbehaving test in
> the query module; the tests currently failing in core are failing for
> other reasons.
> Hudson (the public one I can see *2) is stuck since 22 December, so I
> can't really see how stable the current testing results are. Could
> someone start it?
>
> Cheers,
> Sanne
>
> *1 http://groups.google.com/group/testng-users/browse_thread/thread/c07dee9123c45d52/b525007bed19febd
> *2 http://hudson.jboss.org/hudson/view/Infinispan/job/Infinispan-trunk-JDK6-tcp/
>
>
>
> 2010/1/2 Vladimir Blagojevic<vblagoje at redhat.com>:
>    
>> Sanne, we can use probe utility from JGroups to ping if there are any
>> remaining jgroups members alive ( and therefore CacheManagers). We can
>> do this verification from @afterClass in superclass of all test classes.
>> We just have to somehow record which ports were used by particular
>> TestNG running thread. I will look into this problem as well. We have to
>> do verification somehow automatically, anything else is a futile
>> nightmare....
>>
>> On 09-12-29 8:43 PM, Sanne Grinovero wrote:
>>      
>>> I've found that still some tests are not properly killing the
>>> CacheManager, making other random tests fail,
>>> so I wrote an extension to Infinispan's TestNG listener to identify
>>> the guilty implementations.
>>>
>>> I opened ISPN-323 to track this.
>>>
>>> To identify a non-cleaning test the testNG listener will create a new
>>> CacheManager and verify the number of members,
>>> so this is slowing down tests; to not annoy you it's only enabled in a
>>> new profile "debug-tests":
>>>
>>> mvn -Pdebug-tests clean test
>>>
>>> Also keep in mind that, being a listener, it can't make the test fail:
>>> to find the tests you'll have to grep the build output
>>> for "CacheManagers alive after test!".
>>>
>>> We still need to fix the identified tests, I'll create subtasks of
>>> ISPN-323 for that.
>>>
>>> Cheers,
>>> Sanne
>>>
>>>
>>>        
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>      
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>