[infinispan-dev] Creating cache managers in your unit tests

[Manik Surtani]

On 4 Jan 2010, at 10:48, Sanne Grinovero wrote:

> below:
> 
> 2010/1/4 Vladimir Blagojevic <vblagoje at redhat.com>:
>> Sanne,
>> 
>> I've looked at your listener approach. I think this is the best we can
>> do for now but is there a potential problem with listener approach
>> because we sometimes create cache managers on per test method and
>> sometimes on per test class basis? Can listener handle cases where each
>> test method creates a few cache managers and does not close them properly?
> 
> no the test still needs to be written using the methods explained by
> Manik in first post of this thread,
> this listener is only useful to detect if some test is forgetting to
> cleanup (kill the cachemanager),
> which was happening and is quite annoying to find.
> If the test is going to create a cachemanager on the wrong port (not
> the one dedicated to running thread), it won't detect it :-(

Hmm, that would be a tough one.  But anyway I think it isn't common to create CacheManagers within a test method (and not register it for cleanup).  Lets start with Sanne's approach, identify guilty tests and fix them, and see where we are.  One step at a time.

Cheers
Manik

> 
> Cheers,
> Sanne
> 
>> 
>> Cheers
>> 
>> On 10-01-02 11:47 PM, Sanne Grinovero wrote:
>>> Hi Vladimir,
>>> thanks for the tips, please have a look into what I've already done,
>>> suggestions on improving are welcome:
>>> http://fisheye.jboss.org/changelog/Infinispan?cs=1335
>>> 
>>> The limitation is that it can't make tests fail, it can only print
>>> warnings to the logs.
>>> I didn't use @afterclass because it relies on the test itself to
>>> verify it's correctly written: it doesn't solve the problem of
>>> spotting an incorrectly written test.
>>> (I could as well open them all to have to verify they all extend the
>>> same parent test...)
>>> 
>>> I believe we can run the manual test when we spot "random behavior"
>>> that make you think some test is misbehaving, and keep it disabled
>>> other times.
>>> 
>>> As discussed on a testng forum thread (*1) it doesn't appear to be
>>> possible to change the test result from a listener, I tried even with
>>> a RuntimeException but it's ignoring them.
>>> 
>>> Anyway the results of my effort uncovered just one misbehaving test in
>>> the query module; the tests currently failing in core are failing for
>>> other reasons.
>>> Hudson (the public one I can see *2) is stuck since 22 December, so I
>>> can't really see how stable the current testing results are. Could
>>> someone start it?
>>> 
>>> Cheers,
>>> Sanne
>>> 
>>> *1 http://groups.google.com/group/testng-users/browse_thread/thread/c07dee9123c45d52/b525007bed19febd
>>> *2 http://hudson.jboss.org/hudson/view/Infinispan/job/Infinispan-trunk-JDK6-tcp/
>>> 
>>> 
>>> 
>>> 2010/1/2 Vladimir Blagojevic<vblagoje at redhat.com>:
>>> 
>>>> Sanne, we can use probe utility from JGroups to ping if there are any
>>>> remaining jgroups members alive ( and therefore CacheManagers). We can
>>>> do this verification from @afterClass in superclass of all test classes.
>>>> We just have to somehow record which ports were used by particular
>>>> TestNG running thread. I will look into this problem as well. We have to
>>>> do verification somehow automatically, anything else is a futile
>>>> nightmare....
>>>> 
>>>> On 09-12-29 8:43 PM, Sanne Grinovero wrote:
>>>> 
>>>>> I've found that still some tests are not properly killing the
>>>>> CacheManager, making other random tests fail,
>>>>> so I wrote an extension to Infinispan's TestNG listener to identify
>>>>> the guilty implementations.
>>>>> 
>>>>> I opened ISPN-323 to track this.
>>>>> 
>>>>> To identify a non-cleaning test the testNG listener will create a new
>>>>> CacheManager and verify the number of members,
>>>>> so this is slowing down tests; to not annoy you it's only enabled in a
>>>>> new profile "debug-tests":
>>>>> 
>>>>> mvn -Pdebug-tests clean test
>>>>> 
>>>>> Also keep in mind that, being a listener, it can't make the test fail:
>>>>> to find the tests you'll have to grep the build output
>>>>> for "CacheManagers alive after test!".
>>>>> 
>>>>> We still need to fix the identified tests, I'll create subtasks of
>>>>> ISPN-323 for that.
>>>>> 
>>>>> Cheers,
>>>>> Sanne
>>>>> 
>>>>> 
>>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org