[infinispan-dev] Tests failing: ports shared across modules

[Manik Surtani]

On 9 Dec 2009, at 23:07, Sanne Grinovero wrote:

> Hi Manik,
> sorry for late answer, this got stuck in drafts box some days:
> 
> You're right as the modules are built and tested sequentially it
> shouldn't matter, still it looks like they interact and I have no
> ideas about how that's possible.
> Maybe the cause is simpler, like some test having a leftover cache,
> not properly terminated?
> It could be the case that the shutdown hook of CacheManager is being
> run only at the end of last test, as there's no real shutdown between
> modules, so if some test doesn't properly cleanup
> this could explain the higher-than-expected number of nodes. Will try
> debugging a bit more.

Great, thanks!  Feel free to ping myself or mmarkus on IRC if you want to chat about this some more/bounce ideas around.

> 
> Cheers,
> Sanne
> 
> 
> 2009/12/7 Manik Surtani <manik at jboss.org>:
>> 
>> On 6 Dec 2009, at 13:39, Sanne Grinovero wrote:
>> 
>>> Hello,
>>> Looking into hudson's tests on
>>> http://hudson.jboss.org/hudson/view/Infinispan/job/Infinispan-trunk-JDK6-tcp
>>> it appears the number of failed tests change from build to build even
>>> by documentation changes or other unrelated changes.
>>> Two examples:
>>> 
>>> Build #1031 :
>>> Changes: Typos in javadocs
>>> Test Result (4 failures / -1)
>>> a javadoc change fixed a test?
>>> 
>>> Build #1033 :
>>> Changes: [ISPN-301] (Closing the Lucene Directory will close the cache too)
>>> Test Result (8 failures / +2)
>>> So while I only changed something related to Lucene, the failures in
>>> core increased?
>>> 
>>> In some of the test errors you can find evidence of communication
>>> between test scenarios, like these:
>>> http://hudson.jboss.org/hudson/view/Infinispan/job/Infinispan-trunk-JDK6-tcp/org.infinispan$infinispan-lucene-directory/1025/testReport/(root)/TestSuite/createBeforeClass/
>>> http://hudson.jboss.org/hudson/view/Infinispan/job/Infinispan-trunk-JDK6-tcp/org.infinispan$infinispan-tree/1025/testReport/org.infinispan.api.tree/SyncReplTxTest-createBeforeClass/org_infinispan_api_tree_SyncReplTxTest_createBeforeClass/
>>> 
>>> first one from the Lucene module, second from the Tree module: they
>>> both have the node vmg22.mw.lab.eng.bos.redhat.com-42912
>>> and are complaining about an unexpected number of participants.
>>> Looking into the other errors it always looks like as "someone else"
>>> changed the cache, but there's no evidence, so I think we should solve
>>> the isolation problem first?
>>> 
>>> Both stacktraces show that they're using
>>> org.infinispan.test.MultipleCacheManagersTest.createClusteredCaches(MultipleCacheManagersTest.java:137)
>>> to setup the caches, so it doesn't appear to be a problem with these
>>> two testcases.
>>> 
>>> This kind of interactions don't seem to happen inside a single module,
>>> could it be a classloader problem? A static threadlocal defines the
>>> jgroups port to use, but it's "static" in a per-module world, instead
>>> of globally static?
>>> I've added some logging to
>>> org.infinispan.test.fwk.JGroupsConfigBuilder, 2 snippets of the
>>> result:
>>> [org.infinispan.test.fwk.JGroupsConfigBuilder] (pool-1-thread-1) TCP
>>> bind_port:7900 ClassLoder:org.apache.maven.surefire.booter.IsolatedClassLoader at 2e93d13f
>>> [...many lines..]
>>> [org.infinispan.test.fwk.JGroupsConfigBuilder] (pool-1509-thread-10)
>>> TCP bind_port:7900
>>> ClassLoder:org.apache.maven.surefire.booter.IsolatedClassLoader at 2d1a2259
>>> 
>>> I see the two classloaders being different, and while the threads are
>>> different they are sharing the same bind_port 7900.
>> 
>> Wow, great detective work tracking this down!  Thanks for looking into this!
>> 
>> Interestingly though, isn't each module test suite run sequentially, one after another?  So does it matter that they use separate class loaders and hence separate contents of static variables?
>> 
>>> Looking into http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html
>>> It looks like from Surefire 2.4.3 the default is to use a shared
>>> system classloader, but there's a little warning at the bottom of page
>>> about not being possible to not isolate the classloader while using
>>> forkMode=none
>>> 
>>> ideas?
>>> 
>>> P.S. where can I get the sources of maven-surefire-plugin version2.4.3-JBOSS ?
>> 
>> https://svn.jboss.org/repos/maven/plugins/jboss/trunk/maven-jboss-surefire
>> 
>> or
>> 
>> http://anonsvn.jboss.org/repos/maven/plugins/jboss/trunk/maven-jboss-surefire/
>> 
>> Cheers
>> Manik
>> --
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org