[JBoss-dev] RE: Fixing the testsuite for JRockit
I started looking into some Jrockit failures and I realized that the
execution order of Junit tests inside a class *MAY BE THE REVERSE* when
running with Jrockit, compared to using sun's JVM.
This boils down to Class.getMethods()/getDeclaredMethods() returnings
methods in an undefined order:
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Class.html#getMethods(
)
Other people have stumbled on this before and the verdict seems to be
that tests should be re-written to work in any order:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=142528
http://mail-archive.objectweb.org/howl/2005-11/msg00058.html
I'm certain there are places in our testsuite that really depend on the
execution order, in fact this could explain the majority of jrockit
failures, so unless we isolate and conditionally execute those tests, we
need to fix them!
-----Original Message-----
From: Dimitris Andreadis
Sent: Thursday, September 07, 2006 5:05 PM
To: JBoss.org development list; QA
Subject: Fixing the testsuite for JRockit
An important task for the jbas codebase is to fix the
testsuite when run under the jrockit JVM. This is initially
for the 4.x branch but the fixes should apply verbatim to jboss head.
The starting point is here (see linked tasks & subtasks)
http://jira.jboss.com/jira/browse/JBAS-3598
http://cruisecontrol.jboss.com/cc/buildresults/jboss-4.0-testsuite-1.4
At any time there are anywhere between 54-58 failures, so if
something is in your area please have a go. We'll be
assigning tasks to people, soon.
Same thing for the testsuite when run under sun-jdk5 (around
10 failures)
http://cruisecontrol.jboss.com/cc/buildresults/jboss-4.0-tests
uite-sun-1.5
http://jira.jboss.com/jira/browse/JBAS-3611
There are also a lot of transient failures in parts of the
testsuite that are more sensitive to timing/cpu issues (e.g.
JMS), so subsequent runs often produce different results.
This may also have to do with testsuite runs that interfere
with each other.
If we can (a) improve those parts of the testsuite and (b)
make sure testsuite runs are more isolated from each other,
that would be an improvement.
(I see already a related task from Adrian:
http://jira.jboss.com/jira/browse/JBAS-3221)
Thanks
/Dimitris