[
http://jira.jboss.com/jira/browse/JBAS-4103?page=comments#action_12357495 ]
Aleksandar Kostadinov commented on JBAS-4103:
---------------------------------------------
So where does the difference come from? We have only one target with test names not
renamed after the fix, right?
Can you use some mechanism to count test number and compare to the reported test number
after the testsuite run. Probably counting number of files of all resorce collections and
comparing to the number of resulting xml report files. On any difference (+ AND -) report
to the user - build failed.
This way the same problem cannot occur in the future.
Excuse me if this is something you've already considered :(
testsuite testcases' reports overwritten
----------------------------------------
Key: JBAS-4103
URL:
http://jira.jboss.com/jira/browse/JBAS-4103
Project: JBoss Application Server
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Test Suite
Affects Versions: JBossAS-5.0.0.Beta1, JBossAS-4.0.5.GA
Reporter: Aleksandar Kostadinov
Assigned To: Jaroslaw Kijanowski
Fix For: JBossAS-4.0.5.SP1 , JBossAS-4.2.0.GA, JBossAS-5.0.0.Beta3
When JBoss AS testsuite runs there is xml report produces for every testcase. If the same
testcase is run twice without being renamed it overwrites the result from the previous
run. This could lead to silent failures as if the first run fails and the second not, the
final report will not show any problem.
There appear to be some testcases doing so. We've noticed only
org.jboss.test.jmx.test.RMIAdaptorAuthorizationUnitTestCase to fail the first time and not
the second one, but we'd better avoid the possibility of this happening in the
future.
Affected seem to be many if not all current versions (4.0.4GA, 4.0.5GA, 4.0 branch, 4.2
branch, 5, etc.). See forum for a reference.
There will always be the possibility different targets to include same testcases so we
should probably rename tests run by all but tests-standard-unit target.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira