Darran Lofthouse created AS7-3614: ------------------------------------- Summary: The Ordering of tests in integration/basic seems to affect the successfulness of the tests. Key: AS7-3614 URL: https://issues.jboss.org/browse/AS7-3614 Project: Application Server 7 Issue Type: Bug Components: Test Suite Reporter: Darran Lofthouse Assignee: Andrew Rubinger Fix For: 7.1.0.Final The following early staged of a test run show a clean run: - ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.jboss.as.test.integration.management.cli.HelpTestCase Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.658 sec Running org.jboss.as.test.integration.management.cli.JmsTestCase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.259 sec Running org.jboss.as.test.integration.web.security.WebSecurityJBossWebXmlSecurityRolesTestCase Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.345 sec Running org.jboss.as.test.integration.web.security.WebSecurityFORMTestCase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.913 sec Running org.jboss.as.test.integration.web.security.WebSecurityBASICTestCase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.895 sec Running org.jboss.as.test.integration.web.security.WebSecurityJBossSimpleRoleMappingTestCase Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.576 sec Running org.jboss.as.test.integration.web.security.servlet3.WebSecurityProgrammaticLoginTestCase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.226 sec Running org.jboss.as.test.integration.web.security.WebSecurityCERTTestCase Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.006 sec Running org.jboss.as.test.integration.web.security.tg.TransportGuaranteeTestCase Running locally with a different order I regularly see the following test fail: - ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.jboss.as.test.integration.web.security.WebSecurityJBossWebXmlSecurityRolesTestCase Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.028 sec Running org.jboss.as.test.integration.web.security.servlet3.WebSecurityProgrammaticLoginTestCase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.833 sec Running org.jboss.as.test.integration.web.security.WebSecurityCERTTestCase Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.001 sec Running org.jboss.as.test.integration.web.security.WebSecurityFORMTestCase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.462 sec Running org.jboss.as.test.integration.web.security.tg.TransportGuaranteeTestCase Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.321 sec Running org.jboss.as.test.integration.web.security.WebSecurityJBossSimpleRoleMappingTestCase Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.382 sec <<< FAILURE! The logs seem to indicate a failure to deploy the security domain for the failed test, reviewing the TransportGuaranteeTestCase it does appear that maybe the domain is not removed. Adding a line within WebSecurityJBossSimpleRoleMappingTestCase to remove the domain before adding the domain caused that test to pass. This has been quite tricky to track what is actually happening, I would suggest either trying to use unique names per test so conflicts can not occur or increased validation at the end of the test case to ensure clean up is complete and to fail in the test that leaks the conflicting resource rather than failing a subsequent test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira