[jboss-jira] [JBoss JIRA] (AS7-3614) The Ordering of tests in integration/basic seems to affect the successfulness of the tests.

[Jason Greene (JIRA)]

     [ https://issues.jboss.org/browse/AS7-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Greene updated AS7-3614:
------------------------------

    Fix Version/s: 7.1.1.Final
                       (was: 7.1.0.Final)

    
> The Ordering of tests in integration/basic seems to affect the successfulness of the tests.
> -------------------------------------------------------------------------------------------
>
>                 Key: AS7-3614
>                 URL: https://issues.jboss.org/browse/AS7-3614
>             Project: Application Server 7
>          Issue Type: Bug
>          Components: Test Suite
>            Reporter: Darran Lofthouse
>            Assignee: Andrew Rubinger
>             Fix For: 7.1.1.Final
>
>
> The following early staged of a test run show a clean run: -
> ------------------------------------------------------- T E S T S -------------------------------------------------------
> Running org.jboss.as.test.integration.management.cli.HelpTestCase
> Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.658 sec Running org.jboss.as.test.integration.management.cli.JmsTestCase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.259 sec Running org.jboss.as.test.integration.web.security.WebSecurityJBossWebXmlSecurityRolesTestCase
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.345 sec Running org.jboss.as.test.integration.web.security.WebSecurityFORMTestCase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.913 sec Running org.jboss.as.test.integration.web.security.WebSecurityBASICTestCase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.895 sec Running org.jboss.as.test.integration.web.security.WebSecurityJBossSimpleRoleMappingTestCase Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.576 sec Running org.jboss.as.test.integration.web.security.servlet3.WebSecurityProgrammaticLoginTestCase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.226 sec Running org.jboss.as.test.integration.web.security.WebSecurityCERTTestCase Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.006 sec Running org.jboss.as.test.integration.web.security.tg.TransportGuaranteeTestCase 
> Running locally with a different order I regularly see the following test fail: -
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running org.jboss.as.test.integration.web.security.WebSecurityJBossWebXmlSecurityRolesTestCase
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.028 sec
> Running org.jboss.as.test.integration.web.security.servlet3.WebSecurityProgrammaticLoginTestCase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.833 sec
> Running org.jboss.as.test.integration.web.security.WebSecurityCERTTestCase
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.001 sec
> Running org.jboss.as.test.integration.web.security.WebSecurityFORMTestCase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.462 sec
> Running org.jboss.as.test.integration.web.security.tg.TransportGuaranteeTestCase
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.321 sec
> Running org.jboss.as.test.integration.web.security.WebSecurityJBossSimpleRoleMappingTestCase
> Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.382 sec <<< FAILURE!
> The logs seem to indicate a failure to deploy the security domain for the failed test, reviewing the TransportGuaranteeTestCase it does appear that maybe the domain is not removed.  Adding a line within WebSecurityJBossSimpleRoleMappingTestCase to remove the domain before adding the domain caused that test to pass.
> This has been quite tricky to track what is actually happening, I would suggest either trying to use unique names per test so conflicts can not occur or increased validation at the end of the test case to ensure clean up is complete and to fail in the test that leaks the conflicting resource rather than failing a subsequent test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira