[arquillian-issues] [JBoss JIRA] (ARQ-1356) TestResult isn't updated correctly when TestNG is used

[Jan Papousek (JIRA)]

Jan Papousek created ARQ-1356:
---------------------------------

             Summary: TestResult isn't updated correctly when TestNG is used
                 Key: ARQ-1356
                 URL: https://issues.jboss.org/browse/ARQ-1356
             Project: Arquillian
          Issue Type: Bug
      Security Level: Public (Everyone can see)
          Components: Base Implementation
    Affects Versions: 1.0.3.Final
         Environment: TestNG 6.1.1
            Reporter: Jan Papousek


When a test fails, the state of TestResult is still PASSED and therefore it can't be used in observer:

{code}
public class LifecycleObserver {
    @Inject
    private Instance<TestResult> result;

    public void afterTest(@Observes After event) {
        if (result.get().getStatus().equals(PASSED)) {
            // something
        } else {
            // something else
        }
    }
}
{code}

The problem is [LocalTestExecutor|https://github.com/arquillian/arquillian-core/blob/master/container/test-impl-base/src/main/java/org/jboss/arquillian/container/test/impl/execution/LocalTestExecuter.java] updates the status to FAILED only if an [exception is thrown|https://github.com/arquillian/arquillian-core/blob/master/container/test-impl-base/src/main/java/org/jboss/arquillian/container/test/impl/execution/LocalTestExecuter.java#L68], but [TestMethodExecutor|https://github.com/arquillian/arquillian-core/blob/master/testng/core/src/main/java/org/jboss/arquillian/testng/Arquillian.java#L158] from TestNG implemenation doesn't throw any exception when the test fails. It [memorizes it in ITestResult object|https://github.com/arquillian/arquillian-core/blob/master/testng/core/src/main/java/org/jboss/arquillian/testng/Arquillian.java#L212].


I propose to throw an exception in [the end of test invocation|https://github.com/arquillian/arquillian-core/blob/master/testng/core/src/main/java/org/jboss/arquillian/testng/Arquillian.java#L171], if the test fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira