[arquillian-issues] [JBoss JIRA] (ARQ-2088) When test is skipped because of an assumption, Arquillian's test result is passed

[Dipak Pawar (JIRA)]

    [ https://issues.jboss.org/browse/ARQ-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405481#comment-13405481 ] 

Dipak Pawar commented on ARQ-2088:
----------------------------------

It has nothing to do with core, we have bug in reporter. Look for following events
{code}
(E) ManagerProcessing
	(O) LoadableExtensionLoader.load
	(E) ServiceRegistryLoader
(E) ManagerStarted
	(O) ReporterLifecycleManager.observeFirstEvent
	(E) ExecutionStore
	(O) ConfigurationRegistrar.loadConfiguration
	(E) ArquillianDescriptorImpl
		(O) ProtocolRegistryCreator.createRegistry
		(E) ProtocolRegistry
		(O) ContainerRegistryCreator.createRegistry
		(E) LocalContainerRegistry
		(O) ReporterConfigurator.configure
		(E) ReporterConfiguration
(E) BeforeSuite
	(I) TestContextHandler.createSuiteContext
	(O) ArquillianCoreReporterLifecycleManager.startTestSuite
	(E) TestSuiteSection
		(O) ReporterLifecycleManager.observeEventsForAllSections
	(O) ContainerEventController.execute
	(E) SetupContainers
		(O) ClientDeployerCreator.createClientSideDeployer
		(E) ClientDeployer
		(O) ClientContainerControllerCreator.createClientSideContainerController
		(E) ClientContainerController
		(O) ContainerLifecycleController.setupContainers
	(E) StartSuiteContainers
		(O) ContainerLifecycleController.startSuiteContainers
	(O) EnvironmentReportCreator.startTestSuite
	(E) TestSuiteConfigurationSection
		(O) ReporterLifecycleManager.observeEventsForAllSections
(E) BeforeClass
	(I) TestContextHandler.createSuiteContext
	(I) TestContextHandler.createClassContext
	(E) TestClass
	(O) ArquillianCoreReporterLifecycleManager.startTestClass
	(E) TestClassSection
		(O) ReporterLifecycleManager.observeEventsForAllSections
	(O) ContainerEventController.execute
	(E) StartClassContainers
		(O) ContainerLifecycleController.startClassContainers
	(E) GenerateDeployment
		(O) DeploymentGenerator.generateDeployment
		(E) DeploymentScenario
	(E) DeployManagedDeployments
		(O) ContainerDeployController.deployManaged
	(O) ContainerRestarter.restart
	(O) ClientBeforeAfterLifecycleEventExecuter.on
(E) RulesEnrichment
	(I) TestContextHandler.createSuiteContext
	(I) TestContextHandler.createClassContext
	(E) TestClass
	(I) TestContextHandler.createTestContext
	(I) ContainerEventController.createBeforeContext
	(O) RulesEnricher.enrichRulesAndTestInstance
	(O) ClientBeforeAfterLifecycleEventExecuter.on
(E) BeforeRules
	(I) TestContextHandler.createSuiteContext
	(I) TestContextHandler.createClassContext
	(E) TestClass
	(I) TestContextHandler.createTestContext
	(I) ContainerEventController.createBeforeContext
	(O) RulesEnricher.enrichStatement
	(E) BeforeEnrichment
	(E) AfterEnrichment
	(O) ClientBeforeAfterLifecycleEventExecuter.on
	(E) Before
		(I) TestContextHandler.createSuiteContext
		(I) TestContextHandler.createClassContext
		(E) TestClass
		(I) TestContextHandler.createTestContext
		(I) ContainerEventController.createBeforeContext
		(O) ArquillianCoreReporterLifecycleManager.startTestMethod
		(E) TestMethodSection
			(O) ReporterLifecycleManager.observeEventsForAllSections
		(O) TestInstanceEnricher.enrich
		(E) BeforeEnrichment
		(E) AfterEnrichment
		(O) ClientBeforeAfterLifecycleEventExecuter.on
	(E) Test
		(I) TestContextHandler.createSuiteContext
		(I) TestContextHandler.createClassContext
		(E) TestClass
		(I) TestContextHandler.createTestContext
		(I) ContainerEventController.createTestContext
		(O) ClientTestExecuter.execute
		(E) LocalExecutionEvent
			(O) LocalTestExecuter.execute
This should fail
			(E) TestResult
	(E) After
		(I) TestContextHandler.createSuiteContext
		(I) TestContextHandler.createClassContext
		(E) TestClass
		(I) TestContextHandler.createTestContext
		(I) UpdateTestResultBeforeAfter.update
		(I) ContainerEventController.createAfterContext
		(O) ClientBeforeAfterLifecycleEventExecuter.on
		(O) ArquillianCoreReporterLifecycleManager.stopTestMethod
		(E) TestMethodSection
			(O) ReporterLifecycleManager.observeEventsForAllSections
	(E) ComparisonFailure
	(E) ComparisonFailure
(E) AfterRules
	(I) TestContextHandler.createSuiteContext
	(I) TestContextHandler.createClassContext
	(E) TestClass
	(I) TestContextHandler.createTestContext
	(I) UpdateTestResultBeforeAfter.update
	(I) ContainerEventController.createAfterContext
	(O) ClientBeforeAfterLifecycleEventExecuter.on
	(O) ArquillianCoreReporterLifecycleManager.stopTestMethod
	(E) TestMethodSection
		(O) ReporterLifecycleManager.observeEventsForAllSections
(E) AfterClass
	(I) TestContextHandler.createSuiteContext
	(I) TestContextHandler.createClassContext
	(E) TestClass
	(O) ClientBeforeAfterLifecycleEventExecuter.on
	(O) ContainerEventController.execute
	(E) UnDeployManagedDeployments
		(O) ContainerDeployController.undeployManaged
	(E) StopManualContainers
		(O) ContainerLifecycleController.stopManualContainers
	(E) StopClassContainers
		(O) ContainerLifecycleController.stopClassContainers
	(O) ArquillianCoreReporterLifecycleManager.stopTestClass
	(E) TestClassSection
		(O) ReporterLifecycleManager.observeEventsForAllSections
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.669 sec <<< FAILURE! - in org.arquillian.reporter.FailingTest
runFailingTest(org.arquillian.reporter.FailingTest)  Time elapsed: 0.095 sec  <<< FAILURE!
org.junit.ComparisonFailure: expected:<[0]> but was:<[1]>
	at org.arquillian.reporter.FailingTest.runFailingTest(FailingTest.java:25)

(E) AfterSuite
	(I) TestContextHandler.createSuiteContext
	(O) ContainerEventController.execute
	(E) StopSuiteContainers
		(O) ContainerLifecycleController.stopSuiteContainers
	(O) ArquillianCoreReporterLifecycleManager.stopTestSuite
	(E) TestSuiteSection
		(O) ReporterLifecycleManager.observeEventsForAllSections
(E) ManagerStopping
	(O) ReporterLifecycleManager.observeLastEvent

{code}


We are observing wrong event `AfterTest` in reporter. We should observe `AfterTestLifecycleEvent` from which AfterTest is inhereting,  invoke fire for reporter only if observing event is instance of `AfterTestLifeCycleEvent`

Opened isssue for reporter on [GH|https://github.com/arquillian/arquillian-reporter/issues/20].

Fixed in [PR|https://github.com/arquillian/arquillian-reporter/pull/21]

Thanks,
Dipak

> When test is skipped because of an assumption, Arquillian's test result is passed
> ---------------------------------------------------------------------------------
>
>                 Key: ARQ-2088
>                 URL: https://issues.jboss.org/browse/ARQ-2088
>             Project: Arquillian
>          Issue Type: Enhancement
>          Components: core
>    Affects Versions: 1.1.13.Final
>            Reporter: Matous Jobanek
>
> When you run Arquillian test with an assumption: {{Assume.assumeFalse(true);}} then the test is skipped. The same also say both IDE and maven surefire, but Arquillian's {{TestResult}} retrieved in {{After}} phase contains status Passed.



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)