[arquillian-issues] [JBoss JIRA] (ARQ-919) Document TestNG issues with Arquillian

[Samuel Santos (JIRA)]

     [ https://issues.jboss.org/browse/ARQ-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Samuel Santos updated ARQ-919:
------------------------------

    Description: 
We were using TestNG as the testing framework and we had some issues while trying, for instance, to override the {{@BeforeSuite}} or {{@BeforeClass}} methods. After trying some workarounds we ended up asking Aslak for help/advice. He told us that before/after in TestNG doesn't play nice with Arquillian since it does not give them the control they need. TestNG doesn't go through phases, like before/test/after, those phases only occur if the user has defined them (they work like callbacks); consequently if the user doesn't define a given annotation method, the SPI is never called.

Another issue is that despite TestNG docs stating that tests do not run in parallel:
{quote}If specified, sets the default mechanism used to determine how to use parallel threads when running tests. If not set, default mechanism is not to use parallel threads at all. This can be overridden in the suite definition.{quote}
The truth is that if we do not define dependencies between test classes we end up with {{ClassCastException}} and {{ClassNotFoundException}} errors. Meaning that the same WAR is being deployed to the container before the previous one has been undeployed.

We do think that this is important information that could in some situations/projects invalidate the choice of TestNG as the testing framework. Thereof, it makes sense adding it to the Arquillian documentation. We asked Aslak for his opinion, and he has agreed.

  was:
While working on a project with a big number of layers and abstractions, we ended up using the {{ShrinkWrap.createFromZipFile}} method to deploy the resulting Maven package instead of adding components one by one to the ShrinkWrap archive.

We were using TestNG as the testing framework and we had some issues while trying, for instance, to override the {{@BeforeSuite}} or {{@BeforeClass}} methods. After trying some workarounds we ended up asking Aslak for help/advice. He told us that before/after in TestNG doesn't play nice with Arquillian since it does not give them the control they need. TestNG doesn't go through phases, like before/test/after, those phases only occur if the user has defined them (they work like callbacks); consequently if the user doesn't define a given annotation method, the SPI is never called.

Another issue is that despite TestNG docs stating that tests do not run in parallel:
{quote}If specified, sets the default mechanism used to determine how to use parallel threads when running tests. If not set, default mechanism is not to use parallel threads at all. This can be overridden in the suite definition.{quote}
The truth is that if we do not define dependencies between test classes we end up with {{ClassCastException}} and {{ClassNotFoundException}} errors. Meaning that the same WAR is being deployed to the container before the previous one has been undeployed.

We do think that this is important information that could in some situations/projects invalidate the choice of TestNG as the testing framework. Thereof, it makes sense adding it to the Arquillian documentation. We asked Aslak for his opinion, and he has agreed.


    
> Document TestNG issues with Arquillian
> --------------------------------------
>
>                 Key: ARQ-919
>                 URL: https://issues.jboss.org/browse/ARQ-919
>             Project: Arquillian
>          Issue Type: Task
>      Security Level: Public(Everyone can see) 
>          Components: Documentation
>            Reporter: Samuel Santos
>
> We were using TestNG as the testing framework and we had some issues while trying, for instance, to override the {{@BeforeSuite}} or {{@BeforeClass}} methods. After trying some workarounds we ended up asking Aslak for help/advice. He told us that before/after in TestNG doesn't play nice with Arquillian since it does not give them the control they need. TestNG doesn't go through phases, like before/test/after, those phases only occur if the user has defined them (they work like callbacks); consequently if the user doesn't define a given annotation method, the SPI is never called.
> Another issue is that despite TestNG docs stating that tests do not run in parallel:
> {quote}If specified, sets the default mechanism used to determine how to use parallel threads when running tests. If not set, default mechanism is not to use parallel threads at all. This can be overridden in the suite definition.{quote}
> The truth is that if we do not define dependencies between test classes we end up with {{ClassCastException}} and {{ClassNotFoundException}} errors. Meaning that the same WAR is being deployed to the container before the previous one has been undeployed.
> We do think that this is important information that could in some situations/projects invalidate the choice of TestNG as the testing framework. Thereof, it makes sense adding it to the Arquillian documentation. We asked Aslak for his opinion, and he has agreed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira