[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka updated AS7-5844: ------------------------------ Description: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Categories candidates: * ASTest ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering was: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Categories candidates: * ASTest * Security * CommonCriteria * NonArquillian * Transactions * Multinode * Clustering -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka updated AS7-5844: ------------------------------ Description: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Categories candidates: * ASTest ** EJB ** JPA ** Management ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering was: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Categories candidates: * ASTest ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka updated AS7-5844: ------------------------------ Description: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Example: {code} interface AllTests; interface ATests extends AllTests; interface BTests extends AllTests; interface AaTests extends ATests; @Category(ATests.class) public void ATest(); @Category(AaTests.class) public void AaTest(); @Category(BTests.class) public void BTest(); {code} TODO: Check if test can have multiple categories. Some categories candidates (feel free to extend): * ASTest ** EJB ** JPA ** Management ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering was: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. TODO: Check if test can have multiple categories. {code} interface AllTests; interface ATests extends AllTests; interface BTests extends AllTests; interface AaTests extends ATests; @Category(ATests.class) public void ATest(); @Category(AaTests.class) public void AaTest(); @Category(BTests.class) public void BTest(); {code} Some categories candidates (feel free to extend): * ASTest ** EJB ** JPA ** Management ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka updated AS7-5844: ------------------------------ Description: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Example: {code} interface AllTests; interface ATests extends AllTests; interface BTests extends AllTests; interface AaTests extends ATests; @Category(ATests.class) public void ATest(); @Category(AaTests.class) public void AaTest(); @Category(BTests.class) public void BTest(); {code} It should be possible to have multiple categories: {code} @Categories({Foo.class, Bar.class}) {code} TODO: Check if it also works with FailSafe. Some categories candidates (feel free to extend): * ASTest ** EJB ** JPA ** Management ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering was: Based on AS7-2086 and SUREFIRE-803, we can now create the categories for tests. They would be in testsuite/shared. Example: {code} interface AllTests; interface ATests extends AllTests; interface BTests extends AllTests; interface AaTests extends ATests; @Category(ATests.class) public void ATest(); @Category(AaTests.class) public void AaTest(); @Category(BTests.class) public void BTest(); {code} TODO: Check if test can have multiple categories. Some categories candidates (feel free to extend): * ASTest ** EJB ** JPA ** Management ** Security *** CommonCriteria ** NonArquillian ** Transactions ** Multinode *** Clustering -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka commented on AS7-5844: ----------------------------------- When the category is used for an Arq test, it's class must be added to the deployment. {code} jar.addPackage(CommonCriteria.class.getPackage()); {code} Looking for some more convenient solution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka commented on AS7-5844: ----------------------------------- Pull request sent, https://github.com/jbossas/jboss-as/pull/3513 But this change needs some additional discussion - a reasoning why we need it. TBD. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka edited comment on AS7-5844 at 11/26/12 7:31 PM: ------------------------------------------------------------- The idea/requests behind this is that packages only allow to divide tests into a tree. But sometimes, as Stuart wrote, there are some cross-cutting concerns, see the suggested categories. It's not supposed to replace package-based fine-grained division, but to complement it, to allow define groups of tests they need to run, instead of creating and maintaining list of packages or even classes. Another example where current model has some flaws are smoke tests: Separating the tests physically to different module is artificial and not much semantic: If any tests starts being considered as smoke test, what we do? Move it to different module and change package name. Does that make sense? Wouldn't it be better to just add `(a)Category(Smoke.class)` ? It also collides with the technical solution of separating tests by type of server setup (multinode, ...). If we decided to include a multinode test as smoke, what would we do? Move it to smoke module and create another surefire execution? Doesn't make much sense. Just adding the `Smoke` category and filtering the tests to be run using categories is way better. Last but not least: These categories are totally optional, no one has to go through the tests and add them. It's a tool for use cases when this categorization is necessary, like, security-related tests, it's subset Common Criteria tests, JPA-related tests, "performance smoke tests", etc. Summary: Tests have some cross-cutting concerns, facetes, which can't be covered by a tree structure, by principle. Also, we are bound by technical solutions coming from Maven's and Arquillian's limitations. Having additional way to categorize can only help, without doing any harm. was (Author: ozizka): The idea/requests behind this is that packages only allow to divide tests into a tree. But sometimes, as Stuart wrote, there are some cross-cutting concerns, see the suggested categories. It's not supposed to replace package-based fine-grained division, but to complement it, to allow define groups of tests they need to run, instead of creating and maintaining list of packages or even classes. Another example where current model has some flaws are smoke tests: Separating the tests physically to different module is artificial and not much semantic: If any tests starts being considered as smoke test, what we do? Move it to different module and change package name. Does that make sense? Wouldn't it be better to just add `(a)Category(Smoke.class)` ? It also collides with the technical solution of separating tests by type of server setup (multinode, ...). If we decided to include a multinode test as smoke, what would we do? Move it to smoke module and create another surefire execution? Doesn't make much sense. Just adding the `Smoke` category and filtering the tests to be run using categories is way better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/AS7-5844?page=com.atlassian.jira.plugin.s... ] Ondrej Zizka commented on AS7-5844: ----------------------------------- Documented at https://docs.jboss.org/author/display/AS71/AS+7+Integration+Testsuite+Use... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira