<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.18.3">
</HEAD>
<BODY>
Hi everyone interested,<BR>
<BR>
there are these motivations for grouping the tests:<BR>
<BR>
1) Splitting tests by functional area, e.g. jms, jacorb, cmp.<BR>
2) Splitting tests by purpose, e.g. smoke, basic, stress etc.<BR>
3) Running tests with different configs.<BR>
4) Running various groups of tests.<BR>
<BR>
4) can be done using e.g. -Dtest=org.jboss.as.test.integration.ejb.*TestCase.java (provided -Dtest works fine), so I'll drop it from further considerations).<BR>
<BR>
1-3) is currently done by a combination of modules and Surefire executions.<BR>
The problem with the later is that a failure in one execution prevents the successive from running.<BR>
<BR>
Here are options how that can be solved:<BR>
<BR>
<B>A) Naive solution: Keep status quo, keep tests in the first execution in a good condition.</B><BR>
Pros: * We already have it.<BR>
Cons: * It will fail now and then, and people would need an extra run for the other group, using various params.<BR>
<BR>
<B>B) Megalomanic solution: Make every combination a maven module.</B><BR>
Pros:<BR>
* Avaliable right away.<BR>
Cons:<BR>
* Will result in many modules - e.g. smoke-web, smoke-full, basic-web, basic-full, ...<BR>
* => much harder to maintain, esp. for the product features (need to distribute stuff to too many pom.xml's).<BR>
* Slower testsuite and AS7 build.<BR>
* Would not scale for the cases when each test needs a different config.<BR>
<BR>
<B>C) Optimistic engineer's solution: Use JUnit's new @Category and Arquillians ability to restart a server with other config, based on @ContainerConfig("path-to-config.xml").</B><BR>
Pros:<BR>
* Simple, clean approach; no pom.xml hell.<BR>
* Easy to maintain.<BR>
* Easy to reorganize later.<BR>
* Would get 3) and 4) out of maven<BR>
Cons:<BR>
* Arq's impl of @Category is not tested<BR>
* We would need to put @Category to all tests<BR>
* @ContainerConfig("path-to-config.xml") is not implemented. We could (mis)use @TargetsContainer and @OperatesOnDeployment.<BR>
<BR>
<B>Are there others?</B><BR>
<B>Did I miss some pros or cons?</B><BR>
<B>Which one do you like?</B><BR>
<BR>
Ondra<BR>
<BR>
</BODY>
</HTML>