[jboss-as7-dev] Revisited: Integration TestSuite Organization and Maintenance

[Scott Marlow]

On 08/15/2011 03:27 AM, Andrew Lee Rubinger wrote:
> A summary of where I believe we stand on the 3 points raised here:
>
> 1) TestSuite Organization
>
> Stuart and Kabir have voiced concerns that separating src/main and src/test make authoring more complex.

I think keeping it simple/easy to create new tests, is more important 
than being able to prove that our api/spec-api modules are incomplete. 
We depend on the tests to prove that our AS build works correctly (keep 
it simple/easy so people will create the tests).

>
> While I'll concede that may be true to a small extent, I believe the benefits of separation far outweigh the drawbacks.  By separating these out I uncovered 4 JIRAs which proved that our "api" and "spec-api" modules were incomplete.  In short, paying attention to user dependencies to validate against them is very important.
>
> 2) Run Modes, Test Subsets
>
> I don't think there's been much discussion here, with the exception of the QE team who have provided some use cases that may not be possible given a standard layout.  For instance the CLI and clustering tests are more than the standard "deploy something and make assertions" format we cover in point 1), so these will likely get their own modules as is appropriate.
>
> 3) An authoritative maintainer

If someone sets the standard, by creating many of the unit tests by 
using ARQ correctly.  Others are more likely to follow their example. 
Perhaps someone could tag certain tests as good examples to follow.

>
> No one has commented on this.
>
> By Tuesday evening I need to report back on at least the 3 points above, and also have approval for getting my patch committed.  Failing that, we need to agree on an alternate path forward.  I'm happy with any solution which addresses the respect for dependencies in the testsuite as I've outlined.
>
> S,
> ALR
>