[jboss-as7-dev] Simplifying testsuite structure

[Carlo de Wolf]

I think the main problem is that the goals of each module are undocumented.

On 06/10/2011 09:38 AM, Thomas Diesler wrote:
> Folks,
>
> the current state is
>
> + demos
Targeted at users.
For use in documentation.
>   + api
Using JBoss API.
>   + internals
(Bogus)
>   + legacy
(Bogus)
>   + spec
Using spec API.
> + testsuite
>   + api
Targeted at users.
More extensive JBoss API testing.
>   + benchmark
>   + domain
>   + integration
Targeted at us.
Anything is allowed in the integration tests, so you'll find me doing 
stuff which is not allowed by spec or even advised to users.
>   + smoke
Targeted at us.
Anything is allowed.
>   + spec
Targeted at users.
More extensive spec API testing.

The demos should be run during a testsuite run. Relying on some sort of 
re-usage in smoke is not enough. Re-using bits in integration/smoke 
should not be an issue.

Last but not least, all tests must be documented.
A failing undocumented test is allowed to be deleted on the spot.

Carlo
>   + stress
>
> some of these modules are empty. It is hard to find the "right 
> location" for new tests. The demos are not self verifying. Instead I 
> propose a simplified structure like this
>
> + (demos removed)
> + testsuite
>   + benchmark
>   + domain
>   + integration
>   + smoke
>   + stress
>
> #1 demos removed
>
> The artefacts in the various demos modules are currently reused by 
> testsuite modules (mainly smoke). demos are collapsed into smoke. I 
> believe a well documented smoke testsuite can serve the purpose of 
> demos and be a standalone deliverable. To be standalone you need to 
> add a mvn repository entry to smoke/pom.xml. Users can download this 
> as a binary and run 'mvn test' on it. Test cases should be organised 
> in packages according to their functional area. It is guaranteed that 
> the demos work because the smoke tests run on every build. Smoke tests 
> should be well documented.
>
> #2 Collapse testsuite api+spec into integration
>
> The integration testsuite should only have dependencies on spec and 
> api modules. Where this is not the case (i.e. a test depends on 
> internal impl) the dependency could be flagged and an api module could 
> be made available. Test cases should be organised in packages 
> according to their functional area. There may be test packages that 
> reference jiras (e.g. as835 
> <https://github.com/tdiesler/jboss-as/tree/master/testsuite/integration/src/test/java/org/jboss/as/testsuite/integration/as835>).
>
> The main motivation is, that it is intuitively clear where to put a 
> test. Even more importantly, where to look for already existing test 
> coverage.
>
> cheers
> -thomas
>
>
>
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-as7-dev/attachments/20110613/c00c6cab/attachment.html