[jboss-as7-dev] Simplifying testsuite structure

[Karel Piwko]

Hi All,

I'm a bit confused where to put tests which control container lifecycle
[1]. Some of them will probably use multiple containers (group), so they
would require a specific arquillian.xml configuration.

Either I can create a separate modules for each of the use cases as a
specific arquillian.xml is needed or I can force Arquillian
ConfigurationRegistrar to load the configuration from a different
location.

For the former it would lead to something like:

testsuite / lifecycle-control
testsuite / clustering / domain
testsuite / clustering / standalone 

For the latter, that would lead to multiple surefire plugin execution
for some of the current modules, as it is not possible to force
Arquillian configuration reload, e.g.

testsuite / integration
               / src / test / ... / lifecycle / ...
               / src / test / ... / domain / ...
               / src / test / ... / standalone / ... 

In the pom.xml that would require having multiple <execution>s for
surefire plugin, the activating different Arquillian configuration, e.g.
arquillian-domain-cluster.xml, arquillian-domain-standalone, so a single
Maven command will be required to run differently configured tests.

I personally prefer the latter variant, as it allows to run a single
TestCase within multiple <execution>s, however this might collide with
-Dtest property to select a single test and run it exactly once.

How does that sound to you?

Thanks,

Karel

[1] https://issues.jboss.org/browse/AS7-1011

On Fri, 2011-06-10 at 09:38 +0200, Thomas Diesler wrote:
> Folks,
> 
> the current state is
> 
> + demos
>   + api
>   + internals
>   + legacy
>   + spec
> + testsuite
>   + api
>   + benchmark
>   + domain 
>   + integration
>   + smoke
>   + spec 
>   + stress
> 
> some of these modules are empty. It is hard to find the "right
> location" for new tests. The demos are not self verifying. Instead I
> propose a simplified structure like this
> 
> + (demos removed) 
> + testsuite
>   + benchmark
>   + domain 
>   + integration
>   + smoke
>   + stress
> 
> #1 demos removed
> 
> The artefacts in the various demos modules are currently reused by
> testsuite modules (mainly smoke). demos are collapsed into smoke. I
> believe a well documented smoke testsuite can serve the purpose of
> demos and be a standalone deliverable. To be standalone you need to
> add a mvn repository entry to smoke/pom.xml. Users can download this
> as a binary and run 'mvn test' on it. Test cases should be organised
> in packages according to their functional area. It is guaranteed that
> the demos work because the smoke tests run on every build. Smoke tests
> should be well documented.
> 
> #2 Collapse testsuite api+spec into integration
> 
> The integration testsuite should only have dependencies on spec and
> api modules. Where this is not the case (i.e. a test depends on
> internal impl) the dependency could be flagged and an api module could
> be made available. Test cases should be organised in packages
> according to their functional area. There may be test packages that
> reference jiras (e.g. as835).
> 
> The main motivation is, that it is intuitively clear where to put a
> test. Even more importantly, where to look for already existing test
> coverage.
> 
> cheers
> -thomas
> 
> 
> 
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev