[jboss-as7-dev] AS 7 testsuite proposal - request for feedback

[Richard Achmatowicz]

Hello

Over the past month, there have been discussions going on between Dev 
(Shelly, Andrew, Ondra, Richard) and QA concerning the AS 7 testsuite 
and the final form it should take moving forward. Several options for 
how to organize the testsuite, both in terms of its maven modularization 
and the organization within each maven module, were considered. We 
wanted to come up with a proposal which was scalable and maintainable, 
while at the same time satisfying our all of our testing requirements.

We came up with a proposal, and a description of it can be found here: 
https://docspace.corp.redhat.com/docs/DOC-74146. The document outlines 
initial requirements for the testsuite, and then describes the proposed 
organization of the maven-based testsuite. A prototype version of the 
testsuite (still under some development) exists at git at github.com 
<mailto:git at github.com>:rachmatowicz/jboss-as-testsuite-proposal.git

One of the key differentiators between proposed models is the purpose 
for and the degree to which the testsuite is decomposed into maven 
modules. The proposal presented here uses modules to decompose the 
testsuite into different types of tests (functional vs various types of 
non-functional, tests such as benchmark, stress, soak), but not for 
collecting the same logical collections of tests (e.g. clustering 
functional tests vs domain management functional tests) into different 
logical groups. That separation is achieved within a module through the 
use of maven build profiles. This is a "few modules, large poms" 
solution, as opposed to a "many modules, small poms" solution.

The proposal attempts to be clear about how to achieve many of the most 
common requirements, but there may be omissions. Also, there are still 
certain fundamental questions that need to be dealt with, such as:
* whether or not we support both JUnit and testNG as testing frameworks
* the optimal approach for constructing server configurations of those 
outlined
* is the modularization proposed a good compromise between scalability 
and maintainability

Considered feedback on these issues in particular and on the model as a 
whole would be very helpful. The feedback can be posted as comments to 
the document itself, to facilitate sharing.

With your feedback, we can come up with a model which satisfies our 
requirements and serve us well in future.

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-as7-dev/attachments/20110906/eb4a4de2/attachment-0001.html