[jboss-as7-dev] AS 7 testsuite proposal - request for feedback

Hello Over the past month, there have been discussions going on between Dev (Shelly, Andrew, Ondra, Richard) and QA concerning the AS 7 testsuite and the final form it should take moving forward. Several options for how to organize the testsuite, both in terms of its maven modularization and the organization within each maven module, were considered. We wanted to come up with a proposal which was scalable and maintainable, while at the same time satisfying our all of our testing requirements. We came up with a proposal, and a description of it can be found here: https://docspace.corp.redhat.com/docs/DOC-74146. The document outlines initial requirements for the testsuite, and then describes the proposed organization of the maven-based testsuite. A prototype version of the testsuite (still under some development) exists at git(a)github.com <mailto:git@github.com>:rachmatowicz/jboss-as-testsuite-proposal.git One of the key differentiators between proposed models is the purpose for and the degree to which the testsuite is decomposed into maven modules. The proposal presented here uses modules to decompose the testsuite into different types of tests (functional vs various types of non-functional, tests such as benchmark, stress, soak), but not for collecting the same logical collections of tests (e.g. clustering functional tests vs domain management functional tests) into different logical groups. That separation is achieved within a module through the use of maven build profiles. This is a "few modules, large poms" solution, as opposed to a "many modules, small poms" solution. The proposal attempts to be clear about how to achieve many of the most common requirements, but there may be omissions. Also, there are still certain fundamental questions that need to be dealt with, such as: * whether or not we support both JUnit and testNG as testing frameworks * the optimal approach for constructing server configurations of those outlined * is the modularization proposed a good compromise between scalability and maintainability Considered feedback on these issues in particular and on the model as a whole would be very helpful. The feedback can be posted as comments to the document itself, to facilitate sharing. With your feedback, we can come up with a model which satisfies our requirements and serve us well in future. Richard