[jboss-as7-dev] Simplifying testsuite structure

[Andrew Lee Rubinger]

Hi, guys.

So we have some disagreement towards unification of the testsuite here, 
and I'd like to get that ironed out as soon as we can.  So anyone with 
strong opinions, please weigh.

Thomas has proposed collapsing "spec" and "api" into a single point: 
"integration".  I think this is actually a step back from where we 
should be heading:

https://issues.jboss.org/browse/AS7-999
http://lists.jboss.org/pipermail/jboss-as7-dev/2011-March/000851.html

Now, it's taken me quite some time to do the prereq work in 
ShrinkWrap/Arquillian/AS7 ARQ fork unwinding to get to the point where 
I'm actually gunning fulltime now on that proposal, but the basic 
premise is this:

* Maven modules imply a dependency structure
* We should write tests/demos/samples as our users would
* Be weary of limiting the dependencies to *only* what's required

To this end, I've actually been working in the reverse direction from 
the points outlined in this Thread.  Instead of moving everything into a 
unified project to address all integration testing concerns, I've been 
breaking them into 3 categories:

* spec (Java EE and SE APIs only)
* api (spec, plus APIs we expose as public via AS - Hibernate 
annotations, EJB3 annotations, HornetQ API, etc)
* internals (The whole runtime)

Each of these represent integration tests, and the only difference is 
the dependencies available on the Compilation Classpath.  In other 
words, what a user will see as available in src/main/java.

My work is being done here:

https://github.com/ALRubinger/jboss-as/tree/AS7-999

...though I do need to rebase that and add my latest of today's commits 
to it.

Also, there's a corollary issue to expose out all of the AS7 public APIs 
in one POM:

https://issues.jboss.org/browse/AS7-650

I do feel strongly that when it comes to testing, we must act as users, 
so I suspect this above is the best approach. :)

S,
ALR

On 06/10/2011 03:38 AM, Thomas Diesler wrote:
> Folks,
>
> the current state is
>
> + demos
> + api
> + internals
> + legacy
> + spec
> + testsuite
> + api
> + benchmark
> + domain
> + integration
> + smoke
> + spec
> + stress
>
> some of these modules are empty. It is hard to find the "right location"
> for new tests. The demos are not self verifying. Instead I propose a
> simplified structure like this
>
> + (demos removed)
> + testsuite
> + benchmark
> + domain
> + integration
> + smoke
> + stress
>
> #1 demos removed
>
> The artefacts in the various demos modules are currently reused by
> testsuite modules (mainly smoke). demos are collapsed into smoke. I
> believe a well documented smoke testsuite can serve the purpose of demos
> and be a standalone deliverable. To be standalone you need to add a mvn
> repository entry to smoke/pom.xml. Users can download this as a binary
> and run 'mvn test' on it. Test cases should be organised in packages
> according to their functional area. It is guaranteed that the demos work
> because the smoke tests run on every build. Smoke tests should be well
> documented.
>
> #2 Collapse testsuite api+spec into integration
>
> The integration testsuite should only have dependencies on spec and api
> modules. Where this is not the case (i.e. a test depends on internal
> impl) the dependency could be flagged and an api module could be made
> available. Test cases should be organised in packages according to their
> functional area. There may be test packages that reference jiras (e.g.
> as835
> <https://github.com/tdiesler/jboss-as/tree/master/testsuite/integration/src/test/java/org/jboss/as/testsuite/integration/as835>).
>
> The main motivation is, that it is intuitively clear where to put a
> test. Even more importantly, where to look for already existing test
> coverage.
>
> cheers
> -thomas
>
>
>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev