[jboss-as7-dev] Simplifying testsuite structure

[Jaikiran Pai]

I personally had like the module split that was there in testsuite2, 
where api, spec, and internals were separated out. So yes, I do agree 
with Andrew, that clubbing this all into one single "integration" module 
is going to make it a dumping ground for tests. Having them separate 
looks like a good idea. Once these modules are in place, as Carlo says, 
it's just a matter of documenting explicitly which test module is 
expected to contain what kind of tests.

-Jaikiran
On Tuesday 14 June 2011 10:47 AM, Andrew Lee Rubinger wrote:
> Hi, guys.
>
> So we have some disagreement towards unification of the testsuite here,
> and I'd like to get that ironed out as soon as we can.  So anyone with
> strong opinions, please weigh.
>
> Thomas has proposed collapsing "spec" and "api" into a single point:
> "integration".  I think this is actually a step back from where we
> should be heading:
>
> https://issues.jboss.org/browse/AS7-999
> http://lists.jboss.org/pipermail/jboss-as7-dev/2011-March/000851.html
>
> Now, it's taken me quite some time to do the prereq work in
> ShrinkWrap/Arquillian/AS7 ARQ fork unwinding to get to the point where
> I'm actually gunning fulltime now on that proposal, but the basic
> premise is this:
>
> * Maven modules imply a dependency structure
> * We should write tests/demos/samples as our users would
> * Be weary of limiting the dependencies to *only* what's required
>
> To this end, I've actually been working in the reverse direction from
> the points outlined in this Thread.  Instead of moving everything into a
> unified project to address all integration testing concerns, I've been
> breaking them into 3 categories:
>
> * spec (Java EE and SE APIs only)
> * api (spec, plus APIs we expose as public via AS - Hibernate
> annotations, EJB3 annotations, HornetQ API, etc)
> * internals (The whole runtime)
>
> Each of these represent integration tests, and the only difference is
> the dependencies available on the Compilation Classpath.  In other
> words, what a user will see as available in src/main/java.
>
> My work is being done here:
>
> https://github.com/ALRubinger/jboss-as/tree/AS7-999
>
> ...though I do need to rebase that and add my latest of today's commits
> to it.
>
> Also, there's a corollary issue to expose out all of the AS7 public APIs
> in one POM:
>
> https://issues.jboss.org/browse/AS7-650
>
> I do feel strongly that when it comes to testing, we must act as users,
> so I suspect this above is the best approach. :)
>
> S,
> ALR
>
> On 06/10/2011 03:38 AM, Thomas Diesler wrote:
>> Folks,
>>
>> the current state is
>>
>> + demos
>> + api
>> + internals
>> + legacy
>> + spec
>> + testsuite
>> + api
>> + benchmark
>> + domain
>> + integration
>> + smoke
>> + spec
>> + stress
>>
>> some of these modules are empty. It is hard to find the "right location"
>> for new tests. The demos are not self verifying. Instead I propose a
>> simplified structure like this
>>
>> + (demos removed)
>> + testsuite
>> + benchmark
>> + domain
>> + integration
>> + smoke
>> + stress
>>
>> #1 demos removed
>>
>> The artefacts in the various demos modules are currently reused by
>> testsuite modules (mainly smoke). demos are collapsed into smoke. I
>> believe a well documented smoke testsuite can serve the purpose of demos
>> and be a standalone deliverable. To be standalone you need to add a mvn
>> repository entry to smoke/pom.xml. Users can download this as a binary
>> and run 'mvn test' on it. Test cases should be organised in packages
>> according to their functional area. It is guaranteed that the demos work
>> because the smoke tests run on every build. Smoke tests should be well
>> documented.
>>
>> #2 Collapse testsuite api+spec into integration
>>
>> The integration testsuite should only have dependencies on spec and api
>> modules. Where this is not the case (i.e. a test depends on internal
>> impl) the dependency could be flagged and an api module could be made
>> available. Test cases should be organised in packages according to their
>> functional area. There may be test packages that reference jiras (e.g.
>> as835
>> <https://github.com/tdiesler/jboss-as/tree/master/testsuite/integration/src/test/java/org/jboss/as/testsuite/integration/as835>).
>>
>> The main motivation is, that it is intuitively clear where to put a
>> test. Even more importantly, where to look for already existing test
>> coverage.
>>
>> cheers
>> -thomas
>>
>>
>>
>> --
>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> Thomas Diesler
>> JBoss OSGi Lead
>> JBoss, a division of Red Hat
>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> jboss-as7-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev