#2 Collapse testsuite api+spec into integration

Folks, the current state is + demos + api + internals + legacy + spec + testsuite + api + benchmark + domain + integration + smoke + spec + stress some of these modules are empty. It is hard to find the "right location" for new tests. The demos are not self verifying. Instead I propose a simplified structure like this + (demos removed) + testsuite + benchmark + domain + integration + smoke + stress #1 demos removed The artefacts in the various demos modules are currently reused by testsuite modules (mainly smoke). demos are collapsed into smoke. I believe a well documented smoke testsuite can serve the purpose of demos and be a standalone deliverable. To be standalone you need to add a mvn repository entry to smoke/pom.xml. Users can download this as a binary and run 'mvn test' on it. Test cases should be organised in packages according to their functional area. It is guaranteed that the demos work because the smoke tests run on every build. Smoke tests should be well documented. #2 Collapse testsuite api+spec into integration The integration testsuite should only have dependencies on spec and api modules. Where this is not the case (i.e. a test depends on internal impl) the dependency could be flagged and an api module could be made available. Test cases should be organised in packages according to their functional area. There may be test packages that reference jiras (e.g. as835 <https://github.com/tdiesler/jboss-as/tree/master/testsuite/integration/sr...>). The main motivation is, that it is intuitively clear where to put a test. Even more importantly, where to look for already existing test coverage. cheers -thomas -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Thomas Diesler JBoss OSGi Lead JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Folks, the current state is + demos + api + internals + legacy + spec + testsuite + api + benchmark + domain + integration + smoke + spec + stress some of these modules are empty. It is hard to find the "right location" for new tests. The demos are not self verifying. Instead I propose a simplified structure like this + (demos removed) + testsuite + benchmark + domain + integration + smoke + stress #1 demos removed The artefacts in the various demos modules are currently reused by testsuite modules (mainly smoke). demos are collapsed into smoke. I believe a well documented smoke testsuite can serve the purpose of demos and be a standalone deliverable. To be standalone you need to add a mvn repository entry to smoke/pom.xml. Users can download this as a binary and run 'mvn test' on it. Test cases should be organised in packages according to their functional area. It is guaranteed that the demos work because the smoke tests run on every build. Smoke tests should be well documented. #2 Collapse testsuite api+spec into integration The integration testsuite should only have dependencies on spec and api modules. Where this is not the case (i.e. a test depends on internal impl) the dependency could be flagged and an api module could be made available. Test cases should be organised in packages according to their functional area. There may be test packages that reference jiras (e.g. as835 <https://github.com/tdiesler/jboss-as/tree/master/testsuite/integration/sr...>). The main motivation is, that it is intuitively clear where to put a test. Even more importantly, where to look for already existing test coverage. cheers -thomas -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Thomas Diesler JBoss OSGi Lead JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

I personally had like the module split that was there in testsuite2, where api, spec, and internals were separated out. So yes, I do agree with Andrew, that clubbing this all into one single "integration" module is going to make it a dumping ground for tests. Having them separate looks like a good idea. Once these modules are in place, as Carlo says, it's just a matter of documenting explicitly which test module is expected to contain what kind of tests. -Jaikiran On Tuesday 14 June 2011 10:47 AM, Andrew Lee Rubinger wrote: > Hi, guys. > > So we have some disagreement towards unification of the testsuite here, > and I'd like to get that ironed out as soon as we can. So anyone with > strong opinions, please weigh. > > Thomas has proposed collapsing "spec" and "api" into a single point: > "integration". I think this is actually a step back from where we > should be heading: > > https://issues.jboss.org/browse/AS7-999 > http://lists.jboss.org/pipermail/jboss-as7-dev/2011-March/000851.html > > Now, it's taken me quite some time to do the prereq work in > ShrinkWrap/Arquillian/AS7 ARQ fork unwinding to get to the point where > I'm actually gunning fulltime now on that proposal, but the basic > premise is this: > > * Maven modules imply a dependency structure > * We should write tests/demos/samples as our users would > * Be weary of limiting the dependencies to *only* what's required > > To this end, I've actually been working in the reverse direction from > the points outlined in this Thread. Instead of moving everything into a > unified project to address all integration testing concerns, I've been > breaking them into 3 categories: > > * spec (Java EE and SE APIs only) > * api (spec, plus APIs we expose as public via AS - Hibernate > annotations, EJB3 annotations, HornetQ API, etc) > * internals (The whole runtime) > > Each of these represent integration tests, and the only difference is > the dependencies available on the Compilation Classpath. In other > words, what a user will see as available in src/main/java. > > My work is being done here: > > https://github.com/ALRubinger/jboss-as/tree/AS7-999 > > ...though I do need to rebase that and add my latest of today's commits > to it. > > Also, there's a corollary issue to expose out all of the AS7 public APIs > in one POM: > > https://issues.jboss.org/browse/AS7-650 > > I do feel strongly that when it comes to testing, we must act as users, > so I suspect this above is the best approach. :) > > S, > ALR > > On 06/10/2011 03:38 AM, Thomas Diesler wrote: >> Folks, >> >> the current state is >> >> + demos >> + api >> + internals >> + legacy >> + spec >> + testsuite >> + api >> + benchmark >> + domain >> + integration >> + smoke >> + spec >> + stress >> >> some of these modules are empty. It is hard to find the "right location" >> for new tests. The demos are not self verifying. Instead I propose a >> simplified structure like this >> >> + (demos removed) >> + testsuite >> + benchmark >> + domain >> + integration >> + smoke >> + stress >> >> #1 demos removed >> >> The artefacts in the various demos modules are currently reused by >> testsuite modules (mainly smoke). demos are collapsed into smoke. I >> believe a well documented smoke testsuite can serve the purpose of demos >> and be a standalone deliverable. To be standalone you need to add a mvn >> repository entry to smoke/pom.xml. Users can download this as a binary >> and run 'mvn test' on it. Test cases should be organised in packages >> according to their functional area. It is guaranteed that the demos work >> because the smoke tests run on every build. Smoke tests should be well >> documented. >> >> #2 Collapse testsuite api+spec into integration >> >> The integration testsuite should only have dependencies on spec and api >> modules. Where this is not the case (i.e. a test depends on internal >> impl) the dependency could be flagged and an api module could be made >> available. Test cases should be organised in packages according to their >> functional area. There may be test packages that reference jiras (e.g. >> as835 >> <https://github.com/tdiesler/jboss-as/tree/master/testsuite/integration/sr...>). >> >> The main motivation is, that it is intuitively clear where to put a >> test. Even more importantly, where to look for already existing test >> coverage. >> >> cheers >> -thomas >> >> >> >> -- >> xxxxxxxxxxxxxxxxxxxxxxxxxxxx >> Thomas Diesler >> JBoss OSGi Lead >> JBoss, a division of Red Hat >> xxxxxxxxxxxxxxxxxxxxxxxxxxxx >> >> >> >> _______________________________________________ >> jboss-as7-dev mailing list >> jboss-as7-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Folks, the current state is + demos + api + internals + legacy + spec + testsuite + api + benchmark + domain + integration + smoke + spec + stress some of these modules are empty. It is hard to find the "right location" for new tests. The demos are not self verifying. Instead I propose a simplified structure like this + (demos removed) + testsuite + benchmark + domain + integration + smoke + stress #1 demos removed The artefacts in the various demos modules are currently reused by testsuite modules (mainly smoke). demos are collapsed into smoke. I believe a well documented smoke testsuite can serve the purpose of demos and be a standalone deliverable. To be standalone you need to add a mvn repository entry to smoke/pom.xml. Users can download this as a binary and run 'mvn test' on it. Test cases should be organised in packages according to their functional area. It is guaranteed that the demos work because the smoke tests run on every build. Smoke tests should be well documented. #2 Collapse testsuite api+spec into integration The integration testsuite should only have dependencies on spec and api modules. Where this is not the case (i.e. a test depends on internal impl) the dependency could be flagged and an api module could be made available. Test cases should be organised in packages according to their functional area. There may be test packages that reference jiras (e.g. as835). The main motivation is, that it is intuitively clear where to put a test. Even more importantly, where to look for already existing test coverage. cheers -thomas -- xxxxxxxxxxxxxxxxxxxxxxxxxxxx Thomas Diesler JBoss OSGi Lead JBoss, a division of Red Hat xxxxxxxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev