[jboss-as7-dev] Requirements and Design Proposal: AS7 TestSuite

[Aslak Knutsen]

Shouldn't be a problem to implement a Managed Container v. that use the file based API.

Already see a lot of differences in AS 5 / 6 in regards to how the different APIs (MainDeployer vs ProfileService) register information that Arquillian needs about the deployment. So this is not a bad idea. 

-aslak-

----- Original Message -----
> Andrew, I assume this testing will use the Arquillian API for
> deploying and that the arquillian API will use the yet-to-defined
> (AFAIK) programmatic deployment API directly to do the deployment.
> 
> Any chance that all or some of these tests could also be run in a mode
> where it uses the file based deployment API to get that massaged too
> and by consequence have tests that rely on the same thing many users
> and tools will use ?
> 
> /max
> 
> On Mar 18, 2011, at 24:21, Andrew Lee Rubinger wrote:
> 
> > Looks like a lot of us have different ideas for what the AS7
> > Integration
> > TestSuite should consist of, so I'll kickoff with what I believe is
> > the
> > first design proposal towards getting coverage focused on the
> > end-user
> > (not certifying our own internals).
> >
> > I suspect this breaks down into two categories, which may be
> > modelled as
> > separate modules under the existing "testsuite" aggregator parent:
> >
> > * Specification
> > * AS-specific APIs
> >
> > This isn't difficult work, though I do think it's important we
> > consider
> > some hard rules. IMO we should be developing these suites as if we
> > were
> > application developers, not wearing our server dev hats.
> >
> > ----------------
> >
> > [End Goal]
> >
> > 1) No compile-time dependencies in the module except for what's
> > absolutely necessary.
> >
> > For the spec suite, this means: JDK and EE Spec APIs only in the
> > compilation classpath. Testable asset sources and resources (ie.
> > EJBs,
> > Servlets, etc) would live under src/main/* to enforce that. Only the
> > tests themselves would be located under "src/test/*".
> >
> > The AS-specific API suite may also add in our own APIs to the
> > compilation classpath, but the line should end there. In "test"
> > scope
> > we can place all runtime dependencies.
> >
> > For the specification suite, AS-specific grammars like our own
> > deployment descriptors are fine; these are in many ways equivalent
> > to
> > the TCK porting layer. We're not building a TCK; we're showing that
> > our
> > implementation supports the features advertised.
> >
> > 2) Every single new test created is to have an associated JIRA.
> >
> > We all remember the nightmare it was when the old AS4-6 suite would
> > fall
> > down. We'd comb through each test, at times trying to determine its
> > purpose. By linking to JIRA we get history of intent, which acts as
> > a
> > nice record even in the case that the test isn't so well-documented.
> > I'd argue that tests are a bigger asset than our code, and we should
> > be
> > thinking about these in terms of long-term maintenance to outlive
> > any
> > specific impl.
> >
> > 3) Documentation
> >
> > Alongside the JIRA reference, a quick note about we're looking to
> > accomplish is something I find very helpful. I don't personally buy
> > the
> > argument that code is self-documenting if written well. It gets
> > refactored and stale over time.
> >
> > 4) Run-mode profiles
> >
> > Arquillian provides a wonderful abstraction such that we can get
> > coverage for AS in both remote managed *and* embedded modes without
> > changing the test itself. To certify that everything is working as
> > advertised no matter the runtime, we should be able to run the same
> > suite in standalone, domain, and embedded modes (generally
> > speaking).
> >
> > 5) Porting of AS6 Tests
> >
> > There's no discounting the value this coverage has given us, though
> > I
> > question the purpose of a lot of these tests. I think a great
> > majority
> > of these need to come into the new codebase, refactored to align if
> > needed.
> >
> > ----------------
> >
> > [Current State]
> >
> > Here[1] is an example of what I believe to be a simple, well-written
> > test, with the exception that the tested Servlet and EJB are in the
> > same
> > test source folder.
> >
> > The current "testsuite" aggregator contains modules which mix our
> > end-user certification stuff alongside internals, so I think these
> > should be separated out.
> >
> > A lot of this is set up in some fashion already, but I would like to
> > see us:
> >
> > 1) Agree upon a strict scope for each type of testsuite along the
> > lines
> > of my points above, once we reach agreement
> > 2) Upgrade to ARQ 1.0.0.Alpha5 (which implies ShrinkWrap
> > 1.0.0-alpha-12), just released tonight. Currently AS is on a forked
> > release of ARQ for OSGi purposes, and these changes, if necessary,
> > need
> > to get upstream so we can do upgrades.
> >
> > It's clear that AS7 has made full-steam-ahead progress since last
> > summer, and with a little organization our testsuite can give us a
> > great
> > view of where we stand, from an end-user's perspective, with minimal
> > investment.
> >
> > S,
> > ALR
> >
> >
> > [1]
> > https://github.com/jbossas/jboss-as/blob/master/testsuite/integration/src/test/java/org/jboss/as/testsuite/integration/webejb/ServletInjectionTestCase.java
> > _______________________________________________
> > jboss-as7-dev mailing list
> > jboss-as7-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 
> /max
> http://about.me/maxandersen
> 
> 
> 
> 
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev