Inline.

On 08/11/2011 05:12 PM, Richard Achmatowicz wrote:

Andrew

Can you give a simple, concrete example of a spec test case (citing part

of the J2EE spec, say), which illustrates:

Hehe, my PoC is filled with 'em. :)  But keep in mind we're not directly
necessarily citing the EE specs, because we're *not* building a TCK in AS7.

- what dependencies you want to be on the classpath, as well as those

you don't want to be only the classpath

For this discussion, I'm primarily concerned with the *compilation*
ClassPath, as this is the view users see when building their
applications.  It's about explicitly defining what the AS7 API is.  So
the deps for the compilation ClassPath should be as they are in by
AS7-999 branch and as described in the last email:

* spec - Java SE and Java EE
* api - JBoss-specific extensions to "spec" which comprise our AS7 API
* internals - Anything in the AS7 runtime

- what sample artifacts you would put in src/main and what you would put

in src/test

* src/main - Stuff that is part of the deployment.  Anything that you'd
be including in the @Deployment archive defined by an Arquillian-based
test case.
* src/test - The test infrastructure itself, like the JUnit test and
other util/helpers

- what compile and test execution time constraints you would want to enforce

This would help me to see the motivation for the refactoring and your

planned organization of artifacts in src/main and src/test, which

differ from the standard organization of putting all test related

artifacts in src/test.

Test execution time is an orthogonal concern, and relates instead back
to using "smoke" tests to run by default instead of the entire AS7
integration test suite (which won't scale to run on every build over
time).  IMO it's already taking too long for standard builds.

Again, the motivation for moving/organizing in this fashion is to honor
dependencies and assert that the APIs we export ("api" and "spec-api"
modules) are complete.  For instance, while moving tests around I
discovered that our POMs were incompete:

https://issues.jboss.org/browse/AS7-1489
https://issues.jboss.org/browse/AS7-1493
https://issues.jboss.org/browse/AS7-1479
https://issues.jboss.org/browse/AS7-1478

Also, is it not possible to control what is on the classpath by maven

elements like<classpathDependencyExcludes/>  and the like?

That's for test runtime.  Because Arquillian is starting (or connecting
to) the container in a remote process, we're less concerned with the
client runtime ClassPath, so long as it's enough for Arquillian to do
its thing.

S,
ALR

Richard

On 08/11/2011 07:38 AM, Andrew Lee Rubinger wrote:

Hi guys:

I'd like to reopen the discussion regarding the testsuite organization

and its ongoing maintenance.  This issue dates back a few months with

some debates and differing opinions, so I'll do my best to outline the

guiding principles I'd like to see put in place concisely.

To start off, I've a Proof-of-Concept for many of the following points

now located:

https://github.com/ALRubinger/jboss-as/tree/AS7-999

The relevant JIRA I've been using to track things:

https://issues.jboss.org/browse/AS7-999

So:

1) TestSuite Organization

I believe we need a single top-level categorization by which we may

organize integration tests which are deployment-based and run within the

context of the server.  Because we use Maven modules (which are bound to

a dependency structure), it makes sense to file these modules by the

compile-time dependencies they require.  So in place I've put:

testsuite/spec - Java SE and Java EE APIs only

testsuite/api - AS7 APIs + Spec

testsuite/internals - Use anything in the AS7 runtime in your

deployments; not guaranteed to be back-compat across releases

The primary motivation here is to ensure that the dependencies we export

(ie. "spec-api", and "api" modules) are complete enough for users to

create their own deployments.  In this setup, we act as *users* of our

own APIs, and everything in src/main is limited to the relevant

dependencies.

I know the source of some disagreements earlier centered around placing

the tests right next to the deployments, and some folks consider the

deployments as part of the test itself.  That's not a bad argument at

all, but again consider that we then lose the ability to validate our

tests in the context of our exported APIs.

2) Run Modes, Test Subsets

Because the primary organizational criteria proposed in 1) is by

dependency, these modules will grow large over time.  The AS build over

time will take longer and longer to run.  Additionally, there are

runtime options to consider when starting tests.  So consider the

following requirements:

    * Running the testsuite in IPv6

    * Running only a subset of tests as part of the main build

These lend themselves well to using build profiles.  By default, I think

the "smoke tests" should simply be a set of tests we deem important or

indicative of the general health of AS7 with respect to each subsystem.

   As it stands now, "smoke" is its own module with a bunch of

Embedded-based tests, and I think these should move to the

organizational structure in 1) and instead we can apply some filtering

to make the "smoke" some default set of includes.

3) An authoritative maintainer

I'd like to treat the Arquillian and TestSuite modules as true

subsystems of the Application Server, and as such we'll need someone to

assume the responsibility to review incoming commits/pull requests and

ensure they fit the criteria for acceptance.  Simple things like

consistent package names, using ARQ correctly, and not leaking

dependencies are very important.

So assuming we come to agreement on these points, I'd like to request

push access to the AS7 repo to field testsuite and ARQ-related pull

requests.

...there's much more to discuss (I've more issues to raise alongside the

upcoming EAP requirements), but let's start with those first 3 major

points and my POC, and run from there.

S,

ALR

_______________________________________________

jboss-as7-dev mailing list

jboss-as7-dev@lists.jboss.org

https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________

jboss-as7-dev mailing list

jboss-as7-dev@lists.jboss.org

https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev