On Mon, Jul 25, 2011 at 5:19 PM, Jason Porter <lightguard.jp@gmail.com> wrote:

Seam parent seems like an ideal place to be putting them, we could name the pom files differently and make sure the artifactId is correct in each pom file. It would include arquillian deps mostly I would think, maybe also the build change to run them at the integration-phase instead of the test phase.

I'm not entirely sure how much more we'd be able to pull up and still support each module.

On Mon, Jul 25, 2011 at 15:07, José Rodolfo Freitas <joserodolfo.freitas@gmail.com> wrote:

I like the idea of having containers bom's.

but there's one thing we could try to avoid: repeating all those poms all over the modules (I didn't think about how it could be done, but it'd be awesome), the project structure is already complex.

Maybe the way to reuse this "boms" is still having profiles in seam-parent.

And I want to add that perhaps we could conceive a new module like "seam-testing" to hold all those patterns emerging from this test structure. Classes like deployment helpers and etc etc. I'm afraid that it wouldn't be possible to make the "bom reuse" a case for this module, but I think we should think about what in our tests could be reusable in other modules and other projects.

wdyt?

On Mon, Jul 25, 2011 at 5:43 PM, Ken Finnigan <ken@kenfinnigan.me> wrote:

On Jul 25, 2011, at 16:05, Jason Porter <lightguard.jp@gmail.com> wrote:

I like the idea of what's there in the showcase.

What we'd finally end up with is this:

/api
/src
/main

/java

...
/test
...
/impl
/src
/main
/java
...
/test

...
/testsuite
/common -- pom here
/src
/main
-- tests go here correct? Or under test and just jar them up?

This would be deployment helpers, etc

/internals
/base -- how is this different than common?
/src
/main

This would be base tests that can be run against any container or a base for container specific extensions

....

/<containers

This will be under testsuite as the container def will be common to internals, smoke, etc

On Mon, Jul 25, 2011 at 12:40, Ken Finnigan <ken@kenfinnigan.me> wrote:

All,

Have not had much of a chance to work on this structure as of late, but hope to get back into it tonight.

The other day I took a look at Dan's arquillian showcase and spotted something that might be of benefit to our structure. He had a module that held a separate pom definition for each container type (see here: https://github.com/mojavelinux/arquillian-showcase/tree/master/container-boms ) which I thought would be a good idea to implement for our testing structure. It would enable a single definition of the required libraries for a given container, without the need to repeat it across different test modules.

As an initial structure, to simplify matters, I'm leaning towards:

/testsuite
    /common
    /internals
        /base
        /jboss
        /weld-embedded

And then either as the module decides to split out the tests, or add additional particular tests we can introduce the additional modules of:

/testsuite
    /api
    /benchmark
    /clustering
    /smoke
    /stress

What is everyone's thoughts on the above?

Ken

On Thu, Jul 14, 2011 at 4:47 AM, Jozef Hartinger <jharting@redhat.com> wrote:

I am not against the new structure in general. I just find it unnecessary for many Seam 3 modules. The question is: What benefits does this structure bring to the modules mentioned earlier?

On 07/13/2011 04:44 PM, Ken Finnigan wrote:

It is the plan that this structure would become standard for all Seam 3 modules.

I agree that the container specific requirements for tests can be achieved through profiles. However, I don't believe there is a way to run all tests against all container types within a single module as activating all the required profiles would add every arquillian container to the classpath and it wouldn't know which one to use (Could be wrong on this but that's my understanding).

This seems to be the biggest benefit. However, is it really worth it? Personally, I do not consider running mvn once instead of 3 times (once for each container) a massive improvement. Furthermore, this again only applies to a developer running the tests locally since running module's tests on multiple containers within a single CI run is a non-issue.

It also provides the added benefit of being part of the regular build/test cycle for a module, rather than having to remember to run the tests against each profile (and also remembering the correct names of the profiles).

This is IMHO just a matter of unifying profile names.
Remembering which profiles to run is not an issue for the CI environment. When running the tests locally, you would still need to remember to set path to the container anyway.
Besides, I can see some minor issues with the proposed approach like : How do I set the JBOSS_HOME env. property if the testsuite is run on both 6 and 7 in one go?

Whether the configuration is within a single module and there are multiple profiles, or multiple modules with a single profile, I wouldn't consider any one to be more complex than the other as they simply provide the same configuration in a slightly different fashion.

Well, for many modules the new structure means switching from the integration tests placed within the impl module with single pom file containing configuration for all containers to something like:

/testsuite
pom.xml
/common
pom.xml
/src/main/java - all the tests are here
/src/main/resources - all test resources
/jbossas6 - empty, only contains pom.xml with JBoss AS 6 profile
pom.xml
/jbossas7 - empty, only contains pom.xml with JBoss AS 7 profile
pom.xml
/glassfish - empty, only contains pom.xml with GF profile
pom.xml

On Wed, Jul 13, 2011 at 10:03 AM, Jozef Hartinger <jharting@redhat.com <mailto:jharting@redhat.com>> wrote:

Do you plan to use this new structure in every Seam 3 module? If
so ,I don't think it is a good idea. Modules like Servlet, REST,
Solder, ... have very few container-specific setup requirements.
All that many modules need is the correct Arquillian dependency
which is currently achieved by a Maven profile. Optionally, some
of the modules add or exclude container-specific tests from the
testsuite. This can again be done with Maven profiles.

Since these module do not need any container-specific artifacts
for their testsuite, I find the proposed structure unnecessary
complex. Also, I find splitting the testsuite into categories like
smoke, cluster, internals, etc. quite artificial in this case.

On 07/11/2011 04:41 PM, Ken Finnigan wrote:

Yes it is a new structure for a Seam module for container testing.

Do you mean that you would want to run the three below
scenarios, for all tests, against whichever containers you
define (ie. AS7, SE, etc)? I believe this should be possible
by having base tests that define what the test does (in terms
of the Seam JMS classes) and then separate tests that extend
the base to add the specific libraries for that scenario, ie
OWB + Apache ActiveMQ, which can then be further specialized
for any specific container configuration that is required.

So I would see you needing something like:

\testsuite
\internals
\base (holds the base test cases, with the extended
versions for each scenario of CDI/JMS provider combos)
\jboss (profile for as7, with test case extended from
the base that holds specific container config)
\tomcat (profile plus test extended with container
specific config.

You can then have as many container modules as you want.

Does that make sense?

Over the next day or two I'll be completing the migration of
i18n to option 3 so that it can be used as a base for other
modules.

Ken

On Sun, Jul 10, 2011 at 1:35 PM, John D. Ament
<john.d.ament@gmail.com <mailto:john.d.ament@gmail.com>
<mailto:john.d.ament@gmail.com
<mailto:john.d.ament@gmail.com>>> wrote:

So what exactly is this? Is this a new structure to
support testing?

Here's something I'm interested in. I want to be able to test
seam JMS in an SE type of environment, potentially with
starting
the JMS server in the same VM or a separate VM. Part of this
requires special modules in Seam JMS that connect to remote JMS
providers, some of this requires special testing scenarios.
Ideally, I would want to run the full test suite
repeatedly for
each matching pair. For example:

Seam JMS + OWB + Apache ActiveMQ
Seam JMS + Weld + Hornetq
Seam JMS + Weld + OpenMQ

Is this approach attainable through this setup?

John

On Fri, Jul 8, 2011 at 12:11 PM, Jason Porter
<lightguard.jp <http://lightguard.jp>
<http://lightguard.jp>@gmail.com <http://gmail.com>
<http://gmail.com>> wrote:

I'm liking #3

Sent from my iPhone

On Jul 8, 2011, at 9:28, Ken Finnigan
<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>

<mailto:ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>>> wrote:

I've been doing some further thinking around this
over the
last couple of days and have come up with 3 possible
structures (there are undoubtedly more but that is
as far as
my brain got!):

1) The original idea below. ie.
\testsuite
\api
\internals
\smoke

This setup requires each container, and version
there of, to
be a separate profile. Container specific tests can be
excluded with surefire.

This does get quite messy quickly if there are lots of
containers, and lots of container specific tests
that need to
be written (ie. lots of exclusions in profiles for
tests you
don't want to run)

2) More along the lines of Seam Persistence. ie.
\testsuite
\base
\jboss
\weld-embedded
\jetty

Each version of a container will have their own profile
within a container module.

This nicely breaks down the containers, but makes it
difficult to know what pieces of a module (ie. API,
internals, etc) are being tested on each container.

3) A hybrid approach ie.
\testsuite
\base or common
\internals
\base
\jboss
\jetty

We could either go with a base/common module at the
root of
testsuite, which would hold utilities for artifact
creation
and all common tests or base tests for all modules,
and/or a
base/common for each submodule (ie. api,
internals). Could
take either approach, but a single base/common at the
testsuite root may make it simpler.

Each container module would then have a profile for
each
version, with the version of a container that you
want to
test as part of builds set to be active by default.

I'm personally leaning towards 3 as it gives us a
breakdown
of types of testing, api, internals, smoke,
cluster, etc
while also providing a breakdown of container
specific test
requirements.

Note: All pure unit tests that do not require a
container
would still remain in the appropriate module, ie.
api or
impl, of the project.

Any other options anyone can think of, or tweaks to
any of
the above options?

What does everyone think?

Ken

On Wed, Jul 6, 2011 at 1:43 PM, Jason Porter
<lightguard.jp <http://lightguard.jp>

<http://lightguard.jp>@gmail.com <http://gmail.com>

<http://gmail.com>> wrote:

Adding the rest of the list as I think we're
nearing the
point we want some more feedback.

On Wed, Jul 6, 2011 at 10:35, Ken Finnigan
<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>

<mailto:ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>>>

wrote:

Nasty, read the irc log, not nice at all on
AS7 part!

At the moment I have a structure similar to
persistence, but more following AS7. ie:

\module_parent
\api
\impl
\testsuite
\api
\benchmark
\clustering
\internals
\smoke
\stress

I like the ideas, but I think the structure
should be
along the lines of the containers (perhaps in
addition
to, if we're going to test things like
clustering and perf).

Idea being that pure unit tests (ie. no
container)
remain within either api/impl modules, then any
container testing falls within the testsuite
somewhere. Default test only runs api,
internals and
smoke, others are picked up by a profile.

Having the different containers be different
test modules
also gets us away from having to rely on a
profile (so
you could run each set of tests in one go
instead of
multiple invocations or lots of profiles).

At present most of these folders are just
ideas about
the kinds of tests we could do (along the
AS7 lines),
but they seemed appropriate. Existing i18n
module
tests are now in internals for testing the
implementation. I see smoke as being more
comprehensive tests covering combined use
of features
in the module, and quite possibly
integration with
other modules too.

I do kind of like the idea of all the tests
being inside
the test suite, but I'm also okay with basic
unit tests
living in api and impl. I think we probably
need to do
some lifecycle tweaking and have those tests
under the
testsuite run under the integration phase like
Solder.

Which container is run is then all down to
profile. Not sure whether to continue with
embedded weld being
the default container that is run when no
profile is
specified, or to force a profile to be
specified for
any container testing.

Also had another thought today as to
whether the poms
for the test modules should define
dependencies, or
whether dependent libraries should be added
directly
to deployment artifact as library to prevent
unintended libraries from appearing on test
classpath.

If we go with test modules instead of profiles this
becomes a non issue as they'd just be test deps
for the
module.

Thoughts?

On Wed, Jul 6, 2011 at 11:35 AM, Jason Porter
<lightguard.jp <http://lightguard.jp>

<http://lightguard.jp>@gmail.com <http://gmail.com>

<http://gmail.com>> wrote:

Very cool. I got stuff working for
catch last
night too, but there is a problem if you're
testing deployment errors. It's
actually an AS7
and Arquillian issue. I've opened JIRA
tickets
about both. I think what we'll want to
do is set
up testing similar to what Stuart has
done in
Persistence with multiple modules. We'll
eventually want to test in (probably)
embedded
weld (it's runs quickly, could probably
be like a
smoke test), JBoss AS7, AS6, GF 3.1,
Resin and an
OWB container or embedded OWB. Not all
of those
containers work right now but then we
really
could say we know it all works on each
CDI impl.

Sent from my iPhone

On Jul 6, 2011, at 7:44, Ken Finnigan
<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>

<mailto:ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>>>

wrote:

I actually "stole" some day job
time to try at
work as I thought the problem might
be caused by
the local build of as7 I had in my
repo, but
that wasn't it.

Figured out the problem I was
having with the
aether class not being found was
caused by the
arquillian junit container
dependency being
specified in a parent pom, and then
the managed
container being specified in the
pom running the
tests, which meant Maven decided
the versions of
the jars in the pom running the
tests should
take precedence.

Adding the junit container
dependency to the pom
running the tests fixed that problem.

Should be able to finish up the
test setup some
time tonight to then forward to the
mailing list
for review.

Ken

On Tue, Jul 5, 2011 at 11:13 PM,
Jason Porter
<lightguard.jp <http://lightguard.jp>

<http://lightguard.jp>@gmail.com <http://gmail.com>

<http://gmail.com>> wrote:

Sure, you let me know a time.
I'm working on
getting Catch up to date as
well. BTW,
Arquillian 1.0.0.Final will be
out tomorrow,
not sure about container
support though.

On Tue, Jul 5, 2011 at 20:22,
Ken Finnigan
<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>

<mailto:ken@kenfinnigan.me

<mailto:ken@kenfinnigan.me>>> wrote:

Jason,

If you have some time
tomorrow it would
be appreciated if you could
take a look
at the new testsuite setup
on i18n. Here's the branch:
git://github.com/kenfinnigan/international.git
<http://github.com/kenfinnigan/international.git>
<http://github.com/kenfinnigan/international.git>

I thought I was making
progress but now
I seem to have managed to move
backwards! For some reason
I keep
getting a NoClassDef errors
on aether
classes. It appears that
the classes
are brought in correctly
from the
arquillian junit container
(from
shrinkwrap beta 3), but
then the jboss
as managed container brings
in an older
version of shrinkwrap that
doesn't
include aether.

I'm sure it's a simple fix,
as I've
mirrored it off confbuzz,
servlet and
rest, as well as jbossas,
that all run
as7 arquillian tests, but
can't see the
wood for the trees tonight.
Will be
looking at it again
tomorrow evening,
but if you can spot an easy
fix that
would be great!

Thanks
Ken

On Tue, Jul 5, 2011 at 1:39
PM, Jason
Porter <lightguard.jp
<http://lightguard.jp>
<http://lightguard.jp>@gmail.com <http://gmail.com>

<http://gmail.com>> wrote:

Okay, great. Let me or
Dan know if
you need some help.

On Tue, Jul 5, 2011 at
11:37, Ken
Finnigan
<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>

<mailto:ken@kenfinnigan.me

<mailto:ken@kenfinnigan.me>>> wrote:

Pretty good.

Have the basic
structure for the
module in terms of
what needs to
go where.

Currently going
through and
getting the
existing i18n tests
working in AS7,
which is proving
more of a challenge
than I expected!

Of all the tests
that pass in
weld-ee-embedded,
about a third
actually pass in
AS7! Most of
it, I think, is down to
differing
classpaths, but should
be in a position to
push the
feature branch to
my fork of
i18n in a couple of
days.

Once I've pushed it
to my fork I
was going to send
an email to
everyone on
seam-dev to get
their thoughts on the
layout/setup for
testing.

btw, you don't need
to worry
about cc'ing my
sorstech email
in the future as I
don't use
that much anymore.

Ken

On Tue, Jul 5, 2011
at 1:20 PM,
Jason Porter
<lightguard.jp <http://lightguard.jp>

<http://lightguard.jp>@gmail.com <http://gmail.com>

<http://gmail.com>> wrote:

How goes things
with i18n
and the new
Arquillian?

-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp

Software Engineer
Open Source
Advocate
Author of Seam
Catch - Next
Generation Java
Exception
Handling

PGP key id:
926CCFF5
PGP key
available at:
keyserver.net <http://keyserver.net>
<http://keyserver.net>,
pgp.mit.edu <http://pgp.mit.edu> <http://pgp.mit.edu>

-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp

Software Engineer
Open Source Advocate
Author of Seam Catch - Next
Generation Java
Exception Handling

PGP key id: 926CCFF5
PGP key available at:
keyserver.net <http://keyserver.net>
<http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
<http://pgp.mit.edu>

-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp

Software Engineer
Open Source Advocate
Author of Seam Catch - Next
Generation Java
Exception Handling

PGP key id: 926CCFF5
PGP key available at:
keyserver.net <http://keyserver.net>
<http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
<http://pgp.mit.edu>

-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp

Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java
Exception
Handling

PGP key id: 926CCFF5
PGP key available at: keyserver.net
<http://keyserver.net>
<http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>

<http://pgp.mit.edu>

_______________________________________________
seam-dev mailing list
seam-dev@lists.jboss.org <mailto:seam-dev@lists.jboss.org>
<mailto:seam-dev@lists.jboss.org
<mailto:seam-dev@lists.jboss.org>>

https://lists.jboss.org/mailman/listinfo/seam-dev

_______________________________________________
seam-dev mailing list
seam-dev@lists.jboss.org <mailto:seam-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/seam-dev

_______________________________________________
seam-dev mailing list
seam-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/seam-dev

--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp

Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java Exception Handling

PGP key id: 926CCFF5
PGP key available at: keyserver.net, pgp.mit.edu

Ken

_______________________________________________
seam-dev mailing list
seam-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/seam-dev

--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp

Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java Exception Handling

PGP key id: 926CCFF5
PGP key available at: keyserver.net, pgp.mit.edu