Do you plan to use this new structure in every Seam 3 module? If so ,I don't think it is a good idea. Modules like Servlet, REST, Solder, ... have very few container-specific setup requirements. All that many modules need is the correct Arquillian dependency which is currently achieved by a Maven profile. Optionally, some of the modules add or exclude container-specific tests from the testsuite. This can again be done with Maven profiles.
Since these module do not need any container-specific artifacts for their testsuite, I find the proposed structure unnecessary complex. Also, I find splitting the testsuite into categories like smoke, cluster, internals, etc. quite artificial in this case.
On 07/11/2011 04:41 PM, Ken Finnigan wrote:
Yes it is a new structure for a Seam module for container testing.
Do you mean that you would want to run the three below scenarios, for all tests, against whichever containers you define (ie. AS7, SE, etc)? I believe this should be possible by having base tests that define what the test does (in terms of the Seam JMS classes) and then separate tests that extend the base to add the specific libraries for that scenario, ie OWB + Apache ActiveMQ, which can then be further specialized for any specific container configuration that is required.
So I would see you needing something like:
\testsuite
\internals
\base (holds the base test cases, with the extended versions for each scenario of CDI/JMS provider combos)
\jboss (profile for as7, with test case extended from the base that holds specific container config)
\tomcat (profile plus test extended with container specific config.
You can then have as many container modules as you want.
Does that make sense?
Over the next day or two I'll be completing the migration of i18n to option 3 so that it can be used as a base for other modules.
Ken
On Sun, Jul 10, 2011 at 1:35 PM, John D. Ament <john.d.ament@gmail.com <mailto:john.d.ament@gmail.com>> wrote:
So what exactly is this? Is this a new structure to support testing?
Here's something I'm interested in. I want to be able to test
seam JMS in an SE type of environment, potentially with starting
the JMS server in the same VM or a separate VM. Part of this
requires special modules in Seam JMS that connect to remote JMS
providers, some of this requires special testing scenarios. Ideally, I would want to run the full test suite repeatedly for
each matching pair. For example:
Seam JMS + OWB + Apache ActiveMQ
Seam JMS + Weld + Hornetq
Seam JMS + Weld + OpenMQ
Is this approach attainable through this setup?
John
On Fri, Jul 8, 2011 at 12:11 PM, Jason Porter <lightguard.jp<http://lightguard.jp>@gmail.com <http://gmail.com>> wrote:
I'm liking #3
Sent from my iPhone
On Jul 8, 2011, at 9:28, Ken Finnigan <ken@kenfinnigan.meI've been doing some further thinking around this over the
last couple of days and have come up with 3 possible
structures (there are undoubtedly more but that is as far as
my brain got!):
1) The original idea below. ie.
\testsuite
\api
\internals
\smoke
This setup requires each container, and version there of, to
be a separate profile. Container specific tests can be
excluded with surefire.
This does get quite messy quickly if there are lots of
containers, and lots of container specific tests that need to
be written (ie. lots of exclusions in profiles for tests you
don't want to run)
2) More along the lines of Seam Persistence. ie.
\testsuite
\base
\jboss
\weld-embedded
\jetty
Each version of a container will have their own profile
within a container module.
This nicely breaks down the containers, but makes it
difficult to know what pieces of a module (ie. API,
internals, etc) are being tested on each container.
3) A hybrid approach ie.
\testsuite
\base or common
\internals
\base
\jboss
\jetty
We could either go with a base/common module at the root of
testsuite, which would hold utilities for artifact creation
and all common tests or base tests for all modules, and/or a
base/common for each submodule (ie. api, internals). Could
take either approach, but a single base/common at the
testsuite root may make it simpler.
Each container module would then have a profile for each
version, with the version of a container that you want to
test as part of builds set to be active by default.
I'm personally leaning towards 3 as it gives us a breakdown
of types of testing, api, internals, smoke, cluster, etc
while also providing a breakdown of container specific test
requirements.
Note: All pure unit tests that do not require a container
would still remain in the appropriate module, ie. api or
impl, of the project.
Any other options anyone can think of, or tweaks to any of
the above options?
What does everyone think?
Ken
On Wed, Jul 6, 2011 at 1:43 PM, Jason Porter <lightguard.jp<http://lightguard.jp>@gmail.com <http://gmail.com>> wrote:
Adding the rest of the list as I think we're nearing the
point we want some more feedback.
On Wed, Jul 6, 2011 at 10:35, Ken Finnigan<lightguard.jp <http://lightguard.jp>@gmail.com<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>> wrote:
Nasty, read the irc log, not nice at all on AS7 part!
At the moment I have a structure similar to
persistence, but more following AS7. ie:
\module_parent
\api
\impl
\testsuite
\api
\benchmark
\clustering
\internals
\smoke
\stress
I like the ideas, but I think the structure should be
along the lines of the containers (perhaps in addition
to, if we're going to test things like clustering and perf).
Idea being that pure unit tests (ie. no container)
remain within either api/impl modules, then any
container testing falls within the testsuite
somewhere. Default test only runs api, internals and
smoke, others are picked up by a profile.
Having the different containers be different test modules
also gets us away from having to rely on a profile (so
you could run each set of tests in one go instead of
multiple invocations or lots of profiles).
At present most of these folders are just ideas about
the kinds of tests we could do (along the AS7 lines),
but they seemed appropriate. Existing i18n module
tests are now in internals for testing the
implementation. I see smoke as being more
comprehensive tests covering combined use of features
in the module, and quite possibly integration with
other modules too.
I do kind of like the idea of all the tests being inside
the test suite, but I'm also okay with basic unit tests
living in api and impl. I think we probably need to do
some lifecycle tweaking and have those tests under the
testsuite run under the integration phase like Solder.
Which container is run is then all down to profile. Not sure whether to continue with embedded weld being
the default container that is run when no profile is
specified, or to force a profile to be specified for
any container testing.
Also had another thought today as to whether the poms
for the test modules should define dependencies, or
whether dependent libraries should be added directly
to deployment artifact as library to prevent
unintended libraries from appearing on test classpath.
If we go with test modules instead of profiles this
becomes a non issue as they'd just be test deps for the
module.
Thoughts?
On Wed, Jul 6, 2011 at 11:35 AM, Jason Porter<ken@kenfinnigan.me <mailto:ken@kenfinnigan.me>>
<http://gmail.com>> wrote:
Very cool. I got stuff working for catch last
night too, but there is a problem if you're
testing deployment errors. It's actually an AS7
and Arquillian issue. I've opened JIRA tickets
about both. I think what we'll want to do is set
up testing similar to what Stuart has done in
Persistence with multiple modules. We'll
eventually want to test in (probably) embedded
weld (it's runs quickly, could probably be like a
smoke test), JBoss AS7, AS6, GF 3.1, Resin and an
OWB container or embedded OWB. Not all of those
containers work right now but then we really
could say we know it all works on each CDI impl.
Sent from my iPhone
On Jul 6, 2011, at 7:44, Ken Finnigan
wrote:
I actually "stole" some day job time to try at<lightguard.jp <http://lightguard.jp>@gmail.com
work as I thought the problem might be caused by
the local build of as7 I had in my repo, but
that wasn't it.
Figured out the problem I was having with the
aether class not being found was caused by the
arquillian junit container dependency being
specified in a parent pom, and then the managed
container being specified in the pom running the
tests, which meant Maven decided the versions of
the jars in the pom running the tests should
take precedence.
Adding the junit container dependency to the pom
running the tests fixed that problem.
Should be able to finish up the test setup some
time tonight to then forward to the mailing list
for review.
Ken
On Tue, Jul 5, 2011 at 11:13 PM, Jason Porter
<http://gmail.com>> wrote:
Sure, you let me know a time. I'm working on
getting Catch up to date as well. BTW,
Arquillian 1.0.0.Final will be out tomorrow,
not sure about container support though.
On Tue, Jul 5, 2011 at 20:22, Ken Finnigan
<ken@kenfinnigan.me<mailto:ken@kenfinnigan.me>> wrote:<http://github.com/kenfinnigan/international.git>
Jason,
If you have some time tomorrow it would
be appreciated if you could take a look
at the new testsuite setup on i18n. Here's the branch:
git://github.com/kenfinnigan/international.git<http://lightguard.jp>@gmail.com
I thought I was making progress but now
I seem to have managed to move
backwards! For some reason I keep
getting a NoClassDef errors on aether
classes. It appears that the classes
are brought in correctly from the
arquillian junit container (from
shrinkwrap beta 3), but then the jboss
as managed container brings in an older
version of shrinkwrap that doesn't
include aether.
I'm sure it's a simple fix, as I've
mirrored it off confbuzz, servlet and
rest, as well as jbossas, that all run
as7 arquillian tests, but can't see the
wood for the trees tonight. Will be
looking at it again tomorrow evening,
but if you can spot an easy fix that
would be great!
Thanks
Ken
On Tue, Jul 5, 2011 at 1:39 PM, Jason
Porter <lightguard.jp
<http://gmail.com>> wrote:
Okay, great. Let me or Dan know if
you need some help.
On Tue, Jul 5, 2011 at 11:37, Ken
Finnigan <ken@kenfinnigan.me<http://lightguard.jp>@gmail.com<mailto:ken@kenfinnigan.me>> wrote:
Pretty good.
Have the basic structure for the
module in terms of what needs to
go where.
Currently going through and
getting the existing i18n tests
working in AS7, which is proving
more of a challenge than I expected!
Of all the tests that pass in
weld-ee-embedded, about a third
actually pass in AS7! Most of
it, I think, is down to
differing classpaths, but should
be in a position to push the
feature branch to my fork of
i18n in a couple of days.
Once I've pushed it to my fork I
was going to send an email to
everyone on seam-dev to get
their thoughts on the
layout/setup for testing.
btw, you don't need to worry
about cc'ing my sorstech email
in the future as I don't use
that much anymore.
Ken
On Tue, Jul 5, 2011 at 1:20 PM,
Jason Porter <lightguard.jp<http://keyserver.net>,
<http://gmail.com>> wrote:
How goes things with i18n
and the new Arquillian?
-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next
Generation Java Exception
Handling
PGP key id: 926CCFF5
PGP key available at:
keyserver.net
pgp.mit.edu <http://pgp.mit.edu><http://keyserver.net>, pgp.mit.edu
-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next
Generation Java Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net
<http://pgp.mit.edu><http://keyserver.net>, pgp.mit.edu
-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java
Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net
<http://pgp.mit.edu>
<http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
-- Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java Exception
Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net
_______________________________________________
seam-dev mailing list
seam-dev@lists.jboss.org <mailto:seam-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/seam-dev
_______________________________________________
seam-dev mailing list
seam-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/seam-dev