[seam-dev] Arquillian update
Ken Finnigan
ken at kenfinnigan.me
Mon Jul 25 14:40:14 EDT 2011
All,
Have not had much of a chance to work on this structure as of late, but hope
to get back into it tonight.
The other day I took a look at Dan's arquillian showcase and spotted
something that might be of benefit to our structure. He had a module that
held a separate pom definition for each container type (see here:
https://github.com/mojavelinux/arquillian-showcase/tree/master/container-boms)
which I thought would be a good idea to implement for our testing
structure. It would enable a single definition of the required libraries
for a given container, without the need to repeat it across different test
modules.
As an initial structure, to simplify matters, I'm leaning towards:
/testsuite
/common
/internals
/base
/jboss
/weld-embedded
And then either as the module decides to split out the tests, or add
additional particular tests we can introduce the additional modules of:
/testsuite
/api
/benchmark
/clustering
/smoke
/stress
What is everyone's thoughts on the above?
Ken
On Thu, Jul 14, 2011 at 4:47 AM, Jozef Hartinger <jharting at redhat.com>wrote:
> I am not against the new structure in general. I just find it unnecessary
> for many Seam 3 modules. The question is: What benefits does this structure
> bring to the modules mentioned earlier?
>
>
> On 07/13/2011 04:44 PM, Ken Finnigan wrote:
>
>> It is the plan that this structure would become standard for all Seam 3
>> modules.
>>
>> I agree that the container specific requirements for tests can be achieved
>> through profiles. However, I don't believe there is a way to run all tests
>> against all container types within a single module as activating all the
>> required profiles would add every arquillian container to the classpath and
>> it wouldn't know which one to use (Could be wrong on this but that's my
>> understanding).
>>
> This seems to be the biggest benefit. However, is it really worth it?
> Personally, I do not consider running mvn once instead of 3 times (once for
> each container) a massive improvement. Furthermore, this again only applies
> to a developer running the tests locally since running module's tests on
> multiple containers within a single CI run is a non-issue.
>
>
> It also provides the added benefit of being part of the regular build/test
>> cycle for a module, rather than having to remember to run the tests against
>> each profile (and also remembering the correct names of the profiles).
>>
> This is IMHO just a matter of unifying profile names.
> Remembering which profiles to run is not an issue for the CI environment.
> When running the tests locally, you would still need to remember to set path
> to the container anyway.
> Besides, I can see some minor issues with the proposed approach like : How
> do I set the JBOSS_HOME env. property if the testsuite is run on both 6 and
> 7 in one go?
>
>
> Whether the configuration is within a single module and there are multiple
>> profiles, or multiple modules with a single profile, I wouldn't consider any
>> one to be more complex than the other as they simply provide the same
>> configuration in a slightly different fashion.
>>
> Well, for many modules the new structure means switching from the
> integration tests placed within the impl module with single pom file
> containing configuration for all containers to something like:
>
> /testsuite
> pom.xml
> /common
> pom.xml
> /src/main/java - all the tests are here
> /src/main/resources - all test resources
> /jbossas6 - empty, only contains pom.xml with JBoss AS 6 profile
> pom.xml
> /jbossas7 - empty, only contains pom.xml with JBoss AS 7 profile
> pom.xml
> /glassfish - empty, only contains pom.xml with GF profile
> pom.xml
>
>
>
>> On Wed, Jul 13, 2011 at 10:03 AM, Jozef Hartinger <jharting at redhat.com<mailto:
>> jharting at redhat.com>> wrote:
>>
>> Do you plan to use this new structure in every Seam 3 module? If
>> so ,I don't think it is a good idea. Modules like Servlet, REST,
>> Solder, ... have very few container-specific setup requirements.
>> All that many modules need is the correct Arquillian dependency
>> which is currently achieved by a Maven profile. Optionally, some
>> of the modules add or exclude container-specific tests from the
>> testsuite. This can again be done with Maven profiles.
>>
>> Since these module do not need any container-specific artifacts
>> for their testsuite, I find the proposed structure unnecessary
>> complex. Also, I find splitting the testsuite into categories like
>> smoke, cluster, internals, etc. quite artificial in this case.
>>
>>
>> On 07/11/2011 04:41 PM, Ken Finnigan wrote:
>>
>> Yes it is a new structure for a Seam module for container testing.
>>
>> Do you mean that you would want to run the three below
>> scenarios, for all tests, against whichever containers you
>> define (ie. AS7, SE, etc)? I believe this should be possible
>> by having base tests that define what the test does (in terms
>> of the Seam JMS classes) and then separate tests that extend
>> the base to add the specific libraries for that scenario, ie
>> OWB + Apache ActiveMQ, which can then be further specialized
>> for any specific container configuration that is required.
>>
>> So I would see you needing something like:
>>
>> \testsuite
>> \internals
>> \base (holds the base test cases, with the extended
>> versions for each scenario of CDI/JMS provider combos)
>> \jboss (profile for as7, with test case extended from
>> the base that holds specific container config)
>> \tomcat (profile plus test extended with container
>> specific config.
>>
>> You can then have as many container modules as you want.
>>
>> Does that make sense?
>>
>> Over the next day or two I'll be completing the migration of
>> i18n to option 3 so that it can be used as a base for other
>> modules.
>>
>> Ken
>>
>>
>> On Sun, Jul 10, 2011 at 1:35 PM, John D. Ament
>> <john.d.ament at gmail.com <mailto:john.d.ament at gmail.com**>
>> <mailto:john.d.ament at gmail.com
>> <mailto:john.d.ament at gmail.com**>>> wrote:
>>
>> So what exactly is this? Is this a new structure to
>> support testing?
>>
>> Here's something I'm interested in. I want to be able to test
>> seam JMS in an SE type of environment, potentially with
>> starting
>> the JMS server in the same VM or a separate VM. Part of this
>> requires special modules in Seam JMS that connect to remote JMS
>> providers, some of this requires special testing scenarios.
>> Ideally, I would want to run the full test suite
>> repeatedly for
>> each matching pair. For example:
>>
>> Seam JMS + OWB + Apache ActiveMQ
>> Seam JMS + Weld + Hornetq
>> Seam JMS + Weld + OpenMQ
>>
>> Is this approach attainable through this setup?
>>
>> John
>>
>> On Fri, Jul 8, 2011 at 12:11 PM, Jason Porter
>> <lightguard.jp <http://lightguard.jp>
>> <http://lightguard.jp>@gmail.**com <http://gmail.com> <
>> http://gmail.com>
>> <http://gmail.com>> wrote:
>>
>> I'm liking #3
>>
>> Sent from my iPhone
>>
>> On Jul 8, 2011, at 9:28, Ken Finnigan
>> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
>> <mailto:ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>>> wrote:
>>
>> I've been doing some further thinking around this
>> over the
>> last couple of days and have come up with 3 possible
>> structures (there are undoubtedly more but that is
>> as far as
>> my brain got!):
>>
>> 1) The original idea below. ie.
>> \testsuite
>> \api
>> \internals
>> \smoke
>>
>> This setup requires each container, and version
>> there of, to
>> be a separate profile. Container specific tests can be
>> excluded with surefire.
>>
>> This does get quite messy quickly if there are lots of
>> containers, and lots of container specific tests
>> that need to
>> be written (ie. lots of exclusions in profiles for
>> tests you
>> don't want to run)
>>
>> 2) More along the lines of Seam Persistence. ie.
>> \testsuite
>> \base
>> \jboss
>> \weld-embedded
>> \jetty
>>
>> Each version of a container will have their own profile
>> within a container module.
>>
>> This nicely breaks down the containers, but makes it
>> difficult to know what pieces of a module (ie. API,
>> internals, etc) are being tested on each container.
>>
>> 3) A hybrid approach ie.
>> \testsuite
>> \base or common
>> \internals
>> \base
>> \jboss
>> \jetty
>>
>> We could either go with a base/common module at the
>> root of
>> testsuite, which would hold utilities for artifact
>> creation
>> and all common tests or base tests for all modules,
>> and/or a
>> base/common for each submodule (ie. api,
>> internals). Could
>> take either approach, but a single base/common at the
>> testsuite root may make it simpler.
>>
>> Each container module would then have a profile for
>> each
>> version, with the version of a container that you
>> want to
>> test as part of builds set to be active by default.
>>
>>
>> I'm personally leaning towards 3 as it gives us a
>> breakdown
>> of types of testing, api, internals, smoke,
>> cluster, etc
>> while also providing a breakdown of container
>> specific test
>> requirements.
>>
>> Note: All pure unit tests that do not require a
>> container
>> would still remain in the appropriate module, ie.
>> api or
>> impl, of the project.
>>
>> Any other options anyone can think of, or tweaks to
>> any of
>> the above options?
>>
>> What does everyone think?
>>
>> Ken
>>
>>
>> On Wed, Jul 6, 2011 at 1:43 PM, Jason Porter
>> <lightguard.jp <http://lightguard.jp>
>> <http://lightguard.jp>@gmail.**com <http://gmail.com> <
>> http://gmail.com>
>> <http://gmail.com>> wrote:
>>
>> Adding the rest of the list as I think we're
>> nearing the
>> point we want some more feedback.
>>
>> On Wed, Jul 6, 2011 at 10:35, Ken Finnigan
>> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
>> <mailto:ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>>>
>>
>> wrote:
>>
>> Nasty, read the irc log, not nice at all on
>> AS7 part!
>>
>> At the moment I have a structure similar to
>> persistence, but more following AS7. ie:
>>
>> \module_parent
>> \api
>> \impl
>> \testsuite
>> \api
>> \benchmark
>> \clustering
>> \internals
>> \smoke
>> \stress
>>
>>
>> I like the ideas, but I think the structure
>> should be
>> along the lines of the containers (perhaps in
>> addition
>> to, if we're going to test things like
>> clustering and perf).
>>
>> Idea being that pure unit tests (ie. no
>> container)
>> remain within either api/impl modules, then any
>> container testing falls within the testsuite
>> somewhere. Default test only runs api,
>> internals and
>> smoke, others are picked up by a profile.
>>
>>
>> Having the different containers be different
>> test modules
>> also gets us away from having to rely on a
>> profile (so
>> you could run each set of tests in one go
>> instead of
>> multiple invocations or lots of profiles).
>>
>> At present most of these folders are just
>> ideas about
>> the kinds of tests we could do (along the
>> AS7 lines),
>> but they seemed appropriate. Existing i18n
>> module
>> tests are now in internals for testing the
>> implementation. I see smoke as being more
>> comprehensive tests covering combined use
>> of features
>> in the module, and quite possibly
>> integration with
>> other modules too.
>>
>>
>> I do kind of like the idea of all the tests
>> being inside
>> the test suite, but I'm also okay with basic
>> unit tests
>> living in api and impl. I think we probably
>> need to do
>> some lifecycle tweaking and have those tests
>> under the
>> testsuite run under the integration phase like
>> Solder.
>>
>> Which container is run is then all down to
>> profile. Not sure whether to continue with
>> embedded weld being
>> the default container that is run when no
>> profile is
>> specified, or to force a profile to be
>> specified for
>> any container testing.
>>
>> Also had another thought today as to
>> whether the poms
>> for the test modules should define
>> dependencies, or
>> whether dependent libraries should be added
>> directly
>> to deployment artifact as library to prevent
>> unintended libraries from appearing on test
>> classpath.
>>
>> If we go with test modules instead of profiles this
>> becomes a non issue as they'd just be test deps
>> for the
>> module.
>>
>> Thoughts?
>>
>>
>>
>> On Wed, Jul 6, 2011 at 11:35 AM, Jason Porter
>> <lightguard.jp <http://lightguard.jp>
>> <http://lightguard.jp>@gmail.**com <http://gmail.com> <
>> http://gmail.com>
>>
>> <http://gmail.com>> wrote:
>>
>> Very cool. I got stuff working for
>> catch last
>> night too, but there is a problem if you're
>> testing deployment errors. It's
>> actually an AS7
>> and Arquillian issue. I've opened JIRA
>> tickets
>> about both. I think what we'll want to
>> do is set
>> up testing similar to what Stuart has
>> done in
>> Persistence with multiple modules. We'll
>> eventually want to test in (probably)
>> embedded
>> weld (it's runs quickly, could probably
>> be like a
>> smoke test), JBoss AS7, AS6, GF 3.1,
>> Resin and an
>> OWB container or embedded OWB. Not all
>> of those
>> containers work right now but then we
>> really
>> could say we know it all works on each
>> CDI impl.
>>
>> Sent from my iPhone
>>
>> On Jul 6, 2011, at 7:44, Ken Finnigan
>> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
>> <mailto:ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>>>
>>
>> wrote:
>>
>> I actually "stole" some day job
>> time to try at
>> work as I thought the problem might
>> be caused by
>> the local build of as7 I had in my
>> repo, but
>> that wasn't it.
>>
>> Figured out the problem I was
>> having with the
>> aether class not being found was
>> caused by the
>> arquillian junit container
>> dependency being
>> specified in a parent pom, and then
>> the managed
>> container being specified in the
>> pom running the
>> tests, which meant Maven decided
>> the versions of
>> the jars in the pom running the
>> tests should
>> take precedence.
>>
>> Adding the junit container
>> dependency to the pom
>> running the tests fixed that problem.
>>
>> Should be able to finish up the
>> test setup some
>> time tonight to then forward to the
>> mailing list
>> for review.
>>
>> Ken
>>
>> On Tue, Jul 5, 2011 at 11:13 PM,
>> Jason Porter
>> <lightguard.jp <http://lightguard.jp>
>> <http://lightguard.jp>@gmail.**com <http://gmail.com> <
>> http://gmail.com>
>>
>> <http://gmail.com>> wrote:
>>
>> Sure, you let me know a time.
>> I'm working on
>> getting Catch up to date as
>> well. BTW,
>> Arquillian 1.0.0.Final will be
>> out tomorrow,
>> not sure about container
>> support though.
>>
>>
>> On Tue, Jul 5, 2011 at 20:22,
>> Ken Finnigan
>> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
>> <mailto:ken at kenfinnigan.me
>>
>> <mailto:ken at kenfinnigan.me>>> wrote:
>>
>> Jason,
>>
>> If you have some time
>> tomorrow it would
>> be appreciated if you could
>> take a look
>> at the new testsuite setup
>> on i18n. Here's the branch:
>> git://
>> github.com/kenfinnigan/**international.git<http://github.com/kenfinnigan/international.git>
>> <http://github.com/**kenfinnigan/international.git<http://github.com/kenfinnigan/international.git>
>> >
>> <http://github.com/**kenfinnigan/international.git<http://github.com/kenfinnigan/international.git>
>> >
>>
>>
>> I thought I was making
>> progress but now
>> I seem to have managed to move
>> backwards! For some reason
>> I keep
>> getting a NoClassDef errors
>> on aether
>> classes. It appears that
>> the classes
>> are brought in correctly
>> from the
>> arquillian junit container
>> (from
>> shrinkwrap beta 3), but
>> then the jboss
>> as managed container brings
>> in an older
>> version of shrinkwrap that
>> doesn't
>> include aether.
>>
>> I'm sure it's a simple fix,
>> as I've
>> mirrored it off confbuzz,
>> servlet and
>> rest, as well as jbossas,
>> that all run
>> as7 arquillian tests, but
>> can't see the
>> wood for the trees tonight.
>> Will be
>> looking at it again
>> tomorrow evening,
>> but if you can spot an easy
>> fix that
>> would be great!
>>
>> Thanks
>> Ken
>>
>>
>>
>> On Tue, Jul 5, 2011 at 1:39
>> PM, Jason
>> Porter <lightguard.jp
>> <http://lightguard.jp>
>> <http://lightguard.jp>@gmail.**com <http://gmail.com> <
>> http://gmail.com>
>>
>> <http://gmail.com>> wrote:
>>
>> Okay, great. Let me or
>> Dan know if
>> you need some help.
>>
>>
>> On Tue, Jul 5, 2011 at
>> 11:37, Ken
>> Finnigan
>> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
>> <mailto:ken at kenfinnigan.me
>>
>> <mailto:ken at kenfinnigan.me>>> wrote:
>>
>> Pretty good.
>>
>> Have the basic
>> structure for the
>> module in terms of
>> what needs to
>> go where.
>>
>> Currently going
>> through and
>> getting the
>> existing i18n tests
>> working in AS7,
>> which is proving
>> more of a challenge
>> than I expected!
>>
>> Of all the tests
>> that pass in
>> weld-ee-embedded,
>> about a third
>> actually pass in
>> AS7! Most of
>> it, I think, is down to
>> differing
>> classpaths, but should
>> be in a position to
>> push the
>> feature branch to
>> my fork of
>> i18n in a couple of
>> days.
>>
>> Once I've pushed it
>> to my fork I
>> was going to send
>> an email to
>> everyone on
>> seam-dev to get
>> their thoughts on the
>> layout/setup for
>> testing.
>>
>> btw, you don't need
>> to worry
>> about cc'ing my
>> sorstech email
>> in the future as I
>> don't use
>> that much anymore.
>>
>> Ken
>>
>>
>>
>> On Tue, Jul 5, 2011
>> at 1:20 PM,
>> Jason Porter
>> <lightguard.jp <http://lightguard.jp>
>> <http://lightguard.jp>@gmail.**com <http://gmail.com> <
>> http://gmail.com>
>>
>> <http://gmail.com>> wrote:
>>
>> How goes things
>> with i18n
>> and the new
>> Arquillian?
>>
>> --
>> Jason Porter
>> http://lightguard-jp.blogspot.**com<http://lightguard-jp.blogspot.com>
>> http://twitter.com/**lightguardjp<http://twitter.com/lightguardjp>
>>
>> Software Engineer
>> Open Source
>> Advocate
>> Author of Seam
>> Catch - Next
>> Generation Java
>> Exception
>> Handling
>>
>> PGP key id:
>> 926CCFF5
>> PGP key
>> available at:
>> keyserver.net <http://keyserver.net>
>> <http://keyserver.net>,
>> pgp.mit.edu <http://pgp.mit.edu> <http://pgp.mit.edu>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Jason Porter
>> http://lightguard-jp.blogspot.**com<http://lightguard-jp.blogspot.com>
>> http://twitter.com/**lightguardjp<http://twitter.com/lightguardjp>
>>
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next
>> Generation Java
>> Exception Handling
>>
>> PGP key id: 926CCFF5
>> PGP key available at:
>> keyserver.net <http://keyserver.net>
>> <http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
>> <http://pgp.mit.edu>
>>
>>
>>
>>
>>
>>
>> --
>> Jason Porter
>> http://lightguard-jp.blogspot.**com<http://lightguard-jp.blogspot.com>
>> http://twitter.com/**lightguardjp<http://twitter.com/lightguardjp>
>>
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next
>> Generation Java
>> Exception Handling
>>
>> PGP key id: 926CCFF5
>> PGP key available at:
>> keyserver.net <http://keyserver.net>
>> <http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
>> <http://pgp.mit.edu>
>>
>>
>>
>>
>>
>>
>> -- Jason Porter
>> http://lightguard-jp.blogspot.**com<http://lightguard-jp.blogspot.com>
>> http://twitter.com/**lightguardjp<http://twitter.com/lightguardjp>
>>
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next Generation Java
>> Exception
>> Handling
>>
>> PGP key id: 926CCFF5
>> PGP key available at: keyserver.net
>> <http://keyserver.net>
>> <http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
>> <http://pgp.mit.edu>
>>
>>
>>
>> ______________________________**_________________
>> seam-dev mailing list
>> seam-dev at lists.jboss.org <mailto:seam-dev at lists.jboss.**org<seam-dev at lists.jboss.org>
>> >
>> <mailto:seam-dev at lists.jboss.**org <seam-dev at lists.jboss.org>
>> <mailto:seam-dev at lists.jboss.**org <seam-dev at lists.jboss.org>>>
>>
>> https://lists.jboss.org/**mailman/listinfo/seam-dev<https://lists.jboss.org/mailman/listinfo/seam-dev>
>>
>>
>>
>>
>> ______________________________**_________________
>> seam-dev mailing list
>> seam-dev at lists.jboss.org <mailto:seam-dev at lists.jboss.**org<seam-dev at lists.jboss.org>
>> >
>> https://lists.jboss.org/**mailman/listinfo/seam-dev<https://lists.jboss.org/mailman/listinfo/seam-dev>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/seam-dev/attachments/20110725/7e35b202/attachment-0001.html
More information about the seam-dev
mailing list