[seam-dev] Arquillian update
Ken Finnigan
ken at kenfinnigan.me
Mon Jul 25 16:43:05 EDT 2011
On Jul 25, 2011, at 16:05, Jason Porter <lightguard.jp at gmail.com> wrote:
> I like the idea of what's there in the showcase.
>
> What we'd finally end up with is this:
>
> /api
> /src
> /main
> /java
> ...
> /test
> ...
> /impl
> /src
> /main
> /java
> ...
> /test
> ...
> /testsuite
> /common -- pom here
> /src
> /main
> -- tests go here correct? Or under test and just jar them up?
This would be deployment helpers, etc
> /internals
> /base -- how is this different than common?
> /src
> /main
This would be base tests that can be run against any container or a base for container specific extensions
> ....
> /<containers
This will be under testsuite as the container def will be common to internals, smoke, etc
>
> On Mon, Jul 25, 2011 at 12:40, Ken Finnigan <ken at kenfinnigan.me> wrote:
> All,
>
> Have not had much of a chance to work on this structure as of late, but hope to get back into it tonight.
>
> The other day I took a look at Dan's arquillian showcase and spotted something that might be of benefit to our structure. He had a module that held a separate pom definition for each container type (see here: https://github.com/mojavelinux/arquillian-showcase/tree/master/container-boms ) which I thought would be a good idea to implement for our testing structure. It would enable a single definition of the required libraries for a given container, without the need to repeat it across different test modules.
>
> As an initial structure, to simplify matters, I'm leaning towards:
>
> /testsuite
> /common
> /internals
> /base
> /jboss
> /weld-embedded
>
> And then either as the module decides to split out the tests, or add additional particular tests we can introduce the additional modules of:
>
> /testsuite
> /api
> /benchmark
> /clustering
> /smoke
> /stress
>
> What is everyone's thoughts on the above?
>
> Ken
>
>
>
> On Thu, Jul 14, 2011 at 4:47 AM, Jozef Hartinger <jharting at redhat.com> wrote:
> I am not against the new structure in general. I just find it unnecessary for many Seam 3 modules. The question is: What benefits does this structure bring to the modules mentioned earlier?
>
>
> On 07/13/2011 04:44 PM, Ken Finnigan wrote:
> It is the plan that this structure would become standard for all Seam 3 modules.
>
> I agree that the container specific requirements for tests can be achieved through profiles. However, I don't believe there is a way to run all tests against all container types within a single module as activating all the required profiles would add every arquillian container to the classpath and it wouldn't know which one to use (Could be wrong on this but that's my understanding).
> This seems to be the biggest benefit. However, is it really worth it? Personally, I do not consider running mvn once instead of 3 times (once for each container) a massive improvement. Furthermore, this again only applies to a developer running the tests locally since running module's tests on multiple containers within a single CI run is a non-issue.
>
>
> It also provides the added benefit of being part of the regular build/test cycle for a module, rather than having to remember to run the tests against each profile (and also remembering the correct names of the profiles).
> This is IMHO just a matter of unifying profile names.
> Remembering which profiles to run is not an issue for the CI environment. When running the tests locally, you would still need to remember to set path to the container anyway.
> Besides, I can see some minor issues with the proposed approach like : How do I set the JBOSS_HOME env. property if the testsuite is run on both 6 and 7 in one go?
>
>
> Whether the configuration is within a single module and there are multiple profiles, or multiple modules with a single profile, I wouldn't consider any one to be more complex than the other as they simply provide the same configuration in a slightly different fashion.
> Well, for many modules the new structure means switching from the integration tests placed within the impl module with single pom file containing configuration for all containers to something like:
>
> /testsuite
> pom.xml
> /common
> pom.xml
> /src/main/java - all the tests are here
> /src/main/resources - all test resources
> /jbossas6 - empty, only contains pom.xml with JBoss AS 6 profile
> pom.xml
> /jbossas7 - empty, only contains pom.xml with JBoss AS 7 profile
> pom.xml
> /glassfish - empty, only contains pom.xml with GF profile
> pom.xml
>
>
>
> On Wed, Jul 13, 2011 at 10:03 AM, Jozef Hartinger <jharting at redhat.com <mailto:jharting at redhat.com>> wrote:
>
> Do you plan to use this new structure in every Seam 3 module? If
> so ,I don't think it is a good idea. Modules like Servlet, REST,
> Solder, ... have very few container-specific setup requirements.
> All that many modules need is the correct Arquillian dependency
> which is currently achieved by a Maven profile. Optionally, some
> of the modules add or exclude container-specific tests from the
> testsuite. This can again be done with Maven profiles.
>
> Since these module do not need any container-specific artifacts
> for their testsuite, I find the proposed structure unnecessary
> complex. Also, I find splitting the testsuite into categories like
> smoke, cluster, internals, etc. quite artificial in this case.
>
>
> On 07/11/2011 04:41 PM, Ken Finnigan wrote:
>
> Yes it is a new structure for a Seam module for container testing.
>
> Do you mean that you would want to run the three below
> scenarios, for all tests, against whichever containers you
> define (ie. AS7, SE, etc)? I believe this should be possible
> by having base tests that define what the test does (in terms
> of the Seam JMS classes) and then separate tests that extend
> the base to add the specific libraries for that scenario, ie
> OWB + Apache ActiveMQ, which can then be further specialized
> for any specific container configuration that is required.
>
> So I would see you needing something like:
>
> \testsuite
> \internals
> \base (holds the base test cases, with the extended
> versions for each scenario of CDI/JMS provider combos)
> \jboss (profile for as7, with test case extended from
> the base that holds specific container config)
> \tomcat (profile plus test extended with container
> specific config.
>
> You can then have as many container modules as you want.
>
> Does that make sense?
>
> Over the next day or two I'll be completing the migration of
> i18n to option 3 so that it can be used as a base for other
> modules.
>
> Ken
>
>
> On Sun, Jul 10, 2011 at 1:35 PM, John D. Ament
> <john.d.ament at gmail.com <mailto:john.d.ament at gmail.com>
> <mailto:john.d.ament at gmail.com
> <mailto:john.d.ament at gmail.com>>> wrote:
>
> So what exactly is this? Is this a new structure to
> support testing?
>
> Here's something I'm interested in. I want to be able to test
> seam JMS in an SE type of environment, potentially with
> starting
> the JMS server in the same VM or a separate VM. Part of this
> requires special modules in Seam JMS that connect to remote JMS
> providers, some of this requires special testing scenarios.
> Ideally, I would want to run the full test suite
> repeatedly for
> each matching pair. For example:
>
> Seam JMS + OWB + Apache ActiveMQ
> Seam JMS + Weld + Hornetq
> Seam JMS + Weld + OpenMQ
>
> Is this approach attainable through this setup?
>
> John
>
> On Fri, Jul 8, 2011 at 12:11 PM, Jason Porter
> <lightguard.jp <http://lightguard.jp>
> <http://lightguard.jp>@gmail.com <http://gmail.com>
> <http://gmail.com>> wrote:
>
> I'm liking #3
>
> Sent from my iPhone
>
> On Jul 8, 2011, at 9:28, Ken Finnigan
> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
> <mailto:ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>>> wrote:
>
> I've been doing some further thinking around this
> over the
> last couple of days and have come up with 3 possible
> structures (there are undoubtedly more but that is
> as far as
> my brain got!):
>
> 1) The original idea below. ie.
> \testsuite
> \api
> \internals
> \smoke
>
> This setup requires each container, and version
> there of, to
> be a separate profile. Container specific tests can be
> excluded with surefire.
>
> This does get quite messy quickly if there are lots of
> containers, and lots of container specific tests
> that need to
> be written (ie. lots of exclusions in profiles for
> tests you
> don't want to run)
>
> 2) More along the lines of Seam Persistence. ie.
> \testsuite
> \base
> \jboss
> \weld-embedded
> \jetty
>
> Each version of a container will have their own profile
> within a container module.
>
> This nicely breaks down the containers, but makes it
> difficult to know what pieces of a module (ie. API,
> internals, etc) are being tested on each container.
>
> 3) A hybrid approach ie.
> \testsuite
> \base or common
> \internals
> \base
> \jboss
> \jetty
>
> We could either go with a base/common module at the
> root of
> testsuite, which would hold utilities for artifact
> creation
> and all common tests or base tests for all modules,
> and/or a
> base/common for each submodule (ie. api,
> internals). Could
> take either approach, but a single base/common at the
> testsuite root may make it simpler.
>
> Each container module would then have a profile for
> each
> version, with the version of a container that you
> want to
> test as part of builds set to be active by default.
>
>
> I'm personally leaning towards 3 as it gives us a
> breakdown
> of types of testing, api, internals, smoke,
> cluster, etc
> while also providing a breakdown of container
> specific test
> requirements.
>
> Note: All pure unit tests that do not require a
> container
> would still remain in the appropriate module, ie.
> api or
> impl, of the project.
>
> Any other options anyone can think of, or tweaks to
> any of
> the above options?
>
> What does everyone think?
>
> Ken
>
>
> On Wed, Jul 6, 2011 at 1:43 PM, Jason Porter
> <lightguard.jp <http://lightguard.jp>
> <http://lightguard.jp>@gmail.com <http://gmail.com>
> <http://gmail.com>> wrote:
>
> Adding the rest of the list as I think we're
> nearing the
> point we want some more feedback.
>
> On Wed, Jul 6, 2011 at 10:35, Ken Finnigan
> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
> <mailto:ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>>>
>
> wrote:
>
> Nasty, read the irc log, not nice at all on
> AS7 part!
>
> At the moment I have a structure similar to
> persistence, but more following AS7. ie:
>
> \module_parent
> \api
> \impl
> \testsuite
> \api
> \benchmark
> \clustering
> \internals
> \smoke
> \stress
>
>
> I like the ideas, but I think the structure
> should be
> along the lines of the containers (perhaps in
> addition
> to, if we're going to test things like
> clustering and perf).
>
> Idea being that pure unit tests (ie. no
> container)
> remain within either api/impl modules, then any
> container testing falls within the testsuite
> somewhere. Default test only runs api,
> internals and
> smoke, others are picked up by a profile.
>
>
> Having the different containers be different
> test modules
> also gets us away from having to rely on a
> profile (so
> you could run each set of tests in one go
> instead of
> multiple invocations or lots of profiles).
>
> At present most of these folders are just
> ideas about
> the kinds of tests we could do (along the
> AS7 lines),
> but they seemed appropriate. Existing i18n
> module
> tests are now in internals for testing the
> implementation. I see smoke as being more
> comprehensive tests covering combined use
> of features
> in the module, and quite possibly
> integration with
> other modules too.
>
>
> I do kind of like the idea of all the tests
> being inside
> the test suite, but I'm also okay with basic
> unit tests
> living in api and impl. I think we probably
> need to do
> some lifecycle tweaking and have those tests
> under the
> testsuite run under the integration phase like
> Solder.
>
> Which container is run is then all down to
> profile. Not sure whether to continue with
> embedded weld being
> the default container that is run when no
> profile is
> specified, or to force a profile to be
> specified for
> any container testing.
>
> Also had another thought today as to
> whether the poms
> for the test modules should define
> dependencies, or
> whether dependent libraries should be added
> directly
> to deployment artifact as library to prevent
> unintended libraries from appearing on test
> classpath.
>
> If we go with test modules instead of profiles this
> becomes a non issue as they'd just be test deps
> for the
> module.
>
> Thoughts?
>
>
>
> On Wed, Jul 6, 2011 at 11:35 AM, Jason Porter
> <lightguard.jp <http://lightguard.jp>
> <http://lightguard.jp>@gmail.com <http://gmail.com>
>
> <http://gmail.com>> wrote:
>
> Very cool. I got stuff working for
> catch last
> night too, but there is a problem if you're
> testing deployment errors. It's
> actually an AS7
> and Arquillian issue. I've opened JIRA
> tickets
> about both. I think what we'll want to
> do is set
> up testing similar to what Stuart has
> done in
> Persistence with multiple modules. We'll
> eventually want to test in (probably)
> embedded
> weld (it's runs quickly, could probably
> be like a
> smoke test), JBoss AS7, AS6, GF 3.1,
> Resin and an
> OWB container or embedded OWB. Not all
> of those
> containers work right now but then we
> really
> could say we know it all works on each
> CDI impl.
>
> Sent from my iPhone
>
> On Jul 6, 2011, at 7:44, Ken Finnigan
> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
> <mailto:ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>>>
>
> wrote:
>
> I actually "stole" some day job
> time to try at
> work as I thought the problem might
> be caused by
> the local build of as7 I had in my
> repo, but
> that wasn't it.
>
> Figured out the problem I was
> having with the
> aether class not being found was
> caused by the
> arquillian junit container
> dependency being
> specified in a parent pom, and then
> the managed
> container being specified in the
> pom running the
> tests, which meant Maven decided
> the versions of
> the jars in the pom running the
> tests should
> take precedence.
>
> Adding the junit container
> dependency to the pom
> running the tests fixed that problem.
>
> Should be able to finish up the
> test setup some
> time tonight to then forward to the
> mailing list
> for review.
>
> Ken
>
> On Tue, Jul 5, 2011 at 11:13 PM,
> Jason Porter
> <lightguard.jp <http://lightguard.jp>
> <http://lightguard.jp>@gmail.com <http://gmail.com>
>
> <http://gmail.com>> wrote:
>
> Sure, you let me know a time.
> I'm working on
> getting Catch up to date as
> well. BTW,
> Arquillian 1.0.0.Final will be
> out tomorrow,
> not sure about container
> support though.
>
>
> On Tue, Jul 5, 2011 at 20:22,
> Ken Finnigan
> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
> <mailto:ken at kenfinnigan.me
>
> <mailto:ken at kenfinnigan.me>>> wrote:
>
> Jason,
>
> If you have some time
> tomorrow it would
> be appreciated if you could
> take a look
> at the new testsuite setup
> on i18n. Here's the branch:
> git://github.com/kenfinnigan/international.git
> <http://github.com/kenfinnigan/international.git>
> <http://github.com/kenfinnigan/international.git>
>
>
> I thought I was making
> progress but now
> I seem to have managed to move
> backwards! For some reason
> I keep
> getting a NoClassDef errors
> on aether
> classes. It appears that
> the classes
> are brought in correctly
> from the
> arquillian junit container
> (from
> shrinkwrap beta 3), but
> then the jboss
> as managed container brings
> in an older
> version of shrinkwrap that
> doesn't
> include aether.
>
> I'm sure it's a simple fix,
> as I've
> mirrored it off confbuzz,
> servlet and
> rest, as well as jbossas,
> that all run
> as7 arquillian tests, but
> can't see the
> wood for the trees tonight.
> Will be
> looking at it again
> tomorrow evening,
> but if you can spot an easy
> fix that
> would be great!
>
> Thanks
> Ken
>
>
>
> On Tue, Jul 5, 2011 at 1:39
> PM, Jason
> Porter <lightguard.jp
> <http://lightguard.jp>
> <http://lightguard.jp>@gmail.com <http://gmail.com>
>
> <http://gmail.com>> wrote:
>
> Okay, great. Let me or
> Dan know if
> you need some help.
>
>
> On Tue, Jul 5, 2011 at
> 11:37, Ken
> Finnigan
> <ken at kenfinnigan.me <mailto:ken at kenfinnigan.me>
> <mailto:ken at kenfinnigan.me
>
> <mailto:ken at kenfinnigan.me>>> wrote:
>
> Pretty good.
>
> Have the basic
> structure for the
> module in terms of
> what needs to
> go where.
>
> Currently going
> through and
> getting the
> existing i18n tests
> working in AS7,
> which is proving
> more of a challenge
> than I expected!
>
> Of all the tests
> that pass in
> weld-ee-embedded,
> about a third
> actually pass in
> AS7! Most of
> it, I think, is down to
> differing
> classpaths, but should
> be in a position to
> push the
> feature branch to
> my fork of
> i18n in a couple of
> days.
>
> Once I've pushed it
> to my fork I
> was going to send
> an email to
> everyone on
> seam-dev to get
> their thoughts on the
> layout/setup for
> testing.
>
> btw, you don't need
> to worry
> about cc'ing my
> sorstech email
> in the future as I
> don't use
> that much anymore.
>
> Ken
>
>
>
> On Tue, Jul 5, 2011
> at 1:20 PM,
> Jason Porter
> <lightguard.jp <http://lightguard.jp>
> <http://lightguard.jp>@gmail.com <http://gmail.com>
>
> <http://gmail.com>> wrote:
>
> How goes things
> with i18n
> and the new
> Arquillian?
>
> -- Jason Porter
> http://lightguard-jp.blogspot.com
> http://twitter.com/lightguardjp
>
> Software Engineer
> Open Source
> Advocate
> Author of Seam
> Catch - Next
> Generation Java
> Exception
> Handling
>
> PGP key id:
> 926CCFF5
> PGP key
> available at:
> keyserver.net <http://keyserver.net>
> <http://keyserver.net>,
> pgp.mit.edu <http://pgp.mit.edu> <http://pgp.mit.edu>
>
>
>
>
>
>
>
> -- Jason Porter
> http://lightguard-jp.blogspot.com
> http://twitter.com/lightguardjp
>
> Software Engineer
> Open Source Advocate
> Author of Seam Catch - Next
> Generation Java
> Exception Handling
>
> PGP key id: 926CCFF5
> PGP key available at:
> keyserver.net <http://keyserver.net>
> <http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
> <http://pgp.mit.edu>
>
>
>
>
>
>
> -- Jason Porter
> http://lightguard-jp.blogspot.com
> http://twitter.com/lightguardjp
>
> Software Engineer
> Open Source Advocate
> Author of Seam Catch - Next
> Generation Java
> Exception Handling
>
> PGP key id: 926CCFF5
> PGP key available at:
> keyserver.net <http://keyserver.net>
> <http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
> <http://pgp.mit.edu>
>
>
>
>
>
>
> -- Jason Porter
> http://lightguard-jp.blogspot.com
> http://twitter.com/lightguardjp
>
> Software Engineer
> Open Source Advocate
> Author of Seam Catch - Next Generation Java
> Exception
> Handling
>
> PGP key id: 926CCFF5
> PGP key available at: keyserver.net
> <http://keyserver.net>
> <http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
> <http://pgp.mit.edu>
>
>
>
> _______________________________________________
> seam-dev mailing list
> seam-dev at lists.jboss.org <mailto:seam-dev at lists.jboss.org>
> <mailto:seam-dev at lists.jboss.org
> <mailto:seam-dev at lists.jboss.org>>
>
> https://lists.jboss.org/mailman/listinfo/seam-dev
>
>
>
>
> _______________________________________________
> seam-dev mailing list
> seam-dev at lists.jboss.org <mailto:seam-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/seam-dev
>
>
>
>
> _______________________________________________
> seam-dev mailing list
> seam-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/seam-dev
>
>
>
>
> --
> Jason Porter
> http://lightguard-jp.blogspot.com
> http://twitter.com/lightguardjp
>
> Software Engineer
> Open Source Advocate
> Author of Seam Catch - Next Generation Java Exception Handling
>
> PGP key id: 926CCFF5
> PGP key available at: keyserver.net, pgp.mit.edu
Ken
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/seam-dev/attachments/20110725/09f1c521/attachment-0001.html
More information about the seam-dev
mailing list