I've been doing some further thinking around this over the last couple of days and have come up with 3 possible structures (there are undoubtedly more but that is as far as my brain got!):<br><br>1) The original idea below. ie.<br>
\testsuite<br> \api<br> \internals<br> \smoke<br><br>This setup requires each container, and version there of, to be a separate profile. Container specific tests can be excluded with surefire.<br><br>This does get quite messy quickly if there are lots of containers, and lots of container specific tests that need to be written (ie. lots of exclusions in profiles for tests you don't want to run)<br>
<br>2) More along the lines of Seam Persistence. ie.<br>\testsuite<br> \base<br> \jboss<br> \weld-embedded<br> \jetty<br><br>Each version of a container will have their own profile within a container module.<br>
<br>This nicely breaks down the containers, but makes it difficult to know what pieces of a module (ie. API, internals, etc) are being tested on each container.<br><br>3) A hybrid approach ie.<br>\testsuite<br> \base or common<br>
\internals<br> \base<br> \jboss<br> \jetty<br><br>We could either go with a base/common module at the root of testsuite, which would hold utilities for artifact creation and all common tests or base tests for all modules, and/or a base/common for each submodule (ie. api, internals). Could take either approach, but a single base/common at the testsuite root may make it simpler.<br>
<br>Each container module would then have a profile for each version, with the version of a container that you want to test as part of builds set to be active by default.<br><br><br>I'm personally leaning towards 3 as it gives us a breakdown of types of testing, api, internals, smoke, cluster, etc while also providing a breakdown of container specific test requirements.<br>
<br>Note: All pure unit tests that do not require a container would still remain in the appropriate module, ie. api or impl, of the project.<br><br>Any other options anyone can think of, or tweaks to any of the above options?<br>
<br>What does everyone think?<br><br>Ken<br><br><br><div class="gmail_quote">On Wed, Jul 6, 2011 at 1:43 PM, Jason Porter <span dir="ltr"><<a href="http://lightguard.jp">lightguard.jp</a>@<a href="http://gmail.com">gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>Adding the rest of the list as I think we're nearing the point we want some more feedback.</div><div class="im">
<div><br></div><div>On Wed, Jul 6, 2011 at 10:35, Ken Finnigan <span dir="ltr"><<a href="mailto:ken@kenfinnigan.me" target="_blank">ken@kenfinnigan.me</a>></span> wrote:</div>
</div><div><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Nasty, read the irc log, not nice at all on AS7 part!<br><br>At the moment I have a structure similar to persistence, but more following AS7. ie:<br>
<br>\module_parent<br> \api<br> \impl<br> \testsuite<br> \api<br>
\benchmark<br> \clustering<br> \internals<br> \smoke<br> \stress<br></blockquote><div><br></div></div><div>I like the ideas, but I think the structure should be along the lines of the containers (perhaps in addition to, if we're going to test things like clustering and perf).</div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Idea being that pure unit tests (ie. no container) remain within either api/impl modules, then any container testing falls within the testsuite somewhere. Default test only runs api, internals and smoke, others are picked up by a profile.<br>
</blockquote><div><br></div></div><div>Having the different containers be different test modules also gets us away from having to rely on a profile (so you could run each set of tests in one go instead of multiple invocations or lots of profiles).</div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">At present most of these folders are just ideas about the kinds of tests we could do (along the AS7 lines), but they seemed appropriate. Existing i18n module tests are now in internals for testing the implementation. I see smoke as being more comprehensive tests covering combined use of features in the module, and quite possibly integration with other modules too.<br>
</blockquote><div><br></div></div><div>I do kind of like the idea of all the tests being inside the test suite, but I'm also okay with basic unit tests living in api and impl. I think we probably need to do some lifecycle tweaking and have those tests under the testsuite run under the integration phase like Solder.</div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Which container is run is then all down to profile. Not sure whether to continue with embedded weld being the default container that is run when no profile is specified, or to force a profile to be specified for any container testing.<br>
<br>Also had another thought today as to whether the poms for the test modules should define dependencies, or whether dependent libraries should be added directly to deployment artifact as library to prevent unintended libraries from appearing on test classpath.<br>
</blockquote><div> </div></div><div>If we go with test modules instead of profiles this becomes a non issue as they'd just be test deps for the module.</div><div><div></div><div class="h5"><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thoughts?<div><div></div><div><br><br><br><div class="gmail_quote">On Wed, Jul 6, 2011 at 11:35 AM, Jason Porter <span dir="ltr"><<a href="http://lightguard.jp" target="_blank">lightguard.jp</a>@<a href="http://gmail.com" target="_blank">gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF"><div>Very cool. I got stuff working for catch last night too, but there is a problem if you're testing deployment errors. It's actually an AS7 and Arquillian issue. I've opened JIRA tickets about both. I think what we'll want to do is set up testing similar to what Stuart has done in Persistence with multiple modules. We'll eventually want to test in (probably) embedded weld (it's runs quickly, could probably be like a smoke test), JBoss AS7, AS6, GF 3.1, Resin and an OWB container or embedded OWB. Not all of those containers work right now but then we really could say we know it all works on each CDI impl. </div>
<div><br>Sent from my iPhone</div><div><div></div><div><div><br>On Jul 6, 2011, at 7:44, Ken Finnigan <<a href="mailto:ken@kenfinnigan.me" target="_blank">ken@kenfinnigan.me</a>> wrote:<br><br></div><div>
</div><blockquote type="cite"><div>I actually "stole" some day job time to try at work as I thought the problem might be caused by the local build of as7 I had in my repo, but that wasn't it.<br><br>Figured out the problem I was having with the aether class not being found was caused by the arquillian junit container dependency being specified in a parent pom, and then the managed container being specified in the pom running the tests, which meant Maven decided the versions of the jars in the pom running the tests should take precedence.<br>
<br>Adding the junit container dependency to the pom running the tests fixed that problem.<br><br>Should be able to finish up the test setup some time tonight to then forward to the mailing list for review.<br><br>Ken<br>
<br><div class="gmail_quote">On Tue, Jul 5, 2011 at 11:13 PM, Jason Porter <span dir="ltr"><<a href="http://lightguard.jp" target="_blank"></a><a href="http://lightguard.jp" target="_blank">lightguard.jp</a>@<a href="http://gmail.com" target="_blank"></a><a href="http://gmail.com" target="_blank">gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Sure, you let me know a time. I'm working on getting Catch up to date as well. BTW, Arquillian 1.0.0.Final will be out tomorrow, not sure about container support though.<div><div></div><div><br><br><div class="gmail_quote">
On Tue, Jul 5, 2011 at 20:22, Ken Finnigan <span dir="ltr"><<a href="mailto:ken@kenfinnigan.me" target="_blank"></a><a href="mailto:ken@kenfinnigan.me" target="_blank">ken@kenfinnigan.me</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Jason,<br><br>If you have some time tomorrow it would be appreciated if you could take a look at the new testsuite setup on i18n. Here's the branch: git://<a href="http://github.com/kenfinnigan/international.git" target="_blank"></a><a href="http://github.com/kenfinnigan/international.git" target="_blank">github.com/kenfinnigan/international.git</a><br>
<br>I thought I was making progress but now I seem to have managed to move backwards! For some reason I keep getting a NoClassDef errors on aether classes. It appears that the classes are brought in correctly from the arquillian junit container (from shrinkwrap beta 3), but then the jboss as managed container brings in an older version of shrinkwrap that doesn't include aether.<br>
<br>I'm sure it's a simple fix, as I've mirrored it off confbuzz, servlet and rest, as well as jbossas, that all run as7 arquillian tests, but can't see the wood for the trees tonight. Will be looking at it again tomorrow evening, but if you can spot an easy fix that would be great!<br>
<br>Thanks<br><font color="#888888">Ken</font><div><div></div><div><br><br><br><div class="gmail_quote">On Tue, Jul 5, 2011 at 1:39 PM, Jason Porter <span dir="ltr"><<a href="http://lightguard.jp" target="_blank"></a><a href="http://lightguard.jp" target="_blank">lightguard.jp</a>@<a href="http://gmail.com" target="_blank"></a><a href="http://gmail.com" target="_blank">gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Okay, great. Let me or Dan know if you need some help.<div><div></div><div><br><br><div class="gmail_quote">
On Tue, Jul 5, 2011 at 11:37, Ken Finnigan <span dir="ltr"><<a href="mailto:ken@kenfinnigan.me" target="_blank"></a><a href="mailto:ken@kenfinnigan.me" target="_blank">ken@kenfinnigan.me</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Pretty good.<br><br>Have the basic structure for the module in terms of what needs to go where.<br><br>Currently going through and getting the existing i18n tests working in AS7, which is proving more of a challenge than I expected!<br>
<br>Of all the tests that pass in weld-ee-embedded, about a third actually pass in AS7! Most of it, I think, is down to differing classpaths, but should be in a position to push the feature branch to my fork of i18n in a couple of days.<br>
<br>Once I've pushed it to my fork I was going to send an email to everyone on seam-dev to get their thoughts on the layout/setup for testing.<br><br>btw, you don't need to worry about cc'ing my sorstech email in the future as I don't use that much anymore.<br>
<font color="#888888">
<br>Ken</font><div><div></div><div><br><br><br><div class="gmail_quote">On Tue, Jul 5, 2011 at 1:20 PM, Jason Porter <span dir="ltr"><<a href="http://lightguard.jp" target="_blank"></a><a href="http://lightguard.jp" target="_blank">lightguard.jp</a>@<a href="http://gmail.com" target="_blank"></a><a href="http://gmail.com" target="_blank">gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
How goes things with i18n and the new Arquillian?<br clear="all"><br>-- <br>Jason Porter<br><a href="http://lightguard-jp.blogspot.com" target="_blank"></a><a href="http://lightguard-jp.blogspot.com" target="_blank">http://lightguard-jp.blogspot.com</a><br>
<a href="http://twitter.com/lightguardjp" target="_blank"></a><a href="http://twitter.com/lightguardjp" target="_blank">http://twitter.com/lightguardjp</a><br>
<br>Software Engineer<br>Open Source Advocate<br>Author of Seam Catch - Next Generation Java Exception Handling<br><br>PGP key id: 926CCFF5<br>PGP key available at: <a href="http://keyserver.net" target="_blank"></a><a href="http://keyserver.net" target="_blank">keyserver.net</a>, <a href="http://pgp.mit.edu" target="_blank"></a><a href="http://pgp.mit.edu" target="_blank">pgp.mit.edu</a><br>
</blockquote></div><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Jason Porter<br><a href="http://lightguard-jp.blogspot.com" target="_blank"></a><a href="http://lightguard-jp.blogspot.com" target="_blank">http://lightguard-jp.blogspot.com</a><br>
<a href="http://twitter.com/lightguardjp" target="_blank"></a><a href="http://twitter.com/lightguardjp" target="_blank">http://twitter.com/lightguardjp</a><br>
<br>Software Engineer<br>Open Source Advocate<br>Author of Seam Catch - Next Generation Java Exception Handling<br><br>PGP key id: 926CCFF5<br>PGP key available at: <a href="http://keyserver.net" target="_blank"></a><a href="http://keyserver.net" target="_blank">keyserver.net</a>, <a href="http://pgp.mit.edu" target="_blank"></a><a href="http://pgp.mit.edu" target="_blank">pgp.mit.edu</a><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Jason Porter<br><a href="http://lightguard-jp.blogspot.com" target="_blank"></a><a href="http://lightguard-jp.blogspot.com" target="_blank">http://lightguard-jp.blogspot.com</a><br>
<a href="http://twitter.com/lightguardjp" target="_blank"></a><a href="http://twitter.com/lightguardjp" target="_blank">http://twitter.com/lightguardjp</a><br>
<br>Software Engineer<br>Open Source Advocate<br>Author of Seam Catch - Next Generation Java Exception Handling<br><br>PGP key id: 926CCFF5<br>PGP key available at: <a href="http://keyserver.net" target="_blank"></a><a href="http://keyserver.net" target="_blank">keyserver.net</a>, <a href="http://pgp.mit.edu" target="_blank"></a><a href="http://pgp.mit.edu" target="_blank">pgp.mit.edu</a><br>
</div></div></blockquote></div><br>
</div></blockquote></div></div></div></blockquote></div><br>
</div></div></blockquote></div></div></div><div><div></div><div class="h5"><br><br clear="all"><br>-- <br>Jason Porter<br><a href="http://lightguard-jp.blogspot.com" target="_blank">http://lightguard-jp.blogspot.com</a><br>
<a href="http://twitter.com/lightguardjp" target="_blank">http://twitter.com/lightguardjp</a><br>
<br>Software Engineer<br>Open Source Advocate<br>Author of Seam Catch - Next Generation Java Exception Handling<br><br>PGP key id: 926CCFF5<br>PGP key available at: <a href="http://keyserver.net" target="_blank">keyserver.net</a>, <a href="http://pgp.mit.edu" target="_blank">pgp.mit.edu</a><br>
</div></div></div>
</blockquote></div><br>