<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 12/08/2011, at 9:06 AM, Andrew Lee Rubinger wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Inline.<br><br>On 08/11/2011 05:12 PM, Richard Achmatowicz wrote:<br><blockquote type="cite">Andrew<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Can you give a simple, concrete example of a spec test case (citing part<br></blockquote><blockquote type="cite">of the J2EE spec, say), which illustrates:<br></blockquote><br>Hehe, my PoC is filled with 'em. :) But keep in mind we're not directly <br>necessarily citing the EE specs, because we're *not* building a TCK in AS7.<br><br><blockquote type="cite">- what dependencies you want to be on the classpath, as well as those<br></blockquote><blockquote type="cite">you don't want to be only the classpath<br></blockquote><br>For this discussion, I'm primarily concerned with the *compilation* <br>ClassPath, as this is the view users see when building their <br>applications. It's about explicitly defining what the AS7 API is. So <br>the deps for the compilation ClassPath should be as they are in by <br>AS7-999 branch and as described in the last email:<br><br>* spec - Java SE and Java EE<br>* api - JBoss-specific extensions to "spec" which comprise our AS7 API<br>* internals - Anything in the AS7 runtime<br></div></blockquote><div><br></div><div>I am ok with splitting the tests up in this manner, but I think we will need additional test suite modules for purely technical reasons. </div><div><br></div><div>For instance, at the moment we also have compat for hibernate 3 testing, and timer service, to test the ejb timer service.</div><div><br></div><div>Compat is needed because maven cannot handle having multiple hibernate versions in the same module, and timer service </div><div>is separate because it needs the preview configuration, and need to run multiple times in order to test restoring persistent timers.</div><br><blockquote type="cite"><div><br><blockquote type="cite">- what sample artifacts you would put in src/main and what you would put<br></blockquote><blockquote type="cite">in src/test<br></blockquote><br>* src/main - Stuff that is part of the deployment. Anything that you'd <br>be including in the @Deployment archive defined by an Arquillian-based <br>test case.<br>* src/test - The test infrastructure itself, like the JUnit test and <br>other util/helpers<br></div></blockquote><div><br></div><div>I am very much opposed to this split. IMHO splitting the test classes into two different places makes it much harder to read and write tests. </div><div><br></div><div>I also don't see how having additional test scoped dependencies on the class path is a problem, as in practice this works out to being Arquillian and JUnit. </div><div><br></div><div>Stuart</div><br><blockquote type="cite"><div><br><blockquote type="cite">- what compile and test execution time constraints you would want to enforce<br></blockquote><blockquote type="cite">This would help me to see the motivation for the refactoring and your<br></blockquote><blockquote type="cite">planned organization of artifacts in src/main and src/test, which<br></blockquote><blockquote type="cite">differ from the standard organization of putting all test related<br></blockquote><blockquote type="cite">artifacts in src/test.<br></blockquote><br>Test execution time is an orthogonal concern, and relates instead back <br>to using "smoke" tests to run by default instead of the entire AS7 <br>integration test suite (which won't scale to run on every build over <br>time). IMO it's already taking too long for standard builds.<br><br>Again, the motivation for moving/organizing in this fashion is to honor <br>dependencies and assert that the APIs we export ("api" and "spec-api" <br>modules) are complete. For instance, while moving tests around I <br>discovered that our POMs were incompete:<br><br><a href="https://issues.jboss.org/browse/AS7-1489">https://issues.jboss.org/browse/AS7-1489</a><br>https://issues.jboss.org/browse/AS7-1493<br>https://issues.jboss.org/browse/AS7-1479<br>https://issues.jboss.org/browse/AS7-1478<br></div></blockquote><blockquote type="cite"><div><font class="Apple-style-span" color="#000000"><br></font><blockquote type="cite">Also, is it not possible to control what is on the classpath by maven<br></blockquote><blockquote type="cite">elements like<classpathDependencyExcludes/> and the like?<br></blockquote><br>That's for test runtime. Because Arquillian is starting (or connecting <br>to) the container in a remote process, we're less concerned with the <br>client runtime ClassPath, so long as it's enough for Arquillian to do <br>its thing.<br><br>S,<br>ALR<br><br><blockquote type="cite"><br></blockquote><blockquote type="cite">Richard<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On 08/11/2011 07:38 AM, Andrew Lee Rubinger wrote:<br></blockquote><blockquote type="cite"><blockquote type="cite">Hi guys:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I'd like to reopen the discussion regarding the testsuite organization<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">and its ongoing maintenance. This issue dates back a few months with<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">some debates and differing opinions, so I'll do my best to outline the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">guiding principles I'd like to see put in place concisely.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">To start off, I've a Proof-of-Concept for many of the following points<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">now located:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="https://github.com/ALRubinger/jboss-as/tree/AS7-999">https://github.com/ALRubinger/jboss-as/tree/AS7-999</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">The relevant JIRA I've been using to track things:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="https://issues.jboss.org/browse/AS7-999">https://issues.jboss.org/browse/AS7-999</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">So:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">1) TestSuite Organization<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I believe we need a single top-level categorization by which we may<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">organize integration tests which are deployment-based and run within the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">context of the server. Because we use Maven modules (which are bound to<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">a dependency structure), it makes sense to file these modules by the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">compile-time dependencies they require. So in place I've put:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">testsuite/spec - Java SE and Java EE APIs only<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">testsuite/api - AS7 APIs + Spec<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">testsuite/internals - Use anything in the AS7 runtime in your<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">deployments; not guaranteed to be back-compat across releases<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">The primary motivation here is to ensure that the dependencies we export<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">(ie. "spec-api", and "api" modules) are complete enough for users to<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">create their own deployments. In this setup, we act as *users* of our<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">own APIs, and everything in src/main is limited to the relevant<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">dependencies.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I know the source of some disagreements earlier centered around placing<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">the tests right next to the deployments, and some folks consider the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">deployments as part of the test itself. That's not a bad argument at<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">all, but again consider that we then lose the ability to validate our<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">tests in the context of our exported APIs.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">2) Run Modes, Test Subsets<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Because the primary organizational criteria proposed in 1) is by<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">dependency, these modules will grow large over time. The AS build over<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">time will take longer and longer to run. Additionally, there are<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">runtime options to consider when starting tests. So consider the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">following requirements:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> * Running the testsuite in IPv6<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> * Running only a subset of tests as part of the main build<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">These lend themselves well to using build profiles. By default, I think<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">the "smoke tests" should simply be a set of tests we deem important or<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">indicative of the general health of AS7 with respect to each subsystem.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> As it stands now, "smoke" is its own module with a bunch of<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Embedded-based tests, and I think these should move to the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">organizational structure in 1) and instead we can apply some filtering<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">to make the "smoke" some default set of includes.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">3) An authoritative maintainer<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I'd like to treat the Arquillian and TestSuite modules as true<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">subsystems of the Application Server, and as such we'll need someone to<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">assume the responsibility to review incoming commits/pull requests and<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">ensure they fit the criteria for acceptance. Simple things like<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">consistent package names, using ARQ correctly, and not leaking<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">dependencies are very important.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">So assuming we come to agreement on these points, I'd like to request<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">push access to the AS7 repo to field testsuite and ARQ-related pull<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">requests.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">...there's much more to discuss (I've more issues to raise alongside the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">upcoming EAP requirements), but let's start with those first 3 major<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">points and my POC, and run from there.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">S,<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">ALR<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">_______________________________________________<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">jboss-as7-dev mailing list<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="mailto:jboss-as7-dev@lists.jboss.org">jboss-as7-dev@lists.jboss.org</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="https://lists.jboss.org/mailman/listinfo/jboss-as7-dev">https://lists.jboss.org/mailman/listinfo/jboss-as7-dev</a><br></blockquote></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">jboss-as7-dev mailing list<br></blockquote><blockquote type="cite"><a href="mailto:jboss-as7-dev@lists.jboss.org">jboss-as7-dev@lists.jboss.org</a><br></blockquote><blockquote type="cite"><a href="https://lists.jboss.org/mailman/listinfo/jboss-as7-dev">https://lists.jboss.org/mailman/listinfo/jboss-as7-dev</a><br></blockquote>_______________________________________________<br>jboss-as7-dev mailing list<br><a href="mailto:jboss-as7-dev@lists.jboss.org">jboss-as7-dev@lists.jboss.org</a><br>https://lists.jboss.org/mailman/listinfo/jboss-as7-dev<br></div></blockquote></div><br></body></html>