[jboss-as7-dev] Requirements and Design Proposal: AS7 TestSuite

[Jaikiran Pai]

One more thing (more of a FYI), in AS7 we also have "demos" modules 
which gets pulled in by the tests to make sure the demos are indeed 
working.

-Jaikiran
On Friday 18 March 2011 04:40 PM, Jaikiran Pai wrote:
> On Friday 18 March 2011 12:51 PM, Andrew Lee Rubinger wrote:
>>
>> I suspect this breaks down into two categories, which may be modelled as
>> separate modules under the existing "testsuite" aggregator parent:
>>
>> * Specification
>> * AS-specific APIs
> On a slightly different categorization, we also have to come up with
> tests for stress testing. They can either be for "Specification" related
> tests or even for "AS-specific APIs". Perhaps test "profiles" (need
> _not_ be Maven profile) within those 2 modules would be better instead
> of a separate module?
>
>>
>> [End Goal]
>>
>> 1) No compile-time dependencies in the module except for what's
>> absolutely necessary.
>>
>> For the spec suite, this means: JDK and EE Spec APIs only in the
>> compilation classpath.
> We might also have to include JBoss specific APIs which are exposed to
> the users (for ex: org.jboss.ejb3.annotation which currently isn't there
> but might be introduced soon in AS7).
>> Testable asset sources and resources (ie. EJBs,
>> Servlets, etc) would live under src/main/* to enforce that.  Only the
>> tests themselves would be located under "src/test/*".
> Sounds fine. Although, do we want to end up with all tech specific tests
> (like for EJB3, web, etc...) all in one source path? i.e.
> spec-tests/src/main/org/jboss/spec/<ejb3, web, jsf>....? Or should we
> plan to have different maven modules under the "Spec" tests for each
> individual tech?
>
>>
>> 2) Every single new test created is to have an associated JIRA.
> I'm not quite sure that this will work out. I'm not against it, but it
> might not be practical.
>>
>> By linking to JIRA we get history of intent, which acts as a
>> nice record even in the case that the test isn't so well-documented.
>> I'd argue that tests are a bigger asset than our code, and we should be
>> thinking about these in terms of long-term maintenance to outlive any
>> specific impl.
> I agree about the well-documented tests, but JIRA linking shouldn't
> always be an absolute necessity.
>>
>> 3) Documentation
>>
>> Alongside the JIRA reference, a quick note about we're looking to
>> accomplish is something I find very helpful.  I don't personally buy the
>> argument that code is self-documenting if written well.  It gets
>> refactored and stale over time.
> +1000. A simple brief explanation on what the test is supposed to test
> is really useful.
>>
>> 4) Run-mode profiles
>>
>> Arquillian provides a wonderful abstraction such that we can get
>> coverage for AS in both remote managed *and* embedded modes without
>> changing the test itself.  To certify that everything is working as
>> advertised no matter the runtime, we should be able to run the same
>> suite in standalone, domain, and embedded modes (generally speaking).
> Are these container specific modes that we are talking about? Or are
> these more of test profiles (a.k.a groups) wherein we could perhaps add
> a test under a "stress-test" profile?
>
> -Jaikiran
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev