Why @IntegrationTest? By simply being in the AS Integration Test
module, wouldn't that be self-explanatory?
The use of @IntegrationTest in CDI TCK is basically to separate tests that can run in Weld
SE (mocked EE v) vs the ones that need a full EE container.
Not sure the AS Suite needs that kind of separation.
-aslak-
On 03/20/2011 10:58 AM, Shelly McGowan wrote:
>
>
>
> Why don't we consider what was done for the CDI and BV TCKs.
> Annotations[1] are used to define the test type.
> If it is a spec assertion, the spec reference is clearly identified:
>
> @SpecAssertion(section = "2.1.1", id = "c")
>
> or you could annotate the Test with @IntegrationTest which could be
> used to denote that AS-specific test.
>
> Test (not code) coverage reports can be generated to identify what
> the coverage is for each component. The report
> provides links to fisheye and the test code in svn.
>
>
>
[
1]http://community.jboss.org/wiki/BeanValidationTCK#What_do_all_the_annot...
> Other references:
>
http://docs.jboss.org/hibernate/stable/beanvalidation/tck/reference/html_...
>
http://anonsvn.jboss.org/repos/weld/cdi-tck/trunk/impl/src/main/java/org/...
>
>
> Shelly McGowan
> JBoss, by Red Hat
>
> ----- Original Message -----
> From: "Andrew Lee Rubinger"<andrew.rubinger(a)redhat.com>
> To: jboss-as7-dev(a)lists.jboss.org
> Sent: Saturday, March 19, 2011 5:34:30 PM
> Subject: Re: [jboss-as7-dev] Requirements and Design Proposal: AS7
> TestSuite
>
> It occurs to me I might not have been clear about the JIRA-per-test
> thing.
>
> Say you're working on Feature X. You might commit the feature and
> the
> test alongside one another, citing the same JIRA, with the commit
> message:
>
> "[JBAS-XXX] Implement Feature X and tests".
>
> That's great. No separate issues needed for the feature and the test
> each.
>
> What's no good is going in and adding a bunch of tests for EJB, Web,
> etc
> while we pump up our coverage, alongside a commit message:
>
> "[JBAS-XXX] Add spec coverage to the testsuite"
>
> ...that tells us nothing about each test being made, just that we
> dumped
> 'em in there. Which is obvious, because they're there.
>
> S,
> ALR
>
> On 03/19/2011 05:25 PM, Andrew Lee Rubinger wrote:
>> Inline.
>>
>> On 03/19/2011 03:19 PM, Jason Greene wrote:
>>> I really like the idea of our primary test suite covering API
>>> interaction and your typical unit test which is tightly coupled to
>>> internals. I also like your idea to separate spec Apis from JBoss
>>> aAPIs.
>>>
>>> However, I disagree on the Jira per test requirement. This is only
>>> because I worked with a similar model (all jiras require a test),
>>> and you ended up with lots of duplicate tests, and you always had
>>> to read Jira to understand what was being tested. Also, the tests
>>> ended up being too specific, typically an exact replication of the
>>> reporter's workflow.
>>
>> Consider this: If you had to read JIRA to understand what was being
>> tested, that's a clear violation of point 3): Document the test.
>> And
>> failing good documentation, with no JIRA to turn to, you're flying
>> blind.
>>
>> Also, how exactly does registering a JIRA encourage test
>> duplication?
>> Whether the tests are in an issue tracker or not seems to be
>> unrelated
>> to folks writing duplicate tests.
>>>
>>> IMO a better way to organize it would be by functional area, with
>>> generous descriptions. All spec tests should reference spec
>>> sections, for example. If we have a Jira that demonstrates lack of
>>> coverage, we should translate the problem into meaningful and
>>> reusable tests for the gaps we missed. Those tests should
>>> primarily refer to the API contract and not the Jira. Although
>>> commenting a list of jiras for extra info may be useful provided
>>> the Jira doesn't confuse what the correct behavior should be.
>>
>> I'm a fan of organizing by functional areas and spec references.
>> We'd
>> done that in EJB3 with success: some packages were even named for
>> the
>> spec. Regardless, every commit had a JIRA association.
>>
>> In the end, so long as it's perfectly clear what we're testing and
>> why,
>> I'll have no objections. But I do believe that JIRA is the obvious
>> answer to that question, and requires minimal effort to make a new
>> issue. Especially since every commit is isolated to one concern and
>> linked back to JIRA anyway, right? ;)
>>
>> S,
>> ALR
>>
>>>
>>> Sent from my iPad
>>>
>>> On Mar 18, 2011, at 2:21 AM, Andrew Lee
>>> Rubinger<andrew.rubinger(a)redhat.com> wrote:
>>>
>>>> Looks like a lot of us have different ideas for what the AS7
>>>> Integration
>>>> TestSuite should consist of, so I'll kickoff with what I believe
>>>> is the
>>>> first design proposal towards getting coverage focused on the
>>>> end-user
>>>> (not certifying our own internals).
>>>>
>>>> I suspect this breaks down into two categories, which may be
>>>> modelled as
>>>> separate modules under the existing "testsuite" aggregator
>>>> parent:
>>>>
>>>> * Specification
>>>> * AS-specific APIs
>>>>
>>>> This isn't difficult work, though I do think it's important we
>>>> consider
>>>> some hard rules. IMO we should be developing these suites as if
>>>> we were
>>>> application developers, not wearing our server dev hats.
>>>>
>>>> ----------------
>>>>
>>>> [End Goal]
>>>>
>>>> 1) No compile-time dependencies in the module except for what's
>>>> absolutely necessary.
>>>>
>>>> For the spec suite, this means: JDK and EE Spec APIs only in the
>>>> compilation classpath. Testable asset sources and resources (ie.
>>>> EJBs,
>>>> Servlets, etc) would live under src/main/* to enforce that. Only
>>>> the
>>>> tests themselves would be located under "src/test/*".
>>>>
>>>> The AS-specific API suite may also add in our own APIs to the
>>>> compilation classpath, but the line should end there. In
"test"
>>>> scope
>>>> we can place all runtime dependencies.
>>>>
>>>> For the specification suite, AS-specific grammars like our own
>>>> deployment descriptors are fine; these are in many ways
>>>> equivalent to
>>>> the TCK porting layer. We're not building a TCK; we're showing
>>>> that our
>>>> implementation supports the features advertised.
>>>>
>>>> 2) Every single new test created is to have an associated JIRA.
>>>>
>>>> We all remember the nightmare it was when the old AS4-6 suite
>>>> would fall
>>>> down. We'd comb through each test, at times trying to determine
>>>> its
>>>> purpose. By linking to JIRA we get history of intent, which acts
>>>> as a
>>>> nice record even in the case that the test isn't so
>>>> well-documented.
>>>> I'd argue that tests are a bigger asset than our code, and we
>>>> should be
>>>> thinking about these in terms of long-term maintenance to outlive
>>>> any
>>>> specific impl.
>>>>
>>>> 3) Documentation
>>>>
>>>> Alongside the JIRA reference, a quick note about we're looking to
>>>> accomplish is something I find very helpful. I don't personally
>>>> buy the
>>>> argument that code is self-documenting if written well. It gets
>>>> refactored and stale over time.
>>>>
>>>> 4) Run-mode profiles
>>>>
>>>> Arquillian provides a wonderful abstraction such that we can get
>>>> coverage for AS in both remote managed *and* embedded modes
>>>> without
>>>> changing the test itself. To certify that everything is working
>>>> as
>>>> advertised no matter the runtime, we should be able to run the
>>>> same
>>>> suite in standalone, domain, and embedded modes (generally
>>>> speaking).
>>>>
>>>> 5) Porting of AS6 Tests
>>>>
>>>> There's no discounting the value this coverage has given us,
>>>> though I
>>>> question the purpose of a lot of these tests. I think a great
>>>> majority
>>>> of these need to come into the new codebase, refactored to align
>>>> if needed.
>>>>
>>>> ----------------
>>>>
>>>> [Current State]
>>>>
>>>> Here[1] is an example of what I believe to be a simple,
>>>> well-written
>>>> test, with the exception that the tested Servlet and EJB are in
>>>> the same
>>>> test source folder.
>>>>
>>>> The current "testsuite" aggregator contains modules which mix
our
>>>> end-user certification stuff alongside internals, so I think
>>>> these
>>>> should be separated out.
>>>>
>>>> A lot of this is set up in some fashion already, but I would like
>>>> to see us:
>>>>
>>>> 1) Agree upon a strict scope for each type of testsuite along the
>>>> lines
>>>> of my points above, once we reach agreement
>>>> 2) Upgrade to ARQ 1.0.0.Alpha5 (which implies ShrinkWrap
>>>> 1.0.0-alpha-12), just released tonight. Currently AS is on a
>>>> forked
>>>> release of ARQ for OSGi purposes, and these changes, if
>>>> necessary, need
>>>> to get upstream so we can do upgrades.
>>>>
>>>> It's clear that AS7 has made full-steam-ahead progress since last
>>>> summer, and with a little organization our testsuite can give us
>>>> a great
>>>> view of where we stand, from an end-user's perspective, with
>>>> minimal
>>>> investment.
>>>>
>>>> S,
>>>> ALR
>>>>
>>>>
>>>> [1]
>>>>
https://github.com/jbossas/jboss-as/blob/master/testsuite/integration/src...
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>> jboss-as7-dev(a)lists.jboss.org
>>>>
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> jboss-as7-dev(a)lists.jboss.org
>>>
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>> _______________________________________________
>> jboss-as7-dev mailing list
>> jboss-as7-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
jboss-as7-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev