Author: pete.muir(a)jboss.org Date: 2009-07-23 19:16:08 -0400 (Thu, 23 Jul 2009) New Revision: 3180 Modified: tck/trunk/doc/reference/en-US/introduction.xml Log: ws Modified: tck/trunk/doc/reference/en-US/introduction.xml =================================================================== --- tck/trunk/doc/reference/en-US/introduction.xml 2009-07-23 23:05:00 UTC (rev 3179) +++ tck/trunk/doc/reference/en-US/introduction.xml 2009-07-23 23:16:08 UTC (rev 3180) @@ -1,208 +1,534 @@ <?xml version='1.0' encoding='UTF-8'?> <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]> <chapter id="introduction"> - <title>Introduction</title> - <para>This chapter explains the purpose of a TCK and identifies the foundation elements of the JSR-299 TCK.</para> - <section> - <title>TCK Primer</title> - <para>A TCK, or Technology Compatibility Kit, is one of the three required pieces for any JSR (the other two being the specification document and the reference implementation). The TCK is a set of tools and tests to verify that a licensee&apos;s implementation of the Sun technology conforms to the specification. The tests are the primary component, but the tools serve an equally critical role of providing a framework and/or set of SPIs for executing the tests.</para> - <para>The tests in the TCK are derived from assertions in the written specification document. The assertions materialize as a suite of automated tests that collectively validate whether an implementation complies with the aforementioned assertions, and in turn the specification. For a particular implementation to be certified, all of the required tests must pass (the provided test suite must be run unmodified).</para> - <para>A TCK is entirely implementation agnostic. Ideally, it should validate assertions by consulting the specficiation&apos;s public API. However, when the information returned by the public API is not low-level enough to validate the assertion, the implementation must be consulted directly. In this case, the TCK provides an independent API as part of a porting package which must also be implemented by the licensee. Section 1.3.4 introduces the porting package and section 4.1 covers the requirements for implementing it.</para> - </section> - <section> - <title>Compatibility Testing</title> - <para>The goal of any specification is to eliminate portability problems so long as the program which uses the implementation also conforms to the rules laid out in the specification.</para> - <para>Executing the TCK is a form of compatibility testing. It&apos;s important to understand that compatibility testing is distinctly different from product testing. The TCK is not concerned with robustness, performance or ease of use, and therefore cannot vouch for how well an implementation meets these criteria. What a TCK can do is to ensure the exactness of an implementation as it relates to the specification.</para> - <para>Compatibility testing of any feature relies on both a complete specification and a complete reference implementation. The reference implementation demonstrates how each test can be passed and provides additional context to the licensee during development for the corresponding assertion.</para> - <section> - <title>Why Compatibility Is Important</title> - <para>Java platform compatibility is important to different groups involved with Java technologies for -different reasons: -</para> - <itemizedlist> - <listitem> - <para>Compatibility testing is the means by which Sun ensures that the Java platform does not become -fragmented as it&apos;s ported to different operating systems and hardware. -</para> - </listitem> - <listitem> - <para>Compatibility testing benefits developers working in the Java programming language, enabling -them to write applications once and deploy them across heterogeneous computing - environments without porting.</para> - </listitem> - <listitem> - <para>Compatibility testing enables application users to obtain applications from disparate sources and - deploy them with confidence. -</para> - </listitem> - <listitem> - <para>Conformance testing benefits Java platform implementors by ensuring the same extent of reliability for all Java platform ports. -</para> - </listitem> - </itemizedlist> - <para>The JSR-299 specification goes to great lengths to ensure that programs written for Java EE are compatible and the TCK is rigorous about enforcing the rules the specification lays down.</para> - </section> - </section> - <section> - <title>About the JSR-299 TCK</title> - <para>The JSR-299 TCK is designed as a portable, configurable and automated test suite for -verifying the compatibility of a licensee’s implementation of the JSR-299: Contexts and Dependency Injection for Java EE specification. The test suite is built on TestNG and provides a series of extensions that allow runtime packaging and deployment of JEE artifacts for in-container testing.</para> - <note> - <para>The JSR-299 TCK harness is based on the JBoss Test harness, which provides most of the aforementioned functionality.</para> - </note> - <para>Each test class in the suite acts as a deployable unit. The deployable units, or artifacts, are defined declaratively using annotations. The artifact produced can be either a WAR or an EAR.</para> - <para>The benefit of the declarative approach is that it allows many of the tests to be executed in a standalone implementation of JSR-299, accounting for a boast in developer productivity. However, an implementation is only valid if all tests pass using the in-container execution mode. The standalone mode is merely a developer convenience.</para> - <section> - <title>JSR-299 TCK Specifications and Requirements</title> - <para>This section lists the applicable requirements and specifications for the JSR-299 TCK.</para> - <itemizedlist> - <listitem> - <para><emphasis role="bold">Specification requirements</emphasis> - Software requirements for a JSR-299 implementation are itemized in section 1.2, &quot;Relationship to other specifications&quot; in the JSR-299 specification, with details provided throughout the specification.</para> - </listitem> - <listitem> - <para><emphasis role="bold">Contexts and Dependency Injection for Java EE 1.0 API</emphasis> - The Java API defined in the JSR-299 specification and provided by the reference implementation.</para> - </listitem> - <listitem> - <para><emphasis role="bold">JBoss Test Harness</emphasis> - The JSR-299 TCK requires version 1.0.0 of the JBoss Test Harness</para> - </listitem> - <listitem> - <para><emphasis role="bold">Porting Package</emphasis> - An implementation of SPIs that are required for the test suite to run the in-container tests and at times, consult the JSR-299 implementation</para> - </listitem> - <listitem> - <para><emphasis role="bold">TCK Audit Tool</emphasis> - An itemization of the assertions in the specification documents which are cross referenced by the individual tests. Relevant for producing a test coverage report.</para> - </listitem> - <listitem> - <para><emphasis role="bold">Reference runtime</emphasis> - The designated reference runtimes for compatibility testing of the JSR-299 specification is the Sun Java Platform, Enterprise Edition (Java EE) 5 - reference implementation (RI). - See details at Java EE 5 - (<ulink url="http://java.sun.com/javaee/5/docs/api/">http://java.sun... - </listitem> - <listitem> - <para><emphasis role="bold">Maven 2</emphasis> - Although licensees are free to execute the TCK however they decide, the recommended approach is to use Maven 2. Both the JSR-299 RI and the TCK are build using Maven 2. Furthermore, the TCK runner used to validate the RI, a resource which can be leveraged by licensee to test their own implementations, is also Maven 2 project. Therefore, it&apos;s highly recommended to have Maven 2 installed on the system.</para> - </listitem> - </itemizedlist> - </section> - <section> - <title>JSR-299 TCK Components</title> - <para>The JSR-299 TCK includes the following components:</para> - <itemizedlist> - <listitem> - <para><emphasis role="bold">JBoss Test Harness 1.0.0</emphasis> and related documentation.</para> - </listitem> - <listitem> - <para><emphasis role="bold">TestNG 5.9</emphasis>, the testing framework on which the JBoss Test Harness is based and which provides the extension points for selecting an executing the tests in the test suite.</para> - </listitem> - <listitem> - <para><emphasis role="bold">Porting Package SPIs</emphasis> - Extensions to the JSR-299 SPIs to allow testing of a container.</para> - </listitem> - <listitem> - <para><emphasis role="bold">The test suite</emphasis>, which is a collection of TestNG tests, the TestNG test suite descriptor and supplamental files that provide data for the -automatic running of tests through the JBoss Test harness.</para> - </listitem> - <listitem> - <para><emphasis role="bold">JBoss TCK Runner</emphasis> - An implementation of the JBoss Test Harness deployment APIs for deploying artifacts to JBoss AS 5.1</para> - </listitem> - <listitem> - <para><emphasis role="bold">TCK documentation</emphasis> accompanied by release notes identifying updates between versions.</para> - </listitem> - </itemizedlist> - <para>The JSR-299 TCK has been tested with the following:</para> - <itemizedlist> - <listitem> - <para> Java Platform, Standard Edition, Version 5.0 (Java SE 1.5) -</para> - </listitem> - <listitem> - <para> Java Platform, Enterprise Edition, Version 5 reference implementation (Java EE 5 RI)</para> - </listitem> - <listitem> - <para>JBoss AS 5.1?</para> - </listitem> - </itemizedlist> - <para>The JSR-299 TCK tests run on the following platforms:</para> - <itemizedlist> - <listitem> - <para>Windows XP?</para> - </listitem> - <listitem> - <para>RHEL 5.2?</para> - </listitem> - </itemizedlist> - </section> - <section> - <title>The JBoss Test Harness</title> - <para>JSR-299 TCK leverages the JBoss Test Harness. The JBoss Test Harness is a testing framework based on TestNG that provides a series of extensions that allow runtime packaging and deployment of Java EE artifacts (EAR or WAR) for in-container testing. It&apos;s important to note that the JBoss Test Harness has no relation to or dependency on the JBoss Application Server (AS).</para> - <para>JSR-299 supports Java EE 5, Java EE 6, Java EE 6 Web Profile and the Embeddable EJB 3.1. Accordingly, the TCK will be able to run in any of these environments, though at the time of writing the Embedded EJB (EEJB) mode isn&apos;t operational.</para> + <title>Introduction</title> + + <para> + This chapter explains the purpose of a TCK and identifies the + foundation elements of the JSR-299 TCK. + </para> + <section> + <title>TCK Primer</title> + <para> + A TCK, or Technology Compatibility Kit, is one of the three required + pieces for any JSR (the other two being the specification document and + the reference implementation). The TCK is a set of tools and tests to + verify that a licensee's implementation of the Sun technology + conforms + to the specification. The tests are the primary component, but + the tools + serve an equally critical role of providing a framework + and/or set of + SPIs for executing the tests. + </para> + <para> + The tests in the TCK are derived from assertions in the written + specification document. The assertions materialize as a suite of + automated tests that collectively validate whether an implementation + complies with the aforementioned assertions, and in turn the + specification. For a particular implementation to be certified, all of + the required tests must pass (the provided test suite must be run + unmodified). + </para> + <para> + A TCK is entirely implementation agnostic. Ideally, it should + validate assertions by consulting the specficiation's public API. + However, when the information returned by the public API is not + low-level enough to validate the assertion, the implementation must be + consulted directly. In this case, the TCK provides an independent API + as part of a porting package which must also be implemented by the + licensee. Section 1.3.4 introduces the porting package and section 4.1 + covers the requirements for implementing it. + </para> + </section> + <section> + <title>Compatibility Testing</title> + + <para> + The goal of any specification is to eliminate portability problems + so long as the program which uses the implementation also conforms to + the rules laid out in the specification. + </para> + <para> + Executing the TCK is a form of compatibility testing. It's + important + to understand that compatibility testing is distinctly + different from + product testing. The TCK is not concerned with + robustness, performance + or ease of use, and therefore cannot vouch for + how well an + implementation meets these criteria. What a TCK can do is + to ensure the + exactness of an implementation as it relates to the + specification. + </para> + <para> + Compatibility testing of any feature relies on both a complete + specification and a complete reference implementation. The reference + implementation demonstrates how each test can be passed and provides + additional context to the licensee during development for the + corresponding assertion. + </para> + + <section> + <title>Why Compatibility Is Important</title> + <para> + Java platform compatibility is important to different groups + involved with Java technologies for different reasons: + </para> + <itemizedlist> + <listitem> + <para> + Compatibility testing is the means by which Sun ensures + that the Java platform does not become fragmented as it's + ported to different operating systems and hardware. + </para> + </listitem> + <listitem> + <para> + Compatibility testing benefits developers working in the + Java programming language, enabling them to write applications + once and deploy them across heterogeneous computing + environments without porting. + </para> + </listitem> + <listitem> + <para> + Compatibility testing enables application users to obtain + applications from disparate sources and deploy them with + confidence. + </para> + </listitem> + <listitem> + <para>Conformance testing benefits Java platform implementors by + ensuring the same extent of reliability for all Java platform + ports. + </para> + </listitem> + </itemizedlist> + <para> + The JSR-299 specification goes to great lengths to ensure that + programs written for Java EE are compatible and the TCK is rigorous + about enforcing the rules the specification lays down. + </para> + </section> + + </section> + + <section> + <title>About the JSR-299 TCK</title> + + <para> + The JSR-299 TCK is designed as a portable, configurable and + automated test suite for verifying the compatibility of a licensee’s + implementation of the JSR-299: Contexts and Dependency Injection for + Java EE specification. The test suite is built on TestNG and provides a + series of extensions that allow runtime packaging and deployment of JEE + artifacts for in-container testing. + </para> <note> - <para>You&apos;ll often see the term <emphasis role="italic">in-container</emphasis> used in this reference guide. This term refers to running the test suite in any of the aforementioned environments, whilst <emphasis role="italic">standalone</emphasis> refers to running the tests outside the container via a JSR-299 implementation-specific standalone bootstrap. The standalone mode only runs those tests which don&apos;t require EJB, Servlet or JSF to be present.</para> + <para> + The JSR-299 TCK harness is based on the JBoss Test harness, which + provides most of the aforementioned functionality. + </para> </note> - <para>The last thing Java developers want is yet another testing framework to make their life more complicated. What they do want is a good integration with their Integrated Development Environment (IDE). These days, if a tool doesn&apos;t have an IDE plugin, then it won&apos;t get the attention it deserves. That&apos;s why the JBoss Test Harness is built entirely upon TestNG. TestNG is one of the two prominent test frameworks for Java and TestNG plugins are available for all major IDEs (with the exception of NetBeans) and build tools (Ant and Maven 2).</para> - <para>Because it leverages the existing TestNG ecosystem, there is no need for a special test launcher for the JBoss Test Harness, and in turn the JSR-299 TCK. You simply use the IDE or build tool of your choice. You also get reporting and debugging for free.</para> - <para>The JBoss Test Harness supports the following features:</para> - <itemizedlist> - <listitem> - <para>Test activation via any method supported by the TestNG configuration descriptor (package, group, class)</para> - </listitem> - <listitem> - <para>Exclusion of in-container tests in standalone mode</para> - </listitem> - <listitem> - <para>Exclusion of individual tests labeled as under investigation</para> - </listitem> - <listitem> - <para>Integration with any TestNG plugin (Eclipse, IntelliJ, NetBeans, Ant, Maven)</para> - </listitem> - <listitem> - <para>Automated reporting capability as provided by TestNG</para> - </listitem> - <listitem> - <para>Standalone and in-container test mode</para> - </listitem> - <listitem> - <para>Container pluggability</para> - </listitem> - <listitem> - <para>Declarative packaging of additional resources and classes in artifact</para> - </listitem> - <listitem> - <para>Declarative deployment exception trapping</para> - </listitem> - <listitem> - <para>Artifact dumping for failure and packaging analysis</para> - </listitem> - </itemizedlist> - <para>A test is designated by a method annotated with <literal&gt;(a)org.testng.annotations.Test&lt;/literal&gt; in a class which extends <literal>org.jboss.jsr299.tck.AbstractJSR299Test</literal> and is annotated with <literal&gt;(a)org.jboss.testharness.impl.packaging.Artifact&lt;/literal&gt;.&lt;/para&gt; - <para>The <literal>@Test</literal> annotation is provided by TestNG, the <literal>@Artifact</literal> annotation is provided by the JBoss Test Harness and the <literal>AbstractJSR299Test</literal> is part of the JSR-299 TCK. There is a one-to-one mapping between a TestNG test class and an artifact. The packaging type is defined by the <literal&gt;(a)org.jboss.testharness.impl.packaging.Packaging&lt;/literal&gt; annotation on the test class, defaulting to a WAR if not specified.</para> - <para>Prior to executing the tests for a given class, the JBoss Test Harness packages the class as a deployable artifact (EAR or WAR), along with any extra resources specified, and deploys the artifact to the container. The test execution and results are negotatied via HTTP communication using a thin layer over the TestNG test launcher. The test harness can also catch and affirm expected deployment exceptions. This setup and tear down activity is provided by the super class <literal>org.jboss.jsr299.tck.AbstractJSR299Test</literal>, which all TCK test classes must extend.</para> - <para>If the annotation <literal&gt;(a)org.jboss.testharness.impl.packaging.IntegrationTest&lt;/literal&gt; is not present on the test class, then it means the test class can be executed in standalone mode. In standalone mode, the deployable artifact is assembled on the local classpath and the tests execute in the same JVM as the launcher, just as though it were a regular TestNG test case. The standalone mode is provided merely for convenience and efficiency.</para> - <para>Chapter 4 details how to run the TCK test suite using the JBoss Test Harness.</para> - </section> - <section> - <title>The Porting Package</title> - <para>The JSR-299 TCK relies on an implementation of the porting package to function. The porting package can be divided into two parts. The first part is comprised of extensions to the JSR-299 SPIs to allow testing of a container. The second part must contain implementations of the JBoss Test Harness deployment APIs for deploying test artifacts to a given container.</para> - <para>As mentioned earlier, there are times when the tests need to tap directly into the JSR-299 implementation to manipulate behavior or verify results. The porting package includes a set of SPIs that provide the TCK this level of access without tying the tests to a given implementation.</para> - <para>The four SPI classes in the JSR-299 TCK are as follows:</para> - <itemizedlist> - <listitem> - <para><literal>org.jboss.jsr299.tck.spi.Beans</literal></para> - </listitem> - <listitem> - <para><literal>org.jboss.jsr299.tck.spi.Contexts</literal></para> - </listitem> - <listitem> - <para><literal>org.jboss.jsr299.tck.spi.EL</literal></para> - </listitem> - <listitem> - <para><literal>org.jboss.jsr299.tck.spi.Managers</literal></para> - </listitem> - </itemizedlist> - <para>To run the full TCK you must additionally implement <literal>org.jboss.testharness.spi.Containers</literal>, which handles deploying the test artifact to the container. If you wish to run the TCK in standalone mode, you must also implement <literal>org.jboss.testharness.spi.StandaloneContainers</literal>. Implementations of these APIs are already available for JBoss AS 5.1. Therefore, you only need to implement this part of the porting package if you wish to use a container other than JBoss AS 5.1, for instance GlassFish V3.</para> - <para>The default configuration for the TCK is property-based. Therefore, you specify the implementation class for all of these SPIs in a properties file. There are additional properties for controlling whether the tests are run in standalone or in-container mode. Optionally, you can use system properties to configure the TCK. Details are provided in section 4.1.</para> - </section> - <section> - <title>TCK Audit Tool</title> - <para>The TCK audit is used to list out the assertions identified in the JSR-299 specification. It uses Java annotation processing to match the assertions to testcases in the test suite and produce a coverage report.</para> - <para>The audit document is provided along with the TCK and each assertion matches against at least one test. Each assertion is defined with a reference to a chapter, section and paragraph from the specification document, making it easy for the licensee to locate the language in the specification document that supports the feature being tested.</para> - </section> - </section> + <para> + Each test class in the suite acts as a deployable unit. The + deployable units, or artifacts, are defined declaratively using + annotations. The artifact produced can be either a WAR or an EAR. + </para> + <para> + The benefit of the declarative approach is that it allows many of + the tests to be executed in a standalone implementation of JSR-299, + accounting for a boast in developer productivity. However, an + implementation is only valid if all tests pass using the in-container + execution mode. The standalone mode is merely a developer convenience. + </para> + + <section> + <title>JSR-299 TCK Specifications and Requirements</title> + <para> + This section lists the applicable requirements and specifications + for the JSR-299 TCK. + </para> + <itemizedlist> + <listitem> + <para> + <emphasis role="bold">Specification requirements</emphasis> + - Software requirements for a JSR-299 implementation are + itemized in section 1.2, &quot;Relationship to other + specifications&quot; in the JSR-299 specification, with + details provided throughout the specification. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Contexts and Dependency Injection for + Java EE 1.0 API</emphasis> + - The Java API defined in the JSR-299 specification and + provided by the reference implementation. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">JBoss Test Harness</emphasis> + - The JSR-299 TCK requires version 1.0.0 of the JBoss Test + Harness + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Porting Package</emphasis> + - An implementation of SPIs that are required for the test + suite to run the in-container tests and at times, consult the + JSR-299 implementation + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">TCK Audit Tool</emphasis> + - An itemization of the assertions in the specification + documents which are cross referenced by the individual tests. + Relevant for producing a test coverage report. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Reference runtime</emphasis> + - The designated reference runtimes for compatibility testing + of the JSR-299 specification is the Sun Java Platform, + Enterprise Edition (Java EE) 5 + reference implementation (RI). + See details at Java EE 5 + ( + <ulink url="http://java.sun.com/javaee/5/docs/api/">http://java.sun... + </ulink> + ). + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Maven 2</emphasis> + - Although licensees are free to execute the TCK however they + decide, the recommended approach is to use Maven 2. Both the + JSR-299 RI and the TCK are build using Maven 2. Furthermore, + the TCK runner used to validate the RI, a resource which can + be leveraged by licensee to test their own implementations, is + also Maven 2 project. Therefore, it&apos;s highly recommended + to have Maven 2 installed on the system. + </para> + </listitem> + </itemizedlist> + </section> + + <section> + <title>JSR-299 TCK Components</title> + <para>The JSR-299 TCK includes the following components:</para> + <itemizedlist> + <listitem> + <para> + <emphasis role="bold">JBoss Test Harness 1.0.0</emphasis> + and related documentation. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">TestNG 5.9</emphasis> + , the testing framework on which the JBoss Test Harness is + based and which provides the extension points for selecting an + executing the tests in the test suite. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Porting Package SPIs</emphasis> + - Extensions to the JSR-299 SPIs to allow testing of a + container. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">The test suite</emphasis> + , which is a collection of TestNG tests, the TestNG test suite + descriptor and supplamental files that provide data for the + automatic running of tests through the JBoss Test harness. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">JBoss TCK Runner</emphasis> + - An implementation of the JBoss Test Harness deployment APIs + for deploying artifacts to JBoss AS 5.1 + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">TCK documentation</emphasis> + accompanied by release notes identifying updates between + versions. + </para> + </listitem> + </itemizedlist> + <para> + The JSR-299 TCK has been tested with the following: + </para> + <itemizedlist> + <listitem> + <para> Java Platform, Standard Edition, Version 5.0 (Java SE 1.5) + </para> + </listitem> + <listitem> + <para> Java Platform, Enterprise Edition, Version 5 reference + implementation (Java EE 5 RI)</para> + </listitem> + <listitem> + <para>JBoss AS 5.1?</para> + </listitem> + </itemizedlist> + <para>The JSR-299 TCK tests run on the following platforms:</para> + <itemizedlist> + <listitem> + <para>Windows XP?</para> + </listitem> + <listitem> + <para>RHEL 5.2?</para> + </listitem> + </itemizedlist> + </section> + <section> + <title>The JBoss Test Harness</title> + <para> + JSR-299 TCK leverages the JBoss Test Harness. The JBoss Test + Harness is a testing framework based on TestNG that provides a + series of extensions that allow runtime packaging and deployment of + Java EE artifacts (EAR or WAR) for in-container testing. It&apos;s + important to note that the JBoss Test Harness has no relation to or + dependency on the JBoss Application Server (AS). + </para> + <para> + JSR-299 supports Java EE 5, Java EE 6, Java EE 6 Web Profile and + the Embeddable EJB 3.1. Accordingly, the TCK will be able to run in + any of these environments, though at the time of writing the + Embedded EJB (EEJB) mode isn&apos;t operational. + </para> + <note> + <para> + You'll often see the term + <emphasis role="italic">in-container</emphasis> + used in this reference guide. This term refers to running the + test suite in any of the aforementioned environments, whilst + <emphasis role="italic">standalone</emphasis> + refers to running the tests outside the container via a JSR-299 + implementation-specific standalone bootstrap. The standalone mode + only runs those tests which don&apos;t require EJB, Servlet or + JSF to be present. + </para> + </note> + <para> + The last thing Java developers want is yet another testing + framework to make their life more complicated. What they do want is + a good integration with their Integrated Development Environment + (IDE). These days, if a tool doesn&apos;t have an IDE plugin, then + it won&apos;t get the attention it deserves. That&apos;s why the + JBoss Test Harness is built entirely upon TestNG. TestNG is one of + the two prominent test frameworks for Java and TestNG plugins are + available for all major IDEs (with the exception of NetBeans) and + build tools (Ant and Maven 2). + </para> + <para> + Because it leverages the existing TestNG ecosystem, there is no + need for a special test launcher for the JBoss Test Harness, and in + turn the JSR-299 TCK. You simply use the IDE or build tool of your + choice. You also get reporting and debugging for free. + </para> + <para> + The JBoss Test Harness supports the following features: + </para> + <itemizedlist> + <listitem> + <para> + Test activation via any method supported by the TestNG + configuration descriptor (package, group, class) + </para> + </listitem> + <listitem> + <para> + Exclusion of in-container tests in standalone mode + </para> + </listitem> + <listitem> + <para> + Exclusion of individual tests labeled as under + investigation + </para> + </listitem> + <listitem> + <para> + Integration with any TestNG plugin (Eclipse, IntelliJ, + NetBeans, Ant, Maven) + </para> + </listitem> + <listitem> + <para> + Automated reporting capability as provided by TestNG + </para> + </listitem> + <listitem> + <para> + Standalone and in-container test mode + </para> + </listitem> + <listitem> + <para> + Container pluggability + </para> + </listitem> + <listitem> + <para> + Declarative packaging of additional resources and classes + in artifact + </para> + </listitem> + <listitem> + <para> + Declarative deployment exception trapping + </para> + </listitem> + <listitem> + <para> + Artifact dumping for failure and packaging analysis + </para> + </listitem> + </itemizedlist> + <para> + A test is designated by a method annotated with + <literal&gt;(a)org.testng.annotations.Test&lt;/literal&gt; + in a class which extends + <literal>org.jboss.jsr299.tck.AbstractJSR299Test</literal> + and is annotated with + <literal&gt;(a)org.jboss.testharness.impl.packaging.Artifact&lt;/literal&gt; + . + </para> + <para> + The + <literal>@Test</literal> + annotation is provided by TestNG, the + <literal>@Artifact</literal> + annotation is provided by the JBoss Test Harness and the + <literal>AbstractJSR299Test</literal> + is part of the JSR-299 TCK. There is a one-to-one mapping between a + TestNG test class and an artifact. The packaging type is defined by + the + <literal&gt;(a)org.jboss.testharness.impl.packaging.Packaging&lt;/literal&gt; + annotation on the test class, defaulting to a WAR if not specified. + </para> + <para> + Prior to executing the tests for a given class, the JBoss Test + Harness packages the class as a deployable artifact (EAR or WAR), + along with any extra resources specified, and deploys the artifact + to the container. The test execution and results are negotatied via + HTTP communication using a thin layer over the TestNG test launcher. + The test harness can also catch and affirm expected deployment + exceptions. This setup and tear down activity is provided by the + super class + <literal>org.jboss.jsr299.tck.AbstractJSR299Test</literal> + , which all TCK test classes must extend. + </para> + <para> + If the annotation + <literal&gt;(a)org.jboss.testharness.impl.packaging.IntegrationTest + </literal> + is not present on the test class, then it means the test class can + be executed in standalone mode. In standalone mode, the deployable + artifact is assembled on the local classpath and the tests execute + in the same JVM as the launcher, just as though it were a regular + TestNG test case. The standalone mode is provided merely for + convenience and efficiency. + </para> + <para> + Chapter 4 details how to run the TCK test suite using the JBoss + Test Harness. + </para> + </section> + <section> + <title>The Porting Package</title> + <para> + The JSR-299 TCK relies on an implementation of the porting + package to function. The porting package can be divided into two + parts. The first part is comprised of extensions to the JSR-299 SPIs + to allow testing of a container. The second part must contain + implementations of the JBoss Test Harness deployment APIs for + deploying test artifacts to a given container. + </para> + <para> + As mentioned earlier, there are times when the tests need to tap + directly into the JSR-299 implementation to manipulate behavior or + verify results. The porting package includes a set of SPIs that + provide the TCK this level of access without tying the tests to a + given implementation. + </para> + <para> + The four SPI classes in the JSR-299 TCK are as follows: + </para> + <itemizedlist> + <listitem> + <para> + <literal>org.jboss.jsr299.tck.spi.Beans</literal> + </para> + </listitem> + <listitem> + <para> + <literal>org.jboss.jsr299.tck.spi.Contexts</literal> + </para> + </listitem> + <listitem> + <para> + <literal>org.jboss.jsr299.tck.spi.EL</literal> + </para> + </listitem> + <listitem> + <para> + <literal>org.jboss.jsr299.tck.spi.Managers</literal> + </para> + </listitem> + </itemizedlist> + <para> + To run the full TCK you must additionally implement + <literal>org.jboss.testharness.spi.Containers</literal> + , which handles deploying the test artifact to the container. If you + wish to run the TCK in standalone mode, you must also implement + <literal>org.jboss.testharness.spi.StandaloneContainers</literal> + . Implementations of these APIs are already available for JBoss AS + 5.1. Therefore, you only need to implement this part of the porting + package if you wish to use a container other than JBoss AS 5.1, for + instance GlassFish V3. + </para> + <para> + The default configuration for the TCK is property-based. + Therefore, you specify the implementation class for all of these + SPIs in a properties file. There are additional properties for + controlling whether the tests are run in standalone or in-container + mode. Optionally, you can use system properties to configure the + TCK. Details are provided in section 4.1. + </para> + </section> + <section> + <title>TCK Audit Tool</title> + <para> + The TCK audit is used to list out the assertions identified in + the JSR-299 specification. It uses Java annotation processing to + match the assertions to testcases in the test suite and produce a + coverage report. + </para> + <para> + The audit document is provided along with the TCK and each + assertion matches against at least one test. Each assertion is + defined with a reference to a chapter, section and paragraph from + the specification document, making it easy for the licensee to + locate the language in the specification document that supports the + feature being tested. + </para> + </section> + </section> </chapter>