[hibernate-commits] Hibernate SVN: r17606 - beanvalidation/tck/trunk/src/main/docbook/en-US/harness.
hibernate-commits at lists.jboss.org
hibernate-commits at lists.jboss.org
Thu Oct 1 09:08:50 EDT 2009
Author: hardy.ferentschik
Date: 2009-10-01 09:08:50 -0400 (Thu, 01 Oct 2009)
New Revision: 17606
Modified:
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml
Log:
tck docs
Modified: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml 2009-10-01 12:25:36 UTC (rev 17605)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml 2009-10-01 13:08:50 UTC (rev 17606)
@@ -1,228 +1,214 @@
-<?xml version='1.0' encoding='UTF-8'?>
-<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<chapter id="executing-test-harness">
- <title>Executing a Test Suite</title>
- <para>
- This chapter explains how to execute and debug a test suite built using
- the JBoss Test Harness.
- </para>
- <section id="test-suite-runner">
- <title>Building a test suite runner using Maven 2</title>
- <para>
- The test suite runner project is the magic that makes everything come
- together and allows you to execute the test suite. If you fully
- understand how the JBoss Test Harness functions, and have a good grasp
- on Maven 2, then it's not to difficult to understand how the test suite
- runner project works. Regardless of your background, this guide covers
- what you need to know to get up and running by studying the test suite
- runner used to run the CDI TCK against the CDI RI, Web Beans.
- </para>
- <para>
- The TCK runner for the Web Beans can be found in the jboss-tck-runner
- directory in the Web Beans distribution. The dependencies of the TCK
- runner project for Web Beans are listed in
- <xref linkend="tck-runner-dependencies" />.
- </para>
- <table frame="all" id="tck-runner-dependencies" title="CDI TCK Runner dependencies">
- <title>Web Beans JBoss TCK Runner Dependencies</title>
- <tgroup cols="3">
- <colspec colname="c1" />
- <colspec colname="c2" />
- <colspec colname="c3" />
- <thead>
- <row>
- <entry>Group ID</entry>
- <entry>Artifact ID</entry>
- <entry>Version</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>org.jboss.webbeans</entry>
- <entry>jsr299-api</entry>
- <entry>1.0.0-SNAPSHOT</entry>
- </row>
- <row>
- <entry>org.jboss.jsr299.tck</entry>
- <entry>jsr299-tck-api</entry>
- <entry>1.0.0-SNAPSHOT</entry>
- </row>
- <row>
- <entry>org.jboss.jsr299.tck</entry>
- <entry>jsr299-tck-impl</entry>
- <entry>1.0.0-SNAPSHOT</entry>
- </row>
- <row>
- <entry>org.jboss.webbeans</entry>
- <entry>webbeans-core</entry>
- <entry>1.0.0-SNAPSHOT</entry>
- </row>
- <row>
- <entry>org.jboss.webbeans</entry>
- <entry>webbeans-porting-package</entry>
- <entry>1.0.0-SNAPSHOT</entry>
- </row>
- <row>
- <entry>org.testng</entry>
- <entry>testng (classifier: jdk15)</entry>
- <entry>5.8</entry>
- </row>
- <row>
- <entry>org.jboss.test-harness</entry>
- <entry>jboss-test-harness-jboss-as-51</entry>
- <entry>1.0.0.BETA3</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- <para>
- You can find all of these artifacts in the <ulink
- url="http://repository.jboss.org/maven2">JBoss Maven
- repository</ulink>.
- </para>
- <para>
- You should substituate the webbeans-core and webbeans-porting-package
- artifacts from table 2.2.3 with your own artifacts. You'll also need to
- replace the jboss-test-harness-jboss-as-51 artifact if you are not
- testing your implementation on JBoss AS 5.1. The
- jboss-test-harness-jboss-as-51 artifact contains implementations of the
- <literal>Containers</literal>
- SPI for the JBoss Test Harness for JBoss AS 5.1.
- </para>
- <note>
- <para>
- When running the test suite in the in-container mode, the tests
- will run against libraries installed into the container. In this
- project, Web Beans is only declared as a Maven dependency for when
- the TCK test suite is being executed in standalone mode.
- </para>
- </note>
+ <title>Executing a Test Suite</title>
- <programlisting role="xml"><![CDATA[<plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-dependency-plugin</artifactId>
- <executions>
- <execution>
- <id>copy</id>
- <phase>process-resources</phase>
- <goals>
- <goal>copy</goal>
- </goals>
- <configuration>
- <stripVersion>true</stripVersion>
- <artifactItems>
- <artifactItem>
- <groupId>org.jboss.jsr299.tck</groupId>
- <artifactId>jsr299-tck-impl</artifactId>
- <type>xml</type>
- <classifier>suite</classifier>
- <overWrite>true</overWrite>
- </artifactItem>
- <artifactItem>
- <groupId>org.jboss.webbeans</groupId>
- <artifactId>
- webbeans-porting-package
- </artifactId>
- <overWrite>true</overWrite>
- <outputDirectory>
- ${project.build.directory}/dependency/lib
- </outputDirectory>
- </artifactItem>
- <artifactItem>
- <groupId>org.jboss.webbeans</groupId>
- <artifactId>webbeans-core-test</artifactId>
- <overWrite>true</overWrite>
- <outputDirectory>
- ${project.build.directory}/dependency/lib
- </outputDirectory>
- </artifactItem>
- <artifactItem>
- <groupId>javax.el</groupId>
- <artifactId>el-ri</artifactId>
- <overWrite>true</overWrite>
- <outputDirectory>
- ${project.build.directory}/dependency/lib
- </outputDirectory>
- </artifactItem>
- </artifactItems>
- </configuration>
- </execution>
- </executions>
-</plugin>]]></programlisting>
- <para>
- The target folder for the copies of the dependencies (i.e., the JAR
- files) is declared as the JBoss Test Harness library directory; this
- results in these libraries being added to the test artifact using the
- following property assignment:
- </para>
- <programlisting>org.jboss.testharness.libraryDirectory=target/dependency/lib</programlisting>
- <para>
- We also copy the test suite configuration from the
- local Maven
- repository (groupId=org.jboss.jsr299.tck,
- artifactId=jsr299-tck-impl,
- classifier=suite, type=xml,
- version=1.0.0-SNAPSHOT) to a local
- repository as the TestNG Maven
- plugin expects a local file.
- </para>
- <para>
- The TCK is executed using the Maven TestNG plugin. Maven 2 profiles are
- used to control the properties that are set at the time of the
- execution. For instance, the
- <literal>incontainer</literal>
- profile enables integration tests and disables standalone mode,
- changing the default settings.
- </para>
- <para>
- The jboss-tck-runner project also defines the JBoss Test Harness
- extra configuration directory using the following property:
- </para>
- <programlisting>org.jboss.testharness.container.extraConfigurationDir=../jboss-as</programlisting>
- <para>
- The JBoss Test Harness looks in this directory for either a
- build.properties or local.build.properties file that declares
- additional configuration properties. In particular, the JBoss AS
- Containers implementation looks here to find the
- <literal>jboss.home</literal> property so it knows where the scripts
- are located to start and stop JBoss AS.
- </para>
- </section>
+ <para>This chapter explains how to execute and debug a test suite built
+ using the JBoss Test Harness.</para>
- <section id="dumping-test-artifacts">
- <title>Dumping the Test Artifacts to Disk</title>
- <para>
- As you have learned, when the test suite is executing using
- in-container mode, each test class is packaged as a deployable artifact
- and deployed to the container. The test is then executed within the
- context of the deployed application. This leaves room for errors in
- packaging. When investigating a test failure, it's helpful to be able
- to inspect the artifact after it is generated. The JBoss Test Harness
- can accommodate this type of inspection by "dumping" the generated
- artifact to disk.
- </para>
- <para>
- If you want to write the artifacts to disk, and avoid executing the
- test suite, you can simply execute the main method of the class
- <literal>org.jboss.testharness.api.TCK</literal>.
- For example you could use a Maven profile that is activated when the
- <literal>dumpArtifacts</literal>
- command line property is defined:
- </para>
- <programlisting>mvn test-compile -DdumpArtifacts</programlisting>
- <para>
- The output directory where the artifacts are written is defined by the
- property
- <literal>org.jboss.testharness.outputDirectory</literal>.
- </para>
- <para>
- Once the artifact is written to disk, you have an option of manually
- deploying it to the container. You can execute the tests in the artfact
- by requesting the context path of the application in the browser. If
- you want to execute an individual test method, specify the method name
- in the
- <literal>methodName</literal>
- request parameter (e.g., ?methodName=testMethodName).
- </para>
- </section>
+ <section id="test-suite-runner">
+ <title>Building a test suite runner using Maven 2</title>
+
+ <para>The test suite runner project is the magic that makes everything
+ come together and allows you to execute the test suite. If you fully
+ understand how the JBoss Test Harness functions, and have a good grasp on
+ Maven 2, then it's not to difficult to understand how the test suite
+ runner project works. Regardless of your background, this guide covers
+ what you need to know to get up and running by studying the test suite
+ runner used to run the Bean Validation TCK against the Bean Validation RI,
+ Hibernate Validator.</para>
+
+ <para>The TCK runner for the Hibernate Validator can be found in the
+ <filename>hibernate-validator-tck-runner</filename> directory in the
+ Hibernate Validator checkout. The dependencies of the TCK runner project
+ for Hibernate Validator are listed in <xref
+ linkend="tck-runner-dependencies" />.</para>
+
+ <table frame="all" id="tck-runner-dependencies"
+ title="CDI TCK Runner dependencies">
+ <title>Web Beans JBoss TCK Runner Dependencies</title>
+
+ <tgroup cols="3">
+ <colspec colname="c1" />
+
+ <colspec colname="c2" />
+
+ <colspec colname="c3" />
+
+ <thead>
+ <row>
+ <entry>Group ID</entry>
+
+ <entry>Artifact ID</entry>
+
+ <entry>Version</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry>javax.validation</entry>
+
+ <entry>validation-api</entry>
+
+ <entry>1.0.0.GA</entry>
+ </row>
+
+ <row>
+ <entry>org.hibernate</entry>
+
+ <entry>hibernate-validator</entry>
+
+ <entry>4.0.0.GA</entry>
+ </row>
+
+ <row>
+ <entry>org.hibernate.jsr303.tck</entry>
+
+ <entry>jsr303-tck</entry>
+
+ <entry>1.0.0.GA</entry>
+ </row>
+
+ <row>
+ <entry>org.sl4j</entry>
+
+ <entry>slf4j-log4j12</entry>
+
+ <entry>1.5.6</entry>
+ </row>
+
+ <row>
+ <entry>org.testng</entry>
+
+ <entry>testng (classifier: jdk15)</entry>
+
+ <entry>5.8</entry>
+ </row>
+
+ <row>
+ <entry>org.jboss.test-harness</entry>
+
+ <entry>jboss-test-harness-jboss-as-51</entry>
+
+ <entry>1.0.0</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>You can find all of these artifacts in the <ulink
+ url="http://repository.jboss.org/maven2">JBoss Maven
+ repository</ulink>.</para>
+
+ <para>You should substituate the hibernate-validator artifact from <xref
+ linkend="tck-runner-dependencies" /> with your own artifact. You'll also
+ need to replace the jboss-test-harness-jboss-as-51 artifact if you are not
+ testing your implementation on JBoss AS 5.1. The
+ jboss-test-harness-jboss-as-51 artifact contains implementations of the
+ <literal>Containers</literal> SPI for the JBoss Test Harness for JBoss AS
+ 5.1.</para>
+
+ <note>
+ <para>When running the test suite in the in-container mode, the tests
+ will run against libraries installed into the container. In this
+ project, Hibernate Validator is only declared as a Maven dependency for
+ when the TCK test suite is being executed in standalone mode.</para>
+ </note>
+
+ <programlisting role="xml"><plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>copy</id>
+ <phase>generate-test-sources</phase>
+ <goals>
+ <goal>copy</goal>
+ </goals>
+ <configuration>
+ <stripVersion>true</stripVersion>
+ <artifactItems>
+ <artifactItem>
+ <groupId>org.hibernate.jsr303.tck</groupId>
+ <artifactId>jsr303-tck</artifactId>
+ <type>xml</type>
+ <classifier>suite</classifier>
+ <overWrite>false</overWrite>
+ </artifactItem>
+ <artifactItem>
+ <groupId>javax.validation</groupId>
+ <artifactId>validation-api</artifactId>
+ <overWrite>true</overWrite>
+ <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>org.hibernate</groupId>
+ <artifactId>hibernate-validator</artifactId>
+ <overWrite>true</overWrite>
+ <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ <overWrite>true</overWrite>
+ <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
+ </artifactItem>
+ </artifactItems>
+ </configuration>
+ </execution>
+ </executions>
+</plugin></programlisting>
+
+ <para>The target folder for the copies of the dependencies (i.e., the JAR
+ files) is declared as the JBoss Test Harness library directory; this
+ results in these libraries being added to the test artifact using the
+ following property assignment:</para>
+
+ <programlisting>org.jboss.testharness.libraryDirectory=target/dependency/lib</programlisting>
+
+ <para>We also copy the test suite configuration from the local Maven
+ repository (groupId=org.hibernate.jsr303.tck, artifactId=jsr303-tck,
+ classifier=suite, type=xml, version=1.0.0) to a local repository as the
+ TestNG Maven plugin expects a local file.</para>
+
+ <para>The TCK is executed using the Maven TestNG plugin. Maven 2 profiles
+ are used to control the properties that are set at the time of the
+ execution. For instance, the <literal>incontainer</literal> profile
+ enables integration tests and disables standalone mode, changing the
+ default settings.</para>
+ </section>
+
+ <section id="dumping-test-artifacts">
+ <title>Dumping the Test Artifacts to Disk</title>
+
+ <para>As you have learned, when the test suite is executing using
+ in-container mode, each test class is packaged as a deployable artifact
+ and deployed to the container. The test is then executed within the
+ context of the deployed application. This leaves room for errors in
+ packaging. When investigating a test failure, it's helpful to be able to
+ inspect the artifact after it is generated. The JBoss Test Harness can
+ accommodate this type of inspection by "dumping" the generated artifact to
+ disk.</para>
+
+ <para>If you want to write the artifacts to disk, and avoid executing the
+ test suite, you can simply execute the main method of the class
+ <literal>org.jboss.testharness.api.TCK</literal>. For example you could
+ use a Maven profile that is activated when the
+ <literal>dumpArtifacts</literal> command line property is defined:</para>
+
+ <programlisting>mvn test-compile -DdumpArtifacts</programlisting>
+
+ <para>The output directory where the artifacts are written is defined by
+ the property
+ <literal>org.jboss.testharness.outputDirectory</literal>.</para>
+
+ <para>Once the artifact is written to disk, you have an option of manually
+ deploying it to the container. You can execute the tests in the artfact by
+ requesting the context path of the application in the browser. If you want
+ to execute an individual test method, specify the method name in the
+ <literal>methodName</literal> request parameter (e.g.,
+ ?methodName=testMethodName).</para>
+ </section>
</chapter>
Modified: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml 2009-10-01 12:25:36 UTC (rev 17605)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml 2009-10-01 13:08:50 UTC (rev 17606)
@@ -1,286 +1,226 @@
-<?xml version='1.0' encoding='UTF-8'?>
-<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<chapter id="test-harness-introduction">
- <title>Introduction (JBoss Test Harness)</title>
+ <title>Introduction (JBoss Test Harness)</title>
- <para>
- This chapter explains the purpose of the test harness and describes
- its key features.
- </para>
- <para>
- The JBoss Test Harness is a testing framework based on TestNG that
- provides a series of extensions that allow runtime packaging and
- deployment of Java EE artifacts (EAR or WAR) for in-container testing.
- It's important to note that the JBoss Test Harness has no relation with,
- or dependency on, the JBoss Application Server (JBoss AS).
- </para>
- <note>
- <para>
- You'll often see the term
- <emphasis role="italic">in-container</emphasis>
- used in this reference guide. This term refers to running the test
- suite in any of the aforementioned environments, whilst
- <emphasis role="italic">standalone</emphasis>
- refers to running the tests outside the container via an
- implementation-specific standalone bootstrap. The standalone mode only
- runs those tests which the CDI RI can run without deployment in a Java
- EE container.
- </para>
- </note>
- <para>
- The last thing Java developers want is yet another testing framework to
- make their life more complicated. That's why the JBoss Test Harness is
- built entirely upon TestNG. TestNG is one of two prominent test
- frameworks for Java (the other being JUnit). Furthermore, what developers
- want is a good integration with their Integrated Development Environment
- (IDE). These days, if a tool doesn't have an IDE plugin, then it won't
- get the attention it deserves. TestNG plugins are available for all major
- IDEs and build tools (Ant and Maven 2). Again, a motivating factor for
- extending TestNG.
- </para>
- <para>
- Because it leverages the existing TestNG ecosystem, there is no need for
- a special test launcher for the JBoss Test Harness. You simply use the
- IDE or build tool of your choice (so long as it has TestNG support). You
- also get reporting and debugging for free (various reporting plugins are
- provided for TestNG).
- </para>
- <para>
- You can read more about TestNG at
- <ulink url="http://testng.org/doc/documentation-main.html">testng.org</ulink>.
- </para>
- <para>
- The JBoss Test Harness supports the following features:
- </para>
- <itemizedlist>
- <listitem>
- <para>
- Test activation via any method supported by the TestNG
- configuration descriptor (package, group, class)
- </para>
- </listitem>
- <listitem>
- <para>
- Exclusion of in-container tests in standalone mode
- </para>
- </listitem>
- <listitem>
- <para>
- Exclusion of individual tests labeled as under investigation
- </para>
- </listitem>
- <listitem>
- <para>
- Integration with any TestNG plugin (Eclipse, IntelliJ, Ant, Maven)
- </para>
- </listitem>
- <listitem>
- <para>
- Automated reporting capability as provided by TestNG
- </para>
- </listitem>
- <listitem>
- <para>
- Standalone and in-container test mode
- </para>
- </listitem>
- <listitem>
- <para>
- Container pluggability
- </para>
- </listitem>
- <listitem>
- <para>
- Declarative packaging of additional resources and classes in
- artifact
- </para>
- </listitem>
- <listitem>
- <para>
- Declarative deployment exception trapping
- </para>
- </listitem>
- <listitem>
- <para>
- Artifact dumping for failure and packaging analysis
- </para>
- </listitem>
- </itemizedlist>
- <para>
- A test is designated by a method annotated with
- <literal>@org.testng.annotations.Test</literal>
- in a class which extends
- <literal>org.jboss.testharness.AbstractTest</literal>
- and is annotated with
- <literal>@org.jboss.testharness.impl.packaging.Artifact</literal>.
- </para>
- <note>
- <para>
- Test suites may often choose to extend <literal>AbstractTest</literal>
- and require tests to extend that base class. In fact, both the CDI TCK
- and the Bean Validation TCK provide base classes that extend
- <literal>AbstractTest</literal> to provide functionality specific to
- the needs of the TCK.
- </para>
- </note>
- <para>
- The
- <literal>@Test</literal>
- annotation is provided by TestNG, the
- <literal>@Artifact</literal>
- annotation is provided by the JBoss Test Harness and the
- <literal>AbstractTest</literal>
- is part of the JBoss Test Harness. There is a one-to-one mapping between a
- TestNG test class and an artifact. The packaging type is defined by the
- <literal>@org.jboss.testharness.impl.packaging.Packaging</literal>
- annotation on the test class, defaulting to a WAR if not specified.
- </para>
- <para>
- Prior to executing the tests for a given class, the JBoss Test Harness
- packages the class as a deployable artifact (EAR or WAR), along with any
- extra resources specified, and deploys the artifact to the container. The
- harness provides test execution and result reporting via HTTP
- communication to a simple Servlet using a thin layer over the TestNG test
- launcher. The test harness can also catch and enforce expected deployment
- exceptions. This setup and tear down activity is provided by the super
- class
- <literal>org.jboss.testharness.AbstractTest</literal>,
- which all test classes must extend (directly or indirectly).
- </para>
- <para>
- If the annotation
- <literal>@org.jboss.testharness.impl.packaging.IntegrationTest
- </literal>
- is not present on the test class, then it means the test class can be
- executed in standalone mode. In standalone mode, the deployable artifact
- is assembled on the local classpath and the tests execute in the same JVM
- as the launcher, just as though it were a regular TestNG test case. The
- standalone mode is provided for convenience and efficiency, allowing you
- the speed of mock-based testing and the confidence of an in-container test,
- using the same test objects and tests.
- </para>
-
- <section id="in-container-communication">
- <title>Negotiating the execution of an in-container test</title>
- <para>
- The basic procedure of an in-container test is as follows. The JBoss
- Test Harness produces a deployable artifact from an
- <literal>@Artifact</literal>
- test class and any declared dependent classes, descriptors or other
- resources. Then it deploys the artifact to the container using the
- <literal>Containers</literal>
- SPI, negotiates with the container to execute the test and return the
- result and, finally, undeploys the artifact. TestNG collects the results
- of all the tests run in the typical way and produces a report.
- </para>
- <graphic fileref="images/in-container-execution.png" align="center" />
- <para>
- The question is, how does the JBoss Test Harness negotiate with the
- container to execute the test when TestNG is being invoked locally?
- Technially the mechanism is pluggable, but JBoss Test Harness provides
- a default implementation that uses HTTP communication that you will
- likely use. Here's how the default implementation works.
- </para>
- <para>
- The artifact generator bundles and registers (in the web.xml
- descriptor) an
- <literal>HttpServlet</literal>,
- <literal>org.jboss.testharness.impl.runner.servlet.ServletTestRunner</literal>,
- that responds to test execution GET requests. TestNG running on the
- client side delegates to a test launcher (more on that in a moment)
- which originates these text execution requests to transfer control to
- the container JVM. The name of the test method to be executed is
- specified in a request query parameter named
- <literal>methodName</literal>.
- </para>
- <para>
- When the test execution request is received, the servlet delegates to
- an instance of
- <literal>org.jboss.testharness.impl.runner.TestRunner</literal>,
- passing it the name of the test method.
- <literal>TestRunner</literal>
- reads the name of the test class from the resource
- META-INF/jboss-test-harness.properties, which is bundled in the
- artifact by the artifact generator. It then combines the class name
- and the method name to produce a TestNG test suite and runs the suite
- (within the context of the container).
- </para>
- <para>
- TestNG returns the results of the run as an
- <literal>ITestResult</literal>
- object.
- <literal>ServletTestRunner</literal>
- translates this object into a
- <literal>org.jboss.testharness.api.TestResult</literal>
- and passes it back to the test launcher on the client side by encoding
- the translated object into the response. The object gets encoded as
- either html or a serialized object, depending on the value of the
- <literal>outputMode</literal>
- request parameter that was passed to the servlet. Once the result has
- been transfered to the client-side TestNG, TestNG wraps up the run of
- the test as though it had been executed in the same JVM.
- </para>
- <para>
- There's one piece missing. How does TestNG on the client side know to
- submit a request to the
- <literal>ServletTestRunner</literal>
- servlet to get TestNG to execute the test in the container JVM? That's
- the role of the test launcher.
- </para>
- <para>
- The test launcher is the API that allows test suite to launch the test
- in a pluggable fashion.
- <literal>AbstractTest</literal>,
- the super class of
- <literal>AbtractJSR299Test</literal>,
- implements
- <literal>IHookable</literal>,
- a TestNG interface which allows the execution of the test method to
- be intercepted. Using that mechanism, <literal>AbstractTest</literal> delegates execution
- of the test method (a method annotated with
- <literal>@Test</literal>
- in an
- <literal>@Artifact</literal>
- class) to an implementation of
- <literal>org.jboss.testharness.api.TestLauncher</literal>
- if the tests are being executed in-container. As you might anticipate,
- the implementation is specified using a property with the same name as
- the interface in a META-INF/jboss-test-launcher.properties resource.
- The JBoss Test Harness provides a default implementation,
- <literal>org.jboss.testharness.impl.runner.servlet.ServletTestLauncher</literal>,
- that hooks into the HTTP communication infrastructure described
- above. It invokes the
- <literal>ServletTestRunner</literal>
- servlet for each method annotated with
- <literal>@Test</literal>
- in the
- <literal>@Artifact</literal>
- that is not otherwise disabled.
- </para>
- <para>
- If you wish to implement the runner yourself, you must return a
- <literal>TestResult</literal> as a result of executing the method in
- the container. You must also ensure that any exception which occurs
- during deployment is wrapped as a
- <literal>org.jboss.testharness.api.DeploymentException</literal>, and
- that any communication problem is rethrown as an
- <literal>IOException</literal>. The deployment exception may be
- transformed by an implementation of the
- <literal>org.jboss.testharness.api.DeploymentExceptionTransformer
- </literal> interface, which is specified using the
- <literal>org.jboss.testharness.container.deploymentExceptionTransformer
- </literal> property. The default implementation passes on the original
- exception unchanged. The implementation for JBoss AS used with the CDI
- TCK, on the other hand, deciphers the exception thrown by the JBoss
- deployer and converts it to one of the catagory exceptions defined in
- the CDI TCK API.
- </para>
- <para>
- So in short, JBoss Test Harness takes care of all the interfaces you
- need to execute tests in-container except for the implementation of
- the <literal>Containers</literal> SPI. That is, unless you are
- deploying to one of the containers supported by the JBoss Test Harness
- (TODO we need a table showing supported containers).
- </para>
- </section>
+ <para>This chapter explains the purpose of the test harness and describes
+ its key features.</para>
+
+ <para>The JBoss Test Harness is a testing framework based on TestNG that
+ provides a series of extensions that allow runtime packaging and deployment
+ of Java EE artifacts (EAR or WAR) for in-container testing. It's important
+ to note that the JBoss Test Harness has no relation with, or dependency on,
+ the JBoss Application Server (JBoss AS).</para>
+
+ <note>
+ <para>You'll often see the term <emphasis
+ role="italic">in-container</emphasis> used in this reference guide. This
+ term refers to running the test suite in any of the aforementioned
+ environments, whilst <emphasis role="italic">standalone</emphasis> refers
+ to running the tests outside the container via an implementation-specific
+ standalone bootstrap. The standalone mode only runs those tests which the
+ Bean Validation RI can run without deployment in a Java EE
+ container.</para>
+ </note>
+
+ <para>The last thing Java developers want is yet another testing framework
+ to make their life more complicated. That's why the JBoss Test Harness is
+ built entirely upon TestNG. TestNG is one of two prominent test frameworks
+ for Java (the other being JUnit). Furthermore, what developers want is a
+ good integration with their Integrated Development Environment (IDE). These
+ days, if a tool doesn't have an IDE plugin, then it won't get the attention
+ it deserves. TestNG plugins are available for all major IDEs and build tools
+ (Ant and Maven 2). Again, a motivating factor for extending TestNG.</para>
+
+ <para>Because it leverages the existing TestNG ecosystem, there is no need
+ for a special test launcher for the JBoss Test Harness. You simply use the
+ IDE or build tool of your choice (so long as it has TestNG support). You
+ also get reporting and debugging for free (various reporting plugins are
+ provided for TestNG).</para>
+
+ <para>You can read more about TestNG at <ulink
+ url="http://testng.org/doc/documentation-main.html">testng.org</ulink>.</para>
+
+ <para>The JBoss Test Harness supports the following features:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Test activation via any method supported by the TestNG
+ configuration descriptor (package, group, class)</para>
+ </listitem>
+
+ <listitem>
+ <para>Exclusion of in-container tests in standalone mode</para>
+ </listitem>
+
+ <listitem>
+ <para>Exclusion of individual tests labeled as under
+ investigation</para>
+ </listitem>
+
+ <listitem>
+ <para>Integration with any TestNG plugin (Eclipse, IntelliJ, Ant,
+ Maven)</para>
+ </listitem>
+
+ <listitem>
+ <para>Automated reporting capability as provided by TestNG</para>
+ </listitem>
+
+ <listitem>
+ <para>Standalone and in-container test mode</para>
+ </listitem>
+
+ <listitem>
+ <para>Container pluggability</para>
+ </listitem>
+
+ <listitem>
+ <para>Declarative packaging of additional resources and classes in
+ artifact</para>
+ </listitem>
+
+ <listitem>
+ <para>Declarative deployment exception trapping</para>
+ </listitem>
+
+ <listitem>
+ <para>Artifact dumping for failure and packaging analysis</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>A test is designated by a method annotated with
+ <literal>@org.testng.annotations.Test</literal> in a class which extends
+ <literal>org.jboss.testharness.AbstractTest</literal> and is annotated with
+ <literal>@org.jboss.testharness.impl.packaging.Artifact</literal>.</para>
+
+ <note>
+ <para>Test suites may often choose to extend
+ <literal>AbstractTest</literal> and require tests to extend that base
+ class. In fact, both the CDI TCK and the Bean Validation TCK provide base
+ classes that extend <literal>AbstractTest</literal> to provide
+ functionality specific to the needs of the TCK.</para>
+ </note>
+
+ <para>The <literal>@Test</literal> annotation is provided by TestNG, the
+ <literal>@Artifact</literal> annotation is provided by the JBoss Test
+ Harness and the <literal>AbstractTest</literal> is part of the JBoss Test
+ Harness. There is a one-to-one mapping between a TestNG test class and an
+ artifact. The packaging type is defined by the
+ <literal>@org.jboss.testharness.impl.packaging.Packaging</literal>
+ annotation on the test class, defaulting to a WAR if not specified.</para>
+
+ <para>Prior to executing the tests for a given class, the JBoss Test Harness
+ packages the class as a deployable artifact (EAR or WAR), along with any
+ extra resources specified, and deploys the artifact to the container. The
+ harness provides test execution and result reporting via HTTP communication
+ to a simple Servlet using a thin layer over the TestNG test launcher. The
+ test harness can also catch and enforce expected deployment exceptions. This
+ setup and tear down activity is provided by the super class
+ <literal>org.jboss.testharness.AbstractTest</literal>, which all test
+ classes must extend (directly or indirectly).</para>
+
+ <para>If the annotation
+ <literal>@org.jboss.testharness.impl.packaging.IntegrationTest </literal> is
+ not present on the test class, then it means the test class can be executed
+ in standalone mode. In standalone mode, the deployable artifact is assembled
+ on the local classpath and the tests execute in the same JVM as the
+ launcher, just as though it were a regular TestNG test case. The standalone
+ mode is provided for convenience and efficiency, allowing you the speed of
+ mock-based testing and the confidence of an in-container test, using the
+ same test objects and tests.</para>
+
+ <section id="in-container-communication">
+ <title>Negotiating the execution of an in-container test</title>
+
+ <para>The basic procedure of an in-container test is as follows. The JBoss
+ Test Harness produces a deployable artifact from an
+ <literal>@Artifact</literal> test class and any declared dependent
+ classes, descriptors or other resources. Then it deploys the artifact to
+ the container using the <literal>Containers</literal> SPI, negotiates with
+ the container to execute the test and return the result and, finally,
+ undeploys the artifact. TestNG collects the results of all the tests run
+ in the typical way and produces a report.</para>
+
+ <graphic align="center" fileref="images/in-container-execution.png" />
+
+ <para>The question is, how does the JBoss Test Harness negotiate with the
+ container to execute the test when TestNG is being invoked locally?
+ Technially the mechanism is pluggable, but JBoss Test Harness provides a
+ default implementation that uses HTTP communication that you will likely
+ use. Here's how the default implementation works.</para>
+
+ <para>The artifact generator bundles and registers (in the web.xml
+ descriptor) an <literal>HttpServlet</literal>,
+ <literal>org.jboss.testharness.impl.runner.servlet.ServletTestRunner</literal>,
+ that responds to test execution GET requests. TestNG running on the client
+ side delegates to a test launcher (more on that in a moment) which
+ originates these text execution requests to transfer control to the
+ container JVM. The name of the test method to be executed is specified in
+ a request query parameter named <literal>methodName</literal>.</para>
+
+ <para>When the test execution request is received, the servlet delegates
+ to an instance of
+ <literal>org.jboss.testharness.impl.runner.TestRunner</literal>, passing
+ it the name of the test method. <literal>TestRunner</literal> reads the
+ name of the test class from the resource
+ META-INF/jboss-test-harness.properties, which is bundled in the artifact
+ by the artifact generator. It then combines the class name and the method
+ name to produce a TestNG test suite and runs the suite (within the context
+ of the container).</para>
+
+ <para>TestNG returns the results of the run as an
+ <literal>ITestResult</literal> object.
+ <literal>ServletTestRunner</literal> translates this object into a
+ <literal>org.jboss.testharness.api.TestResult</literal> and passes it back
+ to the test launcher on the client side by encoding the translated object
+ into the response. The object gets encoded as either html or a serialized
+ object, depending on the value of the <literal>outputMode</literal>
+ request parameter that was passed to the servlet. Once the result has been
+ transfered to the client-side TestNG, TestNG wraps up the run of the test
+ as though it had been executed in the same JVM.</para>
+
+ <para>There's one piece missing. How does TestNG on the client side know
+ to submit a request to the <literal>ServletTestRunner</literal> servlet to
+ get TestNG to execute the test in the container JVM? That's the role of
+ the test launcher.</para>
+
+ <para>The test launcher is the API that allows test suite to launch the
+ test in a pluggable fashion. <literal>AbstractTest</literal> implements
+ <literal>IHookable</literal>, a TestNG interface which allows the
+ execution of the test method to be intercepted. Using that mechanism,
+ <literal>AbstractTest</literal> delegates execution of the test method (a
+ method annotated with <literal>@Test</literal> in an
+ <literal>@Artifact</literal> class) to an implementation of
+ <literal>org.jboss.testharness.api.TestLauncher</literal> if the tests are
+ being executed in-container. As you might anticipate, the implementation
+ is specified using a property with the same name as the interface in a
+ META-INF/jboss-test-launcher.properties resource. The JBoss Test Harness
+ provides a default implementation,
+ <literal>org.jboss.testharness.impl.runner.servlet.ServletTestLauncher</literal>,
+ that hooks into the HTTP communication infrastructure described above. It
+ invokes the <literal>ServletTestRunner</literal> servlet for each method
+ annotated with <literal>@Test</literal> in the
+ <literal>@Artifact</literal> that is not otherwise disabled.</para>
+
+ <para>If you wish to implement the runner yourself, you must return a
+ <literal>TestResult</literal> as a result of executing the method in the
+ container. You must also ensure that any exception which occurs during
+ deployment is wrapped as a
+ <literal>org.jboss.testharness.api.DeploymentException</literal>, and that
+ any communication problem is rethrown as an
+ <literal>IOException</literal>. The deployment exception may be
+ transformed by an implementation of the
+ <literal>org.jboss.testharness.api.DeploymentExceptionTransformer
+ </literal> interface, which is specified using the
+ <literal>org.jboss.testharness.container.deploymentExceptionTransformer
+ </literal> property. The default implementation passes on the original
+ exception unchanged. </para>
+
+ <para>So in short, JBoss Test Harness takes care of all the interfaces you
+ need to execute tests in-container except for the implementation of the
+ <literal>Containers</literal> SPI. That is, unless you are deploying to
+ one of the containers supported by the JBoss Test Harness.</para>
+ </section>
</chapter>
More information about the hibernate-commits
mailing list