[hibernate-commits] Hibernate SVN: r17578 - in beanvalidation/tck/trunk: src/main and 5 other directories.
hibernate-commits at lists.jboss.org
hibernate-commits at lists.jboss.org
Tue Sep 29 12:35:00 EDT 2009
Author: hardy.ferentschik
Date: 2009-09-29 12:34:58 -0400 (Tue, 29 Sep 2009)
New Revision: 17578
Added:
beanvalidation/tck/trunk/src/main/docbook/
beanvalidation/tck/trunk/src/main/docbook/en-US/
beanvalidation/tck/trunk/src/main/docbook/en-US/Author_Group.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Info.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Preface.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/appeals-process.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/configuration.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-debugging.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-running.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/executing.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Author_Group.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Info.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Preface.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/configuration.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/images/
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/images/in-container-execution.png
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/harness/master.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/images/
beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-emailable-report.png
beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-plugin-report.png
beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-suite-detail-report.png
beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-summary-report.png
beanvalidation/tck/trunk/src/main/docbook/en-US/installation.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/introduction.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/master.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/part-background.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/part-execution.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/part-test-harness.xml
beanvalidation/tck/trunk/src/main/docbook/en-US/reporting.xml
Modified:
beanvalidation/tck/trunk/
beanvalidation/tck/trunk/pom.xml
Log:
Initial import of TCK doc. Taken from JSR-299
Property changes on: beanvalidation/tck/trunk
___________________________________________________________________
Name: svn:ignore
+ target
Modified: beanvalidation/tck/trunk/pom.xml
===================================================================
--- beanvalidation/tck/trunk/pom.xml 2009-09-29 11:56:52 UTC (rev 17577)
+++ beanvalidation/tck/trunk/pom.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -73,7 +73,72 @@
</extension>
</extensions>
<plugins>
+ <plugin>
+ <groupId>org.jboss.maven.plugins</groupId>
+ <artifactId>maven-jdocbook-plugin</artifactId>
+ <version>2.2.0</version>
+ <extensions>true</extensions>
+ <dependencies>
+ <dependency>
+ <groupId>org.hibernate</groupId>
+ <artifactId>hibernate-jdocbook-style</artifactId>
+ <version>2.0.0</version>
+ <type>jdocbook-style</type>
+ </dependency>
+ </dependencies>
+ <configuration>
+ <sourceDocumentName>master.xml</sourceDocumentName>
+ <sourceDirectory>${basedir}/src/main/docbook</sourceDirectory>
+ <masterTranslation>en-US</masterTranslation>
+ <imageResource>
+ <directory>${basedir}/src/main/docbook/en-US/images</directory>
+ <directory>${basedir}/src/main/docbook/en-US/harness</directory>
+ </imageResource>
+ <formats>
+ <format>
+ <formatName>pdf</formatName>
+ <stylesheetResource>classpath:/xslt/org/hibernate/jdocbook/xslt/pdf.xsl</stylesheetResource>
+ <finalName>beanvalidation-tck.pdf</finalName>
+ </format>
+ <format>
+ <formatName>html_single</formatName>
+ <stylesheetResource>classpath:/xslt/org/hibernate/jdocbook/xslt/xhtml-single.xsl
+ </stylesheetResource>
+ <finalName>index.html</finalName>
+ </format>
+ <format>
+ <formatName>html</formatName>
+ <stylesheetResource>classpath:/xslt/org/hibernate/jdocbook/xslt/xhtml.xsl
+ </stylesheetResource>
+ <finalName>index.html</finalName>
+ </format>
+ </formats>
+ <options>
+ <xincludeSupported>true</xincludeSupported>
+ <xmlTransformerType>saxon</xmlTransformerType>
+ <!-- needed for uri-resolvers; can be ommitted if using 'current' uri scheme -->
+ <!-- could also locate the docbook dependency and inspect its version... -->
+ <docbookVersion>1.72.0</docbookVersion>
+ <localeSeparator>-</localeSeparator>
+ </options>
+ </configuration>
+ <executions>
+ <execution>
+ <id>make-doc</id>
+ <phase>site</phase>
+ <goals>
+ <goal>resources</goal>
+ <goal>generate</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
<plugin>
+ <groupId>org.jboss.maven.plugins</groupId>
+ <artifactId>maven-jdocbook-style-plugin</artifactId>
+ <version>2.0.0</version>
+ </plugin>
+ <plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.2</version>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/Author_Group.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/Author_Group.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/Author_Group.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,32 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE authorgroup PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<authorgroup>
+ <author>
+ <firstname>Gavin</firstname>
+ <surname>King</surname>
+ <affiliation>
+ <jobtitle>JSR-299: Contexts and Dependency Injection (CDI) for Java EE
+ specification lead</jobtitle>
+ <orgname>Red Hat Inc.</orgname>
+ </affiliation>
+ </author>
+ <author>
+ <firstname>Pete</firstname>
+ <surname>Muir</surname>
+ <affiliation>
+ <jobtitle>CDI TCK lead</jobtitle>
+ <orgname>Red Hat Inc.</orgname>
+ </affiliation>
+ </author>
+ <author>
+ <firstname>Dan</firstname>
+ <surname>Allen</surname>
+ <affiliation>
+ <jobtitle>CDI TCK developer</jobtitle>
+ <orgname>Red Hat Inc.</orgname>
+ </affiliation>
+ </author>
+<!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</authorgroup>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Info.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Info.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Info.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,29 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE bookinfo PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<bookinfo>
+ <releaseinfo><literallayout>Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.</literallayout></releaseinfo>
+
+ <title>Technology Compatibility Kit Reference Guide for JSR-303: Bean
+ Validation</title>
+
+ <subtitle>Specification Lead: Red Hat Inc.</subtitle>
+
+ <copyright>
+ <year>2009 </year>
+
+ <holder>Red Hat Middleware, LCC.</holder>
+ </copyright>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</bookinfo>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Preface.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Preface.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/Book_Preface.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,122 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE preface PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<preface id="book-preface">
+ <title>Preface</title>
+
+ <para>This guide describes how to download, install, configure, and run the
+ Technology Compatibility Kit (TCK) used to verify the compatibility of an
+ implementation of the JSR-303: Bean Validation specification.</para>
+
+ <para>The Bean Validation TCK is built atop the JBoss Test Harness, a
+ portable and configurable automated test suite for authoring unit and
+ integration tests in a Java EE environment. The TCK 1.0.0 uses the JBoss
+ Test Harness version 1.0.0 to execute the test suite.</para>
+
+ <para>The Bean Validation TCK is provided under the <ulink
+ url="http://www.apache.org/licenses/LICENSE-2.0">Apache Public License
+ 2.0</ulink>.</para>
+
+ <section id="target-audience">
+ <title>Who Should Use This Book</title>
+
+ <para>This guide is for implementors of the Bean Validation specification
+ to assist in running the test suite that verifies the compatibility of
+ their implementation.</para>
+ </section>
+
+ <section id="before-reading">
+ <title>Before You Read This Book</title>
+
+ <para>The Bean Validation TCK is based on the Bean Validation
+ specification 1.0 (JSR-303). Information about the specification,
+ including links to the specification documents, can be found on the <ulink
+ url="http://jcp.org/en/jsr/detail?id=299">JSR-303 JCP page</ulink>.</para>
+
+ <para>Before running the tests in the Bean Validation TCK, read and become
+ familiar with the JBoss Test Harness Reference Guide (pending), which
+ describes how the test harness functions.</para>
+ </section>
+
+ <section id="book-organization">
+ <title>How This Book Is Organized</title>
+
+ <para>If you are running the Bean Validation TCK for the first time, read
+ <xref linkend="introduction" /> and <xref
+ linkend="test-harness-introduction" /> completely for the necessary
+ background information about the TCK and the JBoss Test Harness,
+ respectively. Once you have reviewed that material, perform the steps
+ outlined in the remaining chapters.</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><xref linkend="introduction" /> gives an overview of the
+ principles that apply generally to all Technology Compatibility Kits
+ (TCKs), outlines the appeals process and describes the Bean Validation
+ TCK architecture and components. It also includes a broad overview of
+ how the TCK is executed and lists the platforms on which the TCK has
+ been tested and verified.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="appeals-process" /> explains the process to be
+ followed by an implementor should they wish to challenge any test in
+ the TCK.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="installation" /> explains where to obtain the
+ required software for the Bean Validation TCK and how to install it.
+ It covers both the primary TCK components as well as tools useful for
+ troubleshooting tests.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="configuration" /> details the configuration of
+ the JBoss Test Harness, how to create a TCK runner for the TCK test
+ suite and the mechanics of how an in-container test is
+ conducted.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="reporting" /> explains the test reports that are
+ generated by the TCK test suite and introduces the TCK audit report as
+ a tool for measuring the completeness of the TCK in testing the
+ JSR-303 specification and in understanding how testcases relate to the
+ specification.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="executing" /> documents how the TCK test suite is
+ executed. It covers both modes supported by the TCK, standalone and
+ in-container, and shows how to dump the generated test artifacts to
+ disk.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="eclipse-running" /> shows how to run individual
+ tests in Eclipse and advises the best way to setup your Eclipse
+ workspace for running the tests.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="eclipse-debugging" /> builds on <xref
+ linkend="eclipse-running" /> by detailing how to debug individual
+ tests in Eclipse.</para>
+ </listitem>
+
+ <listitem>
+ <para><xref linkend="test-harness" /> includes excerpts from the JBoss
+ Test Harness Reference Guide. How to configure the JBoss Test Harness
+ as it relates to the Bean Validation TCK is presented in <xref
+ linkend="configuration" />. However, to aid in debugging or
+ configuring the TCK in your environment, you may want to read in more
+ detail how to use the JBoss Test Harness.</para>
+ </listitem>
+ </itemizedlist>
+ </section>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</preface>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/appeals-process.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/appeals-process.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/appeals-process.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,102 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="appeals-process">
+ <title>Appeals Process</title>
+
+ <para>While the Bean Validation TCK is rigourous about enforcing an
+ implementation's conformance to the JSR-303 specification, it's reasonable
+ to assume that an implementor may discover new and/or better ways to
+ validate the assertions. This chapter covers the appeals process, defined by
+ the Specification Lead, Red Hat Middleware LLC., which allows implementors
+ of the JSR-303 specification to challenge one or more tests defined by the
+ Bean Validation TCK.</para>
+
+ <para>The appeals process identifies who can make challenges to the TCK,
+ what challenges to the TCK may be submitted, how these challenges are
+ submitted, how and by whom challenges are addressed and how accepted
+ challenges to the TCK are managed.</para>
+
+ <para>Following the recent adoption of transparency in the JCP, implementors
+ are encouraged to make their appeals public, which this process facilitates.
+ The JCP community should recognize that issue reports are a central aspect
+ of any good software and it's only natural to point out shortcomings and
+ strive to make improvements. Despite this good faith, not all implementors
+ will be comfortable with a public appeals process. Instructions about how to
+ make a private appeal are therefore provided.</para>
+
+ <section>
+ <title>Who can make challenges to the TCK?</title>
+
+ <para>Any implementor may submit an appeal to challenge one or more tests
+ in the Bean Validation TCK. In fact, members of the JSR-303 Expert Group
+ (EG) encourage this level of participation.</para>
+ </section>
+
+ <section>
+ <title>What challenges to the TCK may be submitted?</title>
+
+ <para>Any test case (e.g., <literal>@Artifact</literal> class,
+ <literal>@Test</literal> method), test case configuration (e.g.,
+ validation.xml), test entities, annotations and other resources may be
+ challenged by an appeal.</para>
+
+ <para>What is generally not challengable are the assertions made by the
+ specification. The specification document is controlled by a separate
+ process and challenges to it should be handled through the JSR-303 EG by
+ sending an e-mail to <ulink
+ url="mailto:jsr299-comments at jcp.org">jsr303-comments at jcp.org</ulink>.</para>
+ </section>
+
+ <section>
+ <title>How these challenges are submitted?</title>
+
+ <para>To submit a challenge, a new issue should be created in the <ulink
+ url="https://jira.jboss.org/jira/browse/WBTCK">HV project</ulink> of the
+ JBoss JIRA using the Issue Type: Bug. The appellant should complete the
+ Summary, Component (TCK Appeal), Environment and Description Field only.
+ Any communication regarding the issue should be pursed in the comments of
+ the filed issue for accurate record.</para>
+
+ <para>To submit an issue in the JBoss JIRA, you must have a (free)
+ JBoss.com member account. You can create a member account using the <ulink
+ url="http://www.jboss.org/index.html?op=checkage&module=user">on-line
+ registration</ulink>.</para>
+
+ <para>If you wish to make a private challenge, you should follow the above
+ procedure, setting the Security Level to Private. Only the issue reporter,
+ TCK Project Lead and designates will be able to view the issue.</para>
+ </section>
+
+ <section>
+ <title>How and by whom challenges are addressed?</title>
+
+ <para>The challenges will be addressed in a timely fashion by the Bean
+ Validation TCK Project Lead, as designated by Specification Lead, Red Hat
+ Middleware LLC. or his/her designate. The appellant can also monitor the
+ process by following the issue report filed in the <ulink
+ url="https://jira.jboss.org/jira/browse/WBTCK">HV project</ulink> of the
+ JBoss JIRA.</para>
+
+ <para>The current TCK Project Lead is listed on the <ulink type=""
+ url="https://jira.jboss.org/jira/browse/WBTCK" userlevel="">HV Project
+ Summary Page</ulink> on the JBoss JIRA.</para>
+ </section>
+
+ <section>
+ <title>How accepted challenges to the TCK are managed?</title>
+
+ <para>Accepted challenges will be acknowledged via the filed issue's
+ comment section. Communication between the Bean Validation TCK Project
+ Lead and the appellant will take place via the issue comments. The issue's
+ status will be set to "Resolved" when the TCK project lead believes the
+ issue to be resolved. The appellant should, within 30 days, either close
+ the issue if they agree, or reopen the issue if they do not believe the
+ issue to be resolved.</para>
+
+ <para>Resolved issue not addressed for 30 days will be closed by the TCK
+ Project Lead. If the TCK Project Lead and appellant are unable to agree on
+ the issue resolution, it will be referred to the JSR-303 specification
+ lead or his/her designate.</para>
+ </section>
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/configuration.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/configuration.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/configuration.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,157 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="configuration">
+ <title>Configuration</title>
+
+ <para>This chapter lays out how to configure the TCK Harness by specifying
+ the SPI implementation classes, defining the target container connection
+ information, and various other switches. You then learn how to setup a TCK
+ runner project that executes the the TCK test suite, putting these settings
+ into practice. Finally, a detailed account of how the JBoss Test Harness
+ negotiates the execution of the tests in the container is given.</para>
+
+ <para>This chapter does not discuss in detail how to use the TCK in
+ standalone mode. The JBoss Test Harness guide provides more on running in
+ standalone mode.</para>
+
+ <section id="tck-harness-properties">
+ <title>TCK Harness Properties</title>
+
+ <para>The JBoss Test Harness allows the test suite to be launched in a
+ pluggable fashion. In order to execute the TCK, the JBoss Test Harness
+ must be configured by specifying implementations of the test launcher and
+ container APIs.</para>
+
+ <para>System properties and/or the resource
+ <code>META-INF/jboss-test-harness.properties</code>, a Java properties
+ file, are used to configure the JBoss Test Harness. You can read more
+ about configuring the JBoss Test Harness in <xref
+ linkend="test-harness-properties" />.</para>
+
+ <para>You should set the following properties:</para>
+
+ <table frame="all">
+ <title>Required JBoss Test Harness Configuration Properties</title>
+
+ <tgroup cols="2">
+ <colspec colname="property" colnum="1" colwidth="5*" />
+
+ <colspec colname="description" colnum="2" colwidth="2*" />
+
+ <thead>
+ <row>
+ <entry>Property = Required/Example Value</entry>
+
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><literal>org.jboss.testharness.libraryDirectory=/path/to/extra/libraries
+ </literal></entry>
+
+ <entry>Directory containing extra JARs you want placed in artifact
+ library directory such as the porting package
+ implementation.</entry>
+ </row>
+
+ <row>
+ <entry><literal>org.jboss.testharness.standalone=false</literal></entry>
+
+ <entry>You must run the tests in-container to pass the TCK</entry>
+ </row>
+
+ <row>
+ <entry><literal>org.jboss.testharness.runIntegrationTests=true
+ </literal></entry>
+
+ <entry>You must run the integration tests to pass the TCK</entry>
+ </row>
+
+ <row>
+ <entry><literal>org.jboss.testharness.spi.Containers=com.acme.AcmeContainer
+ </literal></entry>
+
+ <entry>The container implementation for deploying and executing
+ in-container tests. See <xref
+ linkend="deployment-api-contributions" /></entry>
+ </row>
+
+ <row>
+ <entry><literal>org.jboss.testharness.api.TestLauncher=org.jboss.testharness.impl.runner.servlet.ServletTestLauncher
+ </literal></entry>
+
+ <entry>You should use the <literal>ServletTestLauncher</literal>
+ for Java EE 6 and Java EE 6 Web Profile.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>To run the full TCK you must additionally implement
+ <literal>org.jboss.testharness.spi.Containers</literal>, which handles
+ deploying the test artifact to the container. An implementations of this
+ API is already available for JBoss AS 5.1. Therefore, you only need to
+ implement this part of the porting package if you wish to use another
+ container.</para>
+
+ <note id="deployment-api-contributions">
+ <para>Red Hat Middleware LLC encourages CDI implementators to contribute
+ JBoss Test Harness Deployment API implementations for other containers
+ under the ASL license. Please contact the Bean Validation TCK
+ lead.</para>
+ </note>
+ </section>
+
+ <section>
+ <title>Configuring TestNG to execute the TCK</title>
+
+ <para>The JBoss Test Harness is built atop TestNG, and it's TestNG that is
+ responsible for selecting the tests to execute, the order of execution,
+ and reporting the results. Detailed TestNG documentation can be found at
+ <ulink
+ url="http://testng.org/doc/documentation-main.html">testng.org</ulink>.</para>
+
+ <para>The <literal>tck-tests.xml</literal> artifact provided in the TCK
+ distribution must be run by TestNG 5.9 (described by the TestNG
+ documenation as "with a <literal>testng.xml</literal> file") unmodified
+ for an implementation to pass the TCK. This file also allows tests to be
+ excluded from a run:</para>
+
+ <programlisting><suite name="JSR-299 TCK" verbose="2">
+ <test name="JSR-299 TCK">
+ ...
+ <classes>
+ <class name="org.jboss.jsr299.tck.tests.context.application.ApplicationContextTest">
+ <methods>
+ <exclude name="testApplicationScopeActiveDuringServiceMethod"/>
+ </methods>
+ </class>
+ </classes>
+ ...
+ </test>
+</suite></programlisting>
+
+ <para>TestNG provides extensive reporting information. Depending on the
+ build tool or IDE you use, the reporting will take a different format.
+ Please consult the TestNG documentation and the tool documentation for
+ more information.</para>
+ </section>
+
+ <section>
+ <title>Configuring your build environment to execute the TCK</title>
+
+ <para>It's beyond the scope of this guide to describe in how to set up
+ your build environment to run the TCK. The JBoss Test Harness guide
+ describes how Web Beans uses Maven 2 to execute the Bean Validation TCK.
+ See <xref linkend="test-suite-runner" />. The TestNG documentation
+ provides extensive information on launching TestNG using the Java, Ant,
+ Eclipse or IntellJ IDEA.</para>
+ </section>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-debugging.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-debugging.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-debugging.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,146 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<chapter id="eclipse-debugging">
+ <title>Debugging Tests in Eclipse</title>
+ <para>
+ This chapter explains how to debug standalone and integration tests from
+ the TCK test suite in Eclipse. You should be able to use the lessons
+ learned here to debug tests in an alternate IDE as well.
+ </para>
+ <section>
+ <title>Debugging a standalone test</title>
+ <para>
+ There is almost no difference in how you debug a standalone test
+ from how you run it. With the test class open in the Eclipse editor,
+ simply right click in the editor view and select Debug As > TestNG
+ Test. Eclipse will stop at any breakpoints you set just like it would
+ with any other local debug process.
+ </para>
+ <para>
+ If you plan to step into a class in a Web Beans implementation (or any
+ other dependent library), you must ensure that the source is properly
+ associated with the library. Below are the steps to follow to associate
+ the source of Web Beans with the TestNG debug configuration:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>
+ Select the Run > Debug Configurations... menu from the main
+ menubar
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the name of the test class in the TestNG category
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the Source tab
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the Add... button on the right
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select Java Project
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Check the project the contains the class you want to debug
+ (e.g., webbeans-core)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click OK on the Project Selection window
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click Close on the Debug Configurations window
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>
+ You'll have to complete those steps for any test class you are
+ debugging, though you only have to do it once (the debug configuration
+ hangs around indefinitely).
+ </para>
+ <para>
+ Again, running a test in standalone isn't enough to pass the TCK and
+ cannot be used to run or debug an integration test. Let's look at how
+ to debug a test running in the context of the container.
+ </para>
+ </section>
+ <section>
+ <title>Debugging an integration test</title>
+ <para>
+ In order to debug an integration test, or any test run using in-container
+ mode, the test must be configured to run in-container, as described in
+ <xref linkend="eclipse-in-container" />,
+ and you must attach the IDE debugger to the container. That puts the
+ debugger on both sides of the fence, so to speak.
+ </para>
+ <para>
+ Since setting up a test to run in-container has already been
+ covered, we'll look at how to attach the IDE debugger to the container,
+ and then move on launching the test in debug mode.
+ </para>
+ <section>
+ <title>Attaching the IDE debugger to the container</title>
+ <para>
+ There are two ways to attach the IDE debugger to the container.
+ You can either start the container in debug mode from within the
+ IDE, or you can attach the debugger over a socket connection to a
+ standalone container running with JPDA enabled.
+ </para>
+ <para>
+ The Eclipse Server Tools, a subproject of the Eclipse Web Tools
+ Project (WTP), has support for launching most major application
+ servers, including JBoss AS 5. However, if you are using JBoss AS,
+ you should consider using JBoss Tools instead, which offers tighter
+ integration with JBoss technologies. See either the
+ <ulink url="http://www.eclipse.org/webtools/server/server.php">Server Tools documentation</ulink>
+ or the
+ <ulink url="http://docs.jboss.org/tools/3.0.1.GA/en/as/html/index.html">JBoss Tools documentation</ulink>
+ for how to setup a container and start it in debug mode.
+ </para>
+ <para>
+ See
+ <ulink
+ url="http://maverikpro.wordpress.com/2007/11/26/remote-debug-a-web-application-using-eclipse">this blog entry</ulink>
+ to learn how to start JBoss AS with JPDA enabled and how to get the
+ Eclipse debugger to connect to the remote process.
+ </para>
+ </section>
+ <section>
+ <title>Launching the test in the debugger</title>
+ <para>
+ Once Eclipse is debugging the container, you can set a breakpoint in
+ the test and debug it just like a standalone test. Let's give it a
+ try.
+ </para>
+ <para>
+ Open a test annotated with <literal>@IntegrationTest</literal> in
+ the Eclipse editor, right click in the editor view, and select Debug
+ As > TestNG Test. This time when the IDE hits the breakpoint, it
+ halts the JVM thread of the container rather than the thread that
+ launched the test.
+ </para>
+ <para>
+ Remember that if you need to debug into dependent libraries, the
+ source code for those libraries will need to be registered with the
+ TestNG debug configuration as described in the first section in this
+ chapter.
+ </para>
+ </section>
+ </section>
+<!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-running.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-running.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/eclipse-running.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,447 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<chapter id="eclipse-running">
+ <title>Running Tests in Eclipse</title>
+ <para>
+ This chapter explains how to run individual tests using the Eclipse
+ TestNG plugin. It covers running non-integration tests in standalone mode
+ and integration tests (as well as non-integration tests) in in-container
+ mode. You should be able to use the lessons learned here to debug tests in
+ an alternate IDE as well.
+ </para>
+ <section>
+ <title>Leveraging Eclipse's plugin ecosystem</title>
+ <para>
+ Using an existing test harness (TestNG) allows the tests to be executed
+ and debugged in an Integrated Development Environment (IDE) using
+ available plugins. Using an IDE is also the easiest way to execute a
+ test class in isolation.
+
+ </para>
+ <para>
+ The TCK can be executed in any IDE for which there is a TestNG plugin
+ available. Running a test from the CDI TCK test suite using the Eclipse
+ TestNG plugin is almost as simple as running any other TestNG test. You
+ can also use the plugin to debug a test, which is described in the next
+ chapter.
+ </para>
+ <para>
+ Before running a test from the TCK test suite in Eclipse, you must have
+ the Eclipse <ulink url="http://testng.org">TestNG plugin</ulink> and
+ either the m2eclipse plugin or an Eclipse project generated use the
+ Maven 2 Eclipse plugin (<literal>maven-eclipse-plugin</literal>). Refer
+ to <xref linkend="eclipse-plugins" /> for more information on these
+ plugins.
+ </para>
+
+ <para>
+ With the m2eclipse plugin installed, Eclipse should recognize the
+ CDI TCK projects as valid Eclipse projects (or any Web Beans project
+ for that matter). Import them into the Eclipse workspace at this time.
+ You should also import the Web Beans projects if you want to debug into
+ that code, which is covered later.
+ </para>
+ <tip>
+ <para>
+ If you choose to use the Maven 2 Eclipse plugin
+ (<literal>maven-eclipse-plugin</literal>), you should execute the
+ plugin in both the tck and webbeans projects:
+ </para>
+ <programlisting><![CDATA[cd tck
+mvn clean eclipse:clean eclipse:eclipse -DdownloadSources -DdownloadJavadocs
+cd ../webbeans
+mvn clean eclipse:clean eclipse:eclipse -DdownloadSources -DdownloadJavadocs]]></programlisting>
+ </tip>
+ </section>
+ <section>
+ <title>Readying the Eclipse workspace</title>
+ <para>
+ When setting up your Ecilpse workspace, we recommended creating three
+ workings sets:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>
+ <emphasis role="bold">Web Beans</emphasis> - Groups the CDI API
+ and the CDI RI (i.e., Web Beans) projects
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <emphasis role="bold">CDI TCK</emphasis> - Groups the CDI TCK
+ API and the test suite projects
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <emphasis role="bold">Web Beans JBoss TCK Runner</emphasis> -
+ Groups the porting package implementation and TCK runner projects
+ that are configured to certify Web Beans deployed on JBoss AS 5.1
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>
+ The dependencies between the projects will either be established
+ automatically by the m2eclipse plugin, based on the dependency
+ information in the pom.xml files, or as generated by the <literal>mvn
+ eclipse:eclipse</literal> command.
+ </para>
+ <para>
+ Your workspace should appear as follows:
+ </para>
+ <programlisting><![CDATA[Web Beans
+ jsr299-api
+ webbeans-api
+ webbeans-core
+ webbeans-core-test
+ webbeans-logging
+ webbeans-parent
+ webbeans-spi
+ webbeans-version-matrix
+CDI TCK
+ jsr299-tck-api
+ jsr299-tck-impl
+ jsr299-tck-parent
+Web Beans JBoss TCK Runner
+ webbeans-jboss-tck-runner
+ webbeans-porting-package]]></programlisting>
+ <para>
+ The tests in the TCK test suite are located in the jsr299-tck-impl
+ project. You'll be working within this project in Eclipse when you are
+ developing tests. However, as you learned earlier, there are no references
+ to a CDI implementation in the TCK. So how can you execute an
+ individual test in Eclipse? The secret is that you need to establish a
+ link in Eclipse (not in Maven) between the jsr299-tck-impl project and
+ your TCK runner project, which in this case is
+ webbeans-jboss-tck-runner (the project in the jboss-tck-runner
+ directory).
+ </para>
+ <para>
+ Here are the steps to establish the link:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>
+ Right click on the jsr299-tck-impl project
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select Build Path > Configure Build Path...
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click on the Projects tab
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the Add... button on the right
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Check the TCK runner project (e.g., webbeans-jboss-tck-runner)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the OK button on the Required Project Selection dialog
+ window
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the OK button on the Java Build Path window
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>
+ Of course, the webbeans-jboss-tck-runner also depends on the
+ jsr299-tck-impl at runtime (so it can actually find the tests to
+ execute). But Eclipse doesn't distinguish between build-time and
+ runtime dependencies. As a result, we've created a circular dependency
+ between the projects. In all likelihood, Eclipse will struggle (if not
+ fail) to compile one or more projects. How can we break this cycle?
+ </para>
+ <para>
+ As it turns out, the TCK runner doesn't need to access the tests to
+ build. It only needs its classes, configurations and other dependenices
+ at runtime (when the TestNG plugin executes). Therefore, we can remove
+ the TCK runner's dependence on the jsr299-tck-impl project by following
+ these steps:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>
+ Right click on the webbeans-jboss-tck-runner project
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select Build Path > Configure Build Path...
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click on the Projects tab
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the TCK tests project (jsr299-tck-impl)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the Remove button on the right
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the OK button on the Java Build Path window
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>
+ You are now ready to execute an individual test class (or artifact).
+ Let's start with a test artifact capable of running in standalone mode.
+ </para>
+ </section>
+ <section id="eclipse-standalone">
+ <title>Running a test in standalone mode</title>
+ <para>
+ A standalone test artifact is a class which extends
+ <literal>AbstractJSR299Test</literal>, is annotated with
+ <literal>@Artifact</literal>, but is <emphasis
+ role="italic">not</emphasis> annotated with
+ <literal>@IntegrationTest</literal>. Select a standalone test artifact
+ and open it in the Eclipse editor. Now right click in the editor view
+ and select Run As > TestNG Test. The TestNG view should pop out and
+ you should see all the tests in that artifact pass (if all goes well).
+ </para>
+ <note>
+ <para>
+ If the TCK complains that there is a property missing, close all
+ the projects, open them again, and rebuild. The m2eclipse plugin can
+ be finicky getting everything built correctly the first time.
+ </para>
+ </note>
+ <para>
+ So far you have executed a test in standalone mode. That's not
+ sufficient to pass the TCK. The test must be executed using
+ in-container mode. Using in-container mode is also the only way to
+ execute a test annotated with <literal>@IntegrationTest</literal>
+ (that's what the annotation means) as it requires container resources.
+ </para>
+ <para>
+ Let's see what has to be done to execute an integration test. This
+ will result in the artifact being deployed to the container, which is
+ JBoss AS 5.1 if you are using the JBoss TCK runner.
+ </para>
+ </section>
+ <section id="eclipse-in-container">
+ <title>Running integration tests</title>
+ <para>
+ As you have learned, the JBoss test harness determines how to behave
+ based on the values of numerous system properties or properties defined
+ in META-INF/jboss-test-harness.properties classpath resources. If the
+ property named
+ <literal>org.jboss.testharness.standalone</literal>
+ is not defined, the harness assumes that the test is to be run in
+ standalone mode. In order to run the tests in the container, you need
+ to add a properties file to the classpath that sets the standalone
+ property to false and provides values for other properties required to
+ run an in-container test.
+ </para>
+ <para>
+ The JBoss TCK runner project conveniently provides the properties file
+ src/test/debug-resources/META-INF/jboss-test-harness.properties that
+ contains all of the necessary properties for in-container testing in
+ Eclipse. Assuming you followed the directory structure recommended in
+ <xref linkend="tck-environment"/>, you are good to go. Otherwise, you
+ may have to tune the
+ <literal>org.jboss.testharness.container.extraConfigurationDir</literal> and
+ <literal>org.jboss.testharness.libraryDirectory</literal> properties to
+ point to the relative location of the related projects. The properties
+ should be defined as follows:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>org.jboss.testharness.container.extraConfigurationDir
+ </literal>
+ - the relative path from the jboss-tck-impl project to a
+ directory that contains a build.properties or
+ local.build.properties file defining the location of a JBoss AS
+ 5.1 installation in the
+ <literal>jboss.home</literal>
+ property
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>org.jboss.testharness.libraryDirectory</literal>
+ - the relative path from the jboss-tck-impl project to the
+ target/dependency/lib directory in the TCK runner project.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ The other properties in that file are defined as follows:
+ </para>
+ <programlisting>org.jboss.testharness.standalone=false
+orjboss.testharness.container.forceRestart=false
+orjboss.testharness.runIntegrationTests=true</programlisting>
+ <para>
+ You're now ready to execute an integration test. Select an integration
+ test (a class that extends <literal>AbstractJSR299Test</literal> and is
+ annotated with both <literal>@Artifact</literal> and
+ <literal>@IntegrationTest</literal>) and open it in your Eclipse
+ editor. Follow these steps to execute the class with the TestNG plugin:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>
+ Right click in the editor view and select Run As > TestNG Test
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Observe the test fail because of missing dependencies
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the Run > Run Configurations... menu from the main
+ menubar
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the name of the test class under the TestNG category
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the Classpath tab
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select User Entries in the tree
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the Advanced... button on the right
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select Add Folders and click the OK button
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select the webbeans-jboss-tck-runner/src/test/debug-resources
+ folder
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the OK button on the Folder Selection dialog window
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click on the webbeans-jboss-tck-runner entry
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Move the webbeans-jboss-tck-runner to the first entry using
+ the Up button
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the Run button on the Run Configurations dialog window
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>
+ When you run the test this time, it should pass. If you get a
+ failure that the container (e.g., JBoss AS 5.1) must be run with
+ assertions enabled, you need to stop the container and start it with
+ the -ea flag (or just leave it stopped and the test will start it
+ appropriately).
+ </para>
+ <para>
+ You can simply right click and select Run As > TestNG Test for
+ all subsequent runs for the reason cited earlier, the run configuration
+ for a class is retained indefinitely.
+ </para>
+ <para>
+ Alternatively, you can configure TestNG to execute all tests
+ in-container by default by adding the properties file in the
+ debug-resources folder to the project's classpath as follows:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>
+ Right click on the jsr299-tck-impl project
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Select Build Path > Configure Build Path...
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click on the Libraries tab
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the Add Class Folder... button on the right
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Check the webbeans-jboss-tck-runner/src/test/debug-resources
+ folder
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the OK button on the Class Folder Selection dialog
+ window
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Click the OK button on the Java Build Path window
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>
+ Now you don't have to do any special configuration per test class.
+ </para>
+ <para>
+ You can stop the individual tests from running in-container by
+ reversing the steps above to remove the debug-resources folder from the
+ Eclipse classpath.
+ </para>
+ <para>
+ You have now mastered running the CDI TCK against Web Beans using
+ both Maven 2 and within Eclipse. Now you're likely interested in how to
+ debug a test so that you can efficiently investigate test failures.
+ </para>
+ </section>
+<!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/executing.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/executing.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/executing.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,124 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="executing">
+ <title>Executing the Test Suite</title>
+
+ <para>This chapter explains how to run the TCK on Hibernate Validator as
+ well as your own implementation. The Bean Validation TCK uses the Maven 2
+ TestNG plugin and the JBoss Test Harness to execute the test suite. Learning
+ to execute the test suite from Maven 2 is prerequisite knowlege for running
+ the tests in an IDE, such as Eclipse.</para>
+
+ <section>
+ <title>The Test Suite Runner</title>
+
+ <para>The test suite is executed by the Maven 2 TestNG plugin during the
+ test phase of the Maven 2 life cycle. The execution happens within a TCK
+ runner project (as opposed to the TCK project itself). Hibernate Validator
+ includes a TCK runner project that executes the Bean Validation TCK on
+ Hibernate Validator running inside JBoss AS 5.1. To execute the Bean
+ Validation TCK on your own Bean Validation implementation, you could
+ modify the TCK runner project included with Hibernate Validator to use
+ your Bean Validation implementation as described in <xref
+ linkend="configuration" />.</para>
+ </section>
+
+ <section>
+ <title>Running the Tests In Standalone Mode</title>
+
+ <para>To execute the TCK test suite against Web Beans, first switch to the
+ jboss-tck-runner directory in the extracted Web Beans distribution:</para>
+
+ <programlisting>cd ri/hibernate-validator-tck-runner</programlisting>
+
+ <note>
+ <para>These instructions assume you have extracted the CDI-related
+ software according to the recommendation given in <xref
+ linkend="tck-environment" />.</para>
+ </note>
+
+ <para>Then execute the Maven 2 life cycle through the test phase:</para>
+
+ <programlisting>mvn test</programlisting>
+
+ <para>Without any command-line flags, the test suite is run in standalone
+ mode, which means that any test class with the
+ <literal>@org.jboss.testharness.impl.packaging.IntegrationTest </literal>
+ annotation is skipped. This mode uses the
+ <literal>StandaloneContainers</literal> SPI to invoke the test artifact
+ within a mock Java EE life cycle and capture the results of the test.
+ However, passing the suite in this mode is not sufficient to pass the TCK
+ as a whole. The suite must be passed while executing using the
+ in-container mode.</para>
+ </section>
+
+ <section>
+ <title>Running the Tests In the Container</title>
+
+ <para>To execute the test suite using in-container mode with the JBoss TCK
+ runner, you first have to setup JBoss AS as described in the <xref
+ linkend="tck-in-jboss-as" /> callout.</para>
+
+ <para>Then, execute the TCK runner with Maven 2 as follows:</para>
+
+ <programlisting>mvn test -Dincontainer</programlisting>
+
+ <para>The presence of the <literal>incontainer</literal> property
+ activates a Maven 2 profile that assigns the
+ <literal>org.jboss.testharness.standalone</literal> system property to
+ <literal>false</literal> and the
+ <literal>org.jboss.testharness.runIntegrationTests</literal> system
+ property to <literal>true</literal>, hence activating the in-container
+ test mode. This time, all the test artifacts in the test suite are
+ executed.</para>
+
+ <para>The in-container profile will also start and stop the application
+ server automatically, a feature which the profile activates by setting the
+ <literal>org.jboss.testharness.container.forceRestart</literal> to
+ <literal>true</literal>.</para>
+
+ <para>The in-container mode uses the <literal>Containers</literal> SPI to
+ deploy the test artifact to the container and execute the test in a true
+ Java EE life cycle. The JBoss TCK runner has a dependency on the library
+ that provides an implementation of this interface for JBoss AS 5.1.</para>
+
+ <para>Since in-container tests are executed in a remote JVM, the results
+ of the test must be communicated back to the runner over a
+ container-supported protocol. The JBoss Test Harness provides
+ servlet-based communication over HTTP as described in <xref
+ linkend="incontainer-communication" />.</para>
+ </section>
+
+ <section>
+ <title>Dumping the Test Artifacts</title>
+
+ <para>As you have learned, when the test suite is executing using
+ in-container mode, each test class is packaged as a deployable artifact
+ and deployed to the container. The test is then executed within the
+ context of the deployed application. This leaves room for errors in
+ packaging. When investigating a test failure, you may find it helpful to
+ inspect the artifact after it's generated. The TCK can accommodate this
+ type of inspection by "dumping" the generated artifact to disk.</para>
+
+ <para>The feature just described is activated in the jboss-tck-runner
+ project by appending the <literal>dumpArtifacts</literal> command line
+ property to the end of the command that invokes the Maven 2 test
+ phase.</para>
+
+ <programlisting>mvn test-compile -DdumpArtifacts</programlisting>
+
+ <para>The directory where the artifacts get written is configured using
+ the <literal>org.jboss.testharness.outputDirectory</literal> property. The
+ <literal>dumpArtifacts</literal> profile in the jboss-tck-runner project
+ sets this value to the relative directory path
+ <literal>target/jsr303-artifacts</literal>.</para>
+
+ <para>You can read more about this feature in <xref
+ linkend="dumping-test-artifacts" />.</para>
+ </section>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Author_Group.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Author_Group.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Author_Group.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,22 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE authorgroup PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+
+
+<authorgroup>
+ <author>
+ <firstname>Pete</firstname>
+ <surname>Muir</surname>
+ <affiliation>
+ <jobtitle>JBoss Test Harness lead</jobtitle>
+ <orgname>Red Hat Middleware LLC</orgname>
+ </affiliation>
+ </author>
+ <author>
+ <firstname>Dan</firstname>
+ <surname>Allen</surname>
+ <affiliation>
+ <jobtitle>JBoss Test Harness developer</jobtitle>
+ <orgname>Red Hat Middleware LLC</orgname>
+ </affiliation>
+ </author>
+</authorgroup>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Info.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Info.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Info.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,15 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<!DOCTYPE bookinfo PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<bookinfo>
+ <copyright>
+ <year>2009</year>
+ <holder>
+ Red Hat Middleware LLC, and individual contributors listed as
+ authors.
+ </holder>
+ </copyright>
+
+ <title>JBoss Test Harness Refernce Guide</title>
+ <subtitle></subtitle>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Author_Group.xml" />
+</bookinfo>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Preface.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Preface.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/Book_Preface.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,15 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<!DOCTYPE preface PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<preface id="book-preface">
+ <title>Preface</title>
+
+ <section id="book-organization">
+
+ <title>How This Book Is Organized</title>
+
+ <para>
+ Chapter 1 provides an introduction and explains the motiviations for
+ and goals of the test harness.
+ </para>
+ </section>
+</preface>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/configuration.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/configuration.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/configuration.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,224 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<chapter id="test-harness-configuration">
+ <title>Configuration</title>
+ <para>
+ This chapter lays out how to configure the JBoss Test Harness by
+ specifying the API implementation classes, defining the target container
+ connection information, and various other switches. Finally, a detailed
+ account of how the JBoss Test Harness negotiates the execution of the
+ tests in the container is given.
+ </para>
+ <section id="test-harness-properties">
+ <title>JBoss Test Harness Properties</title>
+ <para>
+ The JBoss Test Harness allows the test suite to be launched in a
+ pluggable fashion. In order to execute a test suite, the JBoss Test
+ Harness must be configured by specifying implementations of the test
+ launcher and container APIs.
+ </para>
+ <para>
+ System properties and/or the resource
+ <code>META-INF/jboss-test-harness.properties</code>,
+ a Java properties file, are used to configure the JBoss Test Harness.
+ The bootstrap configuration builder looks to the property
+ <literal>org.jboss.testharness.api.ConfigurationBuilder</literal>,
+ the first property listed in table 3.1, for the fully qualified class
+ name (FQCN) of a concrete configuration builder implementation to get
+ started. This implementation loads the remaining configuration settings
+ and produces a JBoss Test Harness configuration.
+ </para>
+ <para>
+ For you convenience, the default configuration builder implementation
+ <literal>org.jboss.testharness.impl.PropertiesBasedConfigurationBuilder
+ </literal> is provided, which collates all the JBoss Test Harness
+ configuration settings from system and Java properties. It does so by
+ aggregating the system properties with the properties defined in the
+ META-INF/jboss-test-harness.properties resource in any classpath entry
+ under a single properties map, allowing you to partition the
+ configuration settings as needed.
+ </para>
+ <para>
+ A complete list of configuration properties for the JBoss Test
+ Harness
+ has been itemized in
+ <xref linkend="test-harness-properties-list" />,
+ accompanied by the default value (if any) and a description for each
+ property.
+ </para>
+ <table frame="all" id="test-harness-properties-list" title="Test Harness Configuration Properties">
+ <title>JBoss Test Harness Configuration Properties</title>
+ <tgroup cols="2">
+ <colspec colnum="1" colname="property" colwidth="5*" />
+ <colspec colnum="2" colname="description" colwidth="2*" />
+ <thead>
+ <row>
+ <entry>Property = Default Value</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.api.ConfigurationBuilder=
+ </literal>
+ <literal>org.jboss.testharness.impl.PropertiesBasedConfigurationBuilder
+ </literal>
+ </entry>
+ <entry>
+ The configuration bootstrap class for the JBoss Test
+ Harness.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.testPackage=</literal>
+ </entry>
+ <entry>
+ The top-level Java package containing classes to be
+ tested. Used to determine which artifacts to dump to disk
+ only; not used during running of a suite.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.libraryDirectory=</literal>
+ </entry>
+ <entry>
+ Directory containing extra JARs which should be deployed
+ in
+ the artifact (for example in
+ <literal>WEB-INF/lib</literal>.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.standalone=true</literal>
+ </entry>
+ <entry>
+ Tests are run using standalone mode if true or using
+ in-container mode if false.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.runIntegrationTests=false
+ </literal>
+ </entry>
+ <entry>
+ If true, integration tests are run. In-container mode
+ must be activated.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.spi.Containers=</literal>
+ </entry>
+ <entry>
+ The deployment implementation for setting up and
+ tearing down the container and deploying and undeploying
+ in-container tests.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.host=localhost:8080
+ </literal>
+ </entry>
+ <entry>
+ The host and port on which the container is running.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.connectDelay=5000</literal>
+ </entry>
+ <entry>
+ The timeout (ms) when attempting to connect to the
+ container (e.g. via http).
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.api.TestLauncher=</literal>
+ </entry>
+ <entry>
+ The in-container test launcher, the built in
+ <literal>org.jboss.testharness.impl.runner.servlet.ServletTestLauncher
+ </literal>
+ is provided and suitable for any Servlet environment.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.container.\</literal>
+ <literal>deploymentExceptionTransformer=</literal>
+ </entry>
+ <entry>
+ Provides an interception feature for deployment
+ exceptions,
+ allowing them to be inspected and altered before
+ reporting
+ to the test harness for validation by the test
+ case.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.container.forceRestart=false
+ </literal>
+ </entry>
+ <entry>
+ Whether the container should be restarted before the
+ tests are executed.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.container.extraConfigurationDir=
+ </literal>
+ </entry>
+ <entry>
+ A directory containing a
+ <literal>build.properties</literal>
+ or
+ <literal>local.build.properties</literal>
+ files that define additional properties. Can be used to
+ provide runtime specific properties.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.spi.StandaloneContainers=
+ </literal>
+ </entry>
+ <entry>
+ The container implementation for executing standalone
+ tests.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>dumpArtifacts=false</literal>
+ </entry>
+ <entry>
+ Whether the test artifacts should be written to disk
+ for inspection.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal>org.jboss.testharness.outputDirectory=</literal>
+ <literal>%java.io.tmpdir%/jsr-299-tck/</literal>
+ </entry>
+ <entry>
+ Directory where test artifacts will be written to disk, if
+ <literal>dumpArtifacts</literal>
+ is true.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </section>
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/executing.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,228 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<chapter id="executing-test-harness">
+ <title>Executing a Test Suite</title>
+ <para>
+ This chapter explains how to execute and debug a test suite built using
+ the JBoss Test Harness.
+ </para>
+ <section id="test-suite-runner">
+ <title>Building a test suite runner using Maven 2</title>
+ <para>
+ The test suite runner project is the magic that makes everything come
+ together and allows you to execute the test suite. If you fully
+ understand how the JBoss Test Harness functions, and have a good grasp
+ on Maven 2, then it's not to difficult to understand how the test suite
+ runner project works. Regardless of your background, this guide covers
+ what you need to know to get up and running by studying the test suite
+ runner used to run the CDI TCK against the CDI RI, Web Beans.
+ </para>
+ <para>
+ The TCK runner for the Web Beans can be found in the jboss-tck-runner
+ directory in the Web Beans distribution. The dependencies of the TCK
+ runner project for Web Beans are listed in
+ <xref linkend="tck-runner-dependencies" />.
+ </para>
+ <table frame="all" id="tck-runner-dependencies" title="CDI TCK Runner dependencies">
+ <title>Web Beans JBoss TCK Runner Dependencies</title>
+ <tgroup cols="3">
+ <colspec colname="c1" />
+ <colspec colname="c2" />
+ <colspec colname="c3" />
+ <thead>
+ <row>
+ <entry>Group ID</entry>
+ <entry>Artifact ID</entry>
+ <entry>Version</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>org.jboss.webbeans</entry>
+ <entry>jsr299-api</entry>
+ <entry>1.0.0-SNAPSHOT</entry>
+ </row>
+ <row>
+ <entry>org.jboss.jsr299.tck</entry>
+ <entry>jsr299-tck-api</entry>
+ <entry>1.0.0-SNAPSHOT</entry>
+ </row>
+ <row>
+ <entry>org.jboss.jsr299.tck</entry>
+ <entry>jsr299-tck-impl</entry>
+ <entry>1.0.0-SNAPSHOT</entry>
+ </row>
+ <row>
+ <entry>org.jboss.webbeans</entry>
+ <entry>webbeans-core</entry>
+ <entry>1.0.0-SNAPSHOT</entry>
+ </row>
+ <row>
+ <entry>org.jboss.webbeans</entry>
+ <entry>webbeans-porting-package</entry>
+ <entry>1.0.0-SNAPSHOT</entry>
+ </row>
+ <row>
+ <entry>org.testng</entry>
+ <entry>testng (classifier: jdk15)</entry>
+ <entry>5.8</entry>
+ </row>
+ <row>
+ <entry>org.jboss.test-harness</entry>
+ <entry>jboss-test-harness-jboss-as-51</entry>
+ <entry>1.0.0.BETA3</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ You can find all of these artifacts in the <ulink
+ url="http://repository.jboss.org/maven2">JBoss Maven
+ repository</ulink>.
+ </para>
+ <para>
+ You should substituate the webbeans-core and webbeans-porting-package
+ artifacts from table 2.2.3 with your own artifacts. You'll also need to
+ replace the jboss-test-harness-jboss-as-51 artifact if you are not
+ testing your implementation on JBoss AS 5.1. The
+ jboss-test-harness-jboss-as-51 artifact contains implementations of the
+ <literal>Containers</literal>
+ SPI for the JBoss Test Harness for JBoss AS 5.1.
+ </para>
+ <note>
+ <para>
+ When running the test suite in the in-container mode, the tests
+ will run against libraries installed into the container. In this
+ project, Web Beans is only declared as a Maven dependency for when
+ the TCK test suite is being executed in standalone mode.
+ </para>
+ </note>
+
+ <programlisting role="xml"><![CDATA[<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>copy</id>
+ <phase>process-resources</phase>
+ <goals>
+ <goal>copy</goal>
+ </goals>
+ <configuration>
+ <stripVersion>true</stripVersion>
+ <artifactItems>
+ <artifactItem>
+ <groupId>org.jboss.jsr299.tck</groupId>
+ <artifactId>jsr299-tck-impl</artifactId>
+ <type>xml</type>
+ <classifier>suite</classifier>
+ <overWrite>true</overWrite>
+ </artifactItem>
+ <artifactItem>
+ <groupId>org.jboss.webbeans</groupId>
+ <artifactId>
+ webbeans-porting-package
+ </artifactId>
+ <overWrite>true</overWrite>
+ <outputDirectory>
+ ${project.build.directory}/dependency/lib
+ </outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>org.jboss.webbeans</groupId>
+ <artifactId>webbeans-core-test</artifactId>
+ <overWrite>true</overWrite>
+ <outputDirectory>
+ ${project.build.directory}/dependency/lib
+ </outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>javax.el</groupId>
+ <artifactId>el-ri</artifactId>
+ <overWrite>true</overWrite>
+ <outputDirectory>
+ ${project.build.directory}/dependency/lib
+ </outputDirectory>
+ </artifactItem>
+ </artifactItems>
+ </configuration>
+ </execution>
+ </executions>
+</plugin>]]></programlisting>
+ <para>
+ The target folder for the copies of the dependencies (i.e., the JAR
+ files) is declared as the JBoss Test Harness library directory; this
+ results in these libraries being added to the test artifact using the
+ following property assignment:
+ </para>
+ <programlisting>org.jboss.testharness.libraryDirectory=target/dependency/lib</programlisting>
+ <para>
+ We also copy the test suite configuration from the
+ local Maven
+ repository (groupId=org.jboss.jsr299.tck,
+ artifactId=jsr299-tck-impl,
+ classifier=suite, type=xml,
+ version=1.0.0-SNAPSHOT) to a local
+ repository as the TestNG Maven
+ plugin expects a local file.
+ </para>
+ <para>
+ The TCK is executed using the Maven TestNG plugin. Maven 2 profiles are
+ used to control the properties that are set at the time of the
+ execution. For instance, the
+ <literal>incontainer</literal>
+ profile enables integration tests and disables standalone mode,
+ changing the default settings.
+ </para>
+ <para>
+ The jboss-tck-runner project also defines the JBoss Test Harness
+ extra configuration directory using the following property:
+ </para>
+ <programlisting>org.jboss.testharness.container.extraConfigurationDir=../jboss-as</programlisting>
+ <para>
+ The JBoss Test Harness looks in this directory for either a
+ build.properties or local.build.properties file that declares
+ additional configuration properties. In particular, the JBoss AS
+ Containers implementation looks here to find the
+ <literal>jboss.home</literal> property so it knows where the scripts
+ are located to start and stop JBoss AS.
+ </para>
+ </section>
+
+ <section id="dumping-test-artifacts">
+ <title>Dumping the Test Artifacts to Disk</title>
+ <para>
+ As you have learned, when the test suite is executing using
+ in-container mode, each test class is packaged as a deployable artifact
+ and deployed to the container. The test is then executed within the
+ context of the deployed application. This leaves room for errors in
+ packaging. When investigating a test failure, it's helpful to be able
+ to inspect the artifact after it is generated. The JBoss Test Harness
+ can accommodate this type of inspection by "dumping" the generated
+ artifact to disk.
+ </para>
+ <para>
+ If you want to write the artifacts to disk, and avoid executing the
+ test suite, you can simply execute the main method of the class
+ <literal>org.jboss.testharness.api.TCK</literal>.
+ For example you could use a Maven profile that is activated when the
+ <literal>dumpArtifacts</literal>
+ command line property is defined:
+ </para>
+ <programlisting>mvn test-compile -DdumpArtifacts</programlisting>
+ <para>
+ The output directory where the artifacts are written is defined by the
+ property
+ <literal>org.jboss.testharness.outputDirectory</literal>.
+ </para>
+ <para>
+ Once the artifact is written to disk, you have an option of manually
+ deploying it to the container. You can execute the tests in the artfact
+ by requesting the context path of the application in the browser. If
+ you want to execute an individual test method, specify the method name
+ in the
+ <literal>methodName</literal>
+ request parameter (e.g., ?methodName=testMethodName).
+ </para>
+ </section>
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/images/in-container-execution.png
===================================================================
(Binary files differ)
Property changes on: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/images/in-container-execution.png
___________________________________________________________________
Name: svn:mime-type
+ application/octet-stream
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/introduction.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,286 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<chapter id="test-harness-introduction">
+ <title>Introduction (JBoss Test Harness)</title>
+
+ <para>
+ This chapter explains the purpose of the test harness and describes
+ its key features.
+ </para>
+ <para>
+ The JBoss Test Harness is a testing framework based on TestNG that
+ provides a series of extensions that allow runtime packaging and
+ deployment of Java EE artifacts (EAR or WAR) for in-container testing.
+ It's important to note that the JBoss Test Harness has no relation with,
+ or dependency on, the JBoss Application Server (JBoss AS).
+ </para>
+ <note>
+ <para>
+ You'll often see the term
+ <emphasis role="italic">in-container</emphasis>
+ used in this reference guide. This term refers to running the test
+ suite in any of the aforementioned environments, whilst
+ <emphasis role="italic">standalone</emphasis>
+ refers to running the tests outside the container via an
+ implementation-specific standalone bootstrap. The standalone mode only
+ runs those tests which the CDI RI can run without deployment in a Java
+ EE container.
+ </para>
+ </note>
+ <para>
+ The last thing Java developers want is yet another testing framework to
+ make their life more complicated. That's why the JBoss Test Harness is
+ built entirely upon TestNG. TestNG is one of two prominent test
+ frameworks for Java (the other being JUnit). Furthermore, what developers
+ want is a good integration with their Integrated Development Environment
+ (IDE). These days, if a tool doesn't have an IDE plugin, then it won't
+ get the attention it deserves. TestNG plugins are available for all major
+ IDEs and build tools (Ant and Maven 2). Again, a motivating factor for
+ extending TestNG.
+ </para>
+ <para>
+ Because it leverages the existing TestNG ecosystem, there is no need for
+ a special test launcher for the JBoss Test Harness. You simply use the
+ IDE or build tool of your choice (so long as it has TestNG support). You
+ also get reporting and debugging for free (various reporting plugins are
+ provided for TestNG).
+ </para>
+ <para>
+ You can read more about TestNG at
+ <ulink url="http://testng.org/doc/documentation-main.html">testng.org</ulink>.
+ </para>
+ <para>
+ The JBoss Test Harness supports the following features:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Test activation via any method supported by the TestNG
+ configuration descriptor (package, group, class)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Exclusion of in-container tests in standalone mode
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Exclusion of individual tests labeled as under investigation
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Integration with any TestNG plugin (Eclipse, IntelliJ, Ant, Maven)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Automated reporting capability as provided by TestNG
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Standalone and in-container test mode
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Container pluggability
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Declarative packaging of additional resources and classes in
+ artifact
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Declarative deployment exception trapping
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Artifact dumping for failure and packaging analysis
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ A test is designated by a method annotated with
+ <literal>@org.testng.annotations.Test</literal>
+ in a class which extends
+ <literal>org.jboss.testharness.AbstractTest</literal>
+ and is annotated with
+ <literal>@org.jboss.testharness.impl.packaging.Artifact</literal>.
+ </para>
+ <note>
+ <para>
+ Test suites may often choose to extend <literal>AbstractTest</literal>
+ and require tests to extend that base class. In fact, both the CDI TCK
+ and the Bean Validation TCK provide base classes that extend
+ <literal>AbstractTest</literal> to provide functionality specific to
+ the needs of the TCK.
+ </para>
+ </note>
+ <para>
+ The
+ <literal>@Test</literal>
+ annotation is provided by TestNG, the
+ <literal>@Artifact</literal>
+ annotation is provided by the JBoss Test Harness and the
+ <literal>AbstractTest</literal>
+ is part of the JBoss Test Harness. There is a one-to-one mapping between a
+ TestNG test class and an artifact. The packaging type is defined by the
+ <literal>@org.jboss.testharness.impl.packaging.Packaging</literal>
+ annotation on the test class, defaulting to a WAR if not specified.
+ </para>
+ <para>
+ Prior to executing the tests for a given class, the JBoss Test Harness
+ packages the class as a deployable artifact (EAR or WAR), along with any
+ extra resources specified, and deploys the artifact to the container. The
+ harness provides test execution and result reporting via HTTP
+ communication to a simple Servlet using a thin layer over the TestNG test
+ launcher. The test harness can also catch and enforce expected deployment
+ exceptions. This setup and tear down activity is provided by the super
+ class
+ <literal>org.jboss.testharness.AbstractTest</literal>,
+ which all test classes must extend (directly or indirectly).
+ </para>
+ <para>
+ If the annotation
+ <literal>@org.jboss.testharness.impl.packaging.IntegrationTest
+ </literal>
+ is not present on the test class, then it means the test class can be
+ executed in standalone mode. In standalone mode, the deployable artifact
+ is assembled on the local classpath and the tests execute in the same JVM
+ as the launcher, just as though it were a regular TestNG test case. The
+ standalone mode is provided for convenience and efficiency, allowing you
+ the speed of mock-based testing and the confidence of an in-container test,
+ using the same test objects and tests.
+ </para>
+
+ <section id="in-container-communication">
+ <title>Negotiating the execution of an in-container test</title>
+ <para>
+ The basic procedure of an in-container test is as follows. The JBoss
+ Test Harness produces a deployable artifact from an
+ <literal>@Artifact</literal>
+ test class and any declared dependent classes, descriptors or other
+ resources. Then it deploys the artifact to the container using the
+ <literal>Containers</literal>
+ SPI, negotiates with the container to execute the test and return the
+ result and, finally, undeploys the artifact. TestNG collects the results
+ of all the tests run in the typical way and produces a report.
+ </para>
+ <graphic fileref="images/in-container-execution.png" align="center" />
+ <para>
+ The question is, how does the JBoss Test Harness negotiate with the
+ container to execute the test when TestNG is being invoked locally?
+ Technially the mechanism is pluggable, but JBoss Test Harness provides
+ a default implementation that uses HTTP communication that you will
+ likely use. Here's how the default implementation works.
+ </para>
+ <para>
+ The artifact generator bundles and registers (in the web.xml
+ descriptor) an
+ <literal>HttpServlet</literal>,
+ <literal>org.jboss.testharness.impl.runner.servlet.ServletTestRunner</literal>,
+ that responds to test execution GET requests. TestNG running on the
+ client side delegates to a test launcher (more on that in a moment)
+ which originates these text execution requests to transfer control to
+ the container JVM. The name of the test method to be executed is
+ specified in a request query parameter named
+ <literal>methodName</literal>.
+ </para>
+ <para>
+ When the test execution request is received, the servlet delegates to
+ an instance of
+ <literal>org.jboss.testharness.impl.runner.TestRunner</literal>,
+ passing it the name of the test method.
+ <literal>TestRunner</literal>
+ reads the name of the test class from the resource
+ META-INF/jboss-test-harness.properties, which is bundled in the
+ artifact by the artifact generator. It then combines the class name
+ and the method name to produce a TestNG test suite and runs the suite
+ (within the context of the container).
+ </para>
+ <para>
+ TestNG returns the results of the run as an
+ <literal>ITestResult</literal>
+ object.
+ <literal>ServletTestRunner</literal>
+ translates this object into a
+ <literal>org.jboss.testharness.api.TestResult</literal>
+ and passes it back to the test launcher on the client side by encoding
+ the translated object into the response. The object gets encoded as
+ either html or a serialized object, depending on the value of the
+ <literal>outputMode</literal>
+ request parameter that was passed to the servlet. Once the result has
+ been transfered to the client-side TestNG, TestNG wraps up the run of
+ the test as though it had been executed in the same JVM.
+ </para>
+ <para>
+ There's one piece missing. How does TestNG on the client side know to
+ submit a request to the
+ <literal>ServletTestRunner</literal>
+ servlet to get TestNG to execute the test in the container JVM? That's
+ the role of the test launcher.
+ </para>
+ <para>
+ The test launcher is the API that allows test suite to launch the test
+ in a pluggable fashion.
+ <literal>AbstractTest</literal>,
+ the super class of
+ <literal>AbtractJSR299Test</literal>,
+ implements
+ <literal>IHookable</literal>,
+ a TestNG interface which allows the execution of the test method to
+ be intercepted. Using that mechanism, <literal>AbstractTest</literal> delegates execution
+ of the test method (a method annotated with
+ <literal>@Test</literal>
+ in an
+ <literal>@Artifact</literal>
+ class) to an implementation of
+ <literal>org.jboss.testharness.api.TestLauncher</literal>
+ if the tests are being executed in-container. As you might anticipate,
+ the implementation is specified using a property with the same name as
+ the interface in a META-INF/jboss-test-launcher.properties resource.
+ The JBoss Test Harness provides a default implementation,
+ <literal>org.jboss.testharness.impl.runner.servlet.ServletTestLauncher</literal>,
+ that hooks into the HTTP communication infrastructure described
+ above. It invokes the
+ <literal>ServletTestRunner</literal>
+ servlet for each method annotated with
+ <literal>@Test</literal>
+ in the
+ <literal>@Artifact</literal>
+ that is not otherwise disabled.
+ </para>
+ <para>
+ If you wish to implement the runner yourself, you must return a
+ <literal>TestResult</literal> as a result of executing the method in
+ the container. You must also ensure that any exception which occurs
+ during deployment is wrapped as a
+ <literal>org.jboss.testharness.api.DeploymentException</literal>, and
+ that any communication problem is rethrown as an
+ <literal>IOException</literal>. The deployment exception may be
+ transformed by an implementation of the
+ <literal>org.jboss.testharness.api.DeploymentExceptionTransformer
+ </literal> interface, which is specified using the
+ <literal>org.jboss.testharness.container.deploymentExceptionTransformer
+ </literal> property. The default implementation passes on the original
+ exception unchanged. The implementation for JBoss AS used with the CDI
+ TCK, on the other hand, deciphers the exception thrown by the JBoss
+ deployer and converts it to one of the catagory exceptions defined in
+ the CDI TCK API.
+ </para>
+ <para>
+ So in short, JBoss Test Harness takes care of all the interfaces you
+ need to execute tests in-container except for the implementation of
+ the <literal>Containers</literal> SPI. That is, unless you are
+ deploying to one of the containers supported by the JBoss Test Harness
+ (TODO we need a table showing supported containers).
+ </para>
+ </section>
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/harness/master.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/harness/master.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/harness/master.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,8 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<book lang="en">
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Book_Info.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Book_Preface.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="introduction.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="configuration.xml"/>
+</book>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-emailable-report.png
===================================================================
(Binary files differ)
Property changes on: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-emailable-report.png
___________________________________________________________________
Name: svn:mime-type
+ application/octet-stream
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-plugin-report.png
===================================================================
(Binary files differ)
Property changes on: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-plugin-report.png
___________________________________________________________________
Name: svn:mime-type
+ application/octet-stream
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-suite-detail-report.png
===================================================================
(Binary files differ)
Property changes on: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-suite-detail-report.png
___________________________________________________________________
Name: svn:mime-type
+ application/octet-stream
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-summary-report.png
===================================================================
(Binary files differ)
Property changes on: beanvalidation/tck/trunk/src/main/docbook/en-US/images/testng-summary-report.png
___________________________________________________________________
Name: svn:mime-type
+ application/octet-stream
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/installation.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/installation.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/installation.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,228 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="installation">
+ <title>Installation</title>
+
+ <para>This chapter explains how to obtain the TCK and supporting software
+ and provides recommendations for how to install/extract it on your
+ system.</para>
+
+ <section>
+ <title>Obtaining the Software</title>
+
+ <para>You can obtain a release of the Bean Validation TCK project from the
+ from the <ulink url="http://seamframework.org/Download">download
+ page</ulink> on the Hibernate Validator website. The Bean Validation TCK
+ is distributed as a ZIP file, which contains the TCK artifacts (the test
+ suite binary and source, porting package API binary and source, the test
+ suite descriptor, the audit source and report) in <code>/artifacts</code>,
+ the TCK library dependencies in <code>/lib</code> and documentation in
+ <code>/lib</code>.</para>
+
+ <para>You can also download the currnet source code from <ulink
+ url="http://anonsvn.jboss.org/repos/hibernate/beanvalidation/tck/trunk/">JBoss
+ SVN repository</ulink>.</para>
+
+ <para>The TCK project is available in the JBoss Maven 2 repository as
+ org.hibernate.jsr303.tck<code>:</code>org.hibernate.jsr303.tck; the POM
+ defines all dependencies required to run the TCK.</para>
+
+ <para>Executing the TCK requires a Java EE 5 or better runtime environment
+ (i.e., application server), to which the test artifacts are deployed and
+ the individual tests are invoked. The TCK does not depend on any
+ particular Java EE implementation.</para>
+
+ <para>The JSR-303: Bean Validation reference implementation (RI) project
+ is named Hibernate Validator. You can obtain the latest Hibernate
+ Validator release from the <ulink
+ url="https://www.hibernate.org/6.html">download page</ulink> on Hibernate
+ website.</para>
+
+ <note>
+ <para>Hibernate Validator is not required for running the Bean
+ Validation TCK, but it can be used as a reference for familiarizing
+ yourself with the TCK before testing your own Bean Validation
+ implementation.</para>
+ </note>
+
+ <para>Naturally, to execute Java programs, you must have a Java SE runtime
+ environment. The TCK requires Java 5 or better, which you can obtain from
+ the <ulink url="http://java.sun.com">Java Software</ulink> website.</para>
+ </section>
+
+ <section id="tck-environment">
+ <title>The TCK Environment</title>
+
+ <para>The TCK requires the following two Java runtime environments:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Java 5 or better</para>
+ </listitem>
+
+ <listitem>
+ <para>Java EE 5 or better (e.g., JBoss AS 5.x or GlassFish V3)</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>You should refer to vendor instructions for how to install the
+ runtime.</para>
+
+ <para>The rest of the TCK software can simply be extracted. It's
+ recommended that you create a folder named jsr303 to hold all of the
+ jsr303-related projects. Then, extract the TCK distribution into a
+ subfolder named tck. If you have downloaded the Hibernate Validator
+ distribution, extract it into a sibling folder named ri. The resulting
+ folder structure is shown here:</para>
+
+ <note>
+ <para>This layout is assumed through all descriptions in this reference
+ guide.</para>
+ </note>
+
+ <programlisting>jsr303/
+ ri/
+ tck/</programlisting>
+
+ <para>Each test class is treated as an individual artifact (hence the
+ <literal>@Artifact</literal> annotation on the class). All test methods
+ (i.e., methods annotated with <literal>@Test</literal>) in the test class
+ are run in the application, meaning bean discovery occurs exactly once per
+ artifact and the same BeanManager is used by each test method in the
+ class.</para>
+
+ <tip id="tck-in-jboss-as">
+ <title>Running the TCK against the Bean Validation RI (Hibernate
+ Validator) and JBoss AS</title>
+
+ <para>Web Beans is built as a modular library, and as such can be
+ retro-fitted to Java EE 5 products as required. JBoss AS 5.0 and above
+ releases bundle Web Beans. JBoss AS 5.0 also allows you to upgrade the
+ Web Beans module to the current release (though some functionality may
+ be disabled).</para>
+
+ <para>To install JBoss AS 5.1 and update to the latest Web Beans
+ release:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>First, you should download JBoss AS 5.1 from the JBoss AS
+ <ulink url="http://jboss.org/jbossas/downloads">project
+ page</ulink>.</para>
+ </listitem>
+
+ <listitem>
+ <para>Set the <literal>JBOSS_HOME</literal> environment variable to
+ the location of the JBoss AS software.</para>
+ </listitem>
+
+ <listitem>
+ <para>Change to the webbeans directory.</para>
+ </listitem>
+
+ <listitem>
+ <para>Make sure the <literal>jboss.home</literal> property in the
+ local.build.properties file in the jboss-as directory references a
+ JBoss AS 5.1 installation:</para>
+
+ <programlisting>jboss.home=/path/to/jboss-as-5.1</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>Then, run Ant from the jboss-as directory to install the
+ deployer:</para>
+
+ <programlisting>ant update</programlisting>
+
+ <para>The libraries needed by the deployer are fetched from the
+ Maven 2 repository on demand.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>Web Beans includes a TCK runner that executes the TCK using Web
+ Beans as the CDI implementation and JBoss AS as the Java EE runtime. To
+ run the tck:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>You need to install Maven. You can find documention on how to
+ install Maven 2 in the <ulink
+ url="http://www.sonatype.com/books/maven-book/reference/installation-sect-maven-install.html">Maven:
+ The Definitive Guide</ulink> book published by Sonatype. Web Beans
+ bundles a copy of Maven in the <code>lib/maven</code>
+ directory.</para>
+ </listitem>
+
+ <listitem>
+ <para>Next, instruct Maven to run the TCK:</para>
+
+ <programlisting>cd jboss-tck-runner
+mvn test -Dincontainer</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>TestNG will report, via Maven, the outcome of the run, and
+ report any failures on the console. Details can be found in
+ <code>target/surefire-reports/TestSuite.txt</code>.</para>
+ </listitem>
+ </itemizedlist>
+ </tip>
+ </section>
+
+ <section id="eclipse-plugins">
+ <title>Eclipse Plugins</title>
+
+ <para>Eclipse, or any other IDE, is not required to execute or pass the
+ TCK. However an implementor may wish to execute tests in an IDE to aid
+ debugging the tests. This section introduces two essential Eclipse
+ plugins, TestNG and Maven 2, and points you to resources explaining how to
+ install them.</para>
+
+ <section id="eclipse-testng-plugin">
+ <title>TestNG Plugin</title>
+
+ <para>The TCK is built on the JBoss Test Harness, which is in turn built
+ on TestNG. Therefore, having the TestNG plugin installed in Eclipse is
+ essential. Instructions for using the TestNG update site to add the
+ TestNG plugin to Eclipse are provided on the TestNG <ulink
+ url="http://testng.org/doc/download.html">download page</ulink>. You can
+ find a tutorial that explains how to use the TestNG plugin on the TestNG
+ <ulink url="http://testng.org/doc/eclipse.html">Eclipse
+ page</ulink>.</para>
+ </section>
+
+ <section id="m2eclipse-plugin">
+ <title>Maven 2 Plugin (m2eclipse)</title>
+
+ <para>Another useful plugin is m2eclipse. Both the TCK project and Web
+ Beans are use Maven 2. Therefore, to work with these projects in
+ Eclipse, you may wish to have native support for Maven 2 projects, which
+ the m2eclipse plugin provides. Instructions for using the m2eclipse
+ update site to add the m2eclipse plugin to Eclipse are provided on the
+ m2eclipse home page. For more, read the m2eclipse <ulink
+ url="http://www.sonatype.com/books/m2eclipse-book/reference">reference
+ guide</ulink>.</para>
+
+ <para>m2eclipse is still a rather young project dealing with a complex
+ domain and you may run into problems using it. If that is the case, you
+ can alternatively use the Eclipse plugin for Maven 2 to generate native
+ Eclipse projects from Maven 2 projects.</para>
+
+ <para>If you have Maven 2 installed, you have everything you need. Just
+ execute the following command from any Maven 2 project to produce the
+ Eclipse project files.</para>
+
+ <programlisting>mvn eclipse:eclipse</programlisting>
+ </section>
+
+ <para>Again, the Eclipse plugins are not required to execute the TCK, but
+ can be very helpful when validating an implementation against the TCK test
+ suite and especially when using the modules from the Hibernate Validator
+ project.</para>
+ </section>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/introduction.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/introduction.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/introduction.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,216 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="introduction">
+ <title>Introduction (Bean Validation TCK)</title>
+
+ <para>This chapter explains the purpose of a TCK and identifies the
+ foundation elements of the Bean Validation TCK.</para>
+
+ <section>
+ <title>TCK Primer</title>
+
+ <para>A TCK, or Technology Compatibility Kit, is one of the three required
+ pieces for any JSR (the other two being the specification document and the
+ reference implementation). The TCK is a set of tools and tests to verify
+ that an implementation of the technology conforms to the specification.
+ The tests are the primary component, but the tools serve an equally
+ critical role of providing a framework and/or set of SPIs for executing
+ the tests.</para>
+
+ <para>The tests in the TCK are derived from assertions in the written
+ specification document. The assertions are itemized in an XML document,
+ where they each get assigned a unique identifier, and materialize as a
+ suite of automated tests that collectively validate whether an
+ implementation complies with the aforementioned assertions, and in turn
+ the specification. For a particular implementation to be certified, all of
+ the required tests must pass (i.e., the provided test suite must be run
+ unmodified).</para>
+
+ <para>A TCK is entirely implementation agnostic. It should validate
+ assertions by consulting the specficiation's public API. </para>
+ </section>
+
+ <section>
+ <title>Compatibility Testing</title>
+
+ <para>The goal of any specification is to eliminate portability problems
+ so long as the program which uses the implementation also conforms to the
+ rules laid out in the specification.</para>
+
+ <para>Executing the TCK is a form of compatibility testing. It's important
+ to understand that compatibility testing is distinctly different from
+ product testing. The TCK is not concerned with robustness, performance or
+ ease of use, and therefore cannot vouch for how well an implementation
+ meets these criteria. What a TCK can do is to ensure the exactness of an
+ implementation as it relates to the specification.</para>
+
+ <para>Compatibility testing of any feature relies on both a complete
+ specification and a complete reference implementation. The reference
+ implementation demonstrates how each test can be passed and provides
+ additional context to the implementor during development for the
+ corresponding assertion.</para>
+
+ <section>
+ <title>Why Compatibility Is Important</title>
+
+ <para>Java platform compatibility is important to different groups
+ involved with Java technologies for different reasons:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Compatibility testing is the means by which the JCP ensures
+ that the Java platform does not become fragmented as it's ported to
+ different operating systems and hardware.</para>
+ </listitem>
+
+ <listitem>
+ <para>Compatibility testing benefits developers working in the Java
+ programming language, enabling them to write applications once and
+ deploy them across heterogeneous computing environments without
+ porting.</para>
+ </listitem>
+
+ <listitem>
+ <para>Compatibility testing enables application users to obtain
+ applications from disparate sources and deploy them with
+ confidence.</para>
+ </listitem>
+
+ <listitem>
+ <para>Conformance testing benefits Java platform implementors by
+ ensuring the same extent of reliability for all Java platform
+ ports.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>The Bean Validation specification goes to great lengths to ensure
+ that programs written for Java EE are compatible and the TCK is rigorous
+ about enforcing the rules the specification lays down.</para>
+ </section>
+ </section>
+
+ <section>
+ <title>About the Bean Validation TCK</title>
+
+ <para>The Bean Validation TCK is designed as a portable, configurable and
+ automated test suite for verifying the compatibility of an implementation
+ of the JSR-303: Bean Validation specification. The test suite is built
+ atop TestNG and provides a series of extensions that allow runtime
+ packaging and deployment of JEE artifacts for in-container testing (JBoss
+ Test Harness).</para>
+
+ <note>
+ <para>The Bean Validation TCK harness is based on the JBoss Test
+ harness, which provides most of the aforementioned functionality.</para>
+ </note>
+
+ <para>Each test class in the suite acts as a deployable unit. The
+ deployable units, or artifacts, are defined declaratively using
+ annotations. The artifact produced can be either a WAR or an EAR.</para>
+
+ <para>The declarative approach allows many of the tests to be executed in
+ a standalone implementation of Bean Validation, accounting for a boast in
+ developer productivity. However, an implementation is only valid if all
+ tests pass using the in-container execution mode. The standalone mode is
+ merely a developer convenience.</para>
+
+ <section>
+ <title>Bean Validation TCK Specifications and Requirements</title>
+
+ <para>This section lists the applicable requirements and specifications
+ for the Bean Validation TCK.</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role="bold">Bean Validation API</emphasis> - The
+ Java API defined in the Bean Validation specification and provided
+ by the reference implementation.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">JBoss Test Harness</emphasis> - The Bean
+ Validation TCK requires version 1.0.0 of the JBoss Test Harness. The
+ Harness is based on TestNG 5.x (<ulink
+ url="http://testng.org">http://testng.org</ulink>). You can read
+ more about the harness in <xref linkend="test-harness" />.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">TCK Audit Tool</emphasis> - An
+ itemization of the assertions in the specification documents which
+ are cross referenced by the individual tests. Describes how well the
+ TCK covers the specification.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Reference runtime</emphasis> - The
+ designated reference runtimes for compatibility testing of the Bean
+ Validation specification is the Sun Java Platform, Enterprise
+ Edition (Java EE) 6 reference implementation (RI). See details at
+ Java EE 6 (<ulink
+ url="http://java.sun.com/javaee/6/docs/api/">http://java.sun.com/javaee/6/docs/api/</ulink>).</para>
+ </listitem>
+ </itemizedlist>
+ </section>
+
+ <section>
+ <title>Bean Validation TCK Components</title>
+
+ <para>The Bean Validation TCK includes the following components:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role="bold">JBoss Test Harness 1.0.0</emphasis> and
+ related documentation.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">TestNG 5.9</emphasis>, the testing
+ framework on which the JBoss Test Harness is based and which
+ provides the extension points for selecting an executing the tests
+ in the test suite.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">The test suite</emphasis>, which is a
+ collection of TestNG tests, the TestNG test suite descriptor and
+ supplemental resources that configure CDI and other software
+ components.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">The TCK audit</emphasis> is used to list
+ out the assertions identified in the CDI specification. It matches
+ the assertions to testcases in the test suite by unique identifier
+ and produces a coverage report.</para>
+
+ <para>The audit document is provided along with the TCK. Each
+ assertion is defined with a reference to a chapter, section and
+ paragraph from the specification document, making it easy for the
+ implementor to locate the language in the specification document
+ that supports the feature being tested.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">TCK documentation</emphasis> accompanied
+ by release notes identifying updates between versions.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>The Bean Validation TCK has been tested run on following
+ platforms:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>JBoss AS 5.0.0.GA using Sun Java SE 6 on Red Hat Enterprise
+ Linux 5.2</para>
+ </listitem>
+ </itemizedlist>
+ </section>
+ </section>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/master.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/master.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/master.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<book lang="en">
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Book_Info.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Book_Preface.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="part-background.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="part-execution.xml"/>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="part-test-harness.xml" />
+<!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</book>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/part-background.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/part-background.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/part-background.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE part PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<part>
+ <title>Getting Acquainted with the TCK</title>
+
+ <partintro>
+ <para>The Bean Validation TCK must be used to ensure that your
+ implementation conforms to the Bean Validation specification. This part
+ introduces the TCK, gives some background about its purpose, states the
+ requirements for passing the TCK and outlines the appeals process.</para>
+
+ <para>In this part you will learn where to obtain the Bean Validation TCK
+ and supporting software. You are then presented with recommendations of
+ how to organize and configure the software so that you are ready to
+ execute the TCK.</para>
+
+ <para>Finally, it discusses the reporting provided by the TCK.</para>
+ </partintro>
+
+ <xi:include href="introduction.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <xi:include href="appeals-process.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <xi:include href="installation.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <xi:include href="configuration.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <xi:include href="reporting.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</part>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/part-execution.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/part-execution.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/part-execution.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE part PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<part>
+ <title>Executing and Debugging Tests</title>
+
+ <partintro>
+ <para>In this part you learn how to execute the Bean Validation TCK on the
+ Bean validation reference implementation (Hibernate Validator). First, you
+ are walked through the steps necessary to execute the test suite on
+ Hibernate Validator. Then you discover how to modify the TCK runner to
+ execute the test suite on your own implementation. Finally, you learn how
+ to debug tests from the test suite in Eclipse.</para>
+ </partintro>
+
+ <xi:include href="executing.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <xi:include href="eclipse-running.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <xi:include href="eclipse-debugging.xml"
+ xmlns:xi="http://www.w3.org/2001/XInclude" />
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</part>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/part-test-harness.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/part-test-harness.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/part-test-harness.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,18 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE part PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ ]>
+<part id="test-harness">
+ <title>JBoss Test Harness</title>
+ <partintro>
+ <para>
+ In this part you learn about the JBoss Test Harness through selected
+ chapters from the JBoss Test Harness Reference Guide. You can view the
+ entire JBoss Test Harness Reference Guide at <ulink url="">TODO</ulink>.
+ </para>
+ </partintro>
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="harness/introduction.xml" />
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="harness/configuration.xml" />
+ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="harness/executing.xml" />
+<!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</part>
Added: beanvalidation/tck/trunk/src/main/docbook/en-US/reporting.xml
===================================================================
--- beanvalidation/tck/trunk/src/main/docbook/en-US/reporting.xml (rev 0)
+++ beanvalidation/tck/trunk/src/main/docbook/en-US/reporting.xml 2009-09-29 16:34:58 UTC (rev 17578)
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
+<chapter id="reporting">
+ <title>Reporting</title>
+
+ <para>This chapter covers the two types of reports that can be generated
+ from the TCK, an assertion coverage report and the test execution results.
+ The chapter also justifies why the TCK is good indicator of how accurately
+ an implementation conforms to the JSR-303 specification.</para>
+
+ <section>
+ <title>Bean Validation TCK Coverage Metrics</title>
+
+ <para>The Bean Validation TCK coverage has been measured as
+ follows:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role="bold">Assertion Breadth
+ Coverage</emphasis></para>
+
+ <para>The CDI TCK provides at least 75% coverage of identified
+ assertions with test cases.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Assertion Breadth Coverage
+ Variance</emphasis></para>
+
+ <para>The coverage of specification sub-sections shows at least a
+ normal distribution (centered around 75%).</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Assertion Depth Coverage</emphasis></para>
+
+ <para>The assertion depth coverage has not been measured, as, when an
+ assertion requires more than one testcase, these have been enumerated
+ in an assertion group and so are adequately described by the assertion
+ breadth coverage.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Method Coverage</emphasis></para>
+
+ <para>TODO</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">API Signature Coverage</emphasis></para>
+
+ <para>The CDI TCK covers 100% of all API public methods using the Java
+ CTT Sig Test tool.</para>
+ </listitem>
+ </itemizedlist>
+ </section>
+
+ <section>
+ <title>Bean Validation TCK Coverage Report</title>
+
+ <para>A specification can be distilled into a collection of assertions
+ that define the behavior of the software. This section introduces the Bean
+ Validation TCK coverage report, which documents the relationship between
+ the assertions that have been identified in the JSR-303 specification
+ document and the tests in the TCK test suite.</para>
+
+ <para>The structure of this report is controlled by the assertion
+ document, so we'll start there.</para>
+
+ <section>
+ <title>Bean Validation TCK Assertions</title>
+
+ <para>The Bean Validation TCK developers have analyzed the JSR-303
+ specification document and identified the assertions that are present in
+ each chapter. Here's an example of one such assertion found in section
+ 2.1:</para>
+
+ <blockquote>
+ Every constraint annotation must define a message element of type String
+ </blockquote>
+
+ <para>The assertions are listed in the XML file
+ src/main/resources/tck-audit.xml in the Bean Validation TCK
+ distribution. Each assertion is identified by the section of the
+ specification document in which it resides and assigned a unique
+ paragraph identifier to narrow down the location of the assertion
+ further. To continue with the example, the assertion shown above is
+ listed in the tck-audit.xml file using this XML fragment:</para>
+
+ <programlisting role="XML"><section id="2.1.1" title="Constraint definition properties">
+ ...
+ <assertion id="c">
+ <text>Every constraint annotation must define a message element of type String</text>
+ </assertion>
+ ...
+</section></programlisting>
+
+ <para>The strategy of the Bean Validation TCK is to write a test which
+ validates this assertion when run against an implementation. A test case
+ (a method annotated with <literal>@Test</literal> in an
+ <literal>@Artifact</literal> class) is correlated with an assertion
+ using the
+ <literal>@org.jboss.test.audit.annotations.SpecAssertion</literal>
+ annotation as follows:</para>
+
+ <programlisting role="JAVA">@Test
+ at SpecAssertion(section = "2.1.1", id = "c")
+public void testConstraintDefinitionWithoutMessageParameter() {
+ try {
+ Validator validator = TestUtil.getValidatorUnderTest();
+ validator.validate( new DummyEntityNoMessage() );
+ fail( "The used constraint does not define a message parameter. The validation should have failed." );
+ }
+ catch ( ConstraintDefinitionException e ) {
+ // success
+}
+</programlisting>
+
+ <para>To help evaluate the distribution of coverage for these
+ assertions, the TCK provides a detailed coverage report. This report is
+ also useful to help implementors match tests with the language in the
+ specification that supports the behavior being tested.</para>
+ </section>
+
+ <section>
+ <title>Producing the Coverage Report</title>
+
+ <para>The coverage report is an HTML report generated as part of the TCK
+ project build. Specifically, it is generated by an annotation processor
+ that attaches to the compilation of the classes in the TCK test suite,
+ another tool from the JBoss Test Utils project. You can enable this
+ report by setting the commandline property tck-audit to true when
+ running the Maven 2 build in the tck directory.</para>
+
+ <programlisting>mvn clean install -Dtck-audit=true</programlisting>
+
+ <note>
+ <para>You must run clean first because the annotation processor
+ performs it's work when the test class is being compiled. If
+ compilation is unnecessary, then the assertions referenced in that
+ class will not be discovered.</para>
+ </note>
+
+ <para>The report is written to the file target/coverage.html in the same
+ project. The report has five sections:</para>
+
+ <orderedlist>
+ <listitem>
+ <para><emphasis role="bold">Chapter Summary</emphasis> - List the
+ chapters (that contain assertions) in the specification document
+ along with total assertions, tests and coverage percentage.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Section Summary</emphasis> - Lists the
+ sections (that contain assertions) in the specification document
+ along with total assertions, tests and coverage percentage.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Coverage Detail</emphasis> - Each
+ assertion and the test that covers it, if any.</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Unmatched Tests</emphasis> - A list of
+ tests for which there is no matching assertion (useful during TCK
+ development).</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Unversioned Tests</emphasis> - A list of
+ tests for which there is no <literal>@SpecVersion</literal>
+ annotation on the test class (useful during TCK development).</para>
+ </listitem>
+ </orderedlist>
+
+ <para>The coverage report is color coded to indicate the status of an
+ assertion, or group of assertions. The status codes are as
+ follows:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role="bold">Covered</emphasis> - a test exists for
+ this assertion</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Not covered</emphasis> - no test exists
+ for this assertion</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Problematic</emphasis> - a test exists,
+ but is currently disabled. For example, this may be because the test
+ is under development</para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">Untestable</emphasis> - the assertion
+ has been deemed untestable, a note, explaining why, is normally
+ provided</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>For reasons provided in the <literal>tck-audit.xml</literal>
+ document and presented in the coverage report, some assertions are not
+ testable.</para>
+
+ <para>The coverage report does not give any indication as to whether the
+ tests are passing. That's where the TestNG reports come in.</para>
+ </section>
+
+ <section>
+ <title>TestNG Reports</title>
+
+ <para>As you by now, the CDI TCK test suite is really just a TestNG test
+ suite. That means an execution of the CDI TCK test suite produces all
+ the same reports that TestNG produces. This section will go over those
+ reports and show you were to go to find each of them.</para>
+
+ <section>
+ <title>Maven 2, Surefire and TestNG</title>
+
+ <para>When the CDI TCK test suite is executed during the Maven 2 test
+ phase of the TCK runner project, TestNG is invoked indirectly through
+ the Maven Surefire plugin. Surefire is a test execution abstraction
+ layer capable of executing a mix of tests written for JUnit, TestNG,
+ and other supported test frameworks.</para>
+
+ <para>Why is this relevant? It means two things. First, it means that
+ you are going to get a summary of the test run on the commandline.
+ Here's the output generated when the tests are run using standalone
+ mode.</para>
+
+ <programlisting>-------------------------------------------------------
+ T E S T S
+-------------------------------------------------------
+Running TestSuite
+Tests run: 237, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.062 sec
+
+Results :
+
+Tests run: 237, Failures: 0, Errors: 0, Skipped: 0
+</programlisting>
+
+ <note>
+ <para>The number of tests executed, the execution time, and the
+ output will differ when you run the tests using in-container mode as
+ the CDI TCK requires.</para>
+ </note>
+
+ <para>If the Maven reporting plugin that compliments Surefire is
+ configured properly, Maven will also generate a generic HTML test
+ result report. That report is written to the file test-report.html in
+ the target/surefire-reports directory of the TCK runner project. It
+ shows how many tests were run, how many failed and the success rate of
+ the test run.</para>
+
+ <para>The one drawback of the Maven Surefire report plugin is that it
+ buffers the test failures and puts them in the HTML report rather than
+ outputting them to the commandline. If you are running the test suite
+ to determine if there are any failures, it may be more useful to get
+ this information in the foreground. You can prevent the failures from
+ being redirected to the report using the following commandline
+ switch:</para>
+
+ <programlisting>mvn test -Dsurefire.useFile=false</programlisting>
+
+ <para>The information that the Surefire provides is fairly basic and
+ the detail pales in comparison to what the native TestNG reports
+ provide.</para>
+ </section>
+
+ <section>
+ <title>TestNG HTML Reports</title>
+
+ <para>TestNG produces several HTML reports for a given test run. All
+ the reports can be found in the target/surefire-reports directory in
+ the TCK runner project. Below is a list of the three types of
+ reports:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Test Summary Report</para>
+ </listitem>
+
+ <listitem>
+ <para>Test Suite Detail Report</para>
+ </listitem>
+
+ <listitem>
+ <para>Emailable Report</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>The first report, the test summary report, show below, is
+ written to the file index.html. It produces the same information as
+ the generic Surefire report.</para>
+
+ <graphic align="center" fileref="images/testng-summary-report.png" />
+
+ <para>The summary report links to the test suite detail report, which
+ has a wealth of information. It shows a complete list of test groups
+ along with the classes in each group, which groups were included and
+ excluded, and any exceptions that were raised, whether from a passed
+ or failed test. A partial view of the test suite detail report is
+ shown below.</para>
+
+ <graphic align="center"
+ fileref="images/testng-suite-detail-report.png" />
+
+ <para>The test suite detail report is very useful, but it borderlines
+ on complex. As an alternative, you can have a look at the emailable
+ report, which is a single HTML document that shows much of the same
+ information as the test suite detail report in a more compact layout.
+ A partial view of the emailable report is shown below.</para>
+
+ <graphic align="center" fileref="images/testng-emailable-report.png" />
+
+ <para>Now that you have seen two ways to get test results from the
+ Maven test execution, let's switch over to the IDE, specifically
+ Eclipse, and see how it presents TestNG test results.</para>
+ </section>
+ </section>
+ </section>
+
+ <!--
+vim: ts=3:sw=3:tw=80:set expandtab
+-->
+</chapter>
More information about the hibernate-commits
mailing list