Author: pete.muir(a)jboss.org Date: 2009-07-31 10:07:51 -0400 (Fri, 31 Jul 2009) New Revision: 3384 Modified: tck/trunk/doc/reference/en-US/Book_Preface.xml tck/trunk/doc/reference/en-US/appeals-process.xml tck/trunk/doc/reference/en-US/introduction.xml tck/trunk/doc/reference/en-US/reporting.xml Log: A few updates Modified: tck/trunk/doc/reference/en-US/Book_Preface.xml =================================================================== --- tck/trunk/doc/reference/en-US/Book_Preface.xml 2009-07-31 13:52:43 UTC (rev 3383) +++ tck/trunk/doc/reference/en-US/Book_Preface.xml 2009-07-31 14:07:51 UTC (rev 3384) @@ -86,6 +86,13 @@ </listitem> <listitem> <para> + <xref linkend="appeals-process" /> + explains the process to be followed by an implementor should they + wish to challenge any test in the TCK. + </para> + </listitem> + <listitem> + <para> <xref linkend="installation" /> explains where to obtain the required software for the CDI TCK and how to install it. It covers both the primary TCK components Modified: tck/trunk/doc/reference/en-US/appeals-process.xml =================================================================== --- tck/trunk/doc/reference/en-US/appeals-process.xml 2009-07-31 13:52:43 UTC (rev 3383) +++ tck/trunk/doc/reference/en-US/appeals-process.xml 2009-07-31 14:07:51 UTC (rev 3384) @@ -24,8 +24,7 @@ central aspect of any good software and it's only natural to point out shortcomings and strive to make improvements. Despite this good faith, not all implementors will be comfortable with a public appeals process. - Therefore, further instructions will be forthcoming about how to make a - private appeal. + Instructions about how to make a private appeal are therefore provided. </para> <section> <title>Who can make challenges to the TCK?</title> @@ -67,13 +66,19 @@ url="http://www.jboss.org/index.html?op=checkage&amp;module=user... registration</ulink>. </para> + <para> + If you wish to make a private challenge, you should follow the above + procedure, setting the Security Level to Private. Only the issue + reporter, TCK Project Lead and designates will be able to view the + issue. + </para> </section> <section> <title>How and by whom challenges are addressed?</title> <para> The challenges will be addressed in a timely fashion by the CDI TCK Project Lead, as designated by Specification Lead, Red Hat Middleware - LLC. or his/her appointee. The appellant can also monitor the process + LLC. or his/her designate. The appellant can also monitor the process by following the issue report filed in the <ulink url="https://jira.jboss.org/jira/browse/WBTCK">WBTCK project</ulink> of the JBoss JIRA. @@ -99,7 +104,7 @@ Resolved issue not addressed for 30 days will be closed by the TCK Project Lead. If the TCK Project Lead and appellant are unable to agree on the issue resolution, it will be referred to the JSR-299 - specification lead or his/her desginate. + specification lead or his/her designate. </para> </section> </chapter> Modified: tck/trunk/doc/reference/en-US/introduction.xml =================================================================== --- tck/trunk/doc/reference/en-US/introduction.xml 2009-07-31 13:52:43 UTC (rev 3383) +++ tck/trunk/doc/reference/en-US/introduction.xml 2009-07-31 14:07:51 UTC (rev 3384) @@ -270,29 +270,22 @@ <para> Sun Microsystems, Inc. will implement the porting package for the CDI RI and test the CDI RI on the Java EE Reference - Implementation. + Implementation as well as other platforms. </para> </note> - <para> - The CDI TCK has been tested with the following: - </para> + <para>The CDI TCK has been tested run on following platforms:</para> <itemizedlist> <listitem> - <para>JBoss AS 5.1.0.GA</para> + <para> + JBoss AS 5.1.0.GA using Sun Java SE 6 on Red Hat Enterprise + Linux 5.2 + </para> </listitem> </itemizedlist> - <para>The CDI TCK tests run on the following platforms:</para> - <itemizedlist> - <listitem> - <para>Red Hat Enterprise Linux 5.2</para> - </listitem> - </itemizedlist> <para> CDI supports Java EE 5, Java EE 6, Java EE 6 Web Profile and the - Embeddable EJB 3.1. The TCK will run on any of these runtimes, but - is only part of the CTS for Java EE 6, Java EE 6 Web Profile and - Embeddable EJB 3.1 (SUN: Please confirm these are the CTS the CDI - TCK will run in). + Embeddable EJB 3.1. The TCK will execute on any of these runtimes, + but is only part of the CTS for Java EE 6 and Java EE 6 Web Profile. </para> </section> </section> Modified: tck/trunk/doc/reference/en-US/reporting.xml =================================================================== --- tck/trunk/doc/reference/en-US/reporting.xml 2009-07-31 13:52:43 UTC (rev 3383) +++ tck/trunk/doc/reference/en-US/reporting.xml 2009-07-31 14:07:51 UTC (rev 3384) @@ -8,7 +8,66 @@ chapter also justifies why the TCK is good indicator of how accurately an implementation conforms to the JSR-299 specification. </para> + <section> + <title>CDI TCK Coverage Metrics</title> + + <para> + The CDI TCK coverage has been measured as follows: + </para> + + <itemizedlist> + <listitem> + <para> + <emphasis role="bold">Assertion Breadth Coverage</emphasis> + </para> + <para> + The CDI TCK provides at least 75% coverage of identified + assertions with test cases. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Assertion Breadth Coverage Variance</emphasis> + </para> + <para> + The coverage of specification sub-sections shows at least a + normal distribution (centered around 75%). + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Assertion Depth Coverage</emphasis> + </para> + <para> + The assertion depth coverage has not been measured, as, when an + assertion requires more than one testcase, these have been + enumerated in an assertion group and so are adequately described + by the assertion breadth coverage. + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">Method Coverage</emphasis> + </para> + <para> + TODO + </para> + </listitem> + <listitem> + <para> + <emphasis role="bold">API Signature Coverage</emphasis> + </para> + <para> + The CDI TCK covers 100% of all API public methods using the Java + CTT Sig Test tool. + </para> + </listitem> + </itemizedlist> + + </section> + + <section> <title>CDI TCK Coverage Report</title> <para> A specification can be distilled into a collection of assertions that @@ -66,11 +125,6 @@ }]]></programlisting> <para> - The CDI TCK matches tests to 75% of these assertions, which is - proposed as sufficient coverage to certify a CDI implementation. - </para> - - <para> To help evaluate the distribution of coverage for these assertions, the TCK provides a detailed coverage report. This report is also useful to help implementors match tests with the language in the @@ -151,16 +205,30 @@ <itemizedlist> <listitem> - <para>Covered</para> + <para> + <emphasis role="bold">Covered</emphasis> - a test exists for this + assertion + </para> </listitem> <listitem> - <para>Not covered</para> + <para> + <emphasis role="bold">Not covered</emphasis> - no test exists for this + assertion + </para> </listitem> <listitem> - <para>Unimplemented</para> + <para> + <emphasis role="bold">Problematic</emphasis> - a test exists, but is + currently disabled. For example, this may be because the test + is under development + </para> </listitem> <listitem> - <para>Untestable</para> + <para> + <emphasis role="bold">Untestable</emphasis> - the assertion has been + deemed untestable, a note, explaining why, is normally + provided + </para> </listitem> </itemizedlist> @@ -180,11 +248,10 @@ <title>TestNG Reports</title> <para> - As you know well by now, the CDI TCK test suite is really just a - TestNG test suite. That means an execution of the CDI TCK test - suite produces all the same reports that TestNG produces. This - section will go over those reports and show you were to go - to find each of them. + As you by now, the CDI TCK test suite is really just a TestNG test + suite. That means an execution of the CDI TCK test suite produces + all the same reports that TestNG produces. This section will go over + those reports and show you were to go to find each of them. </para> <section>