[dna-issues] [JBoss JIRA] Commented: (DNA-306) Add ability to run all TCK unit tests within Hudson, showing success and failure rate

[Randall Hauch (JIRA)]

    [ https://jira.jboss.org/jira/browse/DNA-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12458700#action_12458700 ] 

Randall Hauch commented on DNA-306:
-----------------------------------

Added a Hudson build to run every Monday-Friday in the wee hours of the morning.  The latest results are always available here:

http://hudson.qa.jboss.com/hudson/view/DNA/job/DNA%20JCR%20API%20compatibility%20on%20JDK1.5/org.jboss.dna$dna-jcr-tck/lastBuild/testReport/org.jboss.dna.jcr/

Hudson's formatting isn't great, but the top of that page shows that we're passing 35% of the tests (actually, it shows that we're failing 683 of the 1045 tests).  However, at the bottom of that page (below all the individual failures) are some more useful statistics, including a breakdown of the status for each compliance level.

Current status:
- 90.3% pass rate on Level 1
- 12.1% pass rate on Level 2

> Add ability to run all TCK unit tests within Hudson, showing success and failure rate
> -------------------------------------------------------------------------------------
>
>                 Key: DNA-306
>                 URL: https://jira.jboss.org/jira/browse/DNA-306
>             Project: DNA
>          Issue Type: Task
>          Components: Development Environment, Testing
>    Affects Versions: 0.3
>            Reporter: Randall Hauch
>            Priority: Minor
>             Fix For: 0.4
>
>         Attachments: DNA-306.patch
>
>
> Currently our use of the JCR TCK unit tests (again, not the official TCK but a subset of what's run within the TCK) is handled within a JUnit test suite in the "dna-jcr" project, and as we develop new features we're uncommenting tests as we pass them.  This does not give us any information about how we're progressing.  We want to run the tests (perhaps nightly) and make these stats available to let people know how we're doing and proceeding towards Level 1 and Level 2 compatibility.
> Comments, thoughts, and approaches welcome.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira