[dna-dev] Nightly status of JBoss DNA's compatibility with the JCR API

[Randall Hauch]

BTW, you can also see a high-level summary of the results (and a trend  
graph):

http://hudson.qa.jboss.com/hudson/view/DNA/job/DNA%20JCR%20API%20compatibility%20on%20JDK1.5/org.jboss.dna$dna-jcr-tck/


On Mar 23, 2009, at 11:46 PM, Randall Hauch wrote:

> One of the primary objectives of this ongoing release is to support  
> more of the JCR API, including some of the more basic update  
> features.  As part of this effort, we've needed a way to verify that  
> the parts of our implementation we've been working on are behaving  
> correctly per the specification.  We continue to rely upon our own  
> unit tests, but we've also started using a unit test JAR from the  
> Jackrabbit project, which includes 1000+ unit tests that verify the  
> compatibility of any JCR implementation.
>
> To keep things simple, we wanted to include these tests within our  
> Maven builds, whether those are local builds run by developers or  
> builds run by our continuous integration server.  That means that we  
> only want to include those tests that we're actually passing  
> (otherwise the builds will break).  Once we implement a piece of  
> behavior, we uncomment those tests; any regression is automatically  
> identified.
>
> We've made a lot of progress this way, and it's a great approach for  
> those of us down in the trenches.  However, we'd also like to show  
> at a higher level our progress towards JCR API compliance, and that  
> includes the failures as well as the successes.
>
> Well, to make a long story short, we now have a Hudson build that  
> runs these "TCK unit tests".  Right now, this build is running every  
> Monday-Friday in the wee hours of the morning, and the latest  
> results are always available here:
>
> http://hudson.qa.jboss.com/hudson/view/DNA/job/DNA%20JCR%20API%20compatibility%20on%20JDK1.5/org.jboss.dna$dna-jcr-tck/lastBuild/testReport/org.jboss.dna.jcr/
>
> Hudson's formatting isn't great, but the top of that page shows that  
> we're passing 35% of the tests (actually, it shows that we're  
> failing 683 of the 1045 tests).  However, at the bottom of that page  
> (below all the individual failures) are some more useful statistics,  
> including a breakdown of the status for each compliance level.
>
> Current status:
> - 90.3% pass rate on Level 1
> - 12.1% pass rate on Level 2
>
> Watch this space as we quickly start increasing the Level 2 rate!
>
> Best regards,
>
> Randall
>
>
> P.S.  The TCK unit tests are not included in the Maven builds by  
> default.  To run them locally, specify the "jcr-tck" profile.  For  
> example:
> 	$ mvn -P jcr-tck clean install
>
> _______________________________________________
> dna-dev mailing list
> dna-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/dna-dev