UPDATED: Nightly status of JBoss DNA's compatibility with the JCR API
by Randall Hauch
UPDATE: The bad links in the previous email have been corrected.
One of the primary objectives of this ongoing release is to support
more of the JCR API, including some of the more basic update
features. As part of this effort, we've needed a way to verify that
the parts of our implementation we've been working on are behaving
correctly per the specification. We continue to rely upon our own
unit tests, but we've also started using a unit test JAR from the
Jackrabbit project, which includes 1000+ unit tests that verify the
compatibility of any JCR implementation.
To keep things simple, we wanted to include these tests within our
Maven builds, whether those are local builds run by developers or
builds run by our continuous integration server. That means that we
only want to include those tests that we're actually passing
(otherwise the builds will break). Once we implement a piece of
behavior, we uncomment those tests; any regression is automatically
identified.
We've made a lot of progress this way, and it's a great approach for
those of us down in the trenches. However, we'd also like to show at
a higher level our progress towards JCR API compliance, and that
includes the failures as well as the successes.
Well, to make a long story short, we now have a Hudson build that runs
these "TCK unit tests". Right now, this build is running every Monday-
Friday in the wee hours of the morning, and the latest results are
always available here:
http://hudson.jboss.org/hudson/job/DNA%20JCR%20API%20compatibility%20on%2...
Hudson's formatting isn't great, but the top of that page shows that
we're passing 35% of the tests (actually, it shows that we're failing
683 of the 1045 tests). However, at the bottom of that page (below
all the individual failures) are some more useful statistics,
including a breakdown of the status for each compliance level.
Current status:
- 90.3% pass rate on Level 1
- 12.1% pass rate on Level 2
You can also see a higher-level trend graph here:
http://hudson.jboss.org/hudson/job/DNA%20JCR%20API%20compatibility%20on%2...
Watch this space as we quickly start increasing the Level 2 rate!
Best regards,
Randall
P.S. The TCK unit tests are not included in the Maven builds by
default. To run them locally, you have to specify the "jcr-tck"
profile. For example:
$ mvn -P jcr-tck clean install
15 years, 1 month
Nightly status of JBoss DNA's compatibility with the JCR API
by Randall Hauch
One of the primary objectives of this ongoing release is to support
more of the JCR API, including some of the more basic update
features. As part of this effort, we've needed a way to verify that
the parts of our implementation we've been working on are behaving
correctly per the specification. We continue to rely upon our own
unit tests, but we've also started using a unit test JAR from the
Jackrabbit project, which includes 1000+ unit tests that verify the
compatibility of any JCR implementation.
To keep things simple, we wanted to include these tests within our
Maven builds, whether those are local builds run by developers or
builds run by our continuous integration server. That means that we
only want to include those tests that we're actually passing
(otherwise the builds will break). Once we implement a piece of
behavior, we uncomment those tests; any regression is automatically
identified.
We've made a lot of progress this way, and it's a great approach for
those of us down in the trenches. However, we'd also like to show at
a higher level our progress towards JCR API compliance, and that
includes the failures as well as the successes.
Well, to make a long story short, we now have a Hudson build that runs
these "TCK unit tests". Right now, this build is running every Monday-
Friday in the wee hours of the morning, and the latest results are
always available here:
http://hudson.qa.jboss.com/hudson/view/DNA/job/DNA%20JCR%20API%20compatib...
Hudson's formatting isn't great, but the top of that page shows that
we're passing 35% of the tests (actually, it shows that we're failing
683 of the 1045 tests). However, at the bottom of that page (below
all the individual failures) are some more useful statistics,
including a breakdown of the status for each compliance level.
Current status:
- 90.3% pass rate on Level 1
- 12.1% pass rate on Level 2
Watch this space as we quickly start increasing the Level 2 rate!
Best regards,
Randall
P.S. The TCK unit tests are not included in the Maven builds by
default. To run them locally, specify the "jcr-tck" profile. For
example:
$ mvn -P jcr-tck clean install
15 years, 1 month
