[jbosscache-dev] JBossCache 2.0 Beta2 test code coverage analysis

[Manik Surtani]

Hany Mesha wrote:
> Hi all,
> 
> I have run clover once again. This time against the head branch. I did
> 2 data points. One was against right before 2.0 Beta1 and the newer one
> was done yesterday just before 2.0 Beta2. I used the ant target in the
> build all-functionaltests to instrument and capture the code. The first
> data point was done on the test suite where the second data point was
> performed on the test suite and the source code to be able to see how
> much of the source code is actually exercised by the test suite. The
> first data point resulted in 91.8% code coverage where as the second
> data point resulted in 79.4%.
> 
> I also did a gap analysis on the second data point to identify what
> test cases and what source files need to be exercised better to improve
> the overall coverage. I'm targeting a jump from 79.4 to 90 % before
> 2.0GA.
> 
> I invite you to  take a look at the following 2  wiki pages for both
> data points and see what classes and test cases need to provide more
> coverage: 
> Data point 1 (pre beta1) :
> http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCacheHeadCodeCoverageAnalysis
> Data Point 2 (pre beta2) :
> http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCache20Beta2CodeCoverageAnalysis
> 
> 
> On the wiki pages, you'll see the log of my command line test run
> (worth noting that I had 14 test cases reported one or more failure or
> error). Also, you'll only see the overall summary graph presented. If
> you'd like to see the graph details, you can download the detailed HTML
> report attached to the wiki page (*_html.zip) and unzip to drill down to
> a line level of the test cases or classes.
> 
> Finally, I have worked a nice ant build targets for clover that I'd
> like to add to the build script if everyone is OK with that. Those build
> targets provide clover instrumentation and various reporting formats.
> They require adding clover.jar and clover.license to ant-dist directory.
> If you don't prefer adding the files for legal reason, I can just add
> the targets and they're manual targets anyway and should not interfere
> with the automated build but they will be there for future data coverage
> reporting.
> 
> Please let me know you comments and thoughts.
> 
> Cheers,
> 
> Hany
> 
> 
> 
> 
> 
> Hany M. Mesha
> Sr. Software Engineer, Consultant
> Novell Identity Management Engineering
> Toronto, Canada
> hmesha at novell.com
> Mobile: 416-456-6945
> skype: hanymesha
> 
> Novell, Inc.
> SUSE® Linux Enterprise 10
> Your Linux is ready
> http://www.novell.com/linux 
> 
> 
> 
> _______________________________________________
> jbosscache-dev mailing list
> jbosscache-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosscache-dev

Thanks for this, Hany.  This is a useful analysis (insofar as code 
coverage can be considered a useful thing).  It does clearly show areas 
where we do need better coverage (rmi cache loaders, JMX, some pojo 
cache features).  Some of the "red" on the report can be misleading 
though - such as o.j.c.marshall.data (which contain just dummy classes 
for unit tests).

In terms of targets, I think we can only feasibly consider improving the 
test suite after BETA2, so it would make good sense to run the code 
coverage analysis again just before releasing CR1.

Cheers,
-- 
Manik Surtani
Project Lead, JBoss Cache

JBoss, a division of Red Hat

http://labs.jboss.com/jbosscache

Phone: +44 7786 702 706
Email: manik at jboss.com
Skype/Yahoo/AOL: maniksurtani
IRC: manik
MSN: manik at surtani.org
--