[jboss-as7-dev] Revisited: Integration TestSuite Organization and Maintenance

[Andrew Lee Rubinger]

On 08/16/2011 12:54 AM, Jason T. Greene wrote:
> The approach might catch a missing dependency, but it will not catch an
> unnecessary dependency. It seems that only human review can get the most
> accurate results. Alternatively, it might be more correct to actually
> write real API verification tests, that does something like look for an
> expected list of packages, and erroring when an unexpected package is
> found (or a missing one).

I think about this all the time.  Unfortunately, you can't catch a 
leaked API unless you test for all possible leaks.  If anyone's got a 
fine idea how to deal with this, I'm all ears.

WRT using the testsuite as a verification mechanism to validate the 
completeness of the API, yes, it's a convenience mechanism.  But in this 
case, convenience is likely to equal correctness over time (implicit 
maintenance).  Take the case of @SecurityDomain or JavaMail, both 
missing from the Spec-API and API.  If we didn't remember to export it, 
we definitely weren't going to remember to test that it was exported. :) 
  Only when we added tests for EJB3 security or something that used 
javax.mail did we catch it (after I moved to a restricted dependency chain).

S,
ALR
-- 
Andrew Lee Rubinger
Senior Software Engineer
JBoss by Red Hat
Twitter: @ALRubinger
http://about.me/alrubinger