Brian Stansberry wrote: 
Yes and no.

Yes, I definitely agree that the AS testsuite could use beefing up in 
terms of integration testing of some of the stuff the AS brings in.
+1
High level 
integration tests can catch that by luck, but IMHO much better is closer 
unit testing of the deployer class where the problem was. In general, my 
impression is the AS testsuite is overly reliant on integration testing 
and doesn't have enough low level unit testing.
-1

The testsuite is there for regression testing.  An effective regression test is one that uncovers a bug.  A good regression test is one that fails.  In this case, the integration test did that and it wasn't just dumb luck.  I would submit that as regression tests go, integration tests find bugs far more often then their low-level unit-testing counterparts.

Unit tests are written to pass.  The person writing the test also wrote the code.  And good programmers tend to write code that works.

If done right, unit tests do help a programmer figure out what his API needs to look like (TDD).  They also serve as a sanity check for that programmer to make sure his small change didn't mess anything up.  But because they are so fine-grained and isolated, they are unlikely to fail when later run in the testsuite for regression purposes.

The good test (one more likely to fail), is the one that combines the work of two or more programmers.  That is what integration tests do.  We need more emphasis on integration tests, not less.

Furthermore, an integration test goes through real use cases.  It makes sure that at the end of the day, the system can actually do something useful.  And that's the best test of all.
On 02/16/2010 06:26 AM, Dimitris Andreadis wrote:
  
For future releases, shouldn't we enhance the scope of our standard testsuite runs to
uncover issues earlier? Better testing of components like seam/weld/jopr/rest-easy/etc.

Brian Stansberry wrote:
    
Just an FYI on the status of the 6.0.0.M2 release. Rajesh Rajasekaran,
Martin Gencur and the QE folks are doing a great job (thanks!) and found
a couple critical issues not exposed by the testsuite. They proved easy
to fix, and are fixed, but it's delayed things by a day.

      
_______________________________________________
jboss-development mailing list
jboss-development@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development