Just an FYI on the status of the 6.0.0.M2 release. Rajesh Rajasekaran, Martin Gencur and the QE folks are doing a great job (thanks!) and found a couple critical issues not exposed by the testsuite. They proved easy to fix, and are fixed, but it's delayed things by a day.

Stan Silvert wrote: The testsuite is there for regression testing.

A couple of thoughts on this: 1) The testsuite needs to run in a reasonable amount of time, else no one will run it (already a problem today). We need to think about ways to get the main testsuite down to under an hour. This likely means eliminating stress tests, which really don't belong. 2) Tests need to be useful and maintainable. We have several tests that are old, false alarms, or brittle. Stan Silvert wrote: > Brian Stansberry wrote: >> Yes and no. >> >> Yes, I definitely agree that the AS testsuite could use beefing up in >> terms of integration testing of some of the stuff the AS brings in. >> > +1 >> High level >> integration tests can catch that by luck, but IMHO much better is closer >> unit testing of the deployer class where the problem was. In general, my >> impression is the AS testsuite is overly reliant on integration testing >> and doesn't have enough low level unit testing. >> > -1 > > The testsuite is there for regression testing. An effective regression > test is one that uncovers a bug. A good regression test is one that > fails. In this case, the integration test did that and it wasn't just > dumb luck. I would submit that as regression tests go, integration > tests find bugs far more often then their low-level unit-testing > counterparts. > > Unit tests are written to pass. The person writing the test also wrote > the code. And good programmers tend to write code that works. > > If done right, unit tests do help a programmer figure out what his API > needs to look like (TDD). They also serve as a sanity check for that > programmer to make sure his small change didn't mess anything up. But > because they are so fine-grained and isolated, they are unlikely to fail > when later run in the testsuite for regression purposes. > > The good test (one more likely to fail), is the one that combines the > work of two or more programmers. That is what integration tests do. We > need more emphasis on integration tests, not less. > > Furthermore, an integration test goes through real use cases. It makes > sure that at the end of the day, the system can actually do something > useful. And that's the best test of all. >> On 02/16/2010 06:26 AM, Dimitris Andreadis wrote: >> >>> For future releases, shouldn't we enhance the scope of our standard testsuite runs to >>> uncover issues earlier? Better testing of components like seam/weld/jopr/rest-easy/etc. >>> >>> Brian Stansberry wrote: >>> >>>> Just an FYI on the status of the 6.0.0.M2 release. Rajesh Rajasekaran, >>>> Martin Gencur and the QE folks are doing a great job (thanks!) and found >>>> a couple critical issues not exposed by the testsuite. They proved easy >>>> to fix, and are fixed, but it's delayed things by a day. >>>> >>>> >>> _______________________________________________ >>> jboss-development mailing list >>> jboss-development(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/jboss-development >>> >> >> >> > > > ------------------------------------------------------------------------ > > _______________________________________________ > jboss-development mailing list > jboss-development(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-development