One related issue:
Our project's CC run takes a minimum of 3 hours to complete, and we
repeat that against each database we support: MySQL, PostgreSQL, MSSQL,
Sybase, Oracle, i.e. 5 combinations.
The cc machine is so overstretched, that it takes about 8 days (!!) to
complete all of them.
This basically makes the whole concept of continuous build irrelevant,
so until QA gets more machines we pretty much ignore those results, and
just make sure we get clean runs before releases.
Andrig T Miller wrote:
To strengthen your argument, I had 385 build messages in my e-mail
and
here is the breakdown:
79 of the 385 or 20.5% were outright build failures
231 of the 385, or 60% were build completed with test suite failures
52 of the 385, or 13.5% were build that successfully completed
11 of the 385, or 3% were builds that timed out (probably
infrastructure related)
12 of the 385, or 3% were builds that were fixed
Over 80% of all builds across all projects failed or had testsuite
failures.
Andy
On Thu, 2007-06-21 at 11:39 -0400, Bill Burke wrote:
> The productivity of teams like EJB 3.0 has been severely compromised for
> over a year because projects. 60% of the reason I didn't want to work
> on EJB 3 anymore was because I would not commit anything for a week or
> two and come back to find that half of my unit tests were broken. This
> has gotten worse and worse as projects have split off from jboss-head.
>
> There is ZERO peer pressure for breaking the build. Nobody cares.
> Since the testsuites *ALWAYS* fail, nobody is paying attention to
> regressions except for the individual projects where it fails. You even
> have cases where people comment out failing tests! We just *cannot* do
> the refactorings that the majority of teams want to do without a stable
> testsuite.
>
> This has to be fixed immediately. Since there is ZERO peer pressure,
> there needs to be consequences for breaking the build or regressing. I
> propose the following:
>
> 1. Calculate a baseline of passed vs. failed tests
> 2. Tag HEAD
> 3. If there is any testsuite regression or build breakage, freeze SVN
> until the build or regressions are fixed.
> 4. If build or regressions are fixed within 24 hours. Rollback to
> previous tag
> 5. If no regressions or breakage, recalculate baseline and tag head
> 6. GOTO 3
>
> Bill
>
> Dimitris Andreadis wrote:
> > Given the multitude of component updates and our carelessness, it's a
> > rare thing to see jboss AS testsuites run at 100%, A lot of stuff is
> > just checked in without testing it locally, and worse, the CC runs that
> > show failures after the check-ins are ignored.
> >
> > The good new is we finally got there for 4.2.x. (minus
> > occasional/transient timing failures), so let's keep it that way for
> > that branch, and help fix the other ones, especially HEAD.
> >
> > On a broken testsuite it's very "convenient" to just ignore
failures,
> > since "somebody else must have done it".
> >
> > Cheers
> > /Dimitris
> >
> > ===========================
> >
> > View results here ->
> >
http://cruisecontrol.jboss.com/cc/buildresults/jboss-4.2-testsuite-sun-1....
> >
> >
> > BUILD COMPLETE - build.48
> > Date of build: 06/20/2007 22:31:06
> > Time to build: 165 minutes 8 seconds
> >
> > Unit Tests: (4087) Total Errors and Failures: (0)
> > All Tests Passed
> > _______________________________________________
> > jboss-development mailing list
> > jboss-development(a)lists.jboss.org
<mailto:jboss-development@lists.jboss.org>
> >
https://lists.jboss.org/mailman/listinfo/jboss-development
>
>
Andrig (Andy) Miller
VP of Engineering
JBoss, a division of Red Hat
------------------------------------------------------------------------
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development