[wildfly-dev] IMPORTANT - Intermittent Test Ban Redux

[James R. Perkins]

+1

On Wed, Apr 23, 2014 at 3:21 PM, Jason Greene <jason.greene at redhat.com> 
wrote:
> Hi Everyone,
> 
> Almost 2 years ago we agreed to disable all intermittent tests[1]; 
> however, this is still a major problem today, and is still a major 
> hassle for everyone that submits a PR (“retest this please” is 
> now a common part of our process).
> 
> I think it’s finally time to enforce this policy, and disallow all 
> tests that lead to even occasional intermittent failures, even if 
> that leads to loss of coverage. The most fundamental need of our test 
> suite, is to catch regressions, and right now it’s not really 
> serving that purpose.
> 
> So I propose that after 8.1.0.Final is shipped that we immediately 
> disable all tests with a failing history, and create a JIRA issue for 
> the component author to fix them or the bug that is producing the 
> failure. If after 3 weeks, a JIRA issue has for this has made no 
> progress, I propose we completely remove the test from our codebase. 
> 
> In addition, I propose that any repaired test be shown to be 
> intermittent free using a custom CI job (Tomaz can set them) for 
> several runs before accepting them back in.
> 
> Thanks!
> 
> [1] 
> http://wildfly-development.1055759.n5.nabble.com/IMPORTANT-New-Policy-Proposal-No-More-Intermittent-Test-Failures-td5709984.html#a5709993
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
> 
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20140428/dc0aade9/attachment.html