[wildfly-dev] IMPORTANT - Intermittent Test Ban Redux

[Tomaž Cerar]

+1000

Also when we will have stable testsuite, we can have PR testing against
multiple platforms, so you could get linux & windows results directly on PR.
Currently that would just mean lots and lots of wasted hardware resources

--
tomaz


On Thu, Apr 24, 2014 at 12:21 AM, Jason Greene <jason.greene at redhat.com>wrote:

> Hi Everyone,
>
> Almost 2 years ago we agreed to disable all intermittent tests[1];
> however, this is still a major problem today, and is still a major hassle
> for everyone that submits a PR (“retest this please” is now a common part
> of our process).
>
> I think it’s finally time to enforce this policy, and disallow all tests
> that lead to even occasional intermittent failures, even if that leads to
> loss of coverage. The most fundamental need of our test suite, is to catch
> regressions, and right now it’s not really serving that purpose.
>
> So I propose that after 8.1.0.Final is shipped that we immediately disable
> all tests with a failing history, and create a JIRA issue for the component
> author to fix them or the bug that is producing the failure. If after 3
> weeks, a JIRA issue has for this has made no progress, I propose we
> completely remove the test from our codebase.
>
> In addition, I propose that any repaired test be shown to be intermittent
> free using a custom CI job (Tomaz can set them) for several runs before
> accepting them back in.
>
> Thanks!
>
> [1]
> http://wildfly-development.1055759.n5.nabble.com/IMPORTANT-New-Policy-Proposal-No-More-Intermittent-Test-Failures-td5709984.html#a5709993
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20140424/42c8819c/attachment.html