<div dir="ltr"><div><div><div>+1000<br><br></div>Also when we will have stable testsuite, we can have PR testing against multiple platforms, so you could get linux & windows results directly on PR.<br></div>Currently that would just mean lots and lots of wasted hardware resources <br>
<br>--<br></div>tomaz<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 24, 2014 at 12:21 AM, Jason Greene <span dir="ltr"><<a href="mailto:jason.greene@redhat.com" target="_blank">jason.greene@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Everyone,<br>
<br>
Almost 2 years ago we agreed to disable all intermittent tests[1]; however, this is still a major problem today, and is still a major hassle for everyone that submits a PR (“retest this please” is now a common part of our process).<br>
<br>
I think it’s finally time to enforce this policy, and disallow all tests that lead to even occasional intermittent failures, even if that leads to loss of coverage. The most fundamental need of our test suite, is to catch regressions, and right now it’s not really serving that purpose.<br>
<br>
So I propose that after 8.1.0.Final is shipped that we immediately disable all tests with a failing history, and create a JIRA issue for the component author to fix them or the bug that is producing the failure. If after 3 weeks, a JIRA issue has for this has made no progress, I propose we completely remove the test from our codebase.<br>
<br>
In addition, I propose that any repaired test be shown to be intermittent free using a custom CI job (Tomaz can set them) for several runs before accepting them back in.<br>
<br>
Thanks!<br>
<br>
[1] <a href="http://wildfly-development.1055759.n5.nabble.com/IMPORTANT-New-Policy-Proposal-No-More-Intermittent-Test-Failures-td5709984.html#a5709993" target="_blank">http://wildfly-development.1055759.n5.nabble.com/IMPORTANT-New-Policy-Proposal-No-More-Intermittent-Test-Failures-td5709984.html#a5709993</a><br>
--<br>
Jason T. Greene<br>
WildFly Lead / JBoss EAP Platform Architect<br>
JBoss, a division of Red Hat<br>
<br>
<br>
_______________________________________________<br>
wildfly-dev mailing list<br>
<a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a><br>
</blockquote></div><br></div>