<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 30 May 2012, at 18:06, Manik Surtani wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 30 May 2012, at 17:22, Dan Berindei wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><br>Yet there are still lots of tests that have both group="manual" and<br>enabled="false", with descriptions like "Disabled until we can<br>configure Surefire to skip manual tests". You can't blame me for<br>thinking the comment is still valid :)<br></span></blockquote><div><br></div><div>Find out who added this and educate them. :-)</div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><br>So now I have a counter-suggestion:<br>1. Add a 'flaky-' prefix to the group name of tests that fail randomly<br>(or always!) instead of disabling them.<br>2. Always run tests in parallel, forget about sequential runs as they<br>take way too long. Many of the changes we'd have to make so that tests<br>take less time sequentially will make it easier for those tests to<br>fail when they run in parallel.<br>3. Configure a separate build in jenkins to run flaky tests as well,<br>let everyone else run only non-flaky ones.<br>4. Create a JIRA for fixing flaky tests in general, create separate<br>JIRAs only where the fix changes production code.<br>5. Profit!</span></blockquote><br></div><div>+1, that's essentially what I have been getting at.</div></div></blockquote></div>+1, this sounds very reasonable. <div><br></div></body></html>