[infinispan-dev] Let's stop with pull requests

[Sanne Grinovero]

On 31 May 2012 10:33, Galder Zamarreño <galder at redhat.com> wrote:
>
> On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
>
>> I pretty much agree with this; and here's a bit of history.
>>
>> For the large part we have had a stable test suite, but the occasional unpredictability in the suite came in when we introduced the parallel test runner, to allow us to run the (core) suite in under 5 minutes - a suite which otherwise took over 2 hours when run sequentially.
>>
>> We could revert back to just using the sequential test runner if people prefer that - it makes the suite run more predictably and hence easier to debug and maintain - but the drawback is, well, it takes 2 hours to run.
>>
>> Perhaps we should use the parallel suite as a "smoke test", but in the event of any failures, revert to a run using the sequential suite?
>
> I did some thinking on this and here's my view:
>
> There're tests that sometimes are sensitive to thread scheduling. We all have pretty powerful machines and often we won't see these issues, but when we go in CI, we might see them.
>
> What happens is that CI often uses machines that are less powerful, and if running a paralell testsuite in less powerful machines, these thread scheduling errors can come up more often.
>
> One way to solve this would be for individuals to run the testsuite in paralell, and when we go in CI, run it sequentially.
>
> This is what JDG is doing and trust me, it can highlight different issues in our test suite (seen it already), but at least, the thread scheduling issues are less common.
>
> Thoughts?

I also thought it was a matter of "powerful", but the opposite way ;-)
Tests fail very often to me, much worse than on CI, and I always ended
up blaming the more powerful environment I'm running them in.

FYI I'm unable to complete any test run on master, testsuite just
hangs. When it succeeds, I always had more failures than what I see on
CI: even while CI has been occasionally happy (like in February), I
never had a fully stable build, even when running it on a single
thread.

Sorry if that wasn't clear, maybe I should complain about it more often :-P

Seriously, I just think we can design the tests better.

@Dan
sure you can reconfigure the CDI pom.xml to override the surefire settings.

Cheers,
Sanne