[infinispan-dev] Let's stop with pull requests

[Dan Berindei]

On Thu, May 31, 2012 at 12:40 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> On 31 May 2012 10:33, Galder Zamarreño <galder at redhat.com> wrote:
>>
>> On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
>>
>>> I pretty much agree with this; and here's a bit of history.
>>>
>>> For the large part we have had a stable test suite, but the occasional unpredictability in the suite came in when we introduced the parallel test runner, to allow us to run the (core) suite in under 5 minutes - a suite which otherwise took over 2 hours when run sequentially.
>>>
>>> We could revert back to just using the sequential test runner if people prefer that - it makes the suite run more predictably and hence easier to debug and maintain - but the drawback is, well, it takes 2 hours to run.
>>>
>>> Perhaps we should use the parallel suite as a "smoke test", but in the event of any failures, revert to a run using the sequential suite?
>>
>> I did some thinking on this and here's my view:
>>
>> There're tests that sometimes are sensitive to thread scheduling. We all have pretty powerful machines and often we won't see these issues, but when we go in CI, we might see them.
>>
>> What happens is that CI often uses machines that are less powerful, and if running a paralell testsuite in less powerful machines, these thread scheduling errors can come up more often.
>>
>> One way to solve this would be for individuals to run the testsuite in paralell, and when we go in CI, run it sequentially.
>>
>> This is what JDG is doing and trust me, it can highlight different issues in our test suite (seen it already), but at least, the thread scheduling issues are less common.
>>
>> Thoughts?
>
> I also thought it was a matter of "powerful", but the opposite way ;-)
> Tests fail very often to me, much worse than on CI, and I always ended
> up blaming the more powerful environment I'm running them in.
>
> FYI I'm unable to complete any test run on master, testsuite just
> hangs. When it succeeds, I always had more failures than what I see on
> CI: even while CI has been occasionally happy (like in February), I
> never had a fully stable build, even when running it on a single
> thread.
>
> Sorry if that wasn't clear, maybe I should complain about it more often :-P
>
> Seriously, I just think we can design the tests better.
>

Do complain more often :)

I got used to running only the core test suite all the time - I rely
on the Jenkins results to check that I haven't broken everything else.

> @Dan
> sure you can reconfigure the CDI pom.xml to override the surefire settings.
>

Adi checked at the CDI pom.xml and surefire actually *was* configured
to run tests sequentially.
I guess the setting was just ignored for a a few builds - now it's
back to running in one thread and there are no CDI tests failing in
the last build.

Cheers
Dan