[infinispan-dev] Let's stop with pull requests

[Sanne Grinovero]

On 30 May 2012 13:36, Manik Surtani <manik at jboss.org> wrote:
> Well, one problem leads to another, as you are well aware.  Flaky parallel suite leads careless commits.  Yes, we should fix what is broken at the moment but that is not enough since it will get this way again unless we have a stable suite that can be used to ensure quality moving fwd.  The options are:
>
> 1.  We always use the sequential suite.
> 2.  We identify tests that are reliable with the parallel suite (should be at least 90% of the tests IMO) and use this, leaving the sequential suite to BuildHive.

Ah, now I understand. You want to have some tests executed only when
run sequentially.. that looks like a nice compromise, cheers!
that will also speed up our test runs, as not all tests are executed;
we could exclude also some of the slower ones even if they could work
fine in parallel.

Sanne

>
> My pref is #2 but that requires identifying the tests that flake out with the parallel suite.
>
>
>
> On 30 May 2012, at 13:25, Sanne Grinovero wrote:
>
>> It's not really about "which" test mode we should use, as long as
>> people check their commits and have any test that gives enough
>> confidence, which is still not the case even during this discussion:
>>
>> last build Infinispan-master-JDK6-tcp-NON_PARALLEL : 123 failures / +95 failures
>>
>> We need to use non-parallel if that's the only choice we currently
>> have, until parallel is fixed too. I agree that's time consuming, but
>> that's exactly why I'm worried about the fact we leave this go this
>> far. Also it doesn't really take 2 hours anymore, and slow tests can
>> be speed up in many ways.. yes that's an effort, but it will pay off.
>>
>> Cheers,
>> Sanne
>>
>>
>>
>> On 30 May 2012 13:00, Dan Berindei <dan.berindei at gmail.com> wrote:
>>> On Tue, May 29, 2012 at 1:48 PM, Manik Surtani <manik at jboss.org> wrote:
>>>> I pretty much agree with this; and here's a bit of history.
>>>>
>>>> For the large part we have had a stable test suite, but the occasional unpredictability in the suite came in when we introduced the parallel test runner, to allow us to run the (core) suite in under 5 minutes - a suite which otherwise took over 2 hours when run sequentially.
>>>>
>>>> We could revert back to just using the sequential test runner if people prefer that - it makes the suite run more predictably and hence easier to debug and maintain - but the drawback is, well, it takes 2 hours to run.
>>>>
>>>> Perhaps we should use the parallel suite as a "smoke test", but in the event of any failures, revert to a run using the sequential suite?
>>>>
>>>
>>> -1, a smoke test should be something that is not only faster but
>>> always passes, so we could run that on each pull req. Getting a FAIL
>>> from buildhive on each pull request would get tiring real quick.
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik at jboss.org
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev