Well, one problem leads to another, as you are well aware. Flaky parallel suite leads
careless commits. Yes, we should fix what is broken at the moment but that is not enough
since it will get this way again unless we have a stable suite that can be used to ensure
quality moving fwd. The options are:
1. We always use the sequential suite.
2. We identify tests that are reliable with the parallel suite (should be at least 90% of
the tests IMO) and use this, leaving the sequential suite to BuildHive.
My pref is #2 but that requires identifying the tests that flake out with the parallel
suite.
On 30 May 2012, at 13:25, Sanne Grinovero wrote:
It's not really about "which" test mode we should use,
as long as
people check their commits and have any test that gives enough
confidence, which is still not the case even during this discussion:
last build Infinispan-master-JDK6-tcp-NON_PARALLEL : 123 failures / +95 failures
We need to use non-parallel if that's the only choice we currently
have, until parallel is fixed too. I agree that's time consuming, but
that's exactly why I'm worried about the fact we leave this go this
far. Also it doesn't really take 2 hours anymore, and slow tests can
be speed up in many ways.. yes that's an effort, but it will pay off.
Cheers,
Sanne
On 30 May 2012 13:00, Dan Berindei <dan.berindei(a)gmail.com> wrote:
> On Tue, May 29, 2012 at 1:48 PM, Manik Surtani <manik(a)jboss.org> wrote:
>> I pretty much agree with this; and here's a bit of history.
>>
>> For the large part we have had a stable test suite, but the occasional
unpredictability in the suite came in when we introduced the parallel test runner, to
allow us to run the (core) suite in under 5 minutes - a suite which otherwise took over 2
hours when run sequentially.
>>
>> We could revert back to just using the sequential test runner if people prefer
that - it makes the suite run more predictably and hence easier to debug and maintain -
but the drawback is, well, it takes 2 hours to run.
>>
>> Perhaps we should use the parallel suite as a "smoke test", but in the
event of any failures, revert to a run using the sequential suite?
>>
>
> -1, a smoke test should be something that is not only faster but
> always passes, so we could run that on each pull req. Getting a FAIL
> from buildhive on each pull request would get tiring real quick.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev