[infinispan-dev] Let's stop with pull requests

[Sanne Grinovero]

On 30 May 2012 14:25, Dan Berindei <dan.berindei at gmail.com> wrote:
> On Wed, May 30, 2012 at 4:12 PM, Manik Surtani <manik at jboss.org> wrote:
>>
>> On 30 May 2012, at 13:55, Sanne Grinovero wrote:
>>
>>> On 30 May 2012 13:36, Manik Surtani <manik at jboss.org> wrote:
>>>> Well, one problem leads to another, as you are well aware.  Flaky parallel suite leads careless commits.  Yes, we should fix what is broken at the moment but that is not enough since it will get this way again unless we have a stable suite that can be used to ensure quality moving fwd.  The options are:
>>>>
>>>> 1.  We always use the sequential suite.
>>>> 2.  We identify tests that are reliable with the parallel suite (should be at least 90% of the tests IMO) and use this, leaving the sequential suite to BuildHive.
>>>
>>> Ah, now I understand. You want to have some tests executed only when
>>> run sequentially.. that looks like a nice compromise, cheers!
>>> that will also speed up our test runs, as not all tests are executed;
>>> we could exclude also some of the slower ones even if they could work
>>> fine in parallel.
>>
>> Precisely.  No loss in quality, just changing the order of delivery of information to the reviewer.  :)
>>
>
> Ok, first on the list of sequential-only tests: everything that uses
> Arquillian :)
> I was looking at the cdi test failures and it looks like Arquillian
> uses a thread-local variable and that is initialized only on the
> thread that calls their @BeforeSuite method. So every time
> Arquillian.arquillianBefore/AfterClass runs in a different thread, it
> throws a NullPointerException.
>
> I see two issues with your plan, though:
> 1. Buildhive is limited to 15 mins, and a reviewer wouldn't
> necessarily wait for 2 hours to integrate a pull request anyway. So
> the sequential build would be limited to Jenkins runs.
> 2. How do we select which tests run where? I remember we had to
> disable tests precisely because configuring test groups in
> testng/surefire didn't work.

I wouldn't oppose a good plan just because of some minor technical
difficulties. We can solve those in many ways.
# improve Arquillian
# make-your-own BuildHive by using same integration / have them lift
limitations. (i.e. trust Galder and give him some time..)
# fix/workaround TestNG / switch to JUnit / make your own

Sanne