On May 30, 2012, at 3:25 PM, Dan Berindei wrote:
On Wed, May 30, 2012 at 4:12 PM, Manik Surtani
<manik(a)jboss.org> wrote:
>
> On 30 May 2012, at 13:55, Sanne Grinovero wrote:
>
>> On 30 May 2012 13:36, Manik Surtani <manik(a)jboss.org> wrote:
>>> Well, one problem leads to another, as you are well aware. Flaky parallel
suite leads careless commits. Yes, we should fix what is broken at the moment but that is
not enough since it will get this way again unless we have a stable suite that can be used
to ensure quality moving fwd. The options are:
>>>
>>> 1. We always use the sequential suite.
>>> 2. We identify tests that are reliable with the parallel suite (should be at
least 90% of the tests IMO) and use this, leaving the sequential suite to BuildHive.
>>
>> Ah, now I understand. You want to have some tests executed only when
>> run sequentially.. that looks like a nice compromise, cheers!
>> that will also speed up our test runs, as not all tests are executed;
>> we could exclude also some of the slower ones even if they could work
>> fine in parallel.
>
> Precisely. No loss in quality, just changing the order of delivery of information to
the reviewer. :)
>
Ok, first on the list of sequential-only tests: everything that uses
Arquillian :)
I was looking at the cdi test failures and it looks like Arquillian
uses a thread-local variable and that is initialized only on the
thread that calls their @BeforeSuite method. So every time
Arquillian.arquillianBefore/AfterClass runs in a different thread, it
throws a NullPointerException.
I see two issues with your plan, though:
1. Buildhive is limited to 15 mins, and a reviewer wouldn't
necessarily wait for 2 hours to integrate a pull request anyway. So
the sequential build would be limited to Jenkins runs.
^ Just got a msg today that Buildhive has changed default timeout to 120 mins. I'll
modify Buildhive soon (ish).
2. How do we select which tests run where? I remember we had to
disable tests precisely because configuring test groups in
testng/surefire didn't work.
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache