On May 31, 2012, at 11:40 AM, Sanne Grinovero wrote:
On 31 May 2012 10:33, Galder Zamarreño <galder(a)redhat.com>
wrote:
>
> On May 29, 2012, at 12:48 PM, Manik Surtani wrote:
>
>> I pretty much agree with this; and here's a bit of history.
>>
>> For the large part we have had a stable test suite, but the occasional
unpredictability in the suite came in when we introduced the parallel test runner, to
allow us to run the (core) suite in under 5 minutes - a suite which otherwise took over 2
hours when run sequentially.
>>
>> We could revert back to just using the sequential test runner if people prefer
that - it makes the suite run more predictably and hence easier to debug and maintain -
but the drawback is, well, it takes 2 hours to run.
>>
>> Perhaps we should use the parallel suite as a "smoke test", but in the
event of any failures, revert to a run using the sequential suite?
>
> I did some thinking on this and here's my view:
>
> There're tests that sometimes are sensitive to thread scheduling. We all have
pretty powerful machines and often we won't see these issues, but when we go in CI, we
might see them.
>
> What happens is that CI often uses machines that are less powerful, and if running a
paralell testsuite in less powerful machines, these thread scheduling errors can come up
more often.
>
> One way to solve this would be for individuals to run the testsuite in paralell, and
when we go in CI, run it sequentially.
>
> This is what JDG is doing and trust me, it can highlight different issues in our test
suite (seen it already), but at least, the thread scheduling issues are less common.
>
> Thoughts?
I also thought it was a matter of "powerful", but the opposite way ;-)
Tests fail very often to me, much worse than on CI, and I always ended
up blaming the more powerful environment I'm running them in.
FYI I'm unable to complete any test run on master, testsuite just
hangs. When it succeeds, I always had more failures than what I see on
CI: even while CI has been occasionally happy (like in February), I
never had a fully stable build, even when running it on a single
thread.
Sorry if that wasn't clear, maybe I should complain about it more often :-P
More than complain, indicate the tests that fail for you and provide TRACE logs and we can
have a go at fixing them.
Seriously, I just think we can design the tests better.
We can for sure, and we'll be even more successful if we learn what we're doing
wrong. I.e. if you extend SingleCacheManagerTest and do anything other than creating a
cache manager in createCacheManager, you're asking for trouble… (not saying that how
SingleCacheManagerTest is designed is correct, but knowing the things that can go
wrong...)
@Dan
sure you can reconfigure the CDI pom.xml to override the surefire settings.
Cheers,
Sanne
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache