[infinispan-dev] Let's stop with pull requests

[Sanne Grinovero]

On 29 May 2012 10:24, Galder Zamarreño <galder at redhat.com> wrote:
> BuildHive might help somewhat (when it fully works...) but the real problem testsuites will continue to be there:
>
> 1. Some tests fail randomly, due to concurrency issues or any kind.
> 2. Some tests fail due to differences in environments (CI vs your own machine)
>
> The problem with this tests will continue as long as people are individually tasked to tackle these failures. I'm yet to see people proactively fixing this stuff, without any external intervention. As long as this is like this, we'll always be reacting to things.

+1, that's my point. I've been spending much time on the testsuite,
not only in my areas of know-how, and proactively improving things.
Still, if when I leave the project for a month and when I come back
the situation is worse, I feel like I'm fighting windmills. Worst is,
I feel I'm fighting alone and others don't care. Therefore I highly
appreciated your help last week Galder, it's not about the couple
of tests, it's the feeling of someone helping and caring for it.

>
> Either *we all* behave like Sanne suggests, or we carry on like we've done so far.
>
> And then thing is that there's a balance to be found between chasing down this annoying failures, and moving the project forward with new functionality, helping paying customers, helping community, fixing bugs that affect functionality, evangelise….etc.

I think we all want that, but there is a disagreement on how we think
we should act to reach the common goals. Of course YMMV, but in my
mileage the feeling is that the energy spent around failing tests and
working around broken functionality far out weights my
time in developing new improvements.

My new Lucene CacheLoader is an example; the bill for it:

- 3 hours coding it
- 4 hours figuring out problems with configuring it - problems which
were already highlighted by existing tests
- 4 hours fixing tests and debugging core - yea I'm slow at it as I'm
not familiar with it
[desisted from fixing it all]
- 1 hour working around all core issues by hacking hardcoded configuration
- 2 hours testing it: it works for me (but can't contribute it back as
configuration is hacked) and performance is great: much better than
using our other CacheLoaders.

Advantage: ZERO as I can't deliver the value. More especially, I just
had a couple of hours to make this and I'm long overspilled the
timebox (which started in my last weekend anyway), so since I have now
to work on more urgent things, this situation will likely lead to a
couple of months delay in delivering the feature.

On the "motivation" side of things, this also learns me that I'll now
better were to contribute my free time nice ideas in the next weekend.

Conclusion: yes there is a balance. It might work for someone who is
working on Infinispan full time and able to follow all issues and is
familiar with all changes. It doesn't work at all for other people.

Lastly, my position is that tests are far more important than our
implementation code; implementation is rewritten all the time, but I
need the solid testsuite to tell if I'm doing right and make sure our
continual rewriting of internal components make *progress* rather than
pushing evolution back. As I said, we are having a team of impressive
people, and each is having great ideas rewriting things, but nobody
writes perfect code so if we stick with "rewriting" components and
internal services every time to address some performance/functional
problem, we also introduce new bugs. The value in which the project
makes real progress, is the improvement of the tests getting more
solid around all things we fix.

I had this plan to rewrite the FilesystemCacheStore, how would I do
it? it needs a full re-design, so reading the current code won't help
much, but I have some cool ideas I could try out in a weekend,
especially since I have performance tests ready.
Still to want to take advantage of the bugs resolved in the past, I'd
expect to inherit a solid testsuite around it which would enable me to
focus on performance. Still, I'm not going to start this task because
the testsuite is not instilling me enough confidence to be able to
fearlessly address my quest; result once more: you won't get my
patches, not this summer at least.

Just saying, this balance you mention is correct, but is very delicate
and feels very differently for part-time contributors, and I guess
much worse even for potential contributors.

Cheers,
Sanne