On Jul 10, 2014, at 16:12, Sanne Grinovero <sanne(a)infinispan.org> wrote:
On 10 July 2014 15:13, Mircea Markus <mmarkus(a)redhat.com>
wrote:
>
> On Jul 10, 2014, at 15:03, Sanne Grinovero <sanne(a)infinispan.org> wrote:
>
>> The important point for me is that patches don't get merged if they
>> introduce any regression. I hope that rule stays?
>> BTW this matches with the "classic" approach as far as I know it.
>
> yes. A patch might pass the test when integrated and cause intermittent failures
later on, so it's not straight forward to avoid patches introducing regressions, nor
to identify which patch has caused it so that we can roll it back.
It's self-speaking that it's hard to immediately evaluate if a patch
is going to cause intermittent / time-bound failures in the future: we
have no crystal ball, so I'm not making absurd demands. There will
always be cases in which developers will need to ask forgiveness.
But if anything in the test run fails during a review of a patch, and
this patch still gets merged because "the cause is likely unrelated",
or worse "we'll fix that problem later" that's unacceptable and needs
to be investigated further before being merged.
I always did that, and spending a lot of time on things which are not
directly related on my goals, for sake of respect of other developers
in the team and I expect no less from everyone else: when the
testsuite doesn't pass I can't make progress on my own work and need
to necessarily shift to firefighthing. Or go on holidays.
That's what necessarily needs to happen when a bad patch slips in past
our guard, even if doing so you need to reschedule other tasks:
because otherwise it's other people, and more and more people needing
to reschedule their own tasks. Worse yet, the other people who will
need to look at it will not have any of the context to understand what
might be going on. So I hope we're on the same page with this, because
ultimately it's about respect for each person contributing or working
on it.
I understand your frustration and you are absolutely right.
I'm also puzzled on why exactly the current situation is not
sustainable. Sure there is a lot of technical debt to pay, but you
know debts come with interests and it's hard.I've also suggested many
ways to improve our testsuite to make our life easier, like use
Byteman, mock the timers via a TimeService, get rid of TestNG to move
to something more reliable, remove unnecessary dependencies from each
module, use more mocking in areas where we don't actually have an
interest in testing JGroups..
IMO the problem is that ATM we lack the discipline, as a team, to maintain the suite
green. In the last 7 years I've been working on JBossCache and then ISPN it has always
been like this - intermittent failures - and always to big and too ugly of a task to fix
it and maintain it green, especially with the schedule we had ahead of us. Now that the
team is big, this compromise - intermittent failures - really impacts the productivity of
a lot of people. The suite should stay green and we should treat any failure that keep us
away from that as an blocker - a thing we didn't really do up to now.
Nothing like that was done, so you can't say in all fairness that
we
tried to do better. My PR which is the first step to get rid of TestNG
from the Query modules is open since 43 days.. so don't expect me to
fix any problem quickly, it's not particularly motivating.
You can try playing with processes, and I don't disagree with idea,
but when I'm able to find a regression is is that I will do: I'll
revert all commits until the first stable point I find. I've
suggested this approach to you as well, not sure why you don't apply
it more regularly?
Because the last green commit will trigger intermittent failures, try it.
There is no shame in reverting commits, especially
if we do it regularly.
It's stable today, so I'll make a note of this commit id, my crystal
ball is telling that it will be useful soon ;-)
Sanne
>
>>
>> On 10 July 2014 14:04, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>> I just had a chat with Dan and we don't think the current process for the
test suite works. Not hard to see why, the suite is almost never green. So we will adopt a
more classic and simple approach: if a test fails a blocker JIRA is created for it and
assigned to a component lead then team member who'll start working on it
*immediately*. Dan will be watch dog starting today so please expect blocker JIRAs coming
your way and treat them accordingly.
>>>
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (
www.infinispan.org)
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (
www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Cheers,
--
Mircea Markus
Infinispan lead (
www.infinispan.org)