[infinispan-dev] IRC meeting

[Dan Berindei]

On Tue, May 1, 2012 at 6:00 PM, Bela Ban <bban at redhat.com> wrote:
>
> On 5/1/12 4:39 PM, Dan Berindei wrote:
>> Hi Pedro
>
>> A representative failing test is
>> TotalOrderStateTransfer1PcTest.testSTWithWritingNonTxTest (it's
>> actually using implicit transactions, but it doesn't matter). There
>> are two ways I've seen it fail:
>> 1. It appears that SEQUENCER doesn't start delivering TO messages
>> right away after a node joins. It's not very clear to me why this
>> happens, but I see in the logs (http://pastie.org/3843863) that both
>> nodes receive the prepare message, yet only the SEQUENCER on the
>> originator logs that it delivered the message.
>
>
> This might be caused by [1] or [2]. I haven't looked at [2] yet, but I
> do have a fix for [1] but not yet pushed it to the Git repo. Would you
> like me to send you a temp JGroups JAR with [1] included so you can test
> ? Does this occur frequently and is it reproduceable ?
>
> [1] https://issues.jboss.org/browse/JGRP-1455

This doesn't look like it, state transfer has already completed by the
time these messages are dropped - if JGroups was really discarding the
first message on the joiner, state transfer would have failed. I do
see NAKACK2 discarding some messages, but that's before state transfer
even started so I can ignore those:

2012-05-01 17:18:35,063 TRACE (Incoming-1,ISPN,NodeB-17229)
[org.jgroups.protocols.pbcast.NAKACK2] NodeB-17229: message
NodeA-60100::29 was discarded (not yet server)


> [2] https://issues.jboss.org/browse/JGRP-1458
>

That doesn't seem to be the problem either, I have set a breakpoint in
SEQUENCER and it looks like the messages never get there. I have tried
stepping in from NAKACK in the debugger and I couldn't reproduce the
issue. I also tried logging the message in all the protocols in the
stack (NAKACK2/UNICAST2/STABLE/GMS/RSVP/SEQUENCER) to see which one is
dropping the message but again it was too much overhead and I couldn't
reproduce it.