[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban updated JGRP-1461: --------------------------- Fix Version/s: 3.0.10 3.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- Hmm... I think the logic of queueing seqnos in SeqnoTable.Entry is wrong. If we get sequence 5,6,8,7,9 and next_to_receive is 5, then: - 5 and 6 are delivered, next_to_receive=7 - 8 is queued but *delivered* because the seqnos set doesn't have it; this is incorrect ! - 7 is delivered, 8 is removed from seqnos - 9 is delivered, next_to_receive is 10 --> Delivering 8 before 7 is incorrect... however, I don't think this can happen: as SEQUENCER requires a FIFO multicast protocol such as NAKACK or NAKACK2 below it, seqnos will *always* arrive in the right sequence ! What *can* happen though is that a seqno can be delivered twice, e.g. when there's a failover from coord A to B: if C sent seqno 5,6 and 7 to A, and A crashed, then it will resend 5-7 to B. If B broadcasts 5-7, those members that did receive (perhaps) 5, that A managed to broadcast before it dies, will get a duplicate 5 and need to discard it. Because we do receive all seqnos in order, I think we only need to maintain next_to_receive and advance it on message reception. If a seqno is < next_to_receive, we discard it. This way, we can discard the seqnos set. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- I'm making the necessary changes as we speak... I think SeqnoTable should only check if the current seqno is the expected one and if the current seqno is smaller than the next to be delivered (and trash it if true) At some point, I'd like to do a code review of SEQUENCER, would you be interested ? Are you on #jgroups @ irc.freenode.net ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- If SEQUENCER is below the flow control protocols, then forwarding and re-broadcasting won't be subject to flow control. I also got better perf moving it... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- No, we cannot remove seqnos: multiple threads in C could forward seqnos 1,4,3,5,2 to A and A would likely send the broadcasts in that order. So either we order the received seqnos (but how do we know which seqno is the first?), or we have the coordinator assign a global seqno (but how does the sender re-send if the coord crashes ?) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- Re: ordering: If we have 2 message B5 and B6 multicast by B (by the same thread), then B5 will be delivered before B6 everywhere (although messages from other members could sneak in between the 2). This is so because B5 is forwarded to the coordinator (A) via UNICAST. When the send(B5) returns, the unicast message has been assigned a seqno and sent to A. Therefore the send(B6) will get the next seqno in UNICAST. This means that B5 will be delivered by A *before* B6 as UNICAST orders these 2 messages. When processing B5, A broadcasts a message A45. This message is also assigned a seqno that is lower than the seqno assigned form B6 (say it is sent as A50). Therefore we are guaranteed that A46 (B5) is delivered everywhere *before* A50 (B6). When we have different threads sending messages in B concurrently, then the message will be TO-delivered everywhere in the order in which they are assigned seqnos in UNICAST. This is the same behavior as if those messages were sent via NAKACK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban resolved JGRP-1461. ---------------------------- Resolution: Done -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] David Hotham commented on JGRP-1461: ------------------------------------ Isn't this near enough what the SeqnoTable was doing already? ie doesn't reverting d3486c1 leave everything working just fine? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- If we made sure that senders would use monotonically increasing seqnos, then we could re-instate the SeqnoTable... However, this would entail having synchronization around grabing the next seqno and sending it, which would slow down senders (sepeaiclly if we have multiple sender threads)... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- OK, I took the path of less resistance and made sure seqnos are sent as monotonically increasing numbers, so that we only need to check whether a seqno is smaller or equals than the last received seqno. Obliviates the need for a bounded hashset per member; this way we only have 1 Long / member. A quick perf test shows that perf is not worse than the previous impl. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1461?page=com.atlassian.jira.plugin.... ] Bela Ban commented on JGRP-1461: -------------------------------- Hmm, with respect to the same thread sending messages m1 and m2, m1 will always be delivered before m2 because the first send() goes down the stack and assign a seqno in UNICAST that's lower than the one for m2. However, if m1 is sent by thread t1 and m2 by t2, and both are concurrent, then it is undefined which message will be delivered first: the order depends on which messages reaches UNICAST first, but that's fine for concurrent messages. I'll therefore remove the seqno_lock from SEQUENCER and just dish out monotonically increasing *unique* seqnos: say m1 gets #5 and m2 gets #6. If message #6 happens to reach UNICAST first, the forward to the coord means that we'll deliver message m2 *before* m1, but that's fine because the messages were sent concurrently, by different threads. So, as seqnos are now not used to establish order (this is done by forwarding a message to the coord), but only to weed out duplicates, we might see m2 (#6) before m1 (#5). A bounded set at each receiver is used to weed out dupes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira