[ https://issues.jboss.org/browse/JGRP-1495?page=com.atlassian.jira.plugin.... ] David Hotham updated JGRP-1495: ------------------------------- Description: I first raised this, or something very like it, in JGRP-1468; but it got lost among the other fixes that were made in that issue. I've just seen a case where after a merge-view the following happens at a node W: {noformat} SEQUENCER GMS APPLICATION | | | B (view) | | | ------------>| | | |----------------->| | | |----\ | C (view) | | \ | ------------>| | \ | |----------------->| \ | | | \ | C (msg) | | | | ------------>| | | | |---------------------------|-------->| | | | | | | \ | | | \------->| {noformat} - Initially the view is {B,W} - There's a merge view, in which B and C were the old coordinators - B and C both broadcast the new view {B,C,T,W}. In fact both know the physical address of the member shown above (or we're using UDP multicast, if you like); so the view is sent to this node twice. - B's view arrives on thread Incoming-1, and gets as far as GMS - C's view arrives on thread Incoming-2, and gets as far as GMS (where it is dropped, since it is the same view) - A message from C, prompted by the change in view, arrives on thread Incoming-2 - This overtakes the view on thread Incoming-1, and is delivered to the application - Only then does thread Incoming-1 get to deliver the view to the application ie from the application's point of view: C broadcast the view and then the message; whereas at the node shown above the application received the message and then the view. I think that the fix will simply be to put a lock in SEQUENCER around up_prot.up(evt) in SEQUENCER.deliver(). That way messages will be delivered to the application in the same order as they arrive at SEQUENCER. Edit: updated description for clarity was: I first raised this, or something very like it, in JGRP-1468; but it got lost among the other fixes that were made in that issue. I've just seen a case where after a merge-view the following happens: {noformat} SEQUENCER GMS APPLICATION | | | B (view) | | | ------------>| | | |----------------->| | | |----\ | C (view) | | \ | ------------>| | \ | |----------------->| \ | | | \ | C (msg) | | | | ------------>| | | | |---------------------------|-------->| | | | | | | \ | | | \------->| {noformat} - There's a merge view, in which B and C were the old coordinators - B and C both broadcast the new view. In fact both know the physical address of the member shown above (or we're using UDP multicast, if you like); so the view is sent to this node twice. - B's view arrives on thread Incoming-1, and gets as far as GMS - C's view arrives on thread Incoming-2, and gets as far as GMS (where it is dropped, since it is the same view) - A message from C, prompted by the change in view, arrives on thread Incoming-2 - This overtakes the view on thread Incoming-1, and is delivered to the application - Only then does thread Incoming-1 get to deliver the view to the application ie from the application's point of view: C broadcast the view and then the message; whereas at the node shown above the application received the message and then the view. I think that the fix will simply be to put a lock in SEQUENCER around up_prot.up(evt) in SEQUENCER.deliver(). That way messages will be delivered to the application in the same order as they arrive at SEQUENCER. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1495?page=com.atlassian.jira.plugin.... ] David Hotham commented on JGRP-1495: ------------------------------------ That further thinking convinced me that the _real_ question is: why is C broadcasting a message, when it knows perfectly well that it is not coordinator? It should have forwarded to B (who is coordinator). The answer to that is that the variable 'flushing' may not be set soon enough. I've updated my pull request to fix that instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1495?page=com.atlassian.jira.plugin.... ] David Hotham commented on JGRP-1495: ------------------------------------ Then again: while this fix should be sufficient in the example that I saw, C _might_ have been coordinator in the new view; and then we'd still have seen the same problem. So I think some further fix will also be necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1495?page=com.atlassian.jira.plugin.... ] David Hotham edited comment on JGRP-1495 at 7/24/12 5:06 AM: ------------------------------------------------------------- Perhaps the right place to put a lock is GMS.installView(). SEQUENCER or no SEQUENCER, I think the following bug is possible: - View 1 arrives on Thread 1. Gets a far as installView, but new view is not yet sent up and down the stack - View 2 arrives on Thread 2. It's a newer view, and is fully installed and notified to everyone. - Thread 1 picks up, and sends View 1 up and down the stack. Now all other protocols think that the view is View 1, but it should be View 2. Edit: this seems like a separate bug, so I've raised JGRP-1496. Let's leave JGRP-1495 to cover the issue of C wrongly broadcasting that message instead of forwarding it. was (Author: dimbleby): Perhaps the right place to put a lock is GMS.installView(). SEQUENCER or no SEQUENCER, I think the following bug is possible: - View 1 arrives on Thread 1. Gets a far as installView, but new view is not yet sent up and down the stack - View 2 arrives on Thread 2. It's a newer view, and is fully installed and notified to everyone. - Thread 1 picks up, and sends View 1 up and down the stack. Now all other protocols think that the view is View 1, but it should be View 2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JGRP-1495?page=com.atlassian.jira.plugin.... ] Bela Ban resolved JGRP-1495. ---------------------------- Resolution: Done -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira