[jboss-jira] [JBoss JIRA] (JGRP-2197) Merge doesn't work properly

[Allen Zhao (JIRA)]

Allen Zhao created JGRP-2197:
--------------------------------

             Summary: Merge doesn't work properly
                 Key: JGRP-2197
                 URL: https://issues.jboss.org/browse/JGRP-2197
             Project: JGroups
          Issue Type: Bug
    Affects Versions: 3.2.4
            Reporter: Allen Zhao
            Assignee: Bela Ban
            Priority: Critical
         Attachments: jgroupsConfig.xml

Using the attached jGroups configuration file and sample code based on the Draw example, run two MergeTest instances alanz-dev-54589 and  alanz-dev-53593 on one machine, and another instance , AllenMAC-4773 on the second machine; both machines are in the same network connected by a cable.

Initially all three instances are in the same group, then unplug the cable from the second machine, two groups formed as expected: 
        group1:  alanz-dev-54589 and  alanz-dev-53593
        group2: AllenMAC-4773
Then plug the cable into the second machine again, the groups merged into one group with all the three members.

These are good as expected. Keeping doing these by unplug/plug the cable into the second machine, it took around 5 times until the following happened, which is unexpected:
when AllenMAC-4773 merged into the group composed by alanz-dev-54589 and  alanz-dev-53593, it kicked  alanz-dev-54589 out, and the  two groups formed: MergeView::[alanz-dev-53593|14] (2) [alanz-dev-53593, AllenMAC-4773], and alanz-dev-54589 formed a group by itself.

Please see the sample code output from the two instances run on the first machine as follows:


==============alanz-dev-54589===================
-------------------------------------------------------------------
GMS: address=alanz-dev-54589, cluster=draw-cluster, physical address=fe80:0:0:0:1c97:64f4:50a0:20b0%11:63608
-------------------------------------------------------------------
** View=[alanz-dev-54589|0] (1) [alanz-dev-54589]
** View=[alanz-dev-54589|1] (2) [alanz-dev-54589, AllenMAC-4773]
** View=[alanz-dev-54589|2] (3) [alanz-dev-54589, AllenMAC-4773, alanz-dev-53593]
** View=[alanz-dev-54589|3] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|4] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|3] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|3] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|5] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|6] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|5] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|5] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|7] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|8] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|7] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|7] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|9] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|10] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|9] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|9] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|11] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|12] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|11] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|11] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|13] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-53593|15] (3) [alanz-dev-53593, AllenMAC-4773, alanz-dev-54589], 2 subgroups: [alanz-dev-53593|14] (2) [alanz-dev-53593, AllenMAC-4773], [alanz-dev-54589|13] (1) [alanz-dev-54589]




================alanz-dev-53593=======================
---------------------------------------------------------------------
GMS: address=alanz-dev-53593, cluster=draw-cluster, physical address=fe80:0:0:0:1c97:64f4:50a0:20b0%11:63610
-------------------------------------------------------------------
** View=[alanz-dev-54589|2] (3) [alanz-dev-54589, AllenMAC-4773, alanz-dev-53593]
** View=[alanz-dev-54589|3] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|4] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|3] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|3] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|5] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|6] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|5] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|5] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|7] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|8] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|7] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|7] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|9] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|10] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|9] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|9] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|11] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-54589|12] (3) [alanz-dev-54589, alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|11] (2) [alanz-dev-54589, alanz-dev-53593], [AllenMAC-4773|11] (1) [AllenMAC-4773]
** View=[alanz-dev-54589|13] (2) [alanz-dev-54589, alanz-dev-53593]
** MergeView::[alanz-dev-53593|14] (2) [alanz-dev-53593, AllenMAC-4773], 2 subgroups: [alanz-dev-54589|13] (1) [alanz-dev-53593], [AllenMAC-4773|13] (1) [AllenMAC-4773]
** MergeView::[alanz-dev-53593|15] (3) [alanz-dev-53593, AllenMAC-4773, alanz-dev-54589], 2 subgroups: [alanz-dev-53593|14] (2) [alanz-dev-53593, AllenMAC-4773], [alanz-dev-54589|13] (1) [alanz-dev-54589]


The issue happened in our system which uses jGroups for data replication. I turned on all jGroups log, and got some useful log information as the attached. It seems that at some point, a merge request from AllenMAC-4773 was received by alanz-dev-53593, and in that message, mbrs only contained alanz-dev-53593.




--
This message was sent by Atlassian JIRA
(v7.2.3#72005)