[JBoss JIRA] (JGRP-2380) Sometimes cluster members are not discovered when using TCPGOSSIP
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2380?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2380:
--------------------------------
I can't reproduce this (on master). I tried to start the GossipRouter with {{-nio false}} and {{-nio true}}, and it worked in both cases.
If you can post instructions on how to reproduce this, that would be helpful.
A few questions/comments/things to do:
* As I said before, try this with the latest stable release (4.1.5, released this week)
* Try to increase GMS.join_timeout, to see if this helps
* You're apparently using FlagsUUID ({{ ROG (flags=2)(_V=4.0)}}), what for? Are you mixing regular UUIDs and FlagsUUIDs?
* I'm seeing view ids of {{65768}} in your logs, this is very unusual, and only happens when you have a large cluster, or a high rate of joines and leaves.
> Sometimes cluster members are not discovered when using TCPGOSSIP
> -----------------------------------------------------------------
>
> Key: JGRP-2380
> URL: https://issues.jboss.org/browse/JGRP-2380
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 4.0.19
> Reporter: Pavlo Fedyna
> Assignee: Bela Ban
> Priority: Minor
> Fix For: 4.1.5
>
> Attachments: jgroups.xml, logs_failure.txt, logs_success.txt
>
>
> Sometimes new member can't join existing cluster if TCPGOSSIP is used with use_nio property set to true. In such case new member creates its own cluster with only one member of itself. After some period of time MERGE3 protocol merges these two clusters into one, but if min_interval/max_interval values are large, it may take a while.
> For some reason, first try of initial discovery always finishes due to join_timeout. In this case only a few members are discovered with no coordinator.
> If we are lucky enough, GMS prints following log message: "I (WO-KIT-967-28892) am not the first of the nodes, waiting for another client to become coordinator" and makes second attempt to join cluster which now takes a few milliseconds and succeeds (see logs_success.txt). In case of failure, GMS prints "I (WO-KIT-967-14786) am the first of the nodes, will become coordinator" and creates new cluster with only one member (see logs_failure.txt).
> The expectations are that first try of the initial discovery should not fail due to the timeout and it should be as fast as the second one is.
> Workaround: set use_nio to false (or just remove it from the stack configuration)
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (WFLY-11873) JGroups subsystem pbcast.GMS/view attribute only shows the viewId
by Radoslav Husar (Jira)
[ https://issues.jboss.org/browse/WFLY-11873?page=com.atlassian.jira.plugin... ]
Radoslav Husar updated WFLY-11873:
----------------------------------
Fix Version/s: (was: 18.0.0.Final)
> JGroups subsystem pbcast.GMS/view attribute only shows the viewId
> -----------------------------------------------------------------
>
> Key: WFLY-11873
> URL: https://issues.jboss.org/browse/WFLY-11873
> Project: WildFly
> Issue Type: Task
> Components: Clustering
> Affects Versions: 16.0.0.Final
> Reporter: Radoslav Husar
> Assignee: Radoslav Husar
> Priority: Major
> Fix For: 18.0.0.Beta1
>
>
> {code}
> [standalone@127.0.0.1:10190 /] /subsystem=jgroups/channel=ee/protocol=pbcast.GMS:read-resource(include-runtime=true
> {
> "outcome" => "success",
> "result" => {
> "after_creation_hook" => undefined,
> "coord" => false,
> "ergonomics" => true,
> "flush_invoker_class" => undefined,
> "id" => 12,
> "implementation" => "ParticipantGmsImpl",
> "install_view_locally_first" => true,
> "is_leaving" => false,
> "join_timeout" => 3000L,
> "leave_timeout" => 3000L,
> "level" => undefined,
> "local_address" => "node3",
> "log_collect_msgs" => false,
> "log_view_warnings" => true,
> "max_bundling_time" => 50L,
> "max_join_attempts" => 10L,
> "members" => "(2) node2,node3",
> "merge_id" => undefined,
> "merge_in_progress" => false,
> "merge_killer_running" => false,
> "merge_task_running" => false,
> "merge_timeout" => 5000L,
> "num_members" => 2,
> "num_prev_mbrs" => 50,
> "num_prev_views" => 10,
> "number_of_views" => 0,
> "print_local_addr" => false,
> "print_physical_addrs" => true,
> "stats" => false,
> "use_delta_views" => true,
> "use_flush_if_present" => true,
> "view" => "[node2|1]",
> "view_ack_collection_timeout" => 2000L,
> "view_bundling" => true,
> "view_handler_size" => 0,
> "view_handler_suspended" => false
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (WFLY-11873) JGroups subsystem pbcast.GMS/view attribute only shows the viewId
by Radoslav Husar (Jira)
[ https://issues.jboss.org/browse/WFLY-11873?page=com.atlassian.jira.plugin... ]
Radoslav Husar updated WFLY-11873:
----------------------------------
Issue Type: Bug (was: Task)
> JGroups subsystem pbcast.GMS/view attribute only shows the viewId
> -----------------------------------------------------------------
>
> Key: WFLY-11873
> URL: https://issues.jboss.org/browse/WFLY-11873
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 16.0.0.Final
> Reporter: Radoslav Husar
> Assignee: Radoslav Husar
> Priority: Major
> Fix For: 18.0.0.Beta1
>
>
> {code}
> [standalone@127.0.0.1:10190 /] /subsystem=jgroups/channel=ee/protocol=pbcast.GMS:read-resource(include-runtime=true
> {
> "outcome" => "success",
> "result" => {
> "after_creation_hook" => undefined,
> "coord" => false,
> "ergonomics" => true,
> "flush_invoker_class" => undefined,
> "id" => 12,
> "implementation" => "ParticipantGmsImpl",
> "install_view_locally_first" => true,
> "is_leaving" => false,
> "join_timeout" => 3000L,
> "leave_timeout" => 3000L,
> "level" => undefined,
> "local_address" => "node3",
> "log_collect_msgs" => false,
> "log_view_warnings" => true,
> "max_bundling_time" => 50L,
> "max_join_attempts" => 10L,
> "members" => "(2) node2,node3",
> "merge_id" => undefined,
> "merge_in_progress" => false,
> "merge_killer_running" => false,
> "merge_task_running" => false,
> "merge_timeout" => 5000L,
> "num_members" => 2,
> "num_prev_mbrs" => 50,
> "num_prev_views" => 10,
> "number_of_views" => 0,
> "print_local_addr" => false,
> "print_physical_addrs" => true,
> "stats" => false,
> "use_delta_views" => true,
> "use_flush_if_present" => true,
> "view" => "[node2|1]",
> "view_ack_collection_timeout" => 2000L,
> "view_bundling" => true,
> "view_handler_size" => 0,
> "view_handler_suspended" => false
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (WFLY-11873) JGroups subsystem pbcast.GMS/view attribute only shows the viewId
by Radoslav Husar (Jira)
[ https://issues.jboss.org/browse/WFLY-11873?page=com.atlassian.jira.plugin... ]
Radoslav Husar resolved WFLY-11873.
-----------------------------------
Fix Version/s: 18.0.0.Beta1
Resolution: Done
This is fixed for WF18 with JGroups upgrade. Resolving.
> JGroups subsystem pbcast.GMS/view attribute only shows the viewId
> -----------------------------------------------------------------
>
> Key: WFLY-11873
> URL: https://issues.jboss.org/browse/WFLY-11873
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 16.0.0.Final
> Reporter: Radoslav Husar
> Assignee: Radoslav Husar
> Priority: Major
> Fix For: 18.0.0.Beta1
>
>
> {code}
> [standalone@127.0.0.1:10190 /] /subsystem=jgroups/channel=ee/protocol=pbcast.GMS:read-resource(include-runtime=true
> {
> "outcome" => "success",
> "result" => {
> "after_creation_hook" => undefined,
> "coord" => false,
> "ergonomics" => true,
> "flush_invoker_class" => undefined,
> "id" => 12,
> "implementation" => "ParticipantGmsImpl",
> "install_view_locally_first" => true,
> "is_leaving" => false,
> "join_timeout" => 3000L,
> "leave_timeout" => 3000L,
> "level" => undefined,
> "local_address" => "node3",
> "log_collect_msgs" => false,
> "log_view_warnings" => true,
> "max_bundling_time" => 50L,
> "max_join_attempts" => 10L,
> "members" => "(2) node2,node3",
> "merge_id" => undefined,
> "merge_in_progress" => false,
> "merge_killer_running" => false,
> "merge_task_running" => false,
> "merge_timeout" => 5000L,
> "num_members" => 2,
> "num_prev_mbrs" => 50,
> "num_prev_views" => 10,
> "number_of_views" => 0,
> "print_local_addr" => false,
> "print_physical_addrs" => true,
> "stats" => false,
> "use_delta_views" => true,
> "use_flush_if_present" => true,
> "view" => "[node2|1]",
> "view_ack_collection_timeout" => 2000L,
> "view_bundling" => true,
> "view_handler_size" => 0,
> "view_handler_suspended" => false
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (JGRP-2327) UNICAST3: create receiver table when non-first message is received first
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2327?page=com.atlassian.jira.plugin.... ]
Bela Ban resolved JGRP-2327.
----------------------------
Resolution: Done
> UNICAST3: create receiver table when non-first message is received first
> ------------------------------------------------------------------------
>
> Key: JGRP-2327
> URL: https://issues.jboss.org/browse/JGRP-2327
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Priority: Major
> Fix For: 4.1.5
>
>
> * A and B
> * B sends 2 messages to A: B1 and B2
> * A receives B2 _before_ B1 (both B1 and B2 are OOB)
> * The current code has A drop B2 and B1 and send a SEND_FIRST_SEQNO message to B
> * B resends B1, but _not_ B2
> * Retransmission needs to kick in before A receives B2. This might take up to {{xmit_interval * 2}} ms before B2 is retransmitted and delivered
> h4. Scenario (JGRP-2293):
> * A installs a new view
> * B sends a LEAVE-REQ to A (B is leaving, too) on the view installation and a VIEW-ACK for the view. Both messages are unicasts to A and OOB
> * The VIEW-ACK (B2) is received first and dropped, so it will have to be retransmitted
> * This delays the view installation, as A waits for {{view_ack_collection_timeout}} ms until it has received all VIEW-ACKs
> h4. Workaround
> * Set GMS.leave_timeout to a higher value (say 8000ms)
> h4. Solution
> * When B2 is received, and it is not the first message and we don't have a receiver table for B yet, investigate whether we can create the receiver table anyway
> * However, this requires the first seqno from B *to always be 0*
> --> Investigate whether the first seqno in UNICAST3 is always 0, then this solution will work
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months