[jboss-jira] [JBoss JIRA] Commented: (JGRP-706) with dynamic keys in ENCRYPT, a node taking over coordinator can block itself from taking over the view

Tue Mar 11 11:24:58 EDT 2008

    [ http://jira.jboss.com/jira/browse/JGRP-706?page=comments#action_12402152 ] 

Gray Watson commented on JGRP-706:
----------------------------------

The most recent ENCRYPT.java.patch file seems to have resolved the issue.  It makes the assumption that if ENCRYPT gets notified of a TMP_VIEW, the node is trying to become a coordinator so it should assume keyserver duties immediately.  This is working well for us given that our aggressive start/kill script will now run for hours on our staging cluster.

> with dynamic keys in ENCRYPT, a node taking over coordinator can block itself from taking over the view
> -------------------------------------------------------------------------------------------------------
>
>                 Key: JGRP-706
>                 URL: http://jira.jboss.com/jira/browse/JGRP-706
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 2.6.1
>         Environment: Linux test1a 2.6.18-8.el5 #1 SMP Thu Mar 15 19:46:53 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
> CentOS.
>            Reporter: Gray Watson
>         Assigned To: Bela Ban
>             Fix For: 2.6.3, 2.7
>
>         Attachments: ENCRYPT.java.patch, ENCRYPT.patch
>
>
> This is with JGroups 2.6.1 on Linux 2.6.18, and a cluster of 10-20 nodes with several channels on each node.
> Everything is running smoothly, and then we kill the coordinator (of one or more channels), plus the next few nodes in the view, all at about the same time. SUSPECTs happen, and then the highest-ranking remaining node decides to become coordinator. It tries to mcast a new view, but never gets any view ACKs - so nobody else sees the updated view and the channel remains basically unusable. This only happens sometimes, and usually on just some of several channels - even though they share the same stack config.
> Enabling TRACE logging revealed that ENCRYPT.down() is queuing the new view - we use dynamically-generated keys and apparently the view changes have flipped queue_down to 'true'. We see this, over and over:
> 2008-03-02 10:54:47,970 [TRACE] GMS mcasting view {[10.193.48.119:33274|693] [10.193.48.119:33274, 10.193.48.115:52962, 10.192.226.30:45130, 10.193.66.111:43645]} (4 mbrs)
> 2008-03-02 10:54:47,970 [TRACE] ENCRYPT queueing down message as no session key established[dst: <null>, src: <null> (1 headers), size=0 bytes]
> 2008-03-02 10:54:49,972 [WARN] GMS failed to collect all ACKs (4) for view [10.193.48.119:33274|693] [10.193.48.119:33274, 10.193.48.115:52962, 10.192.226.30:45130, 10.193.66.111:43645] after 2000ms, missing ACKs from [10.193.48.119:33274, 10.193.48.115:52962, 10.192.226.30:45130, 10.193.66.111:43645] (received=[]), local_addr=10.193.48.119:33274 

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira