[jboss-user] [Clustering/JBoss] - [org.jgroups.protocols.pbcast.NAKACK] [mach:port] discarded

snedapunk do-not-reply at jboss.com
Wed Jun 27 17:45:08 EDT 2007 

Hello, I am running 5 servers running Jboss-4.0.3SP1 with sticky sessions behind a load balancer. Jgroups version 2.2.7  The problem I am seeing is that after some time, the logs start to fill with  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415.  However the console is showing all of them as members as shown below.  Currently in this case vrx10 is the coordinator.  The only way to  fix is to simply adjust the communication ports of the JBOSS clustering services for each UI machine and restart JBOSS.  However this bas been decided as being unacceptable. Here is a bunch of ouput from org.jgroups.  Any help/advice on resolving this would be greatly appreciated.

In this output it is showing vrx15 as null.
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.UDP] sending message to 230.1.2.7:45592 (src=vrx14:58902), headers are {NAKACK=[NAKACK: MSG, seqno=8263, range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 15123 (15123)], vrx11-in:42955: [484 : 9010 (9010)], vrx13-in:39016: [62 : 11731 (11731)], vrx12-in:42073: [109 : 7002 (7002)], vrx14:58902: [0 : 8262 (8262)], null: [0 : 0], UDP=[UDP:group_addr=Tomcat-Cluster]}



I have verified that these are in fact part of the same cluster by going to jmx-console.
TomcatClusteringCache VRX14*****************************************
 Members: [vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14:58902, vrx15-in:41415]
clusterName = Tomcat-Cluster

jboss:service=UI
[192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099, 192.168.120.145:1099]
(these are the ips of the vrx-in?s note 192.168.120.145:1099 is vrx15-in

TomcatClusteringCache VRX15 : ****************************************
Members:  [vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14-in:58902, vrx15:41415]
clusterName = Tomcat-Cluster

jboss:service=UI
currentView=[192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099, 192.168.120.145:1099]

TomcatClusteringCache:vrx10:*****************************************
Members: [vrx10:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14-in:58902, vrx15-in:41415]

Joss:service=UI
Cordinator = true


Here are the messages found in vrx14, with org.jgroups in warn mode
2007-06-27 14:24:52,219 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,220 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,220 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,221 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,222 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,222 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message from non-member vrx15-in:41415
 [vrx14:58902] discarded message from non-member vrx15-in:41415


This output is from setting vrx14?s org.jgroups to debug.

007-06-27 12:00:26,896 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx10-in:57492#14790>
2007-06-27 12:00:26,897 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx10-in:57492 is: received_msgs: [14790 - 14790 (size=1, missing=0)], delivered_msgs: [427 - 14789]
2007-06-27 12:00:26,897 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=3222, frag_id=3, num_frags=4]
2007-06-27 12:00:26,897 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src:  (1 headers), size = 27872 bytes]
2007-06-27 12:00:26,898 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1182970826892, rsp_expected=false], dest_mbrs=[vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14:58902, vrx15-in:41415]
2007-06-27 12:00:26,898 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jgroups.blocks.RpcDispatcher) with request 1182970826892
2007-06-27 12:00:26,899 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx10-in:57492], method_call: _replicate(prepare(<vrx10-in:57492>:4651, [_put(<vrx10-in:57492>:4651, /JSESSION/localhost/ui/-WKfLv3hhYx7BgSfjSyZbQ**, -WKfLv3hhYx7BgSfjSyZbQ**, [B at ff8bd6, true), _put(<vrx10-in:57492>:4651, /JSESSION/localhost/ui/-WKfLv3hhYx7BgSfjSyZbQ**, VERSION, 35, true)], vrx10-in:57492, true))
2007-06-27 12:00:27,990 DEBUG [org.jgroups.protocols.FD] sending are-you-alive msg to vrx15-in:41406 (additional data: 20 bytes) (own address=vrx14:58891 (additional data: 20 bytes))
2007-06-27 12:00:27,991 DEBUG [org.jgroups.protocols.UDP] sending message to vrx15-in:41406 (additional data: 20 bytes) (src=vrx14:58891 (additional data: 20 bytes)), headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 12:00:27,991 DEBUG [org.jgroups.protocols.UDP] received (ucast) 164 bytes from /192.168.120.145:41406
2007-06-27 12:00:27,992 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx14:58891 (additional data: 20 bytes), src: vrx15-in:41406 (additional data: 20 bytes) (2 headers), size = 0 bytes], headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]} 
2007-06-27 12:00:27,992 DEBUG [org.jgroups.protocols.FD] received ack from vrx15-in:41406 (additional data: 20 bytes)
2007-06-27 12:00:28,103 ERROR [org.jgroups.protocols.pbcast.NAKACK] sender vrx15-in:41415 not found in received_msgs
2007-06-27 12:00:28,104 ERROR [org.jgroups.protocols.pbcast.NAKACK] range is null
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest [vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573 (8573)], vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)], vrx14:58902: [0 : 7814 (7814)], null: [0 : 0] (num_gossip_runs=3, max_gossip_runs=3)
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #7815
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 7814]
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.UDP] sending message to 230.1.2.7:45592 (src=vrx14:58902), headers are {NAKACK=[NAKACK: MSG, seqno=7815, range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573 (8573)], vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)], vrx14:58902: [0 : 7814 (7814)], null: [0 : 0], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.UDP] received (mcast) 477 bytes from /192.168.120.144:58905 (size=477 bytes)
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx14:58902 (3 headers), size = 0 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=7815, range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573 (8573)], vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)], vrx14:58902: [0 : 7814 (7814)], null: [0 : 0], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx14:58902#7815>
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx14:58902 is: received_msgs: [7815 - 7815 (size=1, missing=0)], delivered_msgs: [0 - 7814]
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest vrx10-in:57492#14790 (14790), vrx11-in:42955#8573 (8573), vrx13-in:39016#11375 (11375), vrx12-in:42073#6667 (6667), vrx14:58902#7814 (7814), null#0 (-1) from vrx14:58902
2007-06-27 12:00:28,106 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest from vrx14:58902 (digest=[vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573 (8573)], vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)], vrx14:58902: [0 : 7814 (7814)], null: [0 : 0]) which does not match my own digest ([vrx10-in:57492: [-1 : -1, vrx11-in:42955: [-1 : -1, vrx13-in:39016: [-1 : -1, vrx12-in:42073: [-1 : -1, vrx14:58902: [-1 : -1, vrx15-in:41415: [-1 : -1]): ignoring digest and re-initializing own digest
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.111:42956 (size=8370 bytes)
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx11-in:42955 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=8574, range=null], FRAG=[FRAG: id=1774, frag_id=0, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx11-in:42955#8574>
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx11-in:42955 is: received_msgs: [8574 - 8574 (size=1, missing=0)], delivered_msgs: [484 - 8573]
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.111:42956 (size=8370 bytes)
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=1774, frag_id=0, num_frags=4]
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx11-in:42955 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=8575, range=null], FRAG=[FRAG: id=1774, frag_id=1, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx11-in:42955#8575>
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx11-in:42955 is: received_msgs: [8575 - 8575 (size=1, missing=0)], delivered_msgs: [484 - 8574]
/range is null                                                                                                    
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 2190 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=6998, range=null], FRAG=[FRAG: id=1441, frag_id=3, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx12-in:42073#6998>
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx12-in:42073 is: received_msgs: [6998 - 6998 (size=1, missing=0)], delivered_msgs: [109 - 6997]
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=1441, frag_id=3, num_frags=4]
2007-06-27 12:04:14,887 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src:  (1 headers), size = 26221 bytes]
2007-06-27 12:04:14,887 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1182971054882, rsp_expected=false], dest_mbrs=[vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx14:58902, vrx15-in:41415]
2007-06-27 12:04:14,887 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jgroups.blocks.RpcDispatcher) with request 1182971054882
2007-06-27 12:04:14,888 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx12-in:42073], method_call: _replicate(prepare(<vrx12-in:42073>:2714, [_put(<vrx12-in:42073>:2714, /JSESSION/localhost/ui/YQ9GRvuk5sVrzMpr7bWq6w**, YQ9GRvuk5sVrzMpr7bWq6w**, [B at 1201295, true), _put(<vrx12-in:42073>:2714, /JSESSION/localhost/ui/YQ9GRvuk5sVrzMpr7bWq6w**, VERSION, 37, true)], vrx12-in:42073, true))
2007-06-27 12:04:14,901 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.112:42074 (size=8370 bytes)
2007-06-27 12:04:14,901 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=6999, range=null], FRAG=[FRAG: id=1442, frag_id=0, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,901 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx12-in:42073#6999>
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx12-in:42073 is: received_msgs: [6999 - 6999 (size=1, missing=0)], delivered_msgs: [109 - 6998]
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=1442, frag_id=0, num_frags=4]
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.112:42074 (size=8370 bytes)
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=7000, range=null], FRAG=[FRAG: id=1442, frag_id=1, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx12-in:42073#7000>
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx12-in:42073 is: received_msgs: [7000 - 7000 (size=1, missing=0)], delivered_msgs: [109 - 6999]
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=1442, frag_id=1, num_frags=4]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.112:42074 (size=8370 bytes)
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=7001, range=null], FRAG=[FRAG: id=1442, frag_id=2, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx12-in:42073#7001>
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx12-in:42073 is: received_msgs: [7001 - 7001 (size=1, missing=0)], delivered_msgs: [109 - 7000]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=1442, frag_id=2, num_frags=4]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] received (mcast) 4965 bytes from /192.168.120.112:42074 (size=4965 bytes)
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 4804 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=7002, range=null], FRAG=[FRAG: id=1442, frag_id=3, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received <vrx12-in:42073#7002>
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx12-in:42073 is: received_msgs: [7002 - 7002 (size=1, missing=0)], delivered_msgs: [109 - 7001]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr is [FRAG: id=1442, frag_id=3, num_frags=4]
2007-06-27 12:04:14,904 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src:  (1 headers), size = 28820 bytes]
2007-06-27 12:04:14,904 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1182971054899, rsp_expected=false], dest_mbrs=[vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx14:58902, vrx15-in:41415]
2007-06-27 12:04:14,904 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jgroups.blocks.RpcDispatcher) with request 1182971054899
2007-06-27 12:04:14,905 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx12-in:42073], method_call: _replicate(prepare(<vrx12-in:42073>:2715, [_put(<vrx12-in:42073>:2715, /JSESSION/localhost/ui/P28S4Xe1cjeSM59213DuqQ**, P28S4Xe1cjeSM59213DuqQ**, [B at a419b4, true), _put(<vrx12-in:42073>:2715, /JSESSION/localhost/ui/P28S4Xe1cjeSM59213DuqQ**, VERSION, 6, true)], vrx12-in:42073, true))
2007-06-27 12:04:14,927 DEBUG [org.jgroups.protocols.UDP] received (ucast) 127 bytes from /192.168.120.143:39007
2007-06-27 12:04:14,928 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx14:58891 (additional data: 20 bytes), src: vrx13-in:39007 (additional data: 20 bytes) (2 headers), size = 0 bytes], headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 12:04:14,928 DEBUG [org.jgroups.protocols.UDP] sending message to vrx13-in:39007 (additional data: 20 bytes) (src=vrx14:58891 (additional data: 20 bytes)), headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]}
2007-06-27 12:04:14,967 ERROR [org.jgroups.protocols.pbcast.NAKACK] sender vrx15-in:41415 not found in received_msgs
2007-06-27 12:04:14,967 ERROR [org.jgroups.protocols.pbcast.NAKACK] range is null
2007-06-27 12:04:14,967 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest [vrx10-in:57492: [427 : 15123 (15123)], vrx11-in:42955: [484 : 9010 (9010)], vrx13-in:39016: [62 : 11731 (11731)], vrx12-in:42073: [109 : 7002 (7002)], vrx14:58902: [0 : 8262 (8262)], null: [0 : 0] (num_gossip_runs=3, max_gossip_runs=3)
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #8263
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 8262]
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.UDP] sending message to 230.1.2.7:45592 (src=vrx14:58902), headers are {NAKACK=[NAKACK: MSG, seqno=8263, range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 15123 (15123)], vrx11-in:42955: [484 : 9010 (9010)], vrx13-in:39016: [62 : 11731 (11731)], vrx12-in:42073: [109 : 7002 (7002)], vrx14:58902: [0 : 8262 (8262)], null: [0 : 0], UDP=[UDP:group_addr=Tomcat-Cluster]}




Here is some output when setting vrx15 jgroups to debug
007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx14-in:58902 is: received_msgs: [23099 - 23099 (size=1, missing=0)], delivered_msgs: [0 - 23098]
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr is [FRAG: id=4875, frag_id=0, num_frags=4]
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.144:58905 (size=8370 bytes)
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx14-in:58902 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=23100, range=null], FRAG=[FRAG: id=4875, frag_id=1, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx15:41415] received <vrx14-in:58902#23100>
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx14-in:58902 is: received_msgs: [23100 - 23100 (size=1, missing=0)], delivered_msgs: [0 - 23099]
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr is [FRAG: id=4875, frag_id=1, num_frags=4]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from /192.168.120.144:58905 (size=8370 bytes)
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx14-in:58902 (3 headers), size = 8192 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=23101, range=null], FRAG=[FRAG: id=4875, frag_id=2, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx15:41415] received <vrx14-in:58902#23101>
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] received (mcast) 3721 bytes from /192.168.120.144:58905 (size=3721 bytes)
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx14-in:58902 is: received_msgs: [23101 - 23101 (size=1, missing=0)], delivered_msgs: [0 - 23100]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] message is [dst: 230.1.2.7:45592, src: vrx14-in:58902 (3 headers), size = 3565 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=23102, range=null], FRAG=[FRAG: id=4875, frag_id=3, num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr is [FRAG: id=4875, frag_id=2, num_frags=4]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx15:41415] received <vrx14-in:58902#23102>
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vrx14-in:58902 is: received_msgs: [23102 - 23102 (size=1, missing=0)], delivered_msgs: [0 - 23101]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr is [FRAG: id=4875, frag_id=3, num_frags=4]
2007-06-27 14:40:27,744 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src:  (1 headers), size = 27589 bytes]
2007-06-27 14:40:27,744 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1182980427740, rsp_expected=false], dest_mbrs=[vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx15:41415]
2007-06-27 14:40:27,744 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jgroups.blocks.RpcDispatcher) with request 1182980427740
2007-06-27 14:40:27,745 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx14-in:58902], method_call: _replicate(prepare(<vrx14-in:58902>:8456, [_put(<vrx14-in:58902>:8456, /JSESSION/localhost/ui/+PeD8nZCBNDZofmfeufyyQ**, +PeD8nZCBNDZofmfeufyyQ**, [B at 19f4862, true), _put(<vrx14-in:58902>:8456, /JSESSION/localhost/ui/+PeD8nZCBNDZofmfeufyyQ**, VERSION, 2, true)], vrx14-in:58902, true))
2007-06-27 14:40:29,921 DEBUG [org.jgroups.protocols.UDP] received (ucast) 127 bytes from /192.168.120.144:58891
2007-06-27 14:40:29,922 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406 (additional data: 20 bytes), src: vrx14-in:58891 (additional data: 20 bytes) (2 headers), size = 0 bytes], headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:29,922 DEBUG [org.jgroups.protocols.UDP] sending message to vrx14-in:58891 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)), headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:30,098 DEBUG [org.jgroups.protocols.FD] sending are-you-alive msg to vrx10-in:57485 (additional data: 20 bytes) (own address=vrx15:41406 (additional data: 20 bytes))
2007-06-27 14:40:30,098 DEBUG [org.jgroups.protocols.UDP] sending message to vrx10-in:57485 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)), headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:30,099 DEBUG [org.jgroups.protocols.UDP] received (ucast) 164 bytes from /192.168.120.110:57485
2007-06-27 14:40:30,099 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406 (additional data: 20 bytes), src: vrx10-in:57485 (additional data: 20 bytes) (2 headers), size = 0 bytes], headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]} 
2007-06-27 14:40:30,099 DEBUG [org.jgroups.protocols.FD] received ack from vrx10-in:57485 (additional data: 20 bytes)
2007-06-27 14:40:32,424 DEBUG [org.jgroups.protocols.UDP] received (ucast) 127 bytes from /192.168.120.144:58891
2007-06-27 14:40:32,424 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406 (additional data: 20 bytes), src: vrx14-in:58891 (additional data: 20 bytes) (2 headers), size = 0 bytes], headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:32,424 DEBUG [org.jgroups.protocols.UDP] sending message to vrx14-in:58891 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)), headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:32,600 DEBUG [org.jgroups.protocols.FD] sending are-you-alive msg to vrx10-in:57485 (additional data: 20 bytes) (own address=vrx15:41406 (additional data: 20 bytes))
2007-06-27 14:40:32,600 DEBUG [org.jgroups.protocols.UDP] sending message to vrx10-in:57485 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)), headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:32,601 DEBUG [org.jgroups.protocols.UDP] received (ucast) 164 bytes from /192.168.120.110:57485
2007-06-27 14:40:32,601 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406 (additional data: 20 bytes), src: vrx10-in:57485 (additional data: 20 bytes) (2 headers), size = 0 bytes], headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]} 
2007-06-27 14:40:32,601 DEBUG [org.jgroups.protocols.FD] received ack from vrx10-in:57485 (additional data: 20 bytes)


showHistoryAsXml() from vrx10
   6/27/07 10:09 AM : Partition object created
   
   
      6/27/07 10:09 AM : Initializing partition
   
   
      6/27/07 10:09 AM : Starting partition
   
   
      6/27/07 10:09 AM : New view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099] with viewId: 142 (old view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099] )
   
   
      6/27/07 10:09 AM : setState called on partition
   
   
      6/27/07 10:09 AM : New view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099, 192.168.120.111:1099] with viewId: 143 (old view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099] )
   
   
      6/27/07 10:12 AM : New view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099] with viewId: 144 (old view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099, 192.168.120.111:1099] )
   
   
      6/27/07 10:13 AM : New view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099] with viewId: 145 (old view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099] )
   
   
      6/27/07 10:13 AM : New view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099] with viewId: 146 (old view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099] )
   
   
      6/27/07 10:13 AM : New view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] with viewId: 147 (old view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099] )
   
   
      6/27/07 10:16 AM : New view: [192.168.120.144:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] with viewId: 148 (old view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] )
   
   
      6/27/07 10:16 AM : New view: [192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] with viewId: 149 (old view: [192.168.120.144:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] )
   
   
      6/27/07 10:17 AM : New view: [192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099] with viewId: 150 (old view: [192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] )
   
   
      6/27/07 10:17 AM : getState called on partition
   
   
      6/27/07 10:17 AM : New view: [192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099, 192.168.120.145:1099] with viewId: 151 (old view: [192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099] )
   
   
      6/27/07 10:17 AM : getState called on partition
   




View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4058443#4058443

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4058443

More information about the jboss-user mailing list