[jboss-user] [Clustering/JBoss] - [org.jgroups.protocols.pbcast.NAKACK] [mach:port] discarded

Wednesday, 27 June 2007

Hello, I am running 5 servers running Jboss-4.0.3SP1 with sticky sessions behind a load
balancer. Jgroups version 2.2.7  The problem I am seeing is that after some time, the logs
start to fill with  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] discarded message
from non-member vrx15-in:41415.  However the console is showing all of them as members as
shown below.  Currently in this case vrx10 is the coordinator.  The only way to  fix is to
simply adjust the communication ports of the JBOSS clustering services for each UI machine
and restart JBOSS.  However this bas been decided as being unacceptable. Here is a bunch
of ouput from org.jgroups.  Any help/advice on resolving this would be greatly
appreciated.

In this output it is showing vrx15 as null.
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.UDP] sending message to
230.1.2.7:45592 (src=vrx14:58902), headers are {NAKACK=[NAKACK: MSG, seqno=8263,
range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 15123 (15123)],
vrx11-in:42955: [484 : 9010 (9010)], vrx13-in:39016: [62 : 11731 (11731)], vrx12-in:42073:
[109 : 7002 (7002)], vrx14:58902: [0 : 8262 (8262)], null: [0 : 0],
UDP=[UDP:group_addr=Tomcat-Cluster]}

I have verified that these are in fact part of the same cluster by going to jmx-console.
TomcatClusteringCache VRX14*****************************************
 Members: [vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14:58902,
vrx15-in:41415]
clusterName = Tomcat-Cluster

jboss:service=UI
[192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099,
192.168.120.144:1099, 192.168.120.145:1099]
(these are the ips of the vrx-in?s note 192.168.120.145:1099 is vrx15-in

TomcatClusteringCache VRX15 : ****************************************
Members:  [vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14-in:58902,
vrx15:41415]
clusterName = Tomcat-Cluster

jboss:service=UI
currentView=[192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099,
192.168.120.143:1099, 192.168.120.144:1099, 192.168.120.145:1099]

TomcatClusteringCache:vrx10:*****************************************
Members: [vrx10:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14-in:58902,
vrx15-in:41415]

Joss:service=UI
Cordinator = true

Here are the messages found in vrx14, with org.jgroups in warn mode
2007-06-27 14:24:52,219 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902]
discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,220 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902]
discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,220 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902]
discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,221 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902]
discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,222 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902]
discarded message from non-member vrx15-in:41415
2007-06-27 14:24:52,222 WARN  [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902]
discarded message from non-member vrx15-in:41415
 [vrx14:58902] discarded message from non-member vrx15-in:41415

This output is from setting vrx14?s org.jgroups to debug.

007-06-27 12:00:26,896 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx10-in:57492#14790>
2007-06-27 12:00:26,897 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx10-in:57492 is: received_msgs: [14790 - 14790 (size=1, missing=0)], delivered_msgs:
[427 - 14789]
2007-06-27 12:00:26,897 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=3222, frag_id=3, num_frags=4]
2007-06-27 12:00:26,897 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src: 
(1 headers), size = 27872 bytes]
2007-06-27 12:00:26,898 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header:
name=MessageDispatcher, type=REQ, id=1182970826892, rsp_expected=false],
dest_mbrs=[vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx14:58902, vrx15-in:41415]
2007-06-27 12:00:26,898 DEBUG [org.jgroups.blocks.RequestCorrelator] calling
(org.jgroups.blocks.RpcDispatcher) with request 1182970826892
2007-06-27 12:00:26,899 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx10-in:57492],
method_call: _replicate(prepare(<vrx10-in:57492>:4651,
[_put(<vrx10-in:57492>:4651, /JSESSION/localhost/ui/-WKfLv3hhYx7BgSfjSyZbQ**,
-WKfLv3hhYx7BgSfjSyZbQ**, [B@ff8bd6, true), _put(<vrx10-in:57492>:4651,
/JSESSION/localhost/ui/-WKfLv3hhYx7BgSfjSyZbQ**, VERSION, 35, true)], vrx10-in:57492,
true))
2007-06-27 12:00:27,990 DEBUG [org.jgroups.protocols.FD] sending are-you-alive msg to
vrx15-in:41406 (additional data: 20 bytes) (own address=vrx14:58891 (additional data: 20
bytes))
2007-06-27 12:00:27,991 DEBUG [org.jgroups.protocols.UDP] sending message to
vrx15-in:41406 (additional data: 20 bytes) (src=vrx14:58891 (additional data: 20 bytes)),
headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 12:00:27,991 DEBUG [org.jgroups.protocols.UDP] received (ucast) 164 bytes from
/192.168.120.145:41406
2007-06-27 12:00:27,992 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx14:58891
(additional data: 20 bytes), src: vrx15-in:41406 (additional data: 20 bytes) (2 headers),
size = 0 bytes], headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]} 
2007-06-27 12:00:27,992 DEBUG [org.jgroups.protocols.FD] received ack from vrx15-in:41406
(additional data: 20 bytes)
2007-06-27 12:00:28,103 ERROR [org.jgroups.protocols.pbcast.NAKACK] sender vrx15-in:41415
not found in received_msgs
2007-06-27 12:00:28,104 ERROR [org.jgroups.protocols.pbcast.NAKACK] range is null
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest
[vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573 (8573)],
vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)], vrx14:58902: [0
: 7814 (7814)], null: [0 : 0] (num_gossip_runs=3, max_gossip_runs=3)
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #7815
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 7814]
2007-06-27 12:00:28,104 DEBUG [org.jgroups.protocols.UDP] sending message to
230.1.2.7:45592 (src=vrx14:58902), headers are {NAKACK=[NAKACK: MSG, seqno=7815,
range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 14790 (14790)],
vrx11-in:42955: [484 : 8573 (8573)], vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073:
[109 : 6667 (6667)], vrx14:58902: [0 : 7814 (7814)], null: [0 : 0],
UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.UDP] received (mcast) 477 bytes from
/192.168.120.144:58905 (size=477 bytes)
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx14:58902 (3 headers), size = 0 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=7815, range=null], STABLE=[STABLE_GOSSIP]: digest is
[vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573 (8573)],
vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)], vrx14:58902: [0
: 7814 (7814)], null: [0 : 0], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx14:58902#7815>
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx14:58902 is: received_msgs: [7815 - 7815 (size=1, missing=0)], delivered_msgs: [0 -
7814]
2007-06-27 12:00:28,105 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest
vrx10-in:57492#14790 (14790), vrx11-in:42955#8573 (8573), vrx13-in:39016#11375 (11375),
vrx12-in:42073#6667 (6667), vrx14:58902#7814 (7814), null#0 (-1) from vrx14:58902
2007-06-27 12:00:28,106 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest from
vrx14:58902 (digest=[vrx10-in:57492: [427 : 14790 (14790)], vrx11-in:42955: [484 : 8573
(8573)], vrx13-in:39016: [62 : 11375 (11375)], vrx12-in:42073: [109 : 6667 (6667)],
vrx14:58902: [0 : 7814 (7814)], null: [0 : 0]) which does not match my own digest
([vrx10-in:57492: [-1 : -1, vrx11-in:42955: [-1 : -1, vrx13-in:39016: [-1 : -1,
vrx12-in:42073: [-1 : -1, vrx14:58902: [-1 : -1, vrx15-in:41415: [-1 : -1]): ignoring
digest and re-initializing own digest
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.111:42956 (size=8370 bytes)
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx11-in:42955 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=8574, range=null], FRAG=[FRAG: id=1774, frag_id=0,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx11-in:42955#8574>
2007-06-27 12:00:28,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx11-in:42955 is: received_msgs: [8574 - 8574 (size=1, missing=0)], delivered_msgs: [484
- 8573]
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.111:42956 (size=8370 bytes)
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=1774, frag_id=0, num_frags=4]
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx11-in:42955 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=8575, range=null], FRAG=[FRAG: id=1774, frag_id=1,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx11-in:42955#8575>
2007-06-27 12:00:28,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx11-in:42955 is: received_msgs: [8575 - 8575 (size=1, missing=0)], delivered_msgs: [484
- 8574]
/range is null                                                                            

2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 2190 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=6998, range=null], FRAG=[FRAG: id=1441, frag_id=3,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx12-in:42073#6998>
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx12-in:42073 is: received_msgs: [6998 - 6998 (size=1, missing=0)], delivered_msgs: [109
- 6997]
2007-06-27 12:04:14,886 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=1441, frag_id=3, num_frags=4]
2007-06-27 12:04:14,887 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src: 
(1 headers), size = 26221 bytes]
2007-06-27 12:04:14,887 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header:
name=MessageDispatcher, type=REQ, id=1182971054882, rsp_expected=false],
dest_mbrs=[vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx14:58902, vrx15-in:41415]
2007-06-27 12:04:14,887 DEBUG [org.jgroups.blocks.RequestCorrelator] calling
(org.jgroups.blocks.RpcDispatcher) with request 1182971054882
2007-06-27 12:04:14,888 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx12-in:42073],
method_call: _replicate(prepare(<vrx12-in:42073>:2714,
[_put(<vrx12-in:42073>:2714, /JSESSION/localhost/ui/YQ9GRvuk5sVrzMpr7bWq6w**,
YQ9GRvuk5sVrzMpr7bWq6w**, [B@1201295, true), _put(<vrx12-in:42073>:2714,
/JSESSION/localhost/ui/YQ9GRvuk5sVrzMpr7bWq6w**, VERSION, 37, true)], vrx12-in:42073,
true))
2007-06-27 12:04:14,901 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.112:42074 (size=8370 bytes)
2007-06-27 12:04:14,901 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=6999, range=null], FRAG=[FRAG: id=1442, frag_id=0,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,901 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx12-in:42073#6999>
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx12-in:42073 is: received_msgs: [6999 - 6999 (size=1, missing=0)], delivered_msgs: [109
- 6998]
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=1442, frag_id=0, num_frags=4]
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.112:42074 (size=8370 bytes)
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=7000, range=null], FRAG=[FRAG: id=1442, frag_id=1,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx12-in:42073#7000>
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx12-in:42073 is: received_msgs: [7000 - 7000 (size=1, missing=0)], delivered_msgs: [109
- 6999]
2007-06-27 12:04:14,902 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=1442, frag_id=1, num_frags=4]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.112:42074 (size=8370 bytes)
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=7001, range=null], FRAG=[FRAG: id=1442, frag_id=2,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx12-in:42073#7001>
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx12-in:42073 is: received_msgs: [7001 - 7001 (size=1, missing=0)], delivered_msgs: [109
- 7000]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=1442, frag_id=2, num_frags=4]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] received (mcast) 4965 bytes from
/192.168.120.112:42074 (size=4965 bytes)
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx12-in:42073 (3 headers), size = 4804 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=7002, range=null], FRAG=[FRAG: id=1442, frag_id=3,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx14:58902] received
<vrx12-in:42073#7002>
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx12-in:42073 is: received_msgs: [7002 - 7002 (size=1, missing=0)], delivered_msgs: [109
- 7001]
2007-06-27 12:04:14,903 DEBUG [org.jgroups.protocols.FRAG] [vrx14:58902] received msg, hdr
is [FRAG: id=1442, frag_id=3, num_frags=4]
2007-06-27 12:04:14,904 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src: 
(1 headers), size = 28820 bytes]
2007-06-27 12:04:14,904 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header:
name=MessageDispatcher, type=REQ, id=1182971054899, rsp_expected=false],
dest_mbrs=[vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx14:58902, vrx15-in:41415]
2007-06-27 12:04:14,904 DEBUG [org.jgroups.blocks.RequestCorrelator] calling
(org.jgroups.blocks.RpcDispatcher) with request 1182971054899
2007-06-27 12:04:14,905 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx12-in:42073],
method_call: _replicate(prepare(<vrx12-in:42073>:2715,
[_put(<vrx12-in:42073>:2715, /JSESSION/localhost/ui/P28S4Xe1cjeSM59213DuqQ**,
P28S4Xe1cjeSM59213DuqQ**, [B@a419b4, true), _put(<vrx12-in:42073>:2715,
/JSESSION/localhost/ui/P28S4Xe1cjeSM59213DuqQ**, VERSION, 6, true)], vrx12-in:42073,
true))
2007-06-27 12:04:14,927 DEBUG [org.jgroups.protocols.UDP] received (ucast) 127 bytes from
/192.168.120.143:39007
2007-06-27 12:04:14,928 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx14:58891
(additional data: 20 bytes), src: vrx13-in:39007 (additional data: 20 bytes) (2 headers),
size = 0 bytes], headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 12:04:14,928 DEBUG [org.jgroups.protocols.UDP] sending message to
vrx13-in:39007 (additional data: 20 bytes) (src=vrx14:58891 (additional data: 20 bytes)),
headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]}
2007-06-27 12:04:14,967 ERROR [org.jgroups.protocols.pbcast.NAKACK] sender vrx15-in:41415
not found in received_msgs
2007-06-27 12:04:14,967 ERROR [org.jgroups.protocols.pbcast.NAKACK] range is null
2007-06-27 12:04:14,967 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest
[vrx10-in:57492: [427 : 15123 (15123)], vrx11-in:42955: [484 : 9010 (9010)],
vrx13-in:39016: [62 : 11731 (11731)], vrx12-in:42073: [109 : 7002 (7002)], vrx14:58902: [0
: 8262 (8262)], null: [0 : 0] (num_gossip_runs=3, max_gossip_runs=3)
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #8263
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 8262]
2007-06-27 12:04:14,968 DEBUG [org.jgroups.protocols.UDP] sending message to
230.1.2.7:45592 (src=vrx14:58902), headers are {NAKACK=[NAKACK: MSG, seqno=8263,
range=null], STABLE=[STABLE_GOSSIP]: digest is [vrx10-in:57492: [427 : 15123 (15123)],
vrx11-in:42955: [484 : 9010 (9010)], vrx13-in:39016: [62 : 11731 (11731)], vrx12-in:42073:
[109 : 7002 (7002)], vrx14:58902: [0 : 8262 (8262)], null: [0 : 0],
UDP=[UDP:group_addr=Tomcat-Cluster]}

Here is some output when setting vrx15 jgroups to debug
007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx14-in:58902 is: received_msgs: [23099 - 23099 (size=1, missing=0)], delivered_msgs: [0
- 23098]
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr
is [FRAG: id=4875, frag_id=0, num_frags=4]
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.144:58905 (size=8370 bytes)
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx14-in:58902 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=23100, range=null], FRAG=[FRAG: id=4875, frag_id=1,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx15:41415] received
<vrx14-in:58902#23100>
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx14-in:58902 is: received_msgs: [23100 - 23100 (size=1, missing=0)], delivered_msgs: [0
- 23099]
2007-06-27 14:40:27,742 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr
is [FRAG: id=4875, frag_id=1, num_frags=4]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] received (mcast) 8370 bytes from
/192.168.120.144:58905 (size=8370 bytes)
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx14-in:58902 (3 headers), size = 8192 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=23101, range=null], FRAG=[FRAG: id=4875, frag_id=2,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx15:41415] received
<vrx14-in:58902#23101>
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] received (mcast) 3721 bytes from
/192.168.120.144:58905 (size=3721 bytes)
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx14-in:58902 is: received_msgs: [23101 - 23101 (size=1, missing=0)], delivered_msgs: [0
- 23100]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.UDP] message is [dst:
230.1.2.7:45592, src: vrx14-in:58902 (3 headers), size = 3565 bytes], headers are
{NAKACK=[NAKACK: MSG, seqno=23102, range=null], FRAG=[FRAG: id=4875, frag_id=3,
num_frags=4], UDP=[UDP:group_addr=Tomcat-Cluster]}
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr
is [FRAG: id=4875, frag_id=2, num_frags=4]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vrx15:41415] received
<vrx14-in:58902#23102>
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for
vrx14-in:58902 is: received_msgs: [23102 - 23102 (size=1, missing=0)], delivered_msgs: [0
- 23101]
2007-06-27 14:40:27,743 DEBUG [org.jgroups.protocols.FRAG] [vrx15:41415] received msg, hdr
is [FRAG: id=4875, frag_id=3, num_frags=4]
2007-06-27 14:40:27,744 DEBUG [org.jgroups.protocols.FRAG] assembled_msg is [dst: , src: 
(1 headers), size = 27589 bytes]
2007-06-27 14:40:27,744 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header:
name=MessageDispatcher, type=REQ, id=1182980427740, rsp_expected=false],
dest_mbrs=[vrx10-in:57492, vrx11-in:42955, vrx13-in:39016, vrx12-in:42073, vrx15:41415]
2007-06-27 14:40:27,744 DEBUG [org.jgroups.blocks.RequestCorrelator] calling
(org.jgroups.blocks.RpcDispatcher) with request 1182980427740
2007-06-27 14:40:27,745 DEBUG [org.jgroups.blocks.RpcDispatcher] [sender=vrx14-in:58902],
method_call: _replicate(prepare(<vrx14-in:58902>:8456,
[_put(<vrx14-in:58902>:8456, /JSESSION/localhost/ui/+PeD8nZCBNDZofmfeufyyQ**,
+PeD8nZCBNDZofmfeufyyQ**, [B@19f4862, true), _put(<vrx14-in:58902>:8456,
/JSESSION/localhost/ui/+PeD8nZCBNDZofmfeufyyQ**, VERSION, 2, true)], vrx14-in:58902,
true))
2007-06-27 14:40:29,921 DEBUG [org.jgroups.protocols.UDP] received (ucast) 127 bytes from
/192.168.120.144:58891
2007-06-27 14:40:29,922 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406
(additional data: 20 bytes), src: vrx14-in:58891 (additional data: 20 bytes) (2 headers),
size = 0 bytes], headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:29,922 DEBUG [org.jgroups.protocols.UDP] sending message to
vrx14-in:58891 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)),
headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:30,098 DEBUG [org.jgroups.protocols.FD] sending are-you-alive msg to
vrx10-in:57485 (additional data: 20 bytes) (own address=vrx15:41406 (additional data: 20
bytes))
2007-06-27 14:40:30,098 DEBUG [org.jgroups.protocols.UDP] sending message to
vrx10-in:57485 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)),
headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:30,099 DEBUG [org.jgroups.protocols.UDP] received (ucast) 164 bytes from
/192.168.120.110:57485
2007-06-27 14:40:30,099 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406
(additional data: 20 bytes), src: vrx10-in:57485 (additional data: 20 bytes) (2 headers),
size = 0 bytes], headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]} 
2007-06-27 14:40:30,099 DEBUG [org.jgroups.protocols.FD] received ack from vrx10-in:57485
(additional data: 20 bytes)
2007-06-27 14:40:32,424 DEBUG [org.jgroups.protocols.UDP] received (ucast) 127 bytes from
/192.168.120.144:58891
2007-06-27 14:40:32,424 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406
(additional data: 20 bytes), src: vrx14-in:58891 (additional data: 20 bytes) (2 headers),
size = 0 bytes], headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:32,424 DEBUG [org.jgroups.protocols.UDP] sending message to
vrx14-in:58891 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)),
headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:32,600 DEBUG [org.jgroups.protocols.FD] sending are-you-alive msg to
vrx10-in:57485 (additional data: 20 bytes) (own address=vrx15:41406 (additional data: 20
bytes))
2007-06-27 14:40:32,600 DEBUG [org.jgroups.protocols.UDP] sending message to
vrx10-in:57485 (additional data: 20 bytes) (src=vrx15:41406 (additional data: 20 bytes)),
headers are {FD=[FD: heartbeat], UDP=[UDP:group_addr=UI]}
2007-06-27 14:40:32,601 DEBUG [org.jgroups.protocols.UDP] received (ucast) 164 bytes from
/192.168.120.110:57485
2007-06-27 14:40:32,601 DEBUG [org.jgroups.protocols.UDP] message is [dst: vrx15:41406
(additional data: 20 bytes), src: vrx10-in:57485 (additional data: 20 bytes) (2 headers),
size = 0 bytes], headers are {FD=[FD: heartbeat ack], UDP=[UDP:group_addr=UI]} 
2007-06-27 14:40:32,601 DEBUG [org.jgroups.protocols.FD] received ack from vrx10-in:57485
(additional data: 20 bytes)

showHistoryAsXml() from vrx10
   6/27/07 10:09 AM : Partition object created

      6/27/07 10:09 AM : Initializing partition

      6/27/07 10:09 AM : Starting partition

      6/27/07 10:09 AM : New view: [192.168.120.144:1099, 192.168.120.143:1099,
192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099] with viewId: 142 (old
view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099,
192.168.120.112:1099, 192.168.120.110:1099] )

      6/27/07 10:09 AM : setState called on partition

      6/27/07 10:09 AM : New view: [192.168.120.144:1099, 192.168.120.143:1099,
192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099, 192.168.120.111:1099]
with viewId: 143 (old view: [192.168.120.144:1099, 192.168.120.143:1099,
192.168.120.145:1099, 192.168.120.112:1099, 192.168.120.110:1099] )

      6/27/07 10:12 AM : New view: [192.168.120.144:1099, 192.168.120.143:1099,
192.168.120.145:1099, 192.168.120.110:1099, 192.168.120.111:1099] with viewId: 144 (old
view: [192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099,
192.168.120.112:1099, 192.168.120.110:1099, 192.168.120.111:1099] )

      6/27/07 10:13 AM : New view: [192.168.120.144:1099, 192.168.120.145:1099,
192.168.120.110:1099, 192.168.120.111:1099] with viewId: 145 (old view:
[192.168.120.144:1099, 192.168.120.143:1099, 192.168.120.145:1099, 192.168.120.110:1099,
192.168.120.111:1099] )

      6/27/07 10:13 AM : New view: [192.168.120.144:1099, 192.168.120.145:1099,
192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099] with viewId: 146 (old
view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099,
192.168.120.111:1099] )

      6/27/07 10:13 AM : New view: [192.168.120.144:1099, 192.168.120.145:1099,
192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099]
with viewId: 147 (old view: [192.168.120.144:1099, 192.168.120.145:1099,
192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099] )

      6/27/07 10:16 AM : New view: [192.168.120.144:1099, 192.168.120.110:1099,
192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] with viewId: 148 (old
view: [192.168.120.144:1099, 192.168.120.145:1099, 192.168.120.110:1099,
192.168.120.111:1099, 192.168.120.112:1099, 192.168.120.143:1099] )

      6/27/07 10:16 AM : New view: [192.168.120.110:1099, 192.168.120.111:1099,
192.168.120.112:1099, 192.168.120.143:1099] with viewId: 149 (old view:
[192.168.120.144:1099, 192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099,
192.168.120.143:1099] )

      6/27/07 10:17 AM : New view: [192.168.120.110:1099, 192.168.120.111:1099,
192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099] with viewId: 150 (old
view: [192.168.120.110:1099, 192.168.120.111:1099, 192.168.120.112:1099,
192.168.120.143:1099] )

      6/27/07 10:17 AM : getState called on partition

      6/27/07 10:17 AM : New view: [192.168.120.110:1099, 192.168.120.111:1099,
192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099, 192.168.120.145:1099]
with viewId: 151 (old view: [192.168.120.110:1099, 192.168.120.111:1099,
192.168.120.112:1099, 192.168.120.143:1099, 192.168.120.144:1099] )

      6/27/07 10:17 AM : getState called on partition

View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4058443#...

Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[jboss-user] [Clustering/JBoss] - [org.jgroups.protocols.pbcast.NAKACK] [mach:port] discarded