[
https://issues.jboss.org/browse/JGRP-2382?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-2382:
--------------------------------
The heap shows that UNICAST3 is retaining messages _sent_ to another member. For some
reason, that member doesn't ack any of the sent messages, so they are kept around
until an ACK has been received, or that member dies, or leaves the cluster.
Apparently, that member is _not_ suspected (and removed), so the table for messages sent
to it cannot be cleared.
To further diagnose this issue, logs would be useful, at TRACE level for UNICAST3 and
GMS.
If they have a running system, with the issue present, they could also use probe.sh to
look into the system.
JGroups version 4.0.13.Final.jar is causing memory leaks
--------------------------------------------------------
Key: JGRP-2382
URL:
https://issues.jboss.org/browse/JGRP-2382
Project: JGroups
Issue Type: Feature Request
Affects Versions: 4.0.13
Environment: AIX machine 7.1 with JDK 1.8
Reporter: Rashmi Acharya
Assignee: Bela Ban
Priority: Major
Attachments: dumps_TEST_node1_20190918_after_3_hours.zip,
dumps_TEST_node1_20190918_right_after_restart.zip,
dumps_TEST_node2_20190918_after_3_hours.zip,
dumps_TEST_node2_20190918_right_after_restart.zip
We are observing a constant memory growth and leak with JGroup version 4.0.13 ..
One of our customer is having two node cluster environment and in one node we are
observing org.Group.Messages which contain org.groups.Header and
org.groups.Stack.ipAddress objects.. these are not getting cleared from memory..
We dont see any exception related to Jgroups from logs and but it is causing a gradual
emory growth and OOM.
Here is the Jgroups cluster configuration we have:
dynamic.cluster.property_string
<config
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups
http://www.jgroups.org/schema/jgroups.xsd">
<TCP bind_addr="&HOST_ADDR;"
bind_port="&MULTICAST_NODE_PORT2;"/>
<TCPPING async_discovery="true"
initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0"
send_cache_on_join="true"/>
<MERGE3 min_interval="3000" max_interval="5000" />
<FD_ALL timeout="20000" interval="15000"/>
<FD_SOCK/>
<FD timeout="5000" max_tries="48" />
<VERIFY_SUSPECT timeout="1500"/>
<BARRIER/>
<pbcast.NAKACK2 use_mcast_xmit="false"
discard_delivered_msgs="true"/>
<UNICAST3/>
<pbcast.STABLE desired_avg_gossip="20000" max_bytes="0"
stability_delay="1000"/>
<pbcast.GMS print_local_addr="true" join_timeout="15000" />
</config>
=================================
dynamic.cluster.distribution_property_string
<config
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups
http://www.jgroups.org/schema/jgroups.xsd">
<TCP bind_port="&MULTICAST_NODE_PORT1;" />
<TCPPING async_discovery="true"
initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0"
send_cache_on_join="true"/>
<MERGE3 min_interval="3000" max_interval="5000"/>
<FD_SOCK/>
<FD timeout="5000" max_tries="48"/>
<VERIFY_SUSPECT timeout="1500"/>
<BARRIER/>
<pbcast.NAKACK2 use_mcast_xmit="false"
discard_delivered_msgs="true"/>
<UNICAST3/>
<pbcast.STABLE desired_avg_gossip="20000" max_bytes="0"
stability_delay="1000" />
<pbcast.GMS print_local_addr="true" join_timeout="5000"/>
</config>
================================
dynamic.cluster.lock.protocolStack
<config
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups
http://www.jgroups.org/schema/jgroups.xsd">
<TCP bind_addr="&HOST_ADDR;"
bind_port="&MULTICAST_NODE_PORT3;"/>
<TCPPING async_discovery="true"
initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0"
send_cache_on_join="true"/>
<MERGE3 min_interval="3000" max_interval="5000"/>
<FD_ALL timeout="20000" interval="5000"/>
<FD timeout="5000" max_tries="48"/>
<VERIFY_SUSPECT timeout="1500"/>
<BARRIER/>
<pbcast.NAKACK2 use_mcast_xmit="false"
discard_delivered_msgs="true"/>
<UNICAST3 /> <pbcast.STABLE desired_avg_gossip="20000" />
<pbcast.GMS print_local_addr="true" join_timeout="5000"/>
<FRAG2 frag_size="8096"/>
<CENTRAL_LOCK2/>
</config>
--
This message was sent by Atlassian Jira
(v7.13.5#713005)