[JBoss JIRA] (JGRP-2382) JGroups version 4.0.13.Final.jar is causing memory leaks
by Rashmi Acharya (Jira)
[ https://issues.jboss.org/browse/JGRP-2382?page=com.atlassian.jira.plugin.... ]
Rashmi Acharya commented on JGRP-2382:
--------------------------------------
Is it possible to get into a call with US to help us set/tune these parameters ?
We are part of IBM B2B Sterling Integrator team and we have lot of usecases around Jgroups and It is better to guide what all we need to cover when we upgrade,. Also help us on solving customer Usecase.
> JGroups version 4.0.13.Final.jar is causing memory leaks
> --------------------------------------------------------
>
> Key: JGRP-2382
> URL: https://issues.jboss.org/browse/JGRP-2382
> Project: JGroups
> Issue Type: Feature Request
> Affects Versions: 4.0.13
> Environment: AIX machine 7.1 with JDK 1.8
> Reporter: Rashmi Acharya
> Assignee: Bela Ban
> Priority: Major
> Attachments: dumps_TEST_node1_20190918_after_3_hours.zip, dumps_TEST_node1_20190918_right_after_restart.zip, dumps_TEST_node2_20190918_after_3_hours.zip, dumps_TEST_node2_20190918_right_after_restart.zip
>
>
> We are observing a constant memory growth and leak with JGroup version 4.0.13 ..
> One of our customer is having two node cluster environment and in one node we are observing org.Group.Messages which contain org.groups.Header and org.groups.Stack.ipAddress objects.. these are not getting cleared from memory..
> We dont see any exception related to Jgroups from logs and but it is causing a gradual emory growth and OOM.
> Here is the Jgroups cluster configuration we have:
> dynamic.cluster.property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT2;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000" />
> <FD_ALL timeout="20000" interval="15000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48" />
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000"/>
> <pbcast.GMS print_local_addr="true" join_timeout="15000" />
> </config>
> =================================
> dynamic.cluster.distribution_property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_port="&MULTICAST_NODE_PORT1;" />
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> </config>
> ================================
> dynamic.cluster.lock.protocolStack
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT3;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_ALL timeout="20000" interval="5000"/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3 /> <pbcast.STABLE desired_avg_gossip="20000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> <FRAG2 frag_size="8096"/>
> <CENTRAL_LOCK2/>
> </config>
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (JGRP-2382) JGroups version 4.0.13.Final.jar is causing memory leaks
by Rashmi Acharya (Jira)
[ https://issues.jboss.org/browse/JGRP-2382?page=com.atlassian.jira.plugin.... ]
Rashmi Acharya commented on JGRP-2382:
--------------------------------------
With Jgroup 3.4.0 version we never had UNICAST option and It used to work fine with both load balancing and No memory leaks. If Unicast acknowledgement were processed by other node in the system.
old Property:
distribution_property_string=TCP(bind_port=60961;thread_pool_rejection_policy=run):TCPPING(initial_hosts=harp[60961];port_range=0;timeout=5000;num_initial_members=2):MERGE2(min_interval=3000;max_interval=5000):FD_SOCK:FD(timeout=5000;max_tries=48;level=ERROR):VERIFY_SUSPECT(timeout=1500):pbcast.NAKACK(retransmit_timeout=3000;discard_delivered_msgs=true):pbcast.STABLE(stability_delay=1000;desired_avg_gossip=20000;max_bytes=0):pbcast.GMS(join_timeout=5000;print_local_addr=true)
Now you are suggesting us to add back UNICAST3 ? What about UNICAST protocol ?
FD timeout="5000" max_tries="48"/> is this need to be changed to FD_ALL ?
BARRIER we never had in the system ..
Do you think adding Timeout with UNICAST3 makes any difference w.r.t clearing those objects in memory ? If Yes, we can introduce back UNICAST3 with timeout=5000
> JGroups version 4.0.13.Final.jar is causing memory leaks
> --------------------------------------------------------
>
> Key: JGRP-2382
> URL: https://issues.jboss.org/browse/JGRP-2382
> Project: JGroups
> Issue Type: Feature Request
> Affects Versions: 4.0.13
> Environment: AIX machine 7.1 with JDK 1.8
> Reporter: Rashmi Acharya
> Assignee: Bela Ban
> Priority: Major
> Attachments: dumps_TEST_node1_20190918_after_3_hours.zip, dumps_TEST_node1_20190918_right_after_restart.zip, dumps_TEST_node2_20190918_after_3_hours.zip, dumps_TEST_node2_20190918_right_after_restart.zip
>
>
> We are observing a constant memory growth and leak with JGroup version 4.0.13 ..
> One of our customer is having two node cluster environment and in one node we are observing org.Group.Messages which contain org.groups.Header and org.groups.Stack.ipAddress objects.. these are not getting cleared from memory..
> We dont see any exception related to Jgroups from logs and but it is causing a gradual emory growth and OOM.
> Here is the Jgroups cluster configuration we have:
> dynamic.cluster.property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT2;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000" />
> <FD_ALL timeout="20000" interval="15000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48" />
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000"/>
> <pbcast.GMS print_local_addr="true" join_timeout="15000" />
> </config>
> =================================
> dynamic.cluster.distribution_property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_port="&MULTICAST_NODE_PORT1;" />
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> </config>
> ================================
> dynamic.cluster.lock.protocolStack
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT3;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_ALL timeout="20000" interval="5000"/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3 /> <pbcast.STABLE desired_avg_gossip="20000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> <FRAG2 frag_size="8096"/>
> <CENTRAL_LOCK2/>
> </config>
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (JGRP-2382) JGroups version 4.0.13.Final.jar is causing memory leaks
by Bela Ban (Jira)
[ https://issues.jboss.org/browse/JGRP-2382?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2382:
--------------------------------
* Removing UNICAST3 is a bad idea, as it is required!
* I suggest replace FD with FD_ALL
* Replace NAKACK with NAKACK2
* Remove BARRIER
> JGroups version 4.0.13.Final.jar is causing memory leaks
> --------------------------------------------------------
>
> Key: JGRP-2382
> URL: https://issues.jboss.org/browse/JGRP-2382
> Project: JGroups
> Issue Type: Feature Request
> Affects Versions: 4.0.13
> Environment: AIX machine 7.1 with JDK 1.8
> Reporter: Rashmi Acharya
> Assignee: Bela Ban
> Priority: Major
> Attachments: dumps_TEST_node1_20190918_after_3_hours.zip, dumps_TEST_node1_20190918_right_after_restart.zip, dumps_TEST_node2_20190918_after_3_hours.zip, dumps_TEST_node2_20190918_right_after_restart.zip
>
>
> We are observing a constant memory growth and leak with JGroup version 4.0.13 ..
> One of our customer is having two node cluster environment and in one node we are observing org.Group.Messages which contain org.groups.Header and org.groups.Stack.ipAddress objects.. these are not getting cleared from memory..
> We dont see any exception related to Jgroups from logs and but it is causing a gradual emory growth and OOM.
> Here is the Jgroups cluster configuration we have:
> dynamic.cluster.property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT2;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000" />
> <FD_ALL timeout="20000" interval="15000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48" />
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000"/>
> <pbcast.GMS print_local_addr="true" join_timeout="15000" />
> </config>
> =================================
> dynamic.cluster.distribution_property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_port="&MULTICAST_NODE_PORT1;" />
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> </config>
> ================================
> dynamic.cluster.lock.protocolStack
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT3;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_ALL timeout="20000" interval="5000"/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3 /> <pbcast.STABLE desired_avg_gossip="20000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> <FRAG2 frag_size="8096"/>
> <CENTRAL_LOCK2/>
> </config>
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (JGRP-2382) JGroups version 4.0.13.Final.jar is causing memory leaks
by Rashmi Acharya (Jira)
[ https://issues.jboss.org/browse/JGRP-2382?page=com.atlassian.jira.plugin.... ]
Rashmi Acharya commented on JGRP-2382:
--------------------------------------
We have recommended and removed the UNICAST3 now and changed the Distribution Property String to below value:
Observing the system after this change:
dynamic.cluster.distribution_property_string
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
<TCP bind_port="&MULTICAST_NODE_PORT1;" />
<TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
<MERGE3 min_interval="3000" max_interval="5000"/>
<FD_SOCK/>
<FD timeout="5000" max_tries="48"/>
<VERIFY_SUSPECT timeout="1500"/>
<BARRIER/>
<pbcast.NAKACK retransmit_timeout=3000 discard_delivered_msgs="true"/>
<pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000" />
<pbcast.GMS print_local_addr="true" join_timeout="5000"/>
</config>
> JGroups version 4.0.13.Final.jar is causing memory leaks
> --------------------------------------------------------
>
> Key: JGRP-2382
> URL: https://issues.jboss.org/browse/JGRP-2382
> Project: JGroups
> Issue Type: Feature Request
> Affects Versions: 4.0.13
> Environment: AIX machine 7.1 with JDK 1.8
> Reporter: Rashmi Acharya
> Assignee: Bela Ban
> Priority: Major
> Attachments: dumps_TEST_node1_20190918_after_3_hours.zip, dumps_TEST_node1_20190918_right_after_restart.zip, dumps_TEST_node2_20190918_after_3_hours.zip, dumps_TEST_node2_20190918_right_after_restart.zip
>
>
> We are observing a constant memory growth and leak with JGroup version 4.0.13 ..
> One of our customer is having two node cluster environment and in one node we are observing org.Group.Messages which contain org.groups.Header and org.groups.Stack.ipAddress objects.. these are not getting cleared from memory..
> We dont see any exception related to Jgroups from logs and but it is causing a gradual emory growth and OOM.
> Here is the Jgroups cluster configuration we have:
> dynamic.cluster.property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT2;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000" />
> <FD_ALL timeout="20000" interval="15000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48" />
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000"/>
> <pbcast.GMS print_local_addr="true" join_timeout="15000" />
> </config>
> =================================
> dynamic.cluster.distribution_property_string
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_port="&MULTICAST_NODE_PORT1;" />
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_SOCK/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3/>
> <pbcast.STABLE desired_avg_gossip="20000" max_bytes="0" stability_delay="1000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> </config>
> ================================
> dynamic.cluster.lock.protocolStack
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_addr="&HOST_ADDR;" bind_port="&MULTICAST_NODE_PORT3;"/>
> <TCPPING async_discovery="true" initial_hosts="&CLUSTER_INITIAL_HOSTS;" port_range="0" send_cache_on_join="true"/>
> <MERGE3 min_interval="3000" max_interval="5000"/>
> <FD_ALL timeout="20000" interval="5000"/>
> <FD timeout="5000" max_tries="48"/>
> <VERIFY_SUSPECT timeout="1500"/>
> <BARRIER/>
> <pbcast.NAKACK2 use_mcast_xmit="false" discard_delivered_msgs="true"/>
> <UNICAST3 /> <pbcast.STABLE desired_avg_gossip="20000" />
> <pbcast.GMS print_local_addr="true" join_timeout="5000"/>
> <FRAG2 frag_size="8096"/>
> <CENTRAL_LOCK2/>
> </config>
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (WFLY-12393) The jberet-core module should not depend on the javaee.api module
by Cheng Fang (Jira)
[ https://issues.jboss.org/browse/WFLY-12393?page=com.atlassian.jira.plugin... ]
Cheng Fang commented on WFLY-12393:
-----------------------------------
A list of specific modules from {{javaee.api}} that are needed by {{jberet-core}} module:
{code:xml}
<module name="javax.batch.api"/>
<module name="javax.enterprise.api"/>
<module name="javax.inject.api"/>
<module name="javax.annotation.api"/>
{code}
> The jberet-core module should not depend on the javaee.api module
> -----------------------------------------------------------------
>
> Key: WFLY-12393
> URL: https://issues.jboss.org/browse/WFLY-12393
> Project: WildFly
> Issue Type: Bug
> Components: Batch
> Reporter: Brian Stansberry
> Assignee: Cheng Fang
> Priority: Major
>
> The javaee.api module is just a convenience wrapper around the various EE API modules it depends on and exports. The org.jberet.jberet-core module depends on this; it should be updated to depend on the specific APIs it need.
> Depending on javaee.api is problematic because in a server that has been slimmed using Galleon we only want to depend on modules that are actually needed and not pull in other API.
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months
[JBoss JIRA] (WFLY-12578) Upgrade Yasson from 1.0.3 to 1.0.5
by James Perkins (Jira)
James Perkins created WFLY-12578:
------------------------------------
Summary: Upgrade Yasson from 1.0.3 to 1.0.5
Key: WFLY-12578
URL: https://issues.jboss.org/browse/WFLY-12578
Project: WildFly
Issue Type: Component Upgrade
Components: EE
Reporter: James Perkins
Assignee: James Perkins
Yasson is the Jakarta EE JSON-B implementation and should be upgraded from 1.0.3 to 1.0.5 which is the latest release.
--
This message was sent by Atlassian Jira
(v7.13.5#713005)
6 years, 10 months