[jboss-jira] [JBoss JIRA] (WFLY-6686) JGroups ForkChannel can hold references to Hibernate SessionFactoryImpl which cause memory leak

JGroups ForkChannel can hold references to Hibernate SessionFactoryImpl which cause memory leak ----------------------------------------------------------------------------------------------- Key: WFLY-6686 URL: https://issues.jboss.org/browse/WFLY-6686 Project: WildFly Issue Type: Bug Components: Clustering Environment: Using WF 10 build #2291 (Jun 6, 2016 3:40:44 PM) https://ci.jboss.org/hudson/job/WildFly-latest-master/2291/changes Reporter: Mathieu Lachance Assignee: Paul Ferraro Priority: Blocker Fix For: 10.1.0.Final We are using Hibernate second level cache through JPA configured as such: {code} <properties> <property name="hibernate.cache.use_query_cache" value="true"/> <property name="hibernate.cache.use_second_level_cache" value="true"/> <property name="hibernate.cache.region.factory_class" value="org.jboss.as.jpa.hibernate5.infinispan.SharedInfinispanRegionFactory"/> </properties> {code} After heap dump inspection, it seems that the JGroups ForkChannel identified by "hibernate" can hold a listener that hold the SessionFactoryImpl which then hold the whole application classloader. When undeploying the application, this can lead to classloader leak. I took an heap dump of such scenario and analysed it using eclipse memory analyzer (MAT) and here is the result: {code} Class Name | Ref. Objects | Shallow Heap | Ref. Shallow Heap | Retained Heap ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ channel org.jgroups.JChannel @ 0xc0eac6a0 | 1 | 112 | 136 | 448 '- channel_listeners java.util.concurrent.CopyOnWriteArraySet @ 0xc0ecd260 | 1 | 16 | 136 | 112 '- al java.util.concurrent.CopyOnWriteArrayList @ 0xc0ecd270 | 1 | 24 | 136 | 96 '- array java.lang.Object[2] @ 0xc0ecd2b8 | 1 | 24 | 136 | 24 '- [0] org.jgroups.fork.ForkChannel @ 0xc103d250 | 1 | 120 | 136 | 1 328 '- channel_listeners java.util.concurrent.CopyOnWriteArraySet @ 0xc103d350 | 1 | 16 | 136 | 968 '- al java.util.concurrent.CopyOnWriteArrayList @ 0xc103d360 | 1 | 24 | 136 | 952 '- array java.lang.Object[1] @ 0xc103d3a8 | 1 | 24 | 136 | 880 '- [0] org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher @ 0xc103d1f0| 1 | 96 | 136 | 856 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ {code} To remove that leak I ugily patched JGroups ForkChannel close method as such: {code} /** Closes the fork-channel, essentially setting its state to CLOSED. Note that - contrary to a regular channel - * a closed fork-channel can be connected again: this means re-attaching the fork-channel to the main-channel*/ @Override public void close() { ((ForkProtocolStack)prot_stack).remove(fork_channel_id); if(state == State.CLOSED) return; disconnect(); // leave group if connected prot_stack.destroy(); state=State.CLOSED; notifyChannelClosed(this); this.clearChannelListeners(); // <-- this is the line I added } {code} With that change in place, the memory leak is gone. I highly doubt though this is an acceptable fix. Though it does confirm my theory. I doubt that JGroups is really the culprit -- I'm more in the thinking that the "thing managing" JGroups is the culprit. Since I'm not an expert around that field I've opened the issue against the Wildfly project. Feel free to move it to the proper project. If you need any other informations let me know.