[jboss-jira] [Red Hat JIRA] (JGRP-2135) OOM with JGroups 3.6.11.

OOM with JGroups 3.6.11. ------------------------ Key: JGRP-2135 URL: https://issues.redhat.com/browse/JGRP-2135 Project: JGroups Issue Type: Bug Affects Versions: 3.6.11 Reporter: Zoltan Farkas Assignee: Bela Ban Priority: Major Fix For: 3.6.12, 3.6.20.Final Attachments: AppCluster.xml We are running our JVMs with : -XX:OnOutOfMemoryError="kill -9 %p" we have been experiencing OOMs fairly often, and the OOMs happen at: {code} Object / Stack Frame |Name | Shallow Heap | Retained Heap |Context Class Loader |Is Daemon --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- java.lang.Thread @ 0x81bdf838 |Connection.Receiver [144.77.77.53:50363 - 144.77.77.53:50363],sis-cluster.service,prodpmwsv5-6461| 120 | 456 |sun.misc.Launcher$AppClassLoader @ 0x800175a8|false |- at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48) | | | | | |- at org.jgroups.blocks.cs.TcpConnection$Receiver.run()V (TcpConnection.java:310)| | | | | |- at java.lang.Thread.run()V (Thread.java:745) | | | | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- {code} the Code where it happens is in TcpConnection.java: {code} while(canRun()) { try { int len=in.readInt(); if(buffer == null || buffer.length < len) buffer=new byte[len]; in.readFully(buffer, 0, len); updateLastAccessed(); server.receive(peer_addr, buffer, 0, len); } catch(OutOfMemoryError mem_ex) { t=mem_ex; break; // continue; } catch(IOException io_ex) { t=io_ex; break; } catch(Throwable e) { } } {code} when allocating: buffer=new byte[len]; it looks to me that some invalid large value is received and the process OOMs when allocating a huge byte array Running JVMs without kill on OOM would make this issue "dissapear" in the sense that it is swallowed by: {code} catch(OutOfMemoryError mem_ex) { t=mem_ex; break; // continue; } {code} Handling OutOfMemoryError is a strange implementation choice... instead a size limit should be employed to protect from receiving invalid sizes... My heap limit is 1GB and my heap dumps are 50Mb so the attempted allocation size is huge...