[jboss-jira] [JBoss JIRA] (JGRP-2135) OOM with JGroups 3.6.11.

[Zoltan Farkas (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13341691#comment-13341691 ] 

Zoltan Farkas commented on JGRP-2135:
-------------------------------------

will look into my logs,  I am sure in my env the issue still happens, will probably have to also read whatever comes on that input stream and add it to the exception message and hopefully it will help figuring out what is going on...

In any case my opinion is that catching OOM here is not the right implementation choice...

> OOM with JGroups 3.6.11.
> ------------------------
>
>                 Key: JGRP-2135
>                 URL: https://issues.jboss.org/browse/JGRP-2135
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 3.6.11
>            Reporter: Zoltan Farkas
>            Assignee: Bela Ban
>             Fix For: 3.6.12
>
>
> We are running our JVMs with : -XX:OnOutOfMemoryError="kill -9 %p" 
> we have been experiencing OOMs fairly often, and the OOMs happen at:
> {code}
> Object / Stack Frame                                                              |Name                                                                                             | Shallow Heap | Retained Heap |Context Class Loader                         |Is Daemon
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> java.lang.Thread @ 0x81bdf838                                                     |Connection.Receiver [144.77.77.53:50363 - 144.77.77.53:50363],sis-cluster.service,prodpmwsv5-6461|          120 |           456 |sun.misc.Launcher$AppClassLoader @ 0x800175a8|false
> |- at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48)             |                                                                                                 |              |               |                                             |
> |- at org.jgroups.blocks.cs.TcpConnection$Receiver.run()V (TcpConnection.java:310)|                                                                                                 |              |               |                                             |
> |- at java.lang.Thread.run()V (Thread.java:745)                                   |                                                                                                 |              |               |                                             |
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
> the Code where it happens is in TcpConnection.java:
> {code}
> while(canRun()) {
>                 try {
>                     int len=in.readInt();
>                     if(buffer == null || buffer.length < len)
>                         buffer=new byte[len];
>                     in.readFully(buffer, 0, len);
>                     updateLastAccessed();
>                     server.receive(peer_addr, buffer, 0, len);
>                 }
>                 catch(OutOfMemoryError mem_ex) {
>                     t=mem_ex;
>                     break; // continue;
>                 }
>                 catch(IOException io_ex) {
>                     t=io_ex;
>                     break;
>                 }
>                 catch(Throwable e) {
>                 }
>             }
> {code}
> when allocating:   buffer=new byte[len];
> it looks to me that some invalid large value is received and the process OOMs when allocating a huge byte array
> Running JVMs without kill on OOM would make this issue "dissapear" in the sense that it is swallowed by:
> {code}
>                 catch(OutOfMemoryError mem_ex) {
>                     t=mem_ex;
>                     break; // continue;
>                 }
> {code}
> Handling OutOfMemoryError is a strange implementation choice... 
> instead a size limit should be employed to protect from receiving invalid sizes...
> My heap limit is 1GB and my heap dumps are 50Mb so the attempted allocation size is huge...



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)