<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<body link="#355491" alink="#4262a1" vlink="#355491" style="background: #e2e2e2; margin: 0; padding: 20px;">
<div>
<table cellpadding="0" bgcolor="#FFFFFF" border="0" cellspacing="0" style="border: 1px solid #dadada; margin-bottom: 30px; width: 100%; -moz-border-radius: 6px; -webkit-border-radius: 6px;">
<tbody>
<tr>
<td>
<table border="0" cellpadding="0" cellspacing="0" bgcolor="#FFFFFF" style="border: solid 2px #ccc; background: #dadada; width: 100%; -moz-border-radius: 6px; -webkit-border-radius: 6px;">
<tbody>
<tr>
<td bgcolor="#000000" valign="middle" height="58px" style="border-bottom: 1px solid #ccc; padding: 20px; -moz-border-radius-topleft: 3px; -moz-border-radius-topright: 3px; -webkit-border-top-right-radius: 5px; -webkit-border-top-left-radius: 5px;">
<h1 style="color: #333333; font: bold 22px Arial, Helvetica, sans-serif; margin: 0; display: block !important;">
<!-- To have a header image/logo replace the name below with your img tag -->
<!-- Email clients will render the images when the message is read so any image -->
<!-- must be made available on a public server, so that all recipients can load the image. -->
<a href="http://community.jboss.org/index.jspa" style="text-decoration: none; color: #E1E1E1">JBoss Community</a></h1>
</td>
</tr>
<tr>
<td bgcolor="#FFFFFF" style="font: normal 12px Arial, Helvetica, sans-serif; color:#333333; padding: 20px; -moz-border-radius-bottomleft: 4px; -moz-border-radius-bottomright: 4px; -webkit-border-bottom-right-radius: 5px; -webkit-border-bottom-left-radius: 5px;"><h3 style="margin: 10px 0 5px; font-size: 17px; font-weight: normal;">
way too many SLSB instances eventually cause heap to run out
</h3>
<span style="margin-bottom: 10px;">
created by <a href="http://community.jboss.org/people/ips">Ian Springer</a> in <i>EJB3</i> - <a href="http://community.jboss.org/message/586260#586260">View the full discussion</a>
</span>
<hr style="margin: 20px 0; border: none; background-color: #dadada; height: 1px;">
<div class="jive-rendered-content"><p>In the RHQ 3.0 server, we use JBossAS 4.2.3's bundled EJB3. A user has reported seeing way too many SLSB instances slowly building up in heap and eventually causing the server to run out of heap (see his post from the rhq-devel list below). Can anyone shed some light on what could be causing it? Is it a bug in the EJB container, a bug in our application code, or just some configuration setting we need to adjust?</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>Thanks,</p><p>Ian</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>-----</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><blockquote class="jive-quote"><p>Following up a post from about a month ago.  We were seeing a    persistent slow memory leak in the rhq server in tenured gen space    that eventually led to an out of memory exception after running the    server for about a week.  I captured a heap dump and found hundreds    of thousands of stateless session beans in memory.  Here's a    snapshot from my profiler of a table of classes with greatest number    of instances.  </p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p class="moz-text-html" lang="x-western" style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse;" width="833">      <col width="608"></col> <col span="3" width="75"></col> <tbody>        <tr>          <td height="13" style="border-collapse: collapse;" width="608">Name</td>          <td style="border-collapse: collapse;" width="75">Objects</td>          <td style="border-collapse: collapse;" width="75">Shallow Size</td>          <td style="border-collapse: collapse;" width="75">Retained Size</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.HashMap$Entry</td>          <td align="right" style="border-collapse: collapse;">1939755</td>          <td align="right" style="border-collapse: collapse;">93108240</td>          <td align="right" style="border-collapse: collapse;">189082696</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.HashMap$Entry[]</td>          <td align="right" style="border-collapse: collapse;">1090957</td>          <td align="right" style="border-collapse: collapse;">167796768</td>          <td align="right" style="border-collapse: collapse;">340273520</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.HashMap</td>          <td align="right" style="border-collapse: collapse;">1084265</td>          <td align="right" style="border-collapse: collapse;">69392960</td>          <td align="right" style="border-collapse: collapse;">408521632</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.LinkedList$Entry</td>          <td align="right" style="border-collapse: collapse;">860965</td>          <td align="right" style="border-collapse: collapse;">34438600</td>          <td align="right" style="border-collapse: collapse;">727956072</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.ejb3.BaseSessionContext</td>          <td align="right" style="border-collapse: collapse;">856281</td>          <td align="right" style="border-collapse: collapse;">34251240</td>          <td align="right" style="border-collapse: collapse;">34251240</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor</td>          <td align="right" style="border-collapse: collapse;">856281</td>          <td align="right" style="border-collapse: collapse;">13700496</td>          <td align="right" style="border-collapse: collapse;">13700496</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.common.TransactionInterruptInterceptor</td>          <td align="right" style="border-collapse: collapse;">856281</td>          <td align="right" style="border-collapse: collapse;">13700496</td>          <td align="right" style="border-collapse: collapse;">13700496</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.ejb3.stateless.StatelessBeanContext</td>          <td align="right" style="border-collapse: collapse;">856265</td>          <td align="right" style="border-collapse: collapse;">68501200</td>          <td align="right" style="border-collapse: collapse;">490959040</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.lang.String</td>          <td align="right" style="border-collapse: collapse;">429025</td>          <td align="right" style="border-collapse: collapse;">17161000</td>          <td align="right" style="border-collapse: collapse;">48902064</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">char[]</td>          <td align="right" style="border-collapse: collapse;">379454</td>          <td align="right" style="border-collapse: collapse;">37897872</td>          <td align="right" style="border-collapse: collapse;">37897872</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.lang.Integer</td>          <td align="right" style="border-collapse: collapse;">171633</td>          <td align="right" style="border-collapse: collapse;">4119192</td>          <td align="right" style="border-collapse: collapse;">4119192</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.Hashtable$Entry</td>          <td align="right" style="border-collapse: collapse;">157623</td>          <td align="right" style="border-collapse: collapse;">7565904</td>          <td align="right" style="border-collapse: collapse;">34980432</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.TreeMap$Entry</td>          <td align="right" style="border-collapse: collapse;">105496</td>          <td align="right" style="border-collapse: collapse;">6751744</td>          <td align="right" style="border-collapse: collapse;">14950816</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.lang.String[]</td>          <td align="right" style="border-collapse: collapse;">98401</td>          <td align="right" style="border-collapse: collapse;">4340480</td>          <td align="right" style="border-collapse: collapse;">6555536</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.auth.SubjectManagerBean</td>          <td align="right" style="border-collapse: collapse;">91116</td>          <td align="right" style="border-collapse: collapse;">6560352</td>          <td align="right" style="border-collapse: collapse;">49567104</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.auth.TemporarySessionPasswordGenerator</td>          <td align="right" style="border-collapse: collapse;">91116</td>          <td align="right" style="border-collapse: collapse;">3644640</td>          <td align="right" style="border-collapse: collapse;">43006752</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.authz.AuthorizationManagerBean</td>          <td align="right" style="border-collapse: collapse;">91115</td>          <td align="right" style="border-collapse: collapse;">2186760</td>          <td align="right" style="border-collapse: collapse;">2186760</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.alert.AlertConditionManagerBean</td>          <td align="right" style="border-collapse: collapse;">91084</td>          <td align="right" style="border-collapse: collapse;">2914688</td>          <td align="right" style="border-collapse: collapse;">2914688</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.alert.AlertManagerBean</td>          <td align="right" style="border-collapse: collapse;">90914</td>          <td align="right" style="border-collapse: collapse;">9455056</td>          <td align="right" style="border-collapse: collapse;">9455056</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.alert.AlertDefinitionManagerBean</td>          <td align="right" style="border-collapse: collapse;">90911</td>          <td align="right" style="border-collapse: collapse;">4363728</td>          <td align="right" style="border-collapse: collapse;">4363728</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.alert.AlertConditionLogManagerBean</td>          <td align="right" style="border-collapse: collapse;">90903</td>          <td align="right" style="border-collapse: collapse;">5090568</td>          <td align="right" style="border-collapse: collapse;">5090568</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.alert.CachedConditionManagerBean</td>          <td align="right" style="border-collapse: collapse;">90903</td>          <td align="right" style="border-collapse: collapse;">4363344</td>          <td align="right" style="border-collapse: collapse;">4363344</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.alert.AlertDampeningManagerBean</td>          <td align="right" style="border-collapse: collapse;">90903</td>          <td align="right" style="border-collapse: collapse;">3636120</td>          <td align="right" style="border-collapse: collapse;">3636120</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.security.SecurityAssociation$SubjectContext</td>          <td align="right" style="border-collapse: collapse;">49229</td>          <td align="right" style="border-collapse: collapse;">2362992</td>          <td align="right" style="border-collapse: collapse;">2362992</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.cloud.instance.ServerManagerBean</td>          <td align="right" style="border-collapse: collapse;">39354</td>          <td align="right" style="border-collapse: collapse;">3463152</td>          <td align="right" style="border-collapse: collapse;">3463152</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.rhq.enterprise.server.cloud.CloudManagerBean</td>          <td align="right" style="border-collapse: collapse;">39354</td>          <td align="right" style="border-collapse: collapse;">2833488</td>          <td align="right" style="border-collapse: collapse;">2833488</td>        </tr>      </tbody>    </table><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>    Here are the merged paths from the SubjectManagerBean to GCRoot:</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse;" width="357">      <col width="357"></col> <tbody>        <tr>          <td height="13" style="border-collapse: collapse;" width="357"><All the objects></td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.ejb3.stateless.StatelessBeanContext</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.LinkedList$Entry</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.LinkedList$Entry</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">java.util.LinkedList</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.ejb3.InfinitePool</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.ejb3.ThreadlocalPool</td>        </tr>        <tr>          <td height="13" style="border-collapse: collapse;">org.jboss.ejb3.stateless.StatelessContainer</td>        </tr>      </tbody>    </table><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>    All the other manager beans have similar merged paths.  So I started    to wonder why there were so many slsb's in the ThreadlocalPools and    after some digging found this    (<a class="" href="http://community.jboss.org/message/363520#363520">http://community.jboss.org/message/363520</a>) thread that sort of    describes what I'm seeing.  I still don't know why it's happening    but it gave me something to try.  I changed the Stateless Bean pool    class in ejb3-interceptors-aop.xml from ThreadlocalPool to    StrictMaxPool.  Now when I run the server and watch it with my    profiler I see at max 3 SubjectManagerBeans in memory.  Same appears    to be true for other slsb's.  This isn't a solution to the problem    but I'm hoping someone can shed light on what's really going on.  I    would be happy to upload the heap dump to somewhere public but it's    almost a GB in size.  </p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>    Bala Nair</p><p>    SeaChange International</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p></blockquote></div>
<div style="background-color: #f4f4f4; padding: 10px; margin-top: 20px;">
<p style="margin: 0;">Reply to this message by <a href="http://community.jboss.org/message/586260#586260">going to Community</a></p>
<p style="margin: 0;">Start a new discussion in EJB3 at <a href="http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2029">Community</a></p>
</div></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</body>
</html>