10:01 a.m.
Ian Springer [http://community.jboss.org/people/ips] created the discussion
"way too many SLSB instances eventually cause heap to run out"
To view the discussion, visit: http://community.jboss.org/message/586260#586260
--------------------------------------------------------------
In the RHQ 3.0 server, we use JBossAS 4.2.3's bundled EJB3. A user has reported seeing
way too many SLSB instances slowly building up in heap and eventually causing the server
to run out of heap (see his post from the rhq-devel list below). Can anyone shed some
light on what could be causing it? Is it a bug in the EJB container, a bug in our
application code, or just some configuration setting we need to adjust?
Thanks,
Ian
-----
Following up a post from about a month ago. We were seeing a
persistent slow memory leak in the rhq server in tenured gen space that eventually led
to an out of memory exception after running the server for about a week. I captured a
heap dump and found hundreds of thousands of stateless session beans in memory.
Here's a snapshot from my profiler of a table of classes with greatest number of
instances.
| Name | Objects | Shallow Size |
Retained Size |
| java.util.HashMap$Entry | 1939755 | 93108240 |
189082696 |
| java.util.HashMap$Entry[] | 1090957 | 167796768
| 340273520 |
| java.util.HashMap | 1084265 | 69392960 |
408521632 |
| java.util.LinkedList$Entry | 860965 | 34438600
| 727956072 |
| org.jboss.ejb3.BaseSessionContext | 856281 | 34251240
| 34251240 |
| org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor |
856281 | 13700496 | 13700496 |
| org.rhq.enterprise.server.common.TransactionInterruptInterceptor |
856281 | 13700496 | 13700496 |
| org.jboss.ejb3.stateless.StatelessBeanContext | 856265 |
68501200 | 490959040 |
| java.lang.String | 429025 | 17161000 | 48902064
|
| char[] | 379454 | 37897872 | 37897872
|
| java.lang.Integer | 171633 | 4119192 | 4119192
|
| java.util.Hashtable$Entry | 157623 | 7565904 |
34980432 |
| java.util.TreeMap$Entry | 105496 | 6751744 |
14950816 |
| java.lang.String[] | 98401 | 4340480 | 6555536
|
| org.rhq.enterprise.server.auth.SubjectManagerBean | 91116
| 6560352 | 49567104 |
| org.rhq.enterprise.server.auth.TemporarySessionPasswordGenerator |
91116 | 3644640 | 43006752 |
| org.rhq.enterprise.server.authz.AuthorizationManagerBean | 91115
| 2186760 | 2186760 |
| org.rhq.enterprise.server.alert.AlertConditionManagerBean | 91084
| 2914688 | 2914688 |
| org.rhq.enterprise.server.alert.AlertManagerBean | 90914
| 9455056 | 9455056 |
| org.rhq.enterprise.server.alert.AlertDefinitionManagerBean | 90911
| 4363728 | 4363728 |
| org.rhq.enterprise.server.alert.AlertConditionLogManagerBean |
90903 | 5090568 | 5090568 |
| org.rhq.enterprise.server.alert.CachedConditionManagerBean | 90903
| 4363344 | 4363344 |
| org.rhq.enterprise.server.alert.AlertDampeningManagerBean | 90903
| 3636120 | 3636120 |
| org.jboss.security.SecurityAssociation$SubjectContext | 49229
| 2362992 | 2362992 |
| org.rhq.enterprise.server.cloud.instance.ServerManagerBean | 39354
| 3463152 | 3463152 |
| org.rhq.enterprise.server.cloud.CloudManagerBean | 39354
| 2833488 | 2833488 |
Here are the merged paths from the SubjectManagerBean to GCRoot:
| <All the objects> |
| org.jboss.ejb3.stateless.StatelessBeanContext |
| java.util.LinkedList$Entry |
| java.util.LinkedList$Entry |
| java.util.LinkedList |
| org.jboss.ejb3.InfinitePool |
| org.jboss.ejb3.ThreadlocalPool |
| org.jboss.ejb3.stateless.StatelessContainer |
All the other manager beans have similar merged paths. So I started to wonder why
there were so many slsb's in the ThreadlocalPools and after some digging found
this ( http://community.jboss.org/message/363520#363520
http://community.jboss.org/message/363520) thread that sort of describes what I'm
seeing. I still don't know why it's happening but it gave me something to
try. I changed the Stateless Bean pool class in ejb3-interceptors-aop.xml from
ThreadlocalPool to StrictMaxPool. Now when I run the server and watch it with my
profiler I see at max 3 SubjectManagerBeans in memory. Same appears to be true for
other slsb's. This isn't a solution to the problem but I'm hoping someone
can shed light on what's really going on. I would be happy to upload the heap dump
to somewhere public but it's almost a GB in size.
Bala Nair
SeaChange International
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/586260#586260]
Start a new discussion in EJB3 at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
8:10 a.m.
New subject: [EJB3] - Re: way too many SLSB instances eventually cause heap to run out
Carlo de Wolf [http://community.jboss.org/people/wolfc] created the discussion
"Re: way too many SLSB instances eventually cause heap to run out"
To view the discussion, visit: http://community.jboss.org/message/586896#586896
--------------------------------------------------------------
If somewhere in the call chain there is an unbounded thread pool (for example threads are
created per request) the ThreadlocalPool will keep on instantiating new instances. It is
designed to keep one instance per thread.
You need to identify which piece of code is executed by the unbounded thread pool and use
a StrictMaxPool there.
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/586896#586896]
Start a new discussion in EJB3 at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:34 a.m.
New subject: [EJB3] - Re: way too many SLSB instances eventually cause heap to run out
Robert Geisler [http://community.jboss.org/people/robert.geisler] created the discussion
"Re: way too many SLSB instances eventually cause heap to run out"
To view the discussion, visit: http://community.jboss.org/message/589426#589426
--------------------------------------------------------------
we experienced this behaviour, too. see https://community.jboss.org/message/589030#589030
https://community.jboss.org/message/589030#589030 .
many SLSB instances are created because processing JMS messages allways creates new
threads.
any approach? is there a configuration to limit thread creation??
carlo... this is by design i guess? ; )
so for JMS and EJB Timers StrictMaxPool should be the default configuration...???
regards
robert
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/589426#589426]
Start a new discussion in EJB3 at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
2:59 a.m.
New subject: [EJB3] - Re: way too many SLSB instances eventually cause heap to run out
Carlo de Wolf [http://community.jboss.org/people/wolfc] created the discussion
"Re: way too many SLSB instances eventually cause heap to run out"
To view the discussion, visit: http://community.jboss.org/message/589432#589432
--------------------------------------------------------------
Judging from the other thread you already found my reply here
http://community.jboss.org/message/364033#364033
http://community.jboss.org/message/364033#364033 :)
http://community.jboss.org/thread/110175 Re: Bug: Injected EJB objects instantiated on
every MDB call
http://community.jboss.org/message/364033#364033
http://community.jboss.org/message/364033#364033
http://community.jboss.org/message/364033#364033
http://community.jboss.org/message/364033#364033
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/589432#589432]
Start a new discussion in EJB3 at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&...]
4879
days inactive
4895
days old
3 comments
3 participants
participants (3)
-
Carlo de Wolf -
Ian Springer
-
Robert Geisler