11 a.m.
Have you profiled your application yet? You can get a free trial of yourKit,
or play with the visual JVM in JDK 6..
sburrows wrote:
Hello,
After the introduction of drools into our application, we see our memory
declining over the course of 2-3 days to the point were the garbage
collector thrashes, latency becomes unbearable (> 5 sec) and we have to
restart the servers. We also notice that the garbage collector runs much
more frequently all of the time.
Here's our setup:
- drools 4.0.7
- multi-threaded web application that receives around 10 hits per second
- about 30 rules defined
- we update a global RuleBase nightly by creating a new RuleBase and
replacing the old one, so the old one should be GC'd when there are no
more sessions using it
- we create a new StatefulSession for each web application session and
call dispose() when the web session expires (30min)
- servers have 2GB RAM each
Besides the RuleBase reference to StatefulSession (which is removed by
dispose), are there any other 'hooks' which might cause memory to be
retained longer than the StatefulSession? Is our usage correct? Has
anyone else had this problem? Any other ideas how to resolve this?
Thanks for your help!
sburrows.
--
View this message in context:
http://www.nabble.com/Significant-memory-degradation-in-high-transaction-...
Sent from the drools - user mailing list archive at Nabble.com.
11:37 a.m.
New subject: Significant memory degradation in high transaction system
Yes, we have a YourKit license. If I watch certain objects like the
StateFulSession I can see one created for each web session and deleted each
time the session expires. I've run thousands of transactions through my dev
server in an all night test. The morning results after all sessions had
expired showed little difference from my starting point and nothing
significant in the rules related objects.
We also have a JMX-based free-memory logger in our production environment.
The release that introduced the rule engine in a minimal capacity shows some
memory degradation as well as much higher volatility and a much greater
'swing' when the garbage collector runs. This indicates to me that
something is generating a LOT of temporary objects that are all collected at
the next GC run.
It would be nice to get a confirmation that we have implemented this
correctly. Also I wanted to know of any other connections between a
StatefulSession or the objects added to working memory and the RuleBase. My
concern is that data might be held longer than it should be (ie: data
created in on session is held onto after that session expires) but
eventually does get collected - at least by the time all sessions expire.
Thanks for your help!
sburrows
--
View this message in context:
http://www.nabble.com/Significant-memory-degradation-in-high-transaction-...
Sent from the drools - user mailing list archive at Nabble.com.
11:55 a.m.
New subject: Significant memory degradation in high transaction system
So your app isn't leaking memory?, the GC is just taking longer, or are you
running out of heap space due to a memory leak, etc. and requiring more GC
runs? How do you heap sizes, etc. look over time. What objects are occupying
the most memory?
Also, while I'm thinking about it, it seems like it would be best practice
to put the stateful session release in a try - finally block as a best
practice? ( I haven't seen this in the docs/code samples)
Mike
sburrows wrote:
Yes, we have a YourKit license. If I watch certain objects like the
StateFulSession I can see one created for each web session and deleted
each time the session expires. I've run thousands of transactions through
my dev server in an all night test. The morning results after all
sessions had expired showed little difference from my starting point and
nothing significant in the rules related objects.
We also have a JMX-based free-memory logger in our production environment.
The release that introduced the rule engine in a minimal capacity shows
some memory degradation as well as much higher volatility and a much
greater 'swing' when the garbage collector runs. This indicates to me
that something is generating a LOT of temporary objects that are all
collected at the next GC run.
It would be nice to get a confirmation that we have implemented this
correctly. Also I wanted to know of any other connections between a
StatefulSession or the objects added to working memory and the RuleBase.
My concern is that data might be held longer than it should be (ie: data
created in on session is held onto after that session expires) but
eventually does get collected - at least by the time all sessions expire.
Thanks for your help!
sburrows
--
View this message in context:
http://www.nabble.com/Significant-memory-degradation-in-high-transaction-...
Sent from the drools - user mailing list archive at Nabble.com.
12:17 p.m.
New subject: Significant memory degradation in high transaction system
Unless the leak is somewhere else. But, in general I agree, we haven't found
any leaks so far. Nevertheless, before the release our free-memory never
dipped below 500MB and recovered to 1.6GB during periods of low activity.
Now, within 2-3 days (more quickly during heavier loads) our memory declines
to near 0 and the GC thrashes. The big question is, WHY? If there are no
memory leaks, there must be some other reason memory is being held.
sburrows
Michael Finger wrote:
So your app isn't leaking memory?, the GC is just taking longer, or are
you running out of heap space due to a memory leak, etc. and requiring
more GC runs? How do you heap sizes, etc. look over time. What objects are
occupying the most memory?
Also, while I'm thinking about it, it seems like it would be best practice
to put the stateful session release in a try - finally block as a best
practice? ( I haven't seen this in the docs/code samples)
Mike
--
View this message in context:
http://www.nabble.com/Significant-memory-degradation-in-high-transaction-...
Sent from the drools - user mailing list archive at Nabble.com.
12:23 p.m.
New subject: Significant memory degradation in high transaction system
Hmm. Are you using server side sessions?
sburrows wrote:
Unless the leak is somewhere else. But, in general I agree, we haven't
found any leaks so far. Nevertheless, before the release our free-memory
never dipped below 500MB and recovered to 1.6GB during periods of low
activity. Now, within 2-3 days (more quickly during heavier loads) our
memory declines to near 0 and the GC thrashes. The big question is, WHY?
If there are no memory leaks, there must be some other reason memory is
being held.
sburrows
Michael Finger wrote:
>
> So your app isn't leaking memory?, the GC is just taking longer, or are
> you running out of heap space due to a memory leak, etc. and requiring
> more GC runs? How do you heap sizes, etc. look over time. What objects
> are occupying the most memory?
>
>
> Also, while I'm thinking about it, it seems like it would be best
> practice to put the stateful session release in a try - finally block as
> a best practice? ( I haven't seen this in the docs/code samples)
>
>
> Mike
>
--
View this message in context:
http://www.nabble.com/Significant-memory-degradation-in-high-transaction-...
Sent from the drools - user mailing list archive at Nabble.com.
12:24 p.m.
New subject: Significant memory degradation in high transaction system
Yes.
Michael Finger wrote:
Hmm. Are you using server side sessions?
--
View this message in context:
http://www.nabble.com/Significant-memory-degradation-in-high-transaction-...
Sent from the drools - user mailing list archive at Nabble.com.
5720
days inactive
5720
days old
5 comments
2 participants
participants (2)
-
Michael Finger -
sburrows