[rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects

[Wolfgang Laun]

On 24/02/2013, Julian Klein <julianklein at gmail.com> wrote:
> Ok, thanks.  Here's what I have so far:
>
> 1) I am not retracting facts or setting expiration.  This is by design
> since I am doing the next item.  Do I need to retract globals?

Be careful with globals when you run alike sessions in parallel - the
object(s) is (are) shared.


> 2) I have disposed the session when I ran with a StatefulSession.  I
> understood this means I do not need to retract facts.  I do not attempt to
> reuse the session.  I am now trying to use a StatelessSession.

Don't change things when you try to narrow down one effect. A stateful
session lets you inspect things after fireAllRules has returned.


> 3) Unfortunately, the rule base is very large and this will take a long
> time.  I am hoping to at least get to a point where this runs end-to-end.
>  If it takes several hours, I am ok with that.
> 4) I would expect everything except "eval" statements to take advantage of
> indexing in my rules.   Are you talking about BetaNode indexing?  In all
> cases, I use the property access over getters.
> 5) Since I am using a stateless session, I would expect no recursion.
> 6) Got it.  Thanks.
> 7) Does this only apply with a shared KnowledgeBase?  What if I spawn
> multiple sessions in separate threads?

Don't do this when trying to find a memory leak.

Originally I got the impression you were running sessions one after
the other until running out of memory. Running jmap between sessions
will give you a clear indication of what is left over after dispose().
A jmap is useful to observe trends, and I don't think only
JoinNodeLeftTuple grows with the number of past sessions - don't just
look at the highrunner. Compare its results after 10, 100, 1000,...
single-threaded executions of identical sessions.

-W

>
> All in all, I reduced the allocated heap size and ran jmap as recommended
> by Wolfgang with a 6GB heap, 4 threads running sessions via a ForkJoinPool
> and less than 100K facts.  Here is a snapshot of the top hits.  I ran this
> multiple times, and just like JProfiler, the JoinNodeLeftTuple continue to
> grow and grow.  I would expect this to fluctuate up and down in count and
> size since I am not re-using a session.  Not being familiar with the
> internals of Drools, I am hoping someone could provide a sense of whether
> or not the below points to one of the issues Mark mentioned above.  Also,
> during GC events only Eden space gets freed up so these objects appear to
> be living in Tenured space.  This further concerns me that something is not
> being cleaned up.
>
> I fear it is all not so simple and will continue looking into Mark's list
> for opportunities in my code base as well as work towards simple test case.
>  I appreciate all the time taken to read my rather lengthy emails.  I am
> hoping that this detail will help others as it has me.  Thank you.
>
>  num     #instances         #bytes  class name
> ----------------------------------------------
>    1:      61754567     4940365360  org.drools.reteoo.JoinNodeLeftTuple
>    2:       1954644      109460064  org.drools.reteoo.RightTuple
>    3:        302247       24179760  org.drools.common.AgendaItem
>    4:        447814       21495072  com.mycompany.loader.FactsLoader  <- A
> bunch of callable objects that get executed to load data to send to Drools
> when the processor(s) is (are) idle.
>    5:        302247       19343808
>  org.drools.reteoo.RuleTerminalNodeLeftTuple
>    6:        897091       14353456  java.lang.Integer
>    7:        447814       14330048  java.util.RandomAccessSubList
>    8:        300383       11743152  [C
>    9:        447815       10747560
>  java.util.Collections$SynchronizedRandomAccessList
>
>
>
> On Sat, Feb 23, 2013 at 5:57 PM, Julian Klein <julianklein at gmail.com>
> wrote:
>
>> This is great.  It sounds like I have to go back to the drawing board.
>> It
>> may take a while to work through this list.  I'll circle back with
>> outcomes.
>>
>> Thank you.
>>
>