[rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects

[Mark Proctor]

On 24 Feb 2013, at 04:11, Julian Klein <julianklein at gmail.com> wrote:

> Ok, thanks.  Here's what I have so far:
> 
> 1) I am not retracting facts or setting expiration.  This is by design since I am doing the next item.  Do I need to retract globals?
> 2) I have disposed the session when I ran with a StatefulSession.  I understood this means I do not need to retract facts.  I do not attempt to reuse the session.  I am now trying to use a StatelessSession.
it makes no difference. Stateless is just a convenience wrapper for stateful, that avoids having to call "dispose" at the end.
> 3) Unfortunately, the rule base is very large and this will take a long time.  I am hoping to at least get to a point where this runs end-to-end.  If it takes several hours, I am ok with that.
Don't let an application get this big, before you start testing.
> 4) I would expect everything except "eval" statements to take advantage of indexing in my rules.   Are you talking about BetaNode indexing?  In all cases, I use the property access over getters.
no not everything is indexed. Only '==' operators are indexed, and you must put the field name to the left of the operator. Until recently anything with nested accessors would not be indexed either, but that should have improved in 5.5
> 5) Since I am using a stateless session, I would expect no recursion.
Statless session is just a wrapper for stateful. If the consequences modify or insert data, you get recursions.
> 6) Got it.  Thanks.
> 7) Does this only apply with a shared KnowledgeBase?  What if I spawn multiple sessions in separate threads?
The thread lock is per session.
> 
> All in all, I reduced the allocated heap size and ran jmap as recommended by Wolfgang with a 6GB heap, 4 threads running sessions via a ForkJoinPool and less than 100K facts.  Here is a snapshot of the top hits.  I ran this multiple times, and just like JProfiler, the JoinNodeLeftTuple continue to grow and grow.  I would expect this to fluctuate up and down in count and size since I am not re-using a session.  Not being familiar with the internals of Drools, I am hoping someone could provide a sense of whether or not the below points to one of the issues Mark mentioned above.  Also, during GC events only Eden space gets freed up so these objects appear to be living in Tenured space.  This further concerns me that something is not being cleaned up.  
> 
> I fear it is all not so simple and will continue looking into Mark's list for opportunities in my code base as well as work towards simple test case.  I appreciate all the time taken to read my rather lengthy emails.  I am hoping that this detail will help others as it has me.  Thank you.
> 
>  num     #instances         #bytes  class name
> ----------------------------------------------
>    1:      61754567     4940365360  org.drools.reteoo.JoinNodeLeftTuple
>    2:       1954644      109460064  org.drools.reteoo.RightTuple
>    3:        302247       24179760  org.drools.common.AgendaItem
>    4:        447814       21495072  com.mycompany.loader.FactsLoader  <- A bunch of callable objects that get executed to load data to send to Drools when the processor(s) is (are) idle.
>    5:        302247       19343808  org.drools.reteoo.RuleTerminalNodeLeftTuple
>    6:        897091       14353456  java.lang.Integer
>    7:        447814       14330048  java.util.RandomAccessSubList
>    8:        300383       11743152  [C
>    9:        447815       10747560  java.util.Collections$SynchronizedRandomAccessList
> 
> 
> 
> On Sat, Feb 23, 2013 at 5:57 PM, Julian Klein <julianklein at gmail.com> wrote:
> This is great.  It sounds like I have to go back to the drawing board.  It may take a while to work through this list.  I'll circle back with outcomes.
> 
> Thank you.
> 
> 
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20130224/450e268e/attachment-0001.html