[rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects
Wolfgang Laun
wolfgang.laun at gmail.com
Sun Feb 24 01:12:18 EST 2013
On 24/02/2013, Julian Klein <julianklein at gmail.com> wrote:
> Ok, thanks. Here's what I have so far:
>
> 1) I am not retracting facts or setting expiration. This is by design
> since I am doing the next item. Do I need to retract globals?
Be careful with globals when you run alike sessions in parallel - the
object(s) is (are) shared.
> 2) I have disposed the session when I ran with a StatefulSession. I
> understood this means I do not need to retract facts. I do not attempt to
> reuse the session. I am now trying to use a StatelessSession.
Don't change things when you try to narrow down one effect. A stateful
session lets you inspect things after fireAllRules has returned.
> 3) Unfortunately, the rule base is very large and this will take a long
> time. I am hoping to at least get to a point where this runs end-to-end.
> If it takes several hours, I am ok with that.
> 4) I would expect everything except "eval" statements to take advantage of
> indexing in my rules. Are you talking about BetaNode indexing? In all
> cases, I use the property access over getters.
> 5) Since I am using a stateless session, I would expect no recursion.
> 6) Got it. Thanks.
> 7) Does this only apply with a shared KnowledgeBase? What if I spawn
> multiple sessions in separate threads?
Don't do this when trying to find a memory leak.
Originally I got the impression you were running sessions one after
the other until running out of memory. Running jmap between sessions
will give you a clear indication of what is left over after dispose().
A jmap is useful to observe trends, and I don't think only
JoinNodeLeftTuple grows with the number of past sessions - don't just
look at the highrunner. Compare its results after 10, 100, 1000,...
single-threaded executions of identical sessions.
-W
>
> All in all, I reduced the allocated heap size and ran jmap as recommended
> by Wolfgang with a 6GB heap, 4 threads running sessions via a ForkJoinPool
> and less than 100K facts. Here is a snapshot of the top hits. I ran this
> multiple times, and just like JProfiler, the JoinNodeLeftTuple continue to
> grow and grow. I would expect this to fluctuate up and down in count and
> size since I am not re-using a session. Not being familiar with the
> internals of Drools, I am hoping someone could provide a sense of whether
> or not the below points to one of the issues Mark mentioned above. Also,
> during GC events only Eden space gets freed up so these objects appear to
> be living in Tenured space. This further concerns me that something is not
> being cleaned up.
>
> I fear it is all not so simple and will continue looking into Mark's list
> for opportunities in my code base as well as work towards simple test case.
> I appreciate all the time taken to read my rather lengthy emails. I am
> hoping that this detail will help others as it has me. Thank you.
>
> num #instances #bytes class name
> ----------------------------------------------
> 1: 61754567 4940365360 org.drools.reteoo.JoinNodeLeftTuple
> 2: 1954644 109460064 org.drools.reteoo.RightTuple
> 3: 302247 24179760 org.drools.common.AgendaItem
> 4: 447814 21495072 com.mycompany.loader.FactsLoader <- A
> bunch of callable objects that get executed to load data to send to Drools
> when the processor(s) is (are) idle.
> 5: 302247 19343808
> org.drools.reteoo.RuleTerminalNodeLeftTuple
> 6: 897091 14353456 java.lang.Integer
> 7: 447814 14330048 java.util.RandomAccessSubList
> 8: 300383 11743152 [C
> 9: 447815 10747560
> java.util.Collections$SynchronizedRandomAccessList
>
>
>
> On Sat, Feb 23, 2013 at 5:57 PM, Julian Klein <julianklein at gmail.com>
> wrote:
>
>> This is great. It sounds like I have to go back to the drawing board.
>> It
>> may take a while to work through this list. I'll circle back with
>> outcomes.
>>
>> Thank you.
>>
>
More information about the rules-users
mailing list