Re: [rules-users] GC Overhead Limit Exceeded and 1B JoinLeftNode Objects

On 24/02/2013, Julian Klein <julianklein(a)gmail.com&gt; wrote: > Ok, thanks. Here's what I have so far: > > 1) I am not retracting facts or setting expiration. This is by design > since I am doing the next item. Do I need to retract globals? Be careful with globals when you run alike sessions in parallel - the object(s) is (are) shared. > 2) I have disposed the session when I ran with a StatefulSession. I > understood this means I do not need to retract facts. I do not attempt to > reuse the session. I am now trying to use a StatelessSession. Don't change things when you try to narrow down one effect. A stateful session lets you inspect things after fireAllRules has returned. > 3) Unfortunately, the rule base is very large and this will take a long > time. I am hoping to at least get to a point where this runs end-to-end. > If it takes several hours, I am ok with that. > 4) I would expect everything except "eval" statements to take advantage of > indexing in my rules. Are you talking about BetaNode indexing? In all > cases, I use the property access over getters. > 5) Since I am using a stateless session, I would expect no recursion. > 6) Got it. Thanks. > 7) Does this only apply with a shared KnowledgeBase? What if I spawn > multiple sessions in separate threads? Don't do this when trying to find a memory leak. Originally I got the impression you were running sessions one after the other until running out of memory. Running jmap between sessions will give you a clear indication of what is left over after dispose(). A jmap is useful to observe trends, and I don't think only JoinNodeLeftTuple grows with the number of past sessions - don't just look at the highrunner. Compare its results after 10, 100, 1000,... single-threaded executions of identical sessions. -W > > All in all, I reduced the allocated heap size and ran jmap as recommended > by Wolfgang with a 6GB heap, 4 threads running sessions via a ForkJoinPool > and less than 100K facts. Here is a snapshot of the top hits. I ran this > multiple times, and just like JProfiler, the JoinNodeLeftTuple continue to > grow and grow. I would expect this to fluctuate up and down in count and > size since I am not re-using a session. Not being familiar with the > internals of Drools, I am hoping someone could provide a sense of whether > or not the below points to one of the issues Mark mentioned above. Also, > during GC events only Eden space gets freed up so these objects appear to > be living in Tenured space. This further concerns me that something is not > being cleaned up. > > I fear it is all not so simple and will continue looking into Mark's list > for opportunities in my code base as well as work towards simple test case. > I appreciate all the time taken to read my rather lengthy emails. I am > hoping that this detail will help others as it has me. Thank you. > > num #instances #bytes class name > ---------------------------------------------- > 1: 61754567 4940365360 org.drools.reteoo.JoinNodeLeftTuple > 2: 1954644 109460064 org.drools.reteoo.RightTuple > 3: 302247 24179760 org.drools.common.AgendaItem > 4: 447814 21495072 com.mycompany.loader.FactsLoader <- A > bunch of callable objects that get executed to load data to send to Drools > when the processor(s) is (are) idle. > 5: 302247 19343808 > org.drools.reteoo.RuleTerminalNodeLeftTuple > 6: 897091 14353456 java.lang.Integer > 7: 447814 14330048 java.util.RandomAccessSubList > 8: 300383 11743152 [C > 9: 447815 10747560 > java.util.Collections$SynchronizedRandomAccessList > > > > On Sat, Feb 23, 2013 at 5:57 PM, Julian Klein <julianklein(a)gmail.com&gt; > wrote: > >> This is great. It sounds like I have to go back to the drawing board. >> It >> may take a while to work through this list. I'll circle back with >> outcomes. >> >> Thank you. >> > _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users