Re: [rules-dev] [rules-users] Memory Leak in 5.4.0 and 5.5.0 [was: Garbage collection and sliding windows (Drools 5.5.0 Final)]

If we can get a unit test for this, we are just about to put out 5.6 (which has a lot of important fixes in it already), we'll try and have it fixed for then. Mark On 17 Feb 2013, at 08:55, Wolfgang Laun <wolfgang.laun(a)gmail.com&gt; wrote: > With Drools 5.4.0.Final and 5.5.0.Final, automatic event retraction in > combination with a sliding window is broken, i.e., it suffers from a > memory leak. Full code for a demo has been posted by tai-atari; so I > just add some diagnostics that clearly indicate what goes wrong. > > In the snapshot taken with jmap -histo:live after running the program > for several seconds and the insertion of 15800 AnEvent facts you can > see: > * several classes with 2000 instances each - AnEvents and the ones > required for bookkeeping the future expiry > * EventFactHandle, WindowTupleList, ObjectHashMap$ObjectEntry all > growing without restraint, indicating a memory leak. > > > num #instances #bytes class name > ---------------------------------------------- > 1: 8747 8696096 [S > 2: 26985 4110776 <constMethodKlass> > 3: 26985 2161888 <methodKlass> > 4: 47816 2046304 <symbolKlass> > 5: 2414 1482672 <constantPoolKlass> > 6: 6142 1434280 [I > 7: 13800 1324800 org.drools.common.EventFactHandle > 8: 2414 1056424 <instanceKlassKlass> > 9: 1891 854664 <constantPoolCacheKlass> > 10: 8348 644704 [C > 11: 13800 441600 org.drools.reteoo.WindowTupleList > 12: 2571 426976 [B > 13: 15800 379200 > org.drools.core.util.ObjectHashMap$ObjectEntry > 14: 2672 256512 java.lang.Class > 15: 545 246776 <methodDataKlass> > 16: 7852 188448 java.lang.String > 17: 4084 185496 [[I > 18: 2000 176000 > org.drools.common.PropagationContextImpl > 19: 8 164288 [Lorg.drools.core.util.Entry; > 20: 2000 160000 org.drools.common.AgendaItem > 21: 2005 128320 org.drools.reteoo.WindowTuple > 22: 2000 128000 > org.drools.reteoo.RuleTerminalNodeLeftTuple > 23: 2001 96048 java.util.concurrent.FutureTask$Sync > 24: 2001 80040 > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask > 25: 1165 79152 [J > 26: 229 75112 <objArrayKlassKlass> > 27: 2001 64032 > org.drools.time.impl.JDKTimerService$JDKJobHandle > 28: 2001 64032 > org.drools.time.impl.DefaultTimerJobInstance > 29: 423 54728 [Ljava.lang.Object; > 30: 2150 51600 java.util.LinkedList$Entry > 31: 2052 49248 java.util.LinkedList > 32: 2002 48048 > org.drools.core.util.ObjectHashSet$ObjectEntry > 33: 2001 48024 java.util.Date > 34: 2000 48000 > org.drools.reteoo.ObjectTypeNode$ExpireJobContext > 35: 1958 46992 java.util.HashMap$Entry > 36: 2261 36176 java.lang.Integer > 37: 2006 32096 > java.util.concurrent.atomic.AtomicBoolean > 38: 2001 32016 > org.drools.time.impl.PointInTimeTrigger > 39: 2000 32000 > org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction > 40: 2000 32000 memcons.AnEvent > > > > > On 17/02/2013, tai-atari <p00temkin(a)gmail.com&gt; wrote: >> Hi laune, >> >> Really appreciate you trying to reproduce the issue. Since you are not >> experiencing the same behavior I'm obviously missing something. I've >> attached a very simple example below which reproduces the scenario for >> the >> Drools versions I have tried so far: 5.2, 5.4 and 5.5. >> >> Basically this example runs an infinite loop which inserts 200 events >> about >> every second, and keeps a sliding window of 10 seconds. Every loop will >> print the current wm event count, which will increase to 2000 and stay >> put. >> This is where I expected the expired events to be available for garbage >> collection (since only 2000 are concurrently active in wm). But VisualVM >> shows that org.drools.common.EventFactHandle uses an increasing amount >> of >> memory over time. >> >> Start.java: >> ================================== >> package org.drools.example; >> >> import java.util.Random; >> >> import org.drools.KnowledgeBase; >> import org.drools.KnowledgeBaseConfiguration; >> import org.drools.KnowledgeBaseFactory; >> import org.drools.builder.KnowledgeBuilder; >> import org.drools.builder.KnowledgeBuilderError; >> import org.drools.builder.KnowledgeBuilderErrors; >> import org.drools.builder.KnowledgeBuilderFactory; >> import org.drools.builder.ResourceType; >> import org.drools.conf.EventProcessingOption; >> import org.drools.io.ResourceFactory; >> import org.drools.runtime.StatefulKnowledgeSession; >> import org.drools.runtime.rule.WorkingMemoryEntryPoint; >> >> public class Start { >> >> public static final void main(String[] args) { >> try { >> >> System.out.println("Init"); >> KnowledgeBase kbase = readKnowledgeBase(); >> StatefulKnowledgeSession ksession = >> kbase.newStatefulKnowledgeSession(); >> WorkingMemoryEntryPoint eventStream = >> ksession.getWorkingMemoryEntryPoint("TheEventStream"); >> Random randGen = new Random(); >> >> while( true ) { >> >> // Insert 200 >> int x = 0; >> while( x < 200 ) { >> AnEvent anEvent = new AnEvent(); >> anEvent.setSource(randGen.nextInt()); >> eventStream.insert(anEvent); >> ksession.fireAllRules(); >> x++; >> } >> >> System.out.println("current event count in wm" + ": " + >> eventStream.getFactCount()); >> Thread.sleep(1000); >> } >> >> //ksession.dispose(); >> >> } catch (Throwable t) { >> t.printStackTrace(); >> } >> } >> >> private static KnowledgeBase readKnowledgeBase() throws Exception { >> >> KnowledgeBuilder kbuilder = >> KnowledgeBuilderFactory.newKnowledgeBuilder(); >> kbuilder.add(ResourceFactory.newClassPathResource("Sample.drl"), >> ResourceType.DRL); >> KnowledgeBuilderErrors errors = kbuilder.getErrors(); >> if (errors.size() > 0) { >> for (KnowledgeBuilderError error: errors) { >> System.err.println(error); >> } >> throw new IllegalArgumentException("Could not parse >> knowledge."); >> } >> >> final KnowledgeBaseConfiguration kbConfig = >> KnowledgeBaseFactory.newKnowledgeBaseConfiguration(); >> kbConfig.setOption(EventProcessingOption.STREAM); >> KnowledgeBase kbase = >> KnowledgeBaseFactory.newKnowledgeBase(kbConfig); >> >> kbase.addKnowledgePackages(kbuilder.getKnowledgePackages()); >> return kbase; >> } >> >> } >> ================================== >> >> AnEvent.java: >> ================================== >> package org.drools.example; >> >> public class AnEvent { >> >> private Integer source; >> >> public AnEvent () { >> } >> >> public Integer getSource() { >> return source; >> } >> >> public void setSource(Integer source) { >> this.source = source; >> } >> >> } >> ================================== >> >> Sample.drl >> ================================== >> package org.drools.example >> import org.drools.example.AnEvent; >> >> declare AnEvent >> @role( event ) >> end >> >> rule "Event print" >> when >> AnEvent ( $src: source ) over window:time(10s) from entry-point >> TheEventStream >> then >> System.out.println("---------> Received AnEvent from " + $src); >> end >> ================================== >> >> Also tried using an immutable pojo as you suggested cusmaimatteo, with >> >> AnEvent.java: >> ================================== >> package org.drools.example; >> >> public class AnEvent { >> >> private final Integer source; >> >> public AnEvent (Integer i) { >> this.source = i; >> } >> >> public Integer getSource() { >> return source; >> } >> } >> ================================== >> >> and using instead using >> >> eventStream.insert(new AnEvent(randGen.nextInt())); >> >> but with the same memory issues noted. >> >> >> >> >> >> -- >> View this message in context: >> http://drools.46999.n3.nabble.com/Garbage-collection-and-sliding-windows-... >> Sent from the Drools: User forum mailing list archive at Nabble.com. >> _______________________________________________ >> rules-users mailing list >> rules-users(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/rules-users >> > _______________________________________________ > rules-users mailing list > rules-users(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/rules-users _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users