[rules-users] Memory Leak in 5.4.0 and 5.5.0 [was: Garbage collection and sliding windows (Drools 5.5.0 Final)]
Mark Proctor
mproctor at codehaus.org
Sun Feb 17 10:15:13 EST 2013
If we can get a unit test for this, we are just about to put out 5.6 (which has a lot of important fixes in it already), we'll try and have it fixed for then.
Mark
On 17 Feb 2013, at 08:55, Wolfgang Laun <wolfgang.laun at gmail.com> wrote:
> With Drools 5.4.0.Final and 5.5.0.Final, automatic event retraction in
> combination with a sliding window is broken, i.e., it suffers from a
> memory leak. Full code for a demo has been posted by tai-atari; so I
> just add some diagnostics that clearly indicate what goes wrong.
>
> In the snapshot taken with jmap -histo:live after running the program
> for several seconds and the insertion of 15800 AnEvent facts you can
> see:
> * several classes with 2000 instances each - AnEvents and the ones
> required for bookkeeping the future expiry
> * EventFactHandle, WindowTupleList, ObjectHashMap$ObjectEntry all
> growing without restraint, indicating a memory leak.
>
>
> num #instances #bytes class name
> ----------------------------------------------
> 1: 8747 8696096 [S
> 2: 26985 4110776 <constMethodKlass>
> 3: 26985 2161888 <methodKlass>
> 4: 47816 2046304 <symbolKlass>
> 5: 2414 1482672 <constantPoolKlass>
> 6: 6142 1434280 [I
> 7: 13800 1324800 org.drools.common.EventFactHandle
> 8: 2414 1056424 <instanceKlassKlass>
> 9: 1891 854664 <constantPoolCacheKlass>
> 10: 8348 644704 [C
> 11: 13800 441600 org.drools.reteoo.WindowTupleList
> 12: 2571 426976 [B
> 13: 15800 379200
> org.drools.core.util.ObjectHashMap$ObjectEntry
> 14: 2672 256512 java.lang.Class
> 15: 545 246776 <methodDataKlass>
> 16: 7852 188448 java.lang.String
> 17: 4084 185496 [[I
> 18: 2000 176000 org.drools.common.PropagationContextImpl
> 19: 8 164288 [Lorg.drools.core.util.Entry;
> 20: 2000 160000 org.drools.common.AgendaItem
> 21: 2005 128320 org.drools.reteoo.WindowTuple
> 22: 2000 128000 org.drools.reteoo.RuleTerminalNodeLeftTuple
> 23: 2001 96048 java.util.concurrent.FutureTask$Sync
> 24: 2001 80040
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
> 25: 1165 79152 [J
> 26: 229 75112 <objArrayKlassKlass>
> 27: 2001 64032
> org.drools.time.impl.JDKTimerService$JDKJobHandle
> 28: 2001 64032 org.drools.time.impl.DefaultTimerJobInstance
> 29: 423 54728 [Ljava.lang.Object;
> 30: 2150 51600 java.util.LinkedList$Entry
> 31: 2052 49248 java.util.LinkedList
> 32: 2002 48048
> org.drools.core.util.ObjectHashSet$ObjectEntry
> 33: 2001 48024 java.util.Date
> 34: 2000 48000
> org.drools.reteoo.ObjectTypeNode$ExpireJobContext
> 35: 1958 46992 java.util.HashMap$Entry
> 36: 2261 36176 java.lang.Integer
> 37: 2006 32096 java.util.concurrent.atomic.AtomicBoolean
> 38: 2001 32016 org.drools.time.impl.PointInTimeTrigger
> 39: 2000 32000
> org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction
> 40: 2000 32000 memcons.AnEvent
>
>
>
>
> On 17/02/2013, tai-atari <p00temkin at gmail.com> wrote:
>> Hi laune,
>>
>> Really appreciate you trying to reproduce the issue. Since you are not
>> experiencing the same behavior I'm obviously missing something. I've
>> attached a very simple example below which reproduces the scenario for the
>> Drools versions I have tried so far: 5.2, 5.4 and 5.5.
>>
>> Basically this example runs an infinite loop which inserts 200 events about
>> every second, and keeps a sliding window of 10 seconds. Every loop will
>> print the current wm event count, which will increase to 2000 and stay put.
>> This is where I expected the expired events to be available for garbage
>> collection (since only 2000 are concurrently active in wm). But VisualVM
>> shows that org.drools.common.EventFactHandle uses an increasing amount of
>> memory over time.
>>
>> Start.java:
>> ==================================
>> package org.drools.example;
>>
>> import java.util.Random;
>>
>> import org.drools.KnowledgeBase;
>> import org.drools.KnowledgeBaseConfiguration;
>> import org.drools.KnowledgeBaseFactory;
>> import org.drools.builder.KnowledgeBuilder;
>> import org.drools.builder.KnowledgeBuilderError;
>> import org.drools.builder.KnowledgeBuilderErrors;
>> import org.drools.builder.KnowledgeBuilderFactory;
>> import org.drools.builder.ResourceType;
>> import org.drools.conf.EventProcessingOption;
>> import org.drools.io.ResourceFactory;
>> import org.drools.runtime.StatefulKnowledgeSession;
>> import org.drools.runtime.rule.WorkingMemoryEntryPoint;
>>
>> public class Start {
>>
>> public static final void main(String[] args) {
>> try {
>>
>> System.out.println("Init");
>> KnowledgeBase kbase = readKnowledgeBase();
>> StatefulKnowledgeSession ksession =
>> kbase.newStatefulKnowledgeSession();
>> WorkingMemoryEntryPoint eventStream =
>> ksession.getWorkingMemoryEntryPoint("TheEventStream");
>> Random randGen = new Random();
>>
>> while( true ) {
>>
>> // Insert 200
>> int x = 0;
>> while( x < 200 ) {
>> AnEvent anEvent = new AnEvent();
>> anEvent.setSource(randGen.nextInt());
>> eventStream.insert(anEvent);
>> ksession.fireAllRules();
>> x++;
>> }
>>
>> System.out.println("current event count in wm" + ": " +
>> eventStream.getFactCount());
>> Thread.sleep(1000);
>> }
>>
>> //ksession.dispose();
>>
>> } catch (Throwable t) {
>> t.printStackTrace();
>> }
>> }
>>
>> private static KnowledgeBase readKnowledgeBase() throws Exception {
>>
>> KnowledgeBuilder kbuilder =
>> KnowledgeBuilderFactory.newKnowledgeBuilder();
>> kbuilder.add(ResourceFactory.newClassPathResource("Sample.drl"),
>> ResourceType.DRL);
>> KnowledgeBuilderErrors errors = kbuilder.getErrors();
>> if (errors.size() > 0) {
>> for (KnowledgeBuilderError error: errors) {
>> System.err.println(error);
>> }
>> throw new IllegalArgumentException("Could not parse
>> knowledge.");
>> }
>>
>> final KnowledgeBaseConfiguration kbConfig =
>> KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
>> kbConfig.setOption(EventProcessingOption.STREAM);
>> KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase(kbConfig);
>>
>> kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
>> return kbase;
>> }
>>
>> }
>> ==================================
>>
>> AnEvent.java:
>> ==================================
>> package org.drools.example;
>>
>> public class AnEvent {
>>
>> private Integer source;
>>
>> public AnEvent () {
>> }
>>
>> public Integer getSource() {
>> return source;
>> }
>>
>> public void setSource(Integer source) {
>> this.source = source;
>> }
>>
>> }
>> ==================================
>>
>> Sample.drl
>> ==================================
>> package org.drools.example
>> import org.drools.example.AnEvent;
>>
>> declare AnEvent
>> @role( event )
>> end
>>
>> rule "Event print"
>> when
>> AnEvent ( $src: source ) over window:time(10s) from entry-point
>> TheEventStream
>> then
>> System.out.println("---------> Received AnEvent from " + $src);
>> end
>> ==================================
>>
>> Also tried using an immutable pojo as you suggested cusmaimatteo, with
>>
>> AnEvent.java:
>> ==================================
>> package org.drools.example;
>>
>> public class AnEvent {
>>
>> private final Integer source;
>>
>> public AnEvent (Integer i) {
>> this.source = i;
>> }
>>
>> public Integer getSource() {
>> return source;
>> }
>> }
>> ==================================
>>
>> and using instead using
>>
>> eventStream.insert(new AnEvent(randGen.nextInt()));
>>
>> but with the same memory issues noted.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://drools.46999.n3.nabble.com/Garbage-collection-and-sliding-windows-Drools-5-5-0-Final-tp4022183p4022350.html
>> Sent from the Drools: User forum mailing list archive at Nabble.com.
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
More information about the rules-users
mailing list