[rules-users] Memory Leak in 5.4.0 and 5.5.0 [was: Garbage collection and sliding windows (Drools 5.5.0 Final)]
Wolfgang Laun
wolfgang.laun at gmail.com
Sun Feb 17 10:59:59 EST 2013
Give me a JUnit framework where you see the increasing memory consumption...?
On 17/02/2013, Mark Proctor <mproctor at codehaus.org> wrote:
> If we can get a unit test for this, we are just about to put out 5.6 (which
> has a lot of important fixes in it already), we'll try and have it fixed for
> then.
>
> Mark
> On 17 Feb 2013, at 08:55, Wolfgang Laun <wolfgang.laun at gmail.com> wrote:
>
>> With Drools 5.4.0.Final and 5.5.0.Final, automatic event retraction in
>> combination with a sliding window is broken, i.e., it suffers from a
>> memory leak. Full code for a demo has been posted by tai-atari; so I
>> just add some diagnostics that clearly indicate what goes wrong.
>>
>> In the snapshot taken with jmap -histo:live after running the program
>> for several seconds and the insertion of 15800 AnEvent facts you can
>> see:
>> * several classes with 2000 instances each - AnEvents and the ones
>> required for bookkeeping the future expiry
>> * EventFactHandle, WindowTupleList, ObjectHashMap$ObjectEntry all
>> growing without restraint, indicating a memory leak.
>>
>>
>> num #instances #bytes class name
>> ----------------------------------------------
>> 1: 8747 8696096 [S
>> 2: 26985 4110776 <constMethodKlass>
>> 3: 26985 2161888 <methodKlass>
>> 4: 47816 2046304 <symbolKlass>
>> 5: 2414 1482672 <constantPoolKlass>
>> 6: 6142 1434280 [I
>> 7: 13800 1324800 org.drools.common.EventFactHandle
>> 8: 2414 1056424 <instanceKlassKlass>
>> 9: 1891 854664 <constantPoolCacheKlass>
>> 10: 8348 644704 [C
>> 11: 13800 441600 org.drools.reteoo.WindowTupleList
>> 12: 2571 426976 [B
>> 13: 15800 379200
>> org.drools.core.util.ObjectHashMap$ObjectEntry
>> 14: 2672 256512 java.lang.Class
>> 15: 545 246776 <methodDataKlass>
>> 16: 7852 188448 java.lang.String
>> 17: 4084 185496 [[I
>> 18: 2000 176000
>> org.drools.common.PropagationContextImpl
>> 19: 8 164288 [Lorg.drools.core.util.Entry;
>> 20: 2000 160000 org.drools.common.AgendaItem
>> 21: 2005 128320 org.drools.reteoo.WindowTuple
>> 22: 2000 128000
>> org.drools.reteoo.RuleTerminalNodeLeftTuple
>> 23: 2001 96048 java.util.concurrent.FutureTask$Sync
>> 24: 2001 80040
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
>> 25: 1165 79152 [J
>> 26: 229 75112 <objArrayKlassKlass>
>> 27: 2001 64032
>> org.drools.time.impl.JDKTimerService$JDKJobHandle
>> 28: 2001 64032
>> org.drools.time.impl.DefaultTimerJobInstance
>> 29: 423 54728 [Ljava.lang.Object;
>> 30: 2150 51600 java.util.LinkedList$Entry
>> 31: 2052 49248 java.util.LinkedList
>> 32: 2002 48048
>> org.drools.core.util.ObjectHashSet$ObjectEntry
>> 33: 2001 48024 java.util.Date
>> 34: 2000 48000
>> org.drools.reteoo.ObjectTypeNode$ExpireJobContext
>> 35: 1958 46992 java.util.HashMap$Entry
>> 36: 2261 36176 java.lang.Integer
>> 37: 2006 32096
>> java.util.concurrent.atomic.AtomicBoolean
>> 38: 2001 32016
>> org.drools.time.impl.PointInTimeTrigger
>> 39: 2000 32000
>> org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction
>> 40: 2000 32000 memcons.AnEvent
>>
>>
>>
>>
>> On 17/02/2013, tai-atari <p00temkin at gmail.com> wrote:
>>> Hi laune,
>>>
>>> Really appreciate you trying to reproduce the issue. Since you are not
>>> experiencing the same behavior I'm obviously missing something. I've
>>> attached a very simple example below which reproduces the scenario for
>>> the
>>> Drools versions I have tried so far: 5.2, 5.4 and 5.5.
>>>
>>> Basically this example runs an infinite loop which inserts 200 events
>>> about
>>> every second, and keeps a sliding window of 10 seconds. Every loop will
>>> print the current wm event count, which will increase to 2000 and stay
>>> put.
>>> This is where I expected the expired events to be available for garbage
>>> collection (since only 2000 are concurrently active in wm). But VisualVM
>>> shows that org.drools.common.EventFactHandle uses an increasing amount
>>> of
>>> memory over time.
>>>
>>> Start.java:
>>> ==================================
>>> package org.drools.example;
>>>
>>> import java.util.Random;
>>>
>>> import org.drools.KnowledgeBase;
>>> import org.drools.KnowledgeBaseConfiguration;
>>> import org.drools.KnowledgeBaseFactory;
>>> import org.drools.builder.KnowledgeBuilder;
>>> import org.drools.builder.KnowledgeBuilderError;
>>> import org.drools.builder.KnowledgeBuilderErrors;
>>> import org.drools.builder.KnowledgeBuilderFactory;
>>> import org.drools.builder.ResourceType;
>>> import org.drools.conf.EventProcessingOption;
>>> import org.drools.io.ResourceFactory;
>>> import org.drools.runtime.StatefulKnowledgeSession;
>>> import org.drools.runtime.rule.WorkingMemoryEntryPoint;
>>>
>>> public class Start {
>>>
>>> public static final void main(String[] args) {
>>> try {
>>>
>>> System.out.println("Init");
>>> KnowledgeBase kbase = readKnowledgeBase();
>>> StatefulKnowledgeSession ksession =
>>> kbase.newStatefulKnowledgeSession();
>>> WorkingMemoryEntryPoint eventStream =
>>> ksession.getWorkingMemoryEntryPoint("TheEventStream");
>>> Random randGen = new Random();
>>>
>>> while( true ) {
>>>
>>> // Insert 200
>>> int x = 0;
>>> while( x < 200 ) {
>>> AnEvent anEvent = new AnEvent();
>>> anEvent.setSource(randGen.nextInt());
>>> eventStream.insert(anEvent);
>>> ksession.fireAllRules();
>>> x++;
>>> }
>>>
>>> System.out.println("current event count in wm" + ": " +
>>> eventStream.getFactCount());
>>> Thread.sleep(1000);
>>> }
>>>
>>> //ksession.dispose();
>>>
>>> } catch (Throwable t) {
>>> t.printStackTrace();
>>> }
>>> }
>>>
>>> private static KnowledgeBase readKnowledgeBase() throws Exception {
>>>
>>> KnowledgeBuilder kbuilder =
>>> KnowledgeBuilderFactory.newKnowledgeBuilder();
>>> kbuilder.add(ResourceFactory.newClassPathResource("Sample.drl"),
>>> ResourceType.DRL);
>>> KnowledgeBuilderErrors errors = kbuilder.getErrors();
>>> if (errors.size() > 0) {
>>> for (KnowledgeBuilderError error: errors) {
>>> System.err.println(error);
>>> }
>>> throw new IllegalArgumentException("Could not parse
>>> knowledge.");
>>> }
>>>
>>> final KnowledgeBaseConfiguration kbConfig =
>>> KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
>>> kbConfig.setOption(EventProcessingOption.STREAM);
>>> KnowledgeBase kbase =
>>> KnowledgeBaseFactory.newKnowledgeBase(kbConfig);
>>>
>>> kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
>>> return kbase;
>>> }
>>>
>>> }
>>> ==================================
>>>
>>> AnEvent.java:
>>> ==================================
>>> package org.drools.example;
>>>
>>> public class AnEvent {
>>>
>>> private Integer source;
>>>
>>> public AnEvent () {
>>> }
>>>
>>> public Integer getSource() {
>>> return source;
>>> }
>>>
>>> public void setSource(Integer source) {
>>> this.source = source;
>>> }
>>>
>>> }
>>> ==================================
>>>
>>> Sample.drl
>>> ==================================
>>> package org.drools.example
>>> import org.drools.example.AnEvent;
>>>
>>> declare AnEvent
>>> @role( event )
>>> end
>>>
>>> rule "Event print"
>>> when
>>> AnEvent ( $src: source ) over window:time(10s) from entry-point
>>> TheEventStream
>>> then
>>> System.out.println("---------> Received AnEvent from " + $src);
>>> end
>>> ==================================
>>>
>>> Also tried using an immutable pojo as you suggested cusmaimatteo, with
>>>
>>> AnEvent.java:
>>> ==================================
>>> package org.drools.example;
>>>
>>> public class AnEvent {
>>>
>>> private final Integer source;
>>>
>>> public AnEvent (Integer i) {
>>> this.source = i;
>>> }
>>>
>>> public Integer getSource() {
>>> return source;
>>> }
>>> }
>>> ==================================
>>>
>>> and using instead using
>>>
>>> eventStream.insert(new AnEvent(randGen.nextInt()));
>>>
>>> but with the same memory issues noted.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://drools.46999.n3.nabble.com/Garbage-collection-and-sliding-windows-Drools-5-5-0-Final-tp4022183p4022350.html
>>> Sent from the Drools: User forum mailing list archive at Nabble.com.
>>> _______________________________________________
>>> rules-users mailing list
>>> rules-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
More information about the rules-users
mailing list