[rules-users] Memory Leak in 5.4.0 and 5.5.0 [was: Garbage collection and sliding windows (Drools 5.5.0 Final)]
Wolfgang Laun
wolfgang.laun at gmail.com
Sun Feb 17 11:06:32 EST 2013
This DRL, and just insert AnEvents...
declare AnEvent
@role( event )
end
rule "Event print"
when
$e: AnEvent () over window:time(10s)
then
end
-W
On 17/02/2013, Wolfgang Laun <wolfgang.laun at gmail.com> wrote:
> Give me a JUnit framework where you see the increasing memory
> consumption...?
>
>
>
> On 17/02/2013, Mark Proctor <mproctor at codehaus.org> wrote:
>> If we can get a unit test for this, we are just about to put out 5.6
>> (which
>> has a lot of important fixes in it already), we'll try and have it fixed
>> for
>> then.
>>
>> Mark
>> On 17 Feb 2013, at 08:55, Wolfgang Laun <wolfgang.laun at gmail.com> wrote:
>>
>>> With Drools 5.4.0.Final and 5.5.0.Final, automatic event retraction in
>>> combination with a sliding window is broken, i.e., it suffers from a
>>> memory leak. Full code for a demo has been posted by tai-atari; so I
>>> just add some diagnostics that clearly indicate what goes wrong.
>>>
>>> In the snapshot taken with jmap -histo:live after running the program
>>> for several seconds and the insertion of 15800 AnEvent facts you can
>>> see:
>>> * several classes with 2000 instances each - AnEvents and the ones
>>> required for bookkeeping the future expiry
>>> * EventFactHandle, WindowTupleList, ObjectHashMap$ObjectEntry all
>>> growing without restraint, indicating a memory leak.
>>>
>>>
>>> num #instances #bytes class name
>>> ----------------------------------------------
>>> 1: 8747 8696096 [S
>>> 2: 26985 4110776 <constMethodKlass>
>>> 3: 26985 2161888 <methodKlass>
>>> 4: 47816 2046304 <symbolKlass>
>>> 5: 2414 1482672 <constantPoolKlass>
>>> 6: 6142 1434280 [I
>>> 7: 13800 1324800 org.drools.common.EventFactHandle
>>> 8: 2414 1056424 <instanceKlassKlass>
>>> 9: 1891 854664 <constantPoolCacheKlass>
>>> 10: 8348 644704 [C
>>> 11: 13800 441600 org.drools.reteoo.WindowTupleList
>>> 12: 2571 426976 [B
>>> 13: 15800 379200
>>> org.drools.core.util.ObjectHashMap$ObjectEntry
>>> 14: 2672 256512 java.lang.Class
>>> 15: 545 246776 <methodDataKlass>
>>> 16: 7852 188448 java.lang.String
>>> 17: 4084 185496 [[I
>>> 18: 2000 176000
>>> org.drools.common.PropagationContextImpl
>>> 19: 8 164288 [Lorg.drools.core.util.Entry;
>>> 20: 2000 160000 org.drools.common.AgendaItem
>>> 21: 2005 128320 org.drools.reteoo.WindowTuple
>>> 22: 2000 128000
>>> org.drools.reteoo.RuleTerminalNodeLeftTuple
>>> 23: 2001 96048 java.util.concurrent.FutureTask$Sync
>>> 24: 2001 80040
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
>>> 25: 1165 79152 [J
>>> 26: 229 75112 <objArrayKlassKlass>
>>> 27: 2001 64032
>>> org.drools.time.impl.JDKTimerService$JDKJobHandle
>>> 28: 2001 64032
>>> org.drools.time.impl.DefaultTimerJobInstance
>>> 29: 423 54728 [Ljava.lang.Object;
>>> 30: 2150 51600 java.util.LinkedList$Entry
>>> 31: 2052 49248 java.util.LinkedList
>>> 32: 2002 48048
>>> org.drools.core.util.ObjectHashSet$ObjectEntry
>>> 33: 2001 48024 java.util.Date
>>> 34: 2000 48000
>>> org.drools.reteoo.ObjectTypeNode$ExpireJobContext
>>> 35: 1958 46992 java.util.HashMap$Entry
>>> 36: 2261 36176 java.lang.Integer
>>> 37: 2006 32096
>>> java.util.concurrent.atomic.AtomicBoolean
>>> 38: 2001 32016
>>> org.drools.time.impl.PointInTimeTrigger
>>> 39: 2000 32000
>>> org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction
>>> 40: 2000 32000 memcons.AnEvent
>>>
>>>
>>>
>>>
>>> On 17/02/2013, tai-atari <p00temkin at gmail.com> wrote:
>>>> Hi laune,
>>>>
>>>> Really appreciate you trying to reproduce the issue. Since you are not
>>>> experiencing the same behavior I'm obviously missing something. I've
>>>> attached a very simple example below which reproduces the scenario for
>>>> the
>>>> Drools versions I have tried so far: 5.2, 5.4 and 5.5.
>>>>
>>>> Basically this example runs an infinite loop which inserts 200 events
>>>> about
>>>> every second, and keeps a sliding window of 10 seconds. Every loop will
>>>> print the current wm event count, which will increase to 2000 and stay
>>>> put.
>>>> This is where I expected the expired events to be available for garbage
>>>> collection (since only 2000 are concurrently active in wm). But
>>>> VisualVM
>>>> shows that org.drools.common.EventFactHandle uses an increasing amount
>>>> of
>>>> memory over time.
>>>>
>>>> Start.java:
>>>> ==================================
>>>> package org.drools.example;
>>>>
>>>> import java.util.Random;
>>>>
>>>> import org.drools.KnowledgeBase;
>>>> import org.drools.KnowledgeBaseConfiguration;
>>>> import org.drools.KnowledgeBaseFactory;
>>>> import org.drools.builder.KnowledgeBuilder;
>>>> import org.drools.builder.KnowledgeBuilderError;
>>>> import org.drools.builder.KnowledgeBuilderErrors;
>>>> import org.drools.builder.KnowledgeBuilderFactory;
>>>> import org.drools.builder.ResourceType;
>>>> import org.drools.conf.EventProcessingOption;
>>>> import org.drools.io.ResourceFactory;
>>>> import org.drools.runtime.StatefulKnowledgeSession;
>>>> import org.drools.runtime.rule.WorkingMemoryEntryPoint;
>>>>
>>>> public class Start {
>>>>
>>>> public static final void main(String[] args) {
>>>> try {
>>>>
>>>> System.out.println("Init");
>>>> KnowledgeBase kbase = readKnowledgeBase();
>>>> StatefulKnowledgeSession ksession =
>>>> kbase.newStatefulKnowledgeSession();
>>>> WorkingMemoryEntryPoint eventStream =
>>>> ksession.getWorkingMemoryEntryPoint("TheEventStream");
>>>> Random randGen = new Random();
>>>>
>>>> while( true ) {
>>>>
>>>> // Insert 200
>>>> int x = 0;
>>>> while( x < 200 ) {
>>>> AnEvent anEvent = new AnEvent();
>>>> anEvent.setSource(randGen.nextInt());
>>>> eventStream.insert(anEvent);
>>>> ksession.fireAllRules();
>>>> x++;
>>>> }
>>>>
>>>> System.out.println("current event count in wm" + ": " +
>>>> eventStream.getFactCount());
>>>> Thread.sleep(1000);
>>>> }
>>>>
>>>> //ksession.dispose();
>>>>
>>>> } catch (Throwable t) {
>>>> t.printStackTrace();
>>>> }
>>>> }
>>>>
>>>> private static KnowledgeBase readKnowledgeBase() throws Exception {
>>>>
>>>> KnowledgeBuilder kbuilder =
>>>> KnowledgeBuilderFactory.newKnowledgeBuilder();
>>>> kbuilder.add(ResourceFactory.newClassPathResource("Sample.drl"),
>>>> ResourceType.DRL);
>>>> KnowledgeBuilderErrors errors = kbuilder.getErrors();
>>>> if (errors.size() > 0) {
>>>> for (KnowledgeBuilderError error: errors) {
>>>> System.err.println(error);
>>>> }
>>>> throw new IllegalArgumentException("Could not parse
>>>> knowledge.");
>>>> }
>>>>
>>>> final KnowledgeBaseConfiguration kbConfig =
>>>> KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
>>>> kbConfig.setOption(EventProcessingOption.STREAM);
>>>> KnowledgeBase kbase =
>>>> KnowledgeBaseFactory.newKnowledgeBase(kbConfig);
>>>>
>>>> kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
>>>> return kbase;
>>>> }
>>>>
>>>> }
>>>> ==================================
>>>>
>>>> AnEvent.java:
>>>> ==================================
>>>> package org.drools.example;
>>>>
>>>> public class AnEvent {
>>>>
>>>> private Integer source;
>>>>
>>>> public AnEvent () {
>>>> }
>>>>
>>>> public Integer getSource() {
>>>> return source;
>>>> }
>>>>
>>>> public void setSource(Integer source) {
>>>> this.source = source;
>>>> }
>>>>
>>>> }
>>>> ==================================
>>>>
>>>> Sample.drl
>>>> ==================================
>>>> package org.drools.example
>>>> import org.drools.example.AnEvent;
>>>>
>>>> declare AnEvent
>>>> @role( event )
>>>> end
>>>>
>>>> rule "Event print"
>>>> when
>>>> AnEvent ( $src: source ) over window:time(10s) from entry-point
>>>> TheEventStream
>>>> then
>>>> System.out.println("---------> Received AnEvent from " + $src);
>>>> end
>>>> ==================================
>>>>
>>>> Also tried using an immutable pojo as you suggested cusmaimatteo, with
>>>>
>>>> AnEvent.java:
>>>> ==================================
>>>> package org.drools.example;
>>>>
>>>> public class AnEvent {
>>>>
>>>> private final Integer source;
>>>>
>>>> public AnEvent (Integer i) {
>>>> this.source = i;
>>>> }
>>>>
>>>> public Integer getSource() {
>>>> return source;
>>>> }
>>>> }
>>>> ==================================
>>>>
>>>> and using instead using
>>>>
>>>> eventStream.insert(new AnEvent(randGen.nextInt()));
>>>>
>>>> but with the same memory issues noted.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://drools.46999.n3.nabble.com/Garbage-collection-and-sliding-windows-Drools-5-5-0-Final-tp4022183p4022350.html
>>>> Sent from the Drools: User forum mailing list archive at Nabble.com.
>>>> _______________________________________________
>>>> rules-users mailing list
>>>> rules-users at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>> _______________________________________________
>>> rules-users mailing list
>>> rules-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>
More information about the rules-users
mailing list