[jboss-jira] [JBoss JIRA] Created: (JBRULES-2782) Potential memory leak when rules are often reloaded

[Konstantin Pentarakis (JIRA)]

Potential memory leak when rules are often reloaded
---------------------------------------------------

                 Key: JBRULES-2782
                 URL: https://jira.jboss.org/browse/JBRULES-2782
             Project: Drools
          Issue Type: Bug
      Security Level: Public (Everyone can see)
          Components: drools-core
    Affects Versions: 5.1.1.FINAL
         Environment: Windows 7 x64; JVM: Hotspot; JRE build 1.6.0_22-b04 and JRE build 1.5.0_11-b03
            Reporter: Konstantin Pentarakis
            Assignee: Mark Proctor


Hi!

There could be a small heapsize memory leak.
I encountered the problem when I tried to reload rules very often. The heap-size is permanently growing, eventhough the number of rules is constant! Since the heapsize grows very slowly, this leads to an OutOfMemoryError only, if I have a very large number of rules or if I reload the rules very often.

(code-example see below)

What do I mean by "reloading rules"? I add rules to an existing KnowledgeBase which is hold in memory. The rules come from an external source. For example, I add 500 rules into a KnowledgeBase (first run). Now, I redo this and add the same 500 rules again to the KnowledgeBase (2nd run). And so on. If I try this using a reduced heapsize of 64 MB, this works fine 71 times! But the 72nd time leads to an OutOfMemoryError, because there is no free heapsize left! It doesn't metter whether I remove the previously added rules first and add them afterwords again, or I directly add them.
Using a profiler, I can see, that the heap-consumption constandly increases!
The more rules I use, the less interations are possible (closley linearly). The more max. heapsize I use, the more iterations are possible (of course).
It seams like the Memory is hold by the RETE itself. The KnowledgeBase itself holds the references! If all references to the KnowledgeBase are deleted, the garbage collections frees the lost memory!
I also tried to use sequential mode. But there was no difference at all!

This happens with JRE 1.5 and JRE 1.6.

This also happens with the Drools 5.1.1 KnowledgeBase-Implemantion and the still in Drools 5 existing RuleBase-Implemention. But I didn't tested it yet with previous versions of Drools.

Why do I test this? We are building an application where rules will be created programmatically. There will be a large number of rules (some thousands) and rules can often change. The reload of rules which I tested here is something like a "simulation" of lots of rule changes.


Here is my example. For this example I used a simple Class called "RuleRepository" which has a static String-Array holding the rules as DRL-Strings. I also tried it with files in filesystem - no difference. See the RuleRepository-Class below. I post only 3 dummy-rules here - but that makes no difference. If someone wants to test with more rules, you can simply copy and rename them - or use your own rules!

MassivRuleChangeTest.java:


import java.io.IOException;
import java.util.Collection;

import org.drools.KnowledgeBase;
import org.drools.KnowledgeBaseFactory;
import org.drools.builder.KnowledgeBuilder;
import org.drools.builder.KnowledgeBuilderFactory;
import org.drools.builder.ResourceType;
import org.drools.compiler.DroolsParserException;
import org.drools.definition.KnowledgePackage;
import org.drools.io.ResourceFactory;

public class MassivRuleChangeTest {
	
	public void run() throws Exception {

		KnowledgeBase kBase = createRuleBase();
		Collection<KnowledgePackage> packages;

		long counter = 0;
		int delCounter = 0;
		int ruleCounter = 0;
		while (true) {
			System.out.println(++counter + ":");
			for(int iLoop = 0; iLoop < RuleRepository.getSize(); iLoop++) {
				packages = this.getRulePackage(RuleRepository.getRule(iLoop));
				delCounter = this.cleanKBase(kBase, packages, delCounter);
				kBase.addKnowledgePackages(packages);
				ruleCounter++;
			}
			System.out.println(" - Deleted rules: " + delCounter);
			System.out.println(" - Added rules: " + ruleCounter);
			delCounter = 0;
			ruleCounter = 0;
		}
	}
	
	private int cleanKBase(KnowledgeBase kBase, Collection<KnowledgePackage> packages, int delCounter) {
		if(kBase.getKnowledgePackages().size() > 0) {
			for(KnowledgePackage kPkg : packages) {
				if(kBase.getKnowledgePackage(kPkg.getName()) != null && kBase.getKnowledgePackage(kPkg.getName()).getRules().size() > 0) {
					for(org.drools.definition.rule.Rule rule : kPkg.getRules()) {
						if(kBase.getRule(kPkg.getName(), rule.getName()) != null) {
							kBase.removeRule(kPkg.getName(), rule.getName());
							delCounter++;
						}
					}
					if(kPkg.getRules().size() == 0)
						kBase.removeKnowledgePackage(kPkg.getName());
				}
			}
		}
		return delCounter;
	}
	
	private Collection<KnowledgePackage> getRulePackage(String rule) throws DroolsParserException, IOException {
		
		KnowledgeBuilder builder = KnowledgeBuilderFactory.newKnowledgeBuilder();
			
		builder.add(ResourceFactory.newByteArrayResource(rule.getBytes()), ResourceType.DRL);
		if (builder.hasErrors()) {
			System.out.println(builder.getErrors().toString());
			throw new RuntimeException("Unable to compile " + rule);
		}
		return builder.getKnowledgePackages();
	}

	private KnowledgeBase createRuleBase() throws Exception {

		KnowledgeBase kBase = KnowledgeBaseFactory.newKnowledgeBase();

		return kBase;
	}

	public static void main(String[] args) throws Exception {
		new MassivRuleChangeTest().run();
	}
}

----------------------------------------------------------------------------------------------------------------------------------


RuleRepository.java:


public class RuleRepository {
	private static final String[] rules = {
		"rule \"Rule001\" when #conditions then #actions end",
		"rule \"Rule002\" when #conditions then #actions end",
		"rule \"Rule003\" when #conditions then #actions end",
	};
	
	public static String getRule(int index) {
		return rules[index];
	}
	
	public static int getSize() {
		return rules.length;
	}
	
}




-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira