[JBoss JIRA] Created: (JBRULES-2782) Potential memory leak when rules are often reloaded
by Konstantin Pentarakis (JIRA)
Potential memory leak when rules are often reloaded
---------------------------------------------------
Key: JBRULES-2782
URL: https://jira.jboss.org/browse/JBRULES-2782
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-core
Affects Versions: 5.1.1.FINAL
Environment: Windows 7 x64; JVM: Hotspot; JRE build 1.6.0_22-b04 and JRE build 1.5.0_11-b03
Reporter: Konstantin Pentarakis
Assignee: Mark Proctor
Hi!
There could be a small heapsize memory leak.
I encountered the problem when I tried to reload rules very often. The heap-size is permanently growing, eventhough the number of rules is constant! Since the heapsize grows very slowly, this leads to an OutOfMemoryError only, if I have a very large number of rules or if I reload the rules very often.
(code-example see below)
What do I mean by "reloading rules"? I add rules to an existing KnowledgeBase which is hold in memory. The rules come from an external source. For example, I add 500 rules into a KnowledgeBase (first run). Now, I redo this and add the same 500 rules again to the KnowledgeBase (2nd run). And so on. If I try this using a reduced heapsize of 64 MB, this works fine 71 times! But the 72nd time leads to an OutOfMemoryError, because there is no free heapsize left! It doesn't metter whether I remove the previously added rules first and add them afterwords again, or I directly add them.
Using a profiler, I can see, that the heap-consumption constandly increases!
The more rules I use, the less interations are possible (closley linearly). The more max. heapsize I use, the more iterations are possible (of course).
It seams like the Memory is hold by the RETE itself. The KnowledgeBase itself holds the references! If all references to the KnowledgeBase are deleted, the garbage collections frees the lost memory!
I also tried to use sequential mode. But there was no difference at all!
This happens with JRE 1.5 and JRE 1.6.
This also happens with the Drools 5.1.1 KnowledgeBase-Implemantion and the still in Drools 5 existing RuleBase-Implemention. But I didn't tested it yet with previous versions of Drools.
Why do I test this? We are building an application where rules will be created programmatically. There will be a large number of rules (some thousands) and rules can often change. The reload of rules which I tested here is something like a "simulation" of lots of rule changes.
Here is my example. For this example I used a simple Class called "RuleRepository" which has a static String-Array holding the rules as DRL-Strings. I also tried it with files in filesystem - no difference. See the RuleRepository-Class below. I post only 3 dummy-rules here - but that makes no difference. If someone wants to test with more rules, you can simply copy and rename them - or use your own rules!
MassivRuleChangeTest.java:
import java.io.IOException;
import java.util.Collection;
import org.drools.KnowledgeBase;
import org.drools.KnowledgeBaseFactory;
import org.drools.builder.KnowledgeBuilder;
import org.drools.builder.KnowledgeBuilderFactory;
import org.drools.builder.ResourceType;
import org.drools.compiler.DroolsParserException;
import org.drools.definition.KnowledgePackage;
import org.drools.io.ResourceFactory;
public class MassivRuleChangeTest {
public void run() throws Exception {
KnowledgeBase kBase = createRuleBase();
Collection<KnowledgePackage> packages;
long counter = 0;
int delCounter = 0;
int ruleCounter = 0;
while (true) {
System.out.println(++counter + ":");
for(int iLoop = 0; iLoop < RuleRepository.getSize(); iLoop++) {
packages = this.getRulePackage(RuleRepository.getRule(iLoop));
delCounter = this.cleanKBase(kBase, packages, delCounter);
kBase.addKnowledgePackages(packages);
ruleCounter++;
}
System.out.println(" - Deleted rules: " + delCounter);
System.out.println(" - Added rules: " + ruleCounter);
delCounter = 0;
ruleCounter = 0;
}
}
private int cleanKBase(KnowledgeBase kBase, Collection<KnowledgePackage> packages, int delCounter) {
if(kBase.getKnowledgePackages().size() > 0) {
for(KnowledgePackage kPkg : packages) {
if(kBase.getKnowledgePackage(kPkg.getName()) != null && kBase.getKnowledgePackage(kPkg.getName()).getRules().size() > 0) {
for(org.drools.definition.rule.Rule rule : kPkg.getRules()) {
if(kBase.getRule(kPkg.getName(), rule.getName()) != null) {
kBase.removeRule(kPkg.getName(), rule.getName());
delCounter++;
}
}
if(kPkg.getRules().size() == 0)
kBase.removeKnowledgePackage(kPkg.getName());
}
}
}
return delCounter;
}
private Collection<KnowledgePackage> getRulePackage(String rule) throws DroolsParserException, IOException {
KnowledgeBuilder builder = KnowledgeBuilderFactory.newKnowledgeBuilder();
builder.add(ResourceFactory.newByteArrayResource(rule.getBytes()), ResourceType.DRL);
if (builder.hasErrors()) {
System.out.println(builder.getErrors().toString());
throw new RuntimeException("Unable to compile " + rule);
}
return builder.getKnowledgePackages();
}
private KnowledgeBase createRuleBase() throws Exception {
KnowledgeBase kBase = KnowledgeBaseFactory.newKnowledgeBase();
return kBase;
}
public static void main(String[] args) throws Exception {
new MassivRuleChangeTest().run();
}
}
----------------------------------------------------------------------------------------------------------------------------------
RuleRepository.java:
public class RuleRepository {
private static final String[] rules = {
"rule \"Rule001\" when #conditions then #actions end",
"rule \"Rule002\" when #conditions then #actions end",
"rule \"Rule003\" when #conditions then #actions end",
};
public static String getRule(int index) {
return rules[index];
}
public static int getSize() {
return rules.length;
}
}
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2777) rule with window:length/accumulate/not( before ) fires twice
by Wolfgang Laun (JIRA)
rule with window:length/accumulate/not( before ) fires twice
------------------------------------------------------------
Key: JBRULES-2777
URL: https://jira.jboss.org/browse/JBRULES-2777
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-core
Affects Versions: 5.1.1.FINAL
Reporter: Wolfgang Laun
Assignee: Mark Proctor
Fix For: 5.2.0.M1
STREAM, pseudo-clock, insert facts after advancing the clock to the timestamp in the event:
for( Job job: jobs.getJobs() ){
Date jobTime = job.getTimestamp();
long newClock = jobTime.getTime();
if( newClock > oldClock ){
clock.advanceTime( newClock - oldClock, TimeUnit.MILLISECONDS );
oldClock = newClock;
}
System.out.println( "inserting " + job );
session.insert( job );
System.out.println( "advancing by 1ms" );
clock.advanceTime( 1, TimeUnit.MILLISECONDS );
oldClock++;
System.out.println( "fireAllRules" );
session.fireAllRules();
}
A single rule:
declare Job
@role( event )
@timestamp( timestamp )
@duration( duration )
end
Part of output:
inserting Job: 6 at Wed Nov 03 01:40:00 CET 2010 - 1000
advancing by 1ms
clock: 1288744799999, ts of 1st: 1288742400000 - Average interval: 599 seconds
clock: 1288744799999, ts of 1st: 1288743000000 - Average interval: 449 seconds
fireAllRules
inserting Job: 7 at Wed Nov 03 01:45:00 CET 2010 - 1000
advancing by 1ms
clock: 1288745099999, ts of 1st: 1288743000000 - Average interval: 524 seconds
clock: 1288745099999, ts of 1st: 1288743600000 - Average interval: 374 seconds
fireAllRules
Rule firing is erratic. There is no apparent reason why this should fire twice after advancing the clock.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2882) NPE in EqualityEvaluatorsDefinition$StringEqualEvaluator.evaluateCachedLeft
by Ravi Bhatt (JIRA)
NPE in EqualityEvaluatorsDefinition$StringEqualEvaluator.evaluateCachedLeft
---------------------------------------------------------------------------
Key: JBRULES-2882
URL: https://issues.jboss.org/browse/JBRULES-2882
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-core (expert)
Affects Versions: 5.1.0.FINAL
Environment: OS: Win 7 Professional 64 Bit, 64 Bit Java,
Reporter: Ravi Bhatt
Assignee: Mark Proctor
While working with rules with partitioning turned on.
KnowledgeBaseConfiguration config = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
config.setOption( MultithreadEvaluationOption.YES );
conf.setOption( MaxThreadsOption.get(5) );
Once we start inserting objects into the KSession, somewhere in the middle it throws this error:
Partition task manager caught an unexpected exception: null
Drools is capturing the exception to avoid thread death. Please report stack trace to development team.
java.lang.NullPointerException
at org.drools.base.evaluators.EqualityEvaluatorsDefinition$StringEqualEvaluator.evaluateCachedLeft(EqualityEvaluatorsDefinition.java:1984)
at org.drools.rule.VariableRestriction.isAllowedCachedLeft(VariableRestriction.java:109)
at org.drools.rule.VariableConstraint.isAllowedCachedLeft(VariableConstraint.java:115)
at org.drools.common.SingleBetaConstraints.isAllowedCachedLeft(SingleBetaConstraints.java:142)
at org.drools.reteoo.FromNode.checkConstraintsAndPropagate(FromNode.java:274)
at org.drools.reteoo.FromNode.assertLeftTuple(FromNode.java:138)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.doPropagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:189)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:64)
at org.drools.reteoo.JoinNode.assertLeftTuple(JoinNode.java:85)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.doPropagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:189)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:64)
at org.drools.reteoo.FromNode.checkConstraintsAndPropagate(FromNode.java:279)
at org.drools.reteoo.FromNode.assertLeftTuple(FromNode.java:138)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.doPropagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:189)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:64)
at org.drools.reteoo.JoinNode.assertObject(JoinNode.java:128)
at org.drools.reteoo.PartitionTaskManager$FactAssertAction.execute(PartitionTaskManager.java:274)
at org.drools.reteoo.PartitionTaskManager$PartitionTask.run(PartitionTaskManager.java:118)
at org.drools.concurrent.ExternalExecutorService$ObservableRunnable.run(ExternalExecutorService.java:453)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2877) Disjunction over a set not working
by Oliver Roess (JIRA)
Disjunction over a set not working
----------------------------------
Key: JBRULES-2877
URL: https://issues.jboss.org/browse/JBRULES-2877
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Affects Versions: 5.1.1.FINAL
Reporter: Oliver Roess
Assignee: Mark Proctor
Hello there and a happy new year to everyone reading :)
Why am I here again? Yep, got a problem. Don't know why it doesn't work. I'm using the latest snapshot version.
So here it goes.
A simple rule with conjunction and disjunction but over a set of events.
when
$ldes : Set(size == 2) from collect(LoopDetectorEvent() over window:length(2))
(or LoopDetectorEvent($t1 : transportation == "LKW", $id1 : eventID) from $ldes
LoopDetectorEvent($t2 : transportation == "PKW", $id2 : eventID) from $ldes)
LoopDetectorEvent($t3 : transportation == "BUS", $id3 : eventID) from $ldes
So, what the rule says: fire when there's a BUS event plus either a LKW or PKW event in the set.
But, instead I get a big exception. It's posted at the end.
Interestingly, it works if I don't use the Set, like this...
when
(or $e1 : LoopDetectorEvent($t1 : transportation == "PKW", $id1 : eventID) over window:length(1)
$e2 : LoopDetectorEvent($t2 : transportation == "LKW", $id2 : eventID) over window:length(1))
LoopDetectorEvent($t3 : transportation == "BUS", $id3 : eventID) over window:length(1)
but that rule does not have the same meaning. So it's useless for me.
Besides, it DOES work with the set if I just cut the third event out (the BUS event).
What is wrong here?
Thanks in advance.
Kind regards!
Oliver
Exception in thread "Thread-2" java.lang.ClassCastException: org.drools.reteoo.InitialFactImpl cannot be cast to trafficsys.domain.events.LoopDetectorEvent
at org.drools.base.trafficsys.domain.events.LoopDetectorEvent446196$getTransportation.getValue(Unknown Source)
at org.drools.base.ClassFieldReader.getValue(ClassFieldReader.java:91)
at org.drools.base.evaluators.EqualityEvaluatorsDefinition$StringEqualEvaluator.evaluate(EqualityEvaluatorsDefinition.java:1961)
at org.drools.rule.LiteralRestriction.isAllowed(LiteralRestriction.java:92)
at org.drools.rule.LiteralConstraint.isAllowed(LiteralConstraint.java:109)
at org.drools.reteoo.FromNode.checkConstraintsAndPropagate(FromNode.java:264)
at org.drools.reteoo.FromNode.assertLeftTuple(FromNode.java:138)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.doPropagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:189)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:64)
at org.drools.reteoo.FromNode.checkConstraintsAndPropagate(FromNode.java:279)
at org.drools.reteoo.FromNode.assertLeftTuple(FromNode.java:138)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.doPropagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:189)
at org.drools.reteoo.SingleLeftTupleSinkAdapter.propagateAssertLeftTuple(SingleLeftTupleSinkAdapter.java:64)
at org.drools.reteoo.AccumulateNode.evaluateResultConstraints(AccumulateNode.java:662)
at org.drools.reteoo.AccumulateNode.assertObject(AccumulateNode.java:270)
at org.drools.reteoo.CompositeObjectSinkAdapter.doPropagateAssertObject(CompositeObjectSinkAdapter.java:450)
at org.drools.reteoo.CompositeObjectSinkAdapter.propagateAssertObject(CompositeObjectSinkAdapter.java:378)
at org.drools.reteoo.ObjectTypeNode.assertObject(ObjectTypeNode.java:190)
at org.drools.reteoo.EntryPointNode.assertObject(EntryPointNode.java:145)
at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:1187)
at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:1089)
at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:893)
at org.drools.impl.StatefulKnowledgeSessionImpl.insert(StatefulKnowledgeSessionImpl.java:238)
at trafficsys.SessionInterface.insertAndFire(SessionInterface.java:133)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2871) KnowledgeAgent: unable to determine operator for symbol [after]
by Oscar de Groot (JIRA)
KnowledgeAgent: unable to determine operator for symbol [after]
---------------------------------------------------------------
Key: JBRULES-2871
URL: https://issues.jboss.org/browse/JBRULES-2871
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-core
Affects Versions: 5.1.1.FINAL
Reporter: Oscar de Groot
Assignee: Mark Proctor
When loading a ruleset via a KnowledgeAgent, only the operators defined in org.drools.base.evaluators.Operator (== != >= < <= <) are supported. When loading the same rulebase via a KnowledgeBuilder, all operators are supported.
Upon loading a ruleset that contains another operator (e.g. "after", "before"), the following error is printed:
[2011:01:19 14:01:44:exception]
java.lang.RuntimeException: KnowledgeAgent exception while trying to deserialize KnowledgeDefinitionsPackage
at org.drools.agent.impl.KnowledgeAgentImpl.createPackageFromResource(KnowledgeAgentImpl.java:674)
at org.drools.agent.impl.KnowledgeAgentImpl.addResourcesToKnowledgeBase(KnowledgeAgentImpl.java:899)
...
Caused by: org.drools.RuntimeDroolsException: unable to determine operator for symbol [after]
at org.drools.base.evaluators.Operator.determineOperator(Operator.java:87)
at org.drools.base.evaluators.Operator.readResolve(Operator.java:122)
Using Eclipse's debugging feature, I have determined that KnowledgeBuilder loads the rest of the operators through EvaluatorRegistry, while KnowledgeAgent does nothing to load any operators into org.drools.base.evaluators.Operator. I have tried modifying KnowledgeAgent by adding the code below to the constructor of KnowledgeAgentImpl. This resolves the problem: with this addition, the ruleset with "after"-operator works as intended when loaded via a KnowledgeAgent.
// Load all of Drools' default evaluators instead of just the ones defined
// in org.drools.base.evaluators.Operator
new EvaluatorRegistry();
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2819) DSL parsing/expansion: '\', '$', empty value, empty condition
by Wolfgang Laun (JIRA)
DSL parsing/expansion: '\', '$', empty value, empty condition
---------------------------------------------------------------
Key: JBRULES-2819
URL: https://jira.jboss.org/browse/JBRULES-2819
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-compiler-DSL
Affects Versions: 5.1.1.FINAL
Reporter: Wolfgang Laun
Assignee: Mark Proctor
Fix For: 5.2.0.M1
DSL parser does not accept an empty "value" section, e.g.
[when] a man loves a woman=
Expansion of a rule with an empty condition results in "whenthen".
The DSL parser escapes '$', but not '\' in value sections. It should not modify the value section except for "normalizing" variable references. Variable references should be restricted to a "clean" character sequence (e.g. Java identifier). The current expansion using replaceAll is too fragile.
I have emailed three reworked source files AntlrDSLMappingEntry.java DefaultExpander.java DSLTokenizedMappingFile.java to Edson, but I didn't change the parser itself.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2845) threading issues when using MultithreadEvaluationOption.YES with fusion
by Radai Rosenblatt (JIRA)
threading issues when using MultithreadEvaluationOption.YES with fusion
-----------------------------------------------------------------------
Key: JBRULES-2845
URL: https://issues.jboss.org/browse/JBRULES-2845
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-compiler (fusion), drools-core (expert)
Affects Versions: 5.1.1.FINAL
Environment: dual core processor, jdk 6u21, windows xp
Reporter: Radai Rosenblatt
Assignee: Mark Proctor
Attachments: threadingbug.zip
i have 2 very simple rules and a test case (see attached maven project).
trying to set MultithreadEvaluationOption.YES on the KB causes the test to fail sporadically (~10% of the times?).
the attached project contains a testsuite that runs the same test 8 times (testsuite is defined in RunMe.java). attempting "mvn clean install" runs this suite and usually fails.
commenting out the MultithreadEvaluationOption.YES line in the other testcase (the one referred-to from RunMe) causes the tests to behave as expected (pass).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months
[JBoss JIRA] Created: (JBRULES-2898) incorrect eval() results when accessing global
by Wolfgang Laun (JIRA)
incorrect eval() results when accessing global
----------------------------------------------
Key: JBRULES-2898
URL: https://issues.jboss.org/browse/JBRULES-2898
Project: Drools
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: drools-core
Affects Versions: 5.1.1.FINAL
Reporter: Wolfgang Laun
Assignee: Mark Proctor
Fix For: 5.2.0.M1
Combinations of rules containing an eval CE comparing double > double or double > integer may cause the eval result to be in error. In the attached example, there are 4 rules. If all are used, 2 fire when they should not. If rules 1+2 or 3+4 are used, 1 fires when it should not. If rules 1+4 or 2+3 are used, none misfires.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 7 months