<font size="2"><font face="verdana,sans-serif">Hi again,</font></font><div><font size="2"><font face="verdana,sans-serif"><br></font></font></div><div><font size="2"><font face="verdana,sans-serif">So.. i've tried doing precisely that but i am unable to fire the rules afterwards. I am dividing the XML rule file into smaller valid XML files so that i can compile them individually and write to disk the compiled result. I am using the sample code shown in the docs:</font></font></div>
<div><font size="2"><font face="verdana,sans-serif"><br></font></font></div><div><font size="2"><font face="verdana,sans-serif"><div>// write compiled rules to disk</div><div>ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(filename + ".temp"));</div>
<div>out.writeObject(kbuilder.getKnowledgePackages());</div><div>out.close();</div><div><br></div><div>On a later stage, i load all files holding compiled rules, one at a time, so i can use the knowledge base and create the knowledge session. Here's what i'm doing for each of the generated compiled files:</div>
<div><br></div><div><div>in = new ObjectInputStream(new FileInputStream(filename)); // the ones *.temp</div><div>kpkgs = (Collection<KnowledgePackage>) in.readObject();</div><div>kbase.addKnowledgePackages(kpkgs);</div>
<div>in.close();</div></div><div><br></div><div>This all works apparently well. Drools loads the sum of all rules defined in each smaller XML file. However, whenever i try to insert a fact into memory, i get an exception, just like the following:</div>
<div><br></div><div><div>java.lang.NullPointerException</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.base.ClassFieldReader.getValue(ClassFieldReader.java:91)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.base.evaluators.EqualityEvaluatorsDefinition$StringEqualEvaluator.evaluate(EqualityEvaluatorsDefinition.java:1962)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.rule.LiteralRestriction.isAllowed(LiteralRestriction.java:92)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.rule.OrCompositeRestriction.isAllowed(OrCompositeRestriction.java:25)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.rule.MultiRestrictionFieldConstraint.isAllowed(MultiRestrictionFieldConstraint.java:97)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.reteoo.AlphaNode.assertObject(AlphaNode.java:143)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.reteoo.CompositeObjectSinkAdapter.doPropagateAssertObject(CompositeObjectSinkAdapter.java:360)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.reteoo.CompositeObjectSinkAdapter.propagateAssertObject(CompositeObjectSinkAdapter.java:344)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.reteoo.ObjectTypeNode.assertObject(ObjectTypeNode.java:185)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.reteoo.EntryPointNode.assertObject(EntryPointNode.java:146)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:1046)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:1001)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.common.AbstractWorkingMemory.insert(AbstractWorkingMemory.java:788)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.drools.impl.StatefulKnowledgeSessionImpl.insert(StatefulKnowledgeSessionImpl.java:216)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at pt.itsd.probe.ConfigurationSetup.parseAndCompileConfigFiles(ConfigurationSetup.java:316)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at pt.itsd.probe.ConfigurationSetup.main(ConfigurationSetup.java:131)</div>
</div><div><br></div><div>I have done some testing and I am unable to trace this exception to any particular rule, or event any XML chunck. If i load the entire XML file it takes a lot of memory compiling but it works in the end. Also, if i remove all the <or-restriction-connective> tags, it also works no matter if i load them to kbase.</div>
<div><br></div><div>Is there something i can do? or maybe another way of doing it? Any help is much appreciated.</div><div><br></div><div>I'm using drools 5.0.0 from main download page.</div><div>_ miguel</div><div><br>
</div></font></font><br><div class="gmail_quote">2010/4/3 Wolfgang Laun <span dir="ltr"><<a href="mailto:wolfgang.laun@gmail.com">wolfgang.laun@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
It should be possible to increase heap space to overcome this problem. Of course, if a program wil start swapping, performance will go way down.<br><br>Dividing the rules file is the only way. But even a very simple script of Perl or similar should be able to divide your (IIRC) generated rules file.<br>
<br>-W<br><br><div class="gmail_quote">2010/4/3 miguel machado <span dir="ltr"><<a href="mailto:mls.machado@gmail.com" target="_blank">mls.machado@gmail.com</a>></span><br><blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex">
<div><div></div><div class="h5">
<font size="2"><font face="verdana,sans-serif">hi there,</font></font><div><font size="2"><font face="verdana,sans-serif"><br></font></font></div><div><font face="verdana, sans-serif">So like i said, by pre-compiling the rules and loading them in the main system i was able to reduce a lot memory usage and i can get better results. However, now i've come across a different problem. I just added a fre more dozens of rules and the java program responsible for compiling the DRL file throws a java heap space error, something like an OutOfMemoryException.</font></div>
<div><font face="verdana, sans-serif"><br></font></div><div><font face="verdana, sans-serif">I've read somewhere that dividing the DRL into several files and loading e.g. 500 rules at a time might solve it. Is there another approach to this problem?</font></div>
<div><font face="verdana, sans-serif"><br></font></div><div><font face="verdana, sans-serif">thanks in advance,</font></div><div><font face="verdana, sans-serif">_ miguel</font></div>
<div><font face="verdana, sans-serif"><br></font></div><div><font size="2"><font face="verdana,sans-serif"><br></font></font><br><div class="gmail_quote">On Thu, Apr 1, 2010 at 6:12 PM, miguel machado <span dir="ltr"><<a href="mailto:mls.machado@gmail.com" target="_blank">mls.machado@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex"><font size="2"><font face="verdana,sans-serif">hi again,</font></font><div><font face="verdana, sans-serif"><br>
</font></div>
<div><div class="gmail_quote">2010/4/1 Edson Tirelli <span dir="ltr"><<a href="mailto:ed.tirelli@gmail.com" target="_blank">ed.tirelli@gmail.com</a>></span><div><br>
<blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex"><br> Hmm, let me complement Wolfgang's comment: <br><br>> After you have taken the compiled packages from the KB and
checked for errors, the KB has done his duty the KB ****MUST**** go :-)<br></blockquote><div><br></div></div><div>I agree! But i don't exactly know how to do so... i mean, i call this method which creates the KB from a DRL file and then the kbase and returns the ksession, the KB object declaration scope is just that exact method, and once it reaches 300MB+ in memory, it just stays there... the KB object is no longer in use (or is it?) but memory keeps high. </div>
<div><br></div><div>I've tried setting KB to "null" and then invoking the jvm garbage collector, but to no avail. I wish there was a simple way i could just destroy a particular object sigh</div><div>
<div> </div>
<blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex"><br> The above emphasis is mine. KnowledgeBuilder is a higher level object designed to encapsulate the parsing+compilation steps of resources. It is **not** designed to be reused and doing so might create inconsistencies. The memory spike you see when using the kbuilder is due to the java compiler... </blockquote>
<div><br></div></div><div>I understand that, and i'll have that if i must, but the thing is it's not just a spike, it is prolonged throughout execution, it doesn't reduce afterwards.</div><div><div>
</div><blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex">
Drools itself does not do anything fancy in there other than generate some code (quite inexpensive memory wise) and compile it using either JDT or JANINO (both quite expensive memory wise, when compared to the other steps). <br>
<br> So, I strongly discourage you keeping the KB around and as mentioned by Wolfgang, doing so might keep unnecessary objects in memory.<br></blockquote><div><br></div></div><div>Like i said earlier, it's not that i want to keep KB around, i just can't remove it.</div>
<div>
<div> </div><blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex"><br> Also, just to clarify, deserializing a compiled package means the compiler will not be called (obviously, as everything is already compiled) and that is why it should save you memory.<br>
</blockquote><div><br></div></div><div>Thanks for clarifying that! I've tested with a separate application to build a compiled file and then load it in the main system and it really makes a difference! it's gone to <100MB wow! However, this was just a proof-of-concept, as i'm not sure how i could apply the same technique to the main standalone application. Do i really have to create a separate process/application just for building/compiling the rules, to make sure the compilation-object-junk doesn't stay in memory along with the rest?</div>
<div><br></div><div>Thanks for all the feedback and input so far.</div><div>_ miguel</div><font color="#888888"><div><br></div></font></div><div><br>-- <br>"To understand what is recursion you must first understand recursion"<br>
</div></div>
</blockquote></div><br><br clear="all"><br>-- <br>"To understand what is recursion you must first understand recursion"<br>
</div>
<br></div></div><div class="im">_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></div></blockquote></div><br>
<br>_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>"To understand what is recursion you must first understand recursion"<br>
</div>