Hi All,<br><br>"There is nothing in that which would gain from Drools' Rete algorithm with
its superior "many-patterns-many-object" matching capabilities."<br><br>I know Wolfgang knows all this, but I think we need to clarify this statement for the general public of the list. Assuming all the rules are indeed like he described (very simple, with a single alpha constraint), then there are efficient and simple ways to implement it without a rules engine, but that is not by using "if". At least he would have to use a hashing technique, or the performance would be roughly linear on number of rules ( O(r) ) when in Drools, that does use hashing techniques for alpha constraints, it is constant ( O(1) ). Also, it depends on the non-functional requirements he has, like lifecycle management. <br>
<br> In any case, I decided to take the bate and implement a quick example for this. I changed the HelloWorld example in Drools to generate 20k rules in the same format you have:<br><br>rule "rule X"<br> dialect "mvel"<br>
when<br> Message( message == "X" )<br>then<br> System.out.println( X );<br>end<br> <br> Replacing X by the number of the rule. Then I parse, compile and build a kbase for these rules and fire the rules for a single object. This is **NOT** a benchmark, so I am just giving you a rough idea of the hardware... I am running it on my laptop, with other stuff on. My laptop is a 2-years old lenovo, dual core, 2Gb memory. I am running with -Xmx1024M.<br>
<br> Results I got (in ms):<br><br>1. Generating Package ...done. 163 ms<br>2. Dumping source ...done. 280897 ms<br>3. Parsing and compiling...done. 40557 ms<br>4. Creating kbase ...done. 79512 ms<br>5. Firing rules ...done. 29 ms<br>
<br> Please note that steps 1 and 2 are the rules generation. You don't have that, since your source code is ready for processing, right? Step 3 is executed by the KnowledgeBuilder and step 4 is the kbase.addKnowledgePackages().<br>
<br> Again, this is a very simple case showing that it is possible to parse, compile and load 20k rules quite easily in a drools kbase. Also remember that kbases are designed to be shared, so you should incur that cost once and then create sessions (that are really light to create) reusing the kbase.<br>
<br> Finally, there are obviously room to improve, since the compilation and kbase building is single thread today. We want to move to multi-thread compilation in the future. That should improve things a bit.<br><br> The source code I used is bellow. <br>
<br> Cheers,<br> Edson<br><br><br>public class DroolsTest {<br><br> private static final int NUMBER_OF_RULES = 20000;<br><br> public static final void main(String[] args) {<br> try {<br> // load up the knowledge base<br>
KnowledgeBase kbase = readKnowledgeBase();<br> StatefulKnowledgeSession ksession = kbase.newStatefulKnowledgeSession();<br> //KnowledgeRuntimeLogger logger = KnowledgeRuntimeLoggerFactory.newFileLogger(ksession, "test");<br>
// go !<br> long ts = System.currentTimeMillis();<br> Message message = new Message();<br> message.setMessage("5");<br> message.setStatus(Message.HELLO);<br>
ksession.insert(message);<br> ksession.fireAllRules();<br> System.out.print("5. Firing rules ...");<br> System.out.println("done. "+(System.currentTimeMillis()-ts)+"ms");<br>
//logger.close();<br> } catch (Throwable t) {<br> t.printStackTrace();<br> }<br> }<br><br> private static KnowledgeBase readKnowledgeBase() throws Exception {<br> KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();<br>
<br> System.out.print("1. Generating Package ...");<br> long ts = System.currentTimeMillis();<br> PackageDescr pkg = generatePackageDescr();<br> System.out.println("done. "+(System.currentTimeMillis()-ts)+"ms");<br>
<br> System.out.print("2. Dumping source ...");<br> ts = System.currentTimeMillis();<br> String source = new DrlDumper( ).dump( pkg );<br> //System.out.println(source);<br>
System.out.println("done. "+(System.currentTimeMillis()-ts)+"ms");<br> <br> System.out.print("3. Parsing and compiling...");<br> ts = System.currentTimeMillis();<br>
kbuilder.add(ResourceFactory.newReaderResource( new StringReader( source ) ), ResourceType.DRL);<br> KnowledgeBuilderErrors errors = kbuilder.getErrors();<br> if (errors.size() > 0) {<br> for (KnowledgeBuilderError error: errors) {<br>
System.err.println(error);<br> }<br> throw new IllegalArgumentException("Could not parse knowledge.");<br> }<br> System.out.println("done. "+(System.currentTimeMillis()-ts)+"ms");<br>
<br> System.out.print("4. Creating kbase ...");<br> ts = System.currentTimeMillis();<br> KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase();<br> kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());<br>
System.out.println("done. "+(System.currentTimeMillis()-ts)+"ms");<br> return kbase;<br> }<br> <br> private static PackageDescr generatePackageDescr() {<br> PackageDescr result = new PackageDescr("org.drools.test");<br>
result.addImport( new ImportDescr("com.sample.DroolsTest.Message") );<br> for( int i = 1; i <= NUMBER_OF_RULES; i++ ) {<br> RuleDescr rule = new RuleDescr("rule "+i);<br> AttributeDescr dialect = new AttributeDescr("dialect", "mvel");<br>
AndDescr lhs = new AndDescr();<br> PatternDescr pat = new PatternDescr("Message");<br> FieldConstraintDescr constr = new FieldConstraintDescr("message");<br> LiteralRestrictionDescr restr = new LiteralRestrictionDescr("==",String.valueOf( i ));<br>
constr.addRestriction( restr );<br> pat.addConstraint( constr );<br> lhs.addDescr( pat );<br> rule.addAttribute( dialect );<br> rule.setLhs( lhs );<br> rule.setConsequence( "System.out.println("+i+");\n" );<br>
result.addRule( rule );<br> }<br> return result;<br> }<br><br> public static class Message {<br> <br> public static final int HELLO = 0;<br> public static final int GOODBYE = 1;<br>
<br> private String message;<br><br> private int status;<br><br> public String getMessage() {<br> return this.message;<br> }<br><br> public void setMessage(String message) {<br>
this.message = message;<br> }<br><br> public int getStatus() {<br> return this.status;<br> }<br><br> public void setStatus(int status) {<br> this.status = status;<br>
}<br> <br> }<br><br>}<br><br><br><br><div class="gmail_quote">2010/2/6 Wolfgang Laun <span dir="ltr"><<a href="mailto:wolfgang.laun@gmail.com">wolfgang.laun@gmail.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Have you tried increasing the heap size for the JVM?<br>
<br>
"20K rules [of the same] simple form" sounds as if you are<br>
trying to use a hammer where a screwdriver would be appropriate.<br>
If your description is correct, these 20K rules would be nothing<br>
but 20K if statements, in disguised form. There is nothing<br>
in that which would gain from Drools' Rete algorithm with<br>
its superior "many-patterns-many-object" matching capabilities.<br>
<font color="#888888"><br>
-W<br>
</font><div><div></div><div class="h5"><br>
On Sat, Feb 6, 2010 at 12:42 PM, kpowerinfinity<br>
<<a href="mailto:kpowerinfinity@gmail.com">kpowerinfinity@gmail.com</a>> wrote:<br>
> Hello,<br>
><br>
> We are trying to run a DRL file that has more than 20K rules, all of<br>
> which have the simple form:<br>
><br>
><br>
><br>
> rule "testpeoplenumber"<br>
> salience 100<br>
> dialect "mvel"<br>
> when<br>
> LineItem( inventoryItemCode == "8903210392090")<br>
> then<br>
> InventoryItem fact0 = new InventoryItem();<br>
> fact0.setCategoryCode( "ONPROMO" );<br>
> insert(fact0 );<br>
> end<br>
><br>
> However, the rule file is not getting parsed by the builder, and<br>
> either takes a very long time (well over half an hour), or gives a<br>
> Heap Overflow exception. We tried with a smaller file (about 1K rules,<br>
> and the situation is the same).<br>
><br>
> The parsing code is:<br>
> org.drools.KnowledgeBaseConfiguration kbc =<br>
> org.drools.KnowledgeBaseFactory.newKnowledgeBaseConfiguration(null,<br>
> cl);<br>
> org.drools.KnowledgeBase kbase =<br>
> org.drools.KnowledgeBaseFactory.newKnowledgeBase(kbc);<br>
> org.drools.builder.KnowledgeBuilderConfiguration<br>
> knowledgeBuilderConfig =<br>
> org.drools.builder.KnowledgeBuilderFactory.newKnowledgeBuilderConfiguration(null,<br>
> cl);<br>
> org.drools.builder.KnowledgeBuilder builder =<br>
> org.drools.builder.KnowledgeBuilderFactory.newKnowledgeBuilder(kbase,<br>
> knowledgeBuilderConfig);<br>
> org.drools.builder.impl.KnowledgeBuilderImpl builderImpl =<br>
> builder as org.drools.builder.impl.KnowledgeBuilderImpl;<br>
><br>
> builder.add(org.drools.io.ResourceFactory.newFileResource(filename),<br>
> org.drools.builder.ResourceType.DRL);<br>
><br>
> We are using Drools v5. We've tried both with the java jar as well as<br>
> in .NET (using IKVM to compile it).<br>
><br>
> Any help in understanding why this is happening and how it can be<br>
> avoided will be appreciated.<br>
><br>
> Warm Regards<br>
> Krishna.<br>
><br>
> --<br>
> <a href="http://kpowerinfinity.wordpress.com" target="_blank">http://kpowerinfinity.wordpress.com</a><br>
> <a href="http://www.linkedin.com/in/kpowerinfinity" target="_blank">http://www.linkedin.com/in/kpowerinfinity</a><br>
> _______________________________________________<br>
> rules-users mailing list<br>
> <a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
><br>
<br>
_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br> Edson Tirelli<br> JBoss Drools Core Development<br> JBoss by Red Hat @ <a href="http://www.jboss.com">www.jboss.com</a><br>