<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<title>Scaling to large numbers of rules</title>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Adam,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>There are several ways to deal with large number of rules.
For example, instead of re-compiling the whole rule set every time, you could
just compile the rules that are updated/new, then adding them to the existing
KnowledgeBase instance. Drools is capable to handle such usage, even
during rule execution – but would not recommend to do that. API’s
are also available to remove packages, rules, etc. We are using something
similar for generated rules. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>It is also important to note that it would require persisting
the KnowledgeBase so it could be reloaded after restarting the application.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Just a thought.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>-Ming<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
rules-users-bounces@lists.jboss.org
[mailto:rules-users-bounces@lists.jboss.org] <b>On Behalf Of </b>Adam Sussman<br>
<b>Sent:</b> Wednesday, September 02, 2009 7:21 PM<br>
<b>To:</b> rules-users@lists.jboss.org<br>
<b>Subject:</b> [rules-users] Scaling to large numbers of rules<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p><span style='font-size:10.0pt'>I am hoping that I am doing something wrong
here and that one of you can<br>
point me in the right direction.<br>
<br>
Can anyone provide some advice on scaling up the number of rules in a<br>
single KnowledgeBase? While I have seen all sorts of reports on having<br>
lots of facts, I have not seen anything about having lots of rules.<br>
<br>
I need to get to about 200K rules in a single KnowledgeBase, and also<br>
to run several of these side by side in the same system.<br>
<br>
The problem:<br>
<br>
As the number of rules increases, the time to compile and load them<br>
into memory skyrockets. Now while I realize that the Rete algorithm<br>
complexity is about the number of rules, the times I am seeing are<br>
pretty scary. Also, at about 30k rules, things just fall apart.<br>
<br>
On a 64 bit Linux OS with 2 2.4Ghz processors using a 64bit JVM from<br>
OpenJDK (1.6.1) with 1 Gig of memory allocated to the JVM, loading<br>
from .drl files:<br>
<br>
1000 rules:<br>
KnowledgeBuilder.add:
7 seconds<br>
KnowledgeBase.addKnowledgePackages:
.8 seconds<br>
<br>
10000 rules:<br>
KnowledgeBuilder.add:
79 seconds<br>
KnowledgeBase.addKnowledgePackages:
23 seconds<br>
<br>
15000 rules:<br>
KnowledgeBuilder.add:
138 seconds<br>
KnowledgeBase.addKnowledgePackages: 55 seconds<br>
<br>
20000 rules:<br>
KnowledgeBuilder.add:
488 seconds<br>
KnowledgeBase.addKnowledgePackages:
100 seconds<br>
<br>
30000 rules:<br>
KnowledgeBuilder.add:
out of memory<br>
KnowledgeBase.addKnowledgePackages:
never runs<br>
<br>
At this rate, 200k rules will take 13-14 hours to compile<br>
and 2-3 hours to load into RAM, assuming I can even get<br>
to that many rules. This just is not usable.<br>
<br>
Time to fire all rules is negligible (fortunately!).<br>
<br>
The rules I am testing on are very simple 1-3 variable equality whens<br>
with a simple System.out.println then clause.<br>
<br>
The benchmark code I am running is as follows:<br>
<br>
KnowledgeBase kbase =
KnowledgeBaseFactory.newKnowledgeBase();<br>
KnowledgeBuilder kbuilder =
KnowledgeBuilderFactory.newKnowledgeBuilder();<br>
<br>
kbuilder.add( ResourceFactory.newClassPathResource( drlFile,
RuleRunner.class ), ResourceType.DRL );<br>
<br>
Collection<KnowledgePackage> pkgs =
kbuilder.getKnowledgePackages();<br>
kbase.addKnowledgePackages( pkgs );<br>
<br>
<br>
Sample rule:<br>
<br>
rule "00000005 - random rule"<br>
when<br>
Transaction(someId == 35156 && someOtherId == '79F81FB8134A129F'
&& someCollection contains 'EC3F2A1DCA88')<br>
then<br>
System.out.println("match rule 00000005 - random rule");<br>
end<br>
<br>
Any help would be appreciated.<br>
<br>
Regards,<br>
<br>
Adam Sussman</span><o:p></o:p></p>
<pre>CONFIDENTIALITY NOTICE:<o:p></o:p></pre><pre><o:p> </o:p></pre><pre>This message contains information which may be confidential or privileged. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this transmission in error, please notify me immediately by telephone.<o:p></o:p></pre></div>
</body>
</html>