<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div><div><div>I was running some performance analysis on my rules so I could tune in hopes of scaling up from hundreds of thousands of datapoints to millions of entries and noticed some alarming performance scaling issues when rules are firing and inserting dependent objects. At first I thought it was one of my more complex rules but when I broke down my examples to the simplest possible test cases I found that the central problem was that performance was not linear with number of facts inserted -- it was polynominal (roughly n^2.5).</div><div><br></div><div>My simplest test example was basically each fact inserted initially would trigger a new logical relationship fact to be inserted. No fancy logic. Also it could not trigger itself recursively. I tried three approaches : logical inserts of new facts, manual inserts/retraction, and update of the current fact with new data.</div><div><br></div><div>Updates on current fact were roughly linear assuming I prevented re-firing.</div><div>Manual Inserts and retracts scaled with O(n^2.5)</div><div>LogicalInserts were O(n^3)</div><div><br></div><div>The documentation suggest insertion of dependent facts to simplify logic. But when dealing with tens of thousands of events triggering it did not scale sufficiently. Can anyone tell me what I may have done wrong? I have included rules and data below. </div><div><br></div><div>This was on 5.3 using fireAllRules. The time measured was rule execution only and excluded times due to my testing framework.</div><div><br></div><div>// SIMPLIFIED RULES FILE</div><div><div>declare DataPoint</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>anomaly : boolean</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>entityId : String</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>average : double</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>predict : double</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>predictDev : double</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>timestamp : long</div><div>end</div><div><br></div><div>declare AnomalyFact</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>entityId : String</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>anomalyType : AnomalyType</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ruleName : String</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>timestamp : long</div><div>end</div><div><br></div><div>rule "Traffic Below 4 Deviations"</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>no-loop</div><div> when</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>$datapoint : DataPoint (</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>anomaly==false,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>average < (predict - (predictDev * 4))</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>)</div><div> then</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// update self only. This line was always active.</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>modify($datapoint) {setAnomaly(true)};</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// on logicalInsert test I commented in the following line</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// <span class="Apple-tab-span" style="white-space:pre"> </span>insertLogical( new AnomalyFact ( $datapoint.getEntityId(), AnomalyEnum.LOW, drools.getRule().getName(), $datapoint.getTimestamp()));</div><div><br></div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// on insert test I commented in the following line and added another rule to retract any anomaly with no datapoint</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// <span class="Apple-tab-span" style="white-space:pre"> </span>insert( new AnomalyFact ( $datapoint.getEntityId(), AnomalyEnum.LOW, drools.getRule().getName(), $datapoint.getTimestamp()));</div><div>end</div><div><br></div><div><br></div><div>/* TEST RESULT CONTAINED BELOW</div><div>All data inserted triggered the rule. I was load testing for worst case behavior.</div><div><br></div><div>logicalInsert</div><div>n<span class="Apple-tab-span" style="white-space:pre"> </span>run1<span class="Apple-tab-span" style="white-space:pre"> </span>run2<span class="Apple-tab-span" style="white-space:pre"> </span>run3</div><div>5000<span class="Apple-tab-span" style="white-space:pre"> </span>1293<span class="Apple-tab-span" style="white-space:pre"> </span>1232<span class="Apple-tab-span" style="white-space:pre"> </span>1767</div><div>10000<span class="Apple-tab-span" style="white-space:pre"> </span>9458<span class="Apple-tab-span" style="white-space:pre"> </span>8210<span class="Apple-tab-span" style="white-space:pre"> </span>10079</div><div>15000<span class="Apple-tab-span" style="white-space:pre"> </span>24389<span class="Apple-tab-span" style="white-space:pre"> </span>31050<span class="Apple-tab-span" style="white-space:pre"> </span>24311</div><div>30000<span class="Apple-tab-span" style="white-space:pre"> </span>134936</div><div><br></div><div>manual insert/retract</div><div>n<span class="Apple-tab-span" style="white-space:pre"> </span>run1<span class="Apple-tab-span" style="white-space:pre"> </span>run2<span class="Apple-tab-span" style="white-space:pre"> </span>run3</div><div>5000<span class="Apple-tab-span" style="white-space:pre"> </span>1137<span class="Apple-tab-span" style="white-space:pre"> </span>1094<span class="Apple-tab-span" style="white-space:pre"> </span>1169</div><div>10000<span class="Apple-tab-span" style="white-space:pre"> </span>8090<span class="Apple-tab-span" style="white-space:pre"> </span>5862<span class="Apple-tab-span" style="white-space:pre"> </span>5616</div><div>15000<span class="Apple-tab-span" style="white-space:pre"> </span>16741<span class="Apple-tab-span" style="white-space:pre"> </span>16728<span class="Apple-tab-span" style="white-space:pre"> </span>15874</div><div>30000<span class="Apple-tab-span" style="white-space:pre"> </span>76034</div><div><br></div><div>update on current object only</div><div>n<span class="Apple-tab-span" style="white-space:pre"> </span>run1<span class="Apple-tab-span" style="white-space:pre"> </span>run2<span class="Apple-tab-span" style="white-space:pre"> </span>run3</div><div>5000<span class="Apple-tab-span" style="white-space:pre"> </span>417<span class="Apple-tab-span" style="white-space:pre"> </span>444<span class="Apple-tab-span" style="white-space:pre"> </span>413</div><div>10000<span class="Apple-tab-span" style="white-space:pre"> </span>600</div><div>15000<span class="Apple-tab-span" style="white-space:pre"> </span>712<span class="Apple-tab-span" style="white-space:pre"> </span>697<span class="Apple-tab-span" style="white-space:pre"> </span>688</div><div>30000<span class="Apple-tab-span" style="white-space:pre"> </span>999<span class="Apple-tab-span" style="white-space:pre"> </span>970<span class="Apple-tab-span" style="white-space:pre"> </span>1041<span class="Apple-tab-span" style="white-space:pre"> </span>1016<span class="Apple-tab-span" style="white-space:pre"> </span>1163</div><div>60000<span class="Apple-tab-span" style="white-space:pre"> </span>1420</div><div>100000<span class="Apple-tab-span" style="white-space:pre"> </span>1930</div></div><div><br></div><div>Thanks for any help!</div><div>*/</div><div></div></div></div></body></html>