<div dir="ltr">Thanks Wolfgang.<div><br></div><div style>RE (1): Most of the rules with accumulate have a "blocking" fact. Sounds like it would be useful to verify the blocking fact is working as expected. I will give this a try.</div>
<div style><br></div><div style>RE (4): Most of the accumulates in these rules are exactly what you describe… they are used at the end of processing to provide a "summary" of sorts. I think I understand what you mean by converting this to "on foot". I was tempted to try this but thought if there was any way to determine the exact rule which has the problematic accumulations I could focus on converting only that rule. Converting all of the rules is a large commitment. Would be great to be able to verify it will pay off before making the investment.</div>
<div style><br></div><div style>Ryan </div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jan 9, 2013 at 8:59 AM, Wolfgang Laun <span dir="ltr"><<a href="mailto:wolfgang.laun@gmail.com" target="_blank">wolfgang.laun@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">A few pointers:<br>
<br>
(1) Agenda groups do not prevent rules from other groups than the<br>
current one to be constantly evaluated. (I guess that using an initial<br>
"blocking" fact in the costly rules might be an alternative, but I've<br>
never tried and benchmarked.)<br>
<br>
(2) Accumulating into a Set is more costly than into a List as it<br>
needs to maintain a map like Map<Whatever,Integer> to permit the<br>
"reverse" operation when an element drops out of the accumulation,<br>
i.e., to decrease the map's value.<br>
<br>
(3) All accumulations with reverse maintain a map for fact ids to<br>
whatever, in order to be able to do the reverse operation.<br>
<br>
(4) All accumulates with highly volatile domain with a substantial<br>
cardinality are costly. If you need the accumulated values only at the<br>
end, an evaluation "on foot" might be considered. (This is very<br>
simple, if it needs to be done only once.)<br>
<span class="HOEnZb"><font color="#888888"><br>
-W<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On 09/01/2013, Ryan Crumley <<a href="mailto:crumley@gmail.com">crumley@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> I am investigating performance of a Drools 5.4 stateful knowledge session.<br>
> This session has about 200 rules, 200k facts and takes about 1 hour to run<br>
> to completion. Looking at the profile there is a hotspot that consumes<br>
> almost 65% of the cpu time: java.util.AbstractList.hashCode().<br>
><br>
> Here is the full stack:<br>
><br>
><br>
> com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cbDefaultConsequenceInvoker.evaluate(KnowledgeHelper,<br>
> WorkingMemory)<br>
><br>
> com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cbDefaultConsequenceInvokerGenerated.evaluate(KnowledgeHelper,<br>
> WorkingMemory)<br>
><br>
> com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cb.defaultConsequence(KnowledgeHelper,<br>
> List, FactHandle, GradingFact, FactHandle, ReportNode, FactHandle,<br>
> WeightsHolder, FactHandle, Logger)<br>
> org.drools.base.DefaultKnowledgeHelper.update(FactHandle, long)<br>
> org.drools.common.NamedEntryPoint.update(FactHandle, Object, long,<br>
> Activation)<br>
> org.drools.common.NamedEntryPoint.update(FactHandle, Object, long,<br>
> Activation)<br>
><br>
> org.drools.common.PropagationContextImpl.evaluateActionQueue(InternalWorkingMemory)<br>
><br>
> org.drools.reteoo.ReteooWorkingMemory$EvaluateResultConstraints.execute(InternalWorkingMemory)<br>
><br>
> org.drools.reteoo.AccumulateNode.evaluateResultConstraints(AccumulateNode$ActivitySource,<br>
> LeftTuple, PropagationContext, InternalWorkingMemory,<br>
> AccumulateNode$AccumulateMemory, AccumulateNode$AccumulateContext, boolean)<br>
> org.drools.common.DefaultFactHandle.setObject(Object)<br>
> java.util.AbstractList.hashCode()<br>
><br>
> I believe the following clues can be extracted:<br>
><br>
> - "Rule_Set_weights" was fired and a fact was modified (confirmed by<br>
> examining the rule definition)<br>
> - The fact modification caused the pre-conditions for other rules to be<br>
> computed.<br>
> - One of these rules has an accumulate condition that accumulates into an<br>
> AbstractList.<br>
> - This list is very very large. So large that looping through the elements<br>
> in the list and aggregating the hashCode of individual elements dominates<br>
> execution time (the individual element hashCode doesn't even show up in the<br>
> profile… either its very fast or maybe its identify hashCode which the<br>
> profiler might filter?).<br>
> - Accumulate is either working on a large set of data or the same<br>
> accumulate is evaluated many many times.<br>
><br>
> Is my analysis correct? Are there clues that I am missing?<br>
><br>
> I have 15 rules that use accumulate… However none accumulate with a result<br>
> of List. Most accumulate using sum() and count() (result of Number). A few<br>
> use collectSet(). A few more aggregate into a result with a custom type.<br>
><br>
> A few other notes:<br>
> - All accumulate conditions are the last condition in the WHEN clause.<br>
> - I use agenda groups to separate fact processing into phases. Rules that<br>
> accumulate are in a separate agenda group from rules that modify/insert<br>
> facts that are used in accumulation. I hope this prevents the accumulate<br>
> condition from being evaluated until all the rules that modify the facts<br>
> accumulate needs are done firing. I suspect this may not be working as I<br>
> expect. I haven't put together an example to investigate.<br>
> - When accumulating into a set, the rule condition looks like this:<br>
> $factName : Set() from accumulate( FactMatch( $field : field ),<br>
> collectionSet( $field ) )<br>
><br>
> How can I narrow down this further?<br>
><br>
> Are there any general rules to follow to optimize use of accumulate in<br>
> conditions?<br>
><br>
> Thanks,<br>
><br>
> Ryan<br>
><br>
<br>
</div></div><div class="HOEnZb"><div class="h5">_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
</div></div></blockquote></div><br></div>