[rules-users] Rule performance with accumulate

[Ryan Crumley]

Thanks Wolfgang.

RE (1): Most of the rules with accumulate have a "blocking" fact. Sounds
like it would be useful to verify the blocking fact is working as expected.
I will give this a try.

RE (4): Most of the accumulates in these rules are exactly what you
describe… they are used at the end of processing to provide a "summary" of
sorts. I think I understand what you mean by converting this to "on foot".
I was tempted to try this but thought if there was any way to determine the
exact rule which has the problematic accumulations I could focus on
converting only that rule. Converting all of the rules is a large
commitment. Would be great to be able to verify it will pay off before
making the investment.

Ryan


On Wed, Jan 9, 2013 at 8:59 AM, Wolfgang Laun <wolfgang.laun at gmail.com>wrote:

> A few pointers:
>
> (1) Agenda groups do not prevent rules from other groups than the
> current one to be constantly evaluated. (I guess that using an initial
> "blocking" fact in the costly rules might be an alternative, but I've
> never tried and benchmarked.)
>
> (2) Accumulating into a Set is more costly than into a List as it
> needs to maintain a map like Map<Whatever,Integer> to permit the
> "reverse" operation when an element drops out of the accumulation,
> i.e., to decrease the map's value.
>
> (3) All accumulations with reverse maintain a map for fact ids to
> whatever, in order to be able to do the reverse operation.
>
> (4) All accumulates with highly volatile domain with a substantial
> cardinality are costly. If you need the accumulated values only at the
> end, an evaluation "on foot" might be considered. (This is very
> simple, if it needs to be done only once.)
>
> -W
>
> On 09/01/2013, Ryan Crumley <crumley at gmail.com> wrote:
> > Hi,
> >
> > I am investigating performance of a Drools 5.4 stateful knowledge
> session.
> > This session has about 200 rules, 200k facts and takes about 1 hour to
> run
> > to completion. Looking at the profile there is a hotspot that consumes
> > almost 65% of the cpu time: java.util.AbstractList.hashCode().
> >
> > Here is the full stack:
> >
> >
> >
> com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cbDefaultConsequenceInvoker.evaluate(KnowledgeHelper,
> > WorkingMemory)
> >
> >
> com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cbDefaultConsequenceInvokerGenerated.evaluate(KnowledgeHelper,
> > WorkingMemory)
> >
> >
> com.company.rules.engine.Rule_Set_weights_08b44ce519a74b58ab3f85735b2987cb.defaultConsequence(KnowledgeHelper,
> > List, FactHandle, GradingFact, FactHandle, ReportNode, FactHandle,
> > WeightsHolder, FactHandle, Logger)
> >    org.drools.base.DefaultKnowledgeHelper.update(FactHandle, long)
> >    org.drools.common.NamedEntryPoint.update(FactHandle, Object, long,
> > Activation)
> >    org.drools.common.NamedEntryPoint.update(FactHandle, Object, long,
> > Activation)
> >
> >
> org.drools.common.PropagationContextImpl.evaluateActionQueue(InternalWorkingMemory)
> >
> >
> org.drools.reteoo.ReteooWorkingMemory$EvaluateResultConstraints.execute(InternalWorkingMemory)
> >
> >
> org.drools.reteoo.AccumulateNode.evaluateResultConstraints(AccumulateNode$ActivitySource,
> > LeftTuple, PropagationContext, InternalWorkingMemory,
> > AccumulateNode$AccumulateMemory, AccumulateNode$AccumulateContext,
> boolean)
> >    org.drools.common.DefaultFactHandle.setObject(Object)
> >    java.util.AbstractList.hashCode()
> >
> > I believe the following clues can be extracted:
> >
> > - "Rule_Set_weights" was fired and a fact was modified (confirmed by
> > examining the rule definition)
> > - The fact modification caused the pre-conditions for other rules to be
> > computed.
> > - One of these rules has an accumulate condition that accumulates into an
> > AbstractList.
> > - This list is very very large. So large that looping through the
> elements
> > in the list and aggregating the hashCode of individual elements dominates
> > execution time (the individual element hashCode doesn't even show up in
> the
> > profile… either its very fast or maybe its identify hashCode which the
> > profiler might filter?).
> > - Accumulate is either working on a large set of data or the same
> > accumulate is evaluated many many times.
> >
> > Is my analysis correct? Are there clues that I am missing?
> >
> > I have 15 rules that use accumulate… However none accumulate with a
> result
> > of List. Most accumulate using sum() and count() (result of Number). A
> few
> > use collectSet(). A few more aggregate into a result with a custom type.
> >
> > A few other notes:
> > - All accumulate conditions are the last condition in the WHEN clause.
> > - I use agenda groups to separate fact processing into phases. Rules that
> > accumulate are in a separate agenda group from rules that modify/insert
> > facts that are used in accumulation. I hope this prevents the accumulate
> > condition from being evaluated until all the rules that modify the facts
> > accumulate needs are done firing. I suspect this may not be working as I
> > expect. I haven't put together an example to investigate.
> > - When accumulating into a set, the rule condition looks like this:
> >     $factName : Set() from accumulate( FactMatch( $field : field ),
> > collectionSet( $field ) )
> >
> > How can I narrow down this further?
> >
> > Are there any general rules to follow to optimize use of accumulate in
> > conditions?
> >
> > Thanks,
> >
> > Ryan
> >
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20130109/b17a3fd0/attachment-0001.html