[rules-users] Rule set execution performance and memory consumption issues

[Wolfgang Laun]

Guete Morge Joe,

from that other thread I can recall that you are using "from
<expression>", which (not knowing the rest) would be high up on my
suspect list.

And, in order to learn more, some monitoring should be considered. The
number of activation creations and deletions (not by firing) can
indicate wasted work that might be avoided.

Cheers
-W

On 05/08/2013, Joe Ammann <joe at pyx.ch> wrote:
> Hi Wolfgang
>
> On 08/04/2013 03:10 PM, Wolfgang Laun wrote:
>> A simple test shows that, with the rules as shown, you can insert 100k
>> Assignments and an equal number of SourceAssignments (with ~300
>> InstrumentList facts) in a matter of seconds, irrespective of the
>> number of hits and  misses, either way. I suspect that there is some
>> other fly in the ointment.
> I suspect this too, but am unable to lay my fingers on it,
> unfortunately. That's why I started to question my basic pattern. But
> all my test trying to get to the core of the problem showed the same as
> your quick tests - there is absolutely no fundamental problem handling a
> couple of 100'000 objects.
>>
>> There is no "Instrument" pattern in any of the rules, which makes your
>> post slightly inconsistent in itself.
> Right - I was trying to explain the pattern which I suspected to be the
> culprit, without posting a dozen Java classes and a couple of dozens of
> rules. Always a difficult thing to do. The real rules have many more
> patterns, after matching the basic objects, e.g.
>
> rule "R1110: MatchExistingAssignment"
> when
>     sa : SourcingAssignment( )
>     a : Assignment( instrumentId == sa.instrumentId, instrumentListId ==
> sa.instrumentListId )
>     i  : Instrument( id == instrumentId, foType == FOTYPE.Stock,
> statusCd == INSTRUMENT_STATUS.Draft )
>     not DQAttribute ( rootId == i.id, attributePath ==
> Instrument._ATTR_statusCd )
>     ....
>  then
>
> I think I have in the meantime found the main issue - it was one single
> rule that caused the majority of the problem. It was written really
> stupidly ... The time is now down to ~ 2 minutes. Still I see in the log
>
> 2013-08-05 00:36:56.224 [ain] DEBUG:                         Check:
> start inserting objects into working memory {|||||| - |}
> 2013-08-05 00:36:56.225 [ain] DEBUG:                         Check: next
> object type: SourcingInstrument, (12668 instances) {|||||| - |}
> 2013-08-05 00:36:56.701 [ain] DEBUG:                         Check: next
> object type: SourcingInstrumentList, (38 instances) {|||||| - |}
> 2013-08-05 00:36:56.707 [ain] DEBUG:                         Check: next
> object type: SourcingAssignment, (194693 instances) {|||||| - |}
> 2013-08-05 00:36:57.573 [ain] DEBUG:                         Check: next
> object type: Instrument, (17876 instances) {|||||| - |}
> 2013-08-05 00:37:41.009 [ain] DEBUG:                         Check: next
> object type: InstrumentList, (49 instances) {|||||| - |}
> 2013-08-05 00:37:41.359 [ain] DEBUG:                         Check:
> finished inserting objects into working memory {|||||| - |}
> 2013-08-05 00:37:41.360 [ain] DEBUG:                         Check: fire
> validate staging rules {|||||| - |}
> 2013-08-05 00:37:43.502 [ain] DEBUG:                         Check:
> validate staging rules finished, 147674 executed {|||||| - |}
> 2013-08-05 00:37:43.502 [ain] DEBUG:                         Check:
> start inserting objects into working memory {|||||| - |}
> 2013-08-05 00:37:43.502 [ain] DEBUG:                         Check: next
> object type: Assignment, (120006 instances) {|||||| - |}
> 2013-08-05 00:37:50.114 [ain] DEBUG:                         Check:
> finished inserting objects into working memory {|||||| - |}
>
> And I'm wondering a bit why inserting 17876 Instruments takes 45
> seconds, while inserting 194693 SourcingAssignments takes less than a
> second.
>
> I must admit that I have absolutely no clue how I would systematically
> debug such a situation and try to find out which rules are causing the
> "problem". I stumbled over this one stupid rule just by going through
> them one by one. I'm trying to get some experience to cope with such
> problems - since I'm not the only one writing the rules, I continue to
> see such "unexplicable" slowdowns.
>
> Sigh - I guess I still have A LOT to learn about Drools :-)
>
> Thanks for your help Wolfgang - and BTW: your whitepaper about rule
> patterns is a major reference for my learning.
>
> --
> CU, Joe
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>