[rules-users] Drools 4 poor performance scaling?
Mark Proctor
mproctor at codehaus.org
Sat Jul 19 11:03:48 EDT 2008
It's more related to the issues with accumulate.
At the moment ever single propagation executes both the action and the
result and propagates the result to the next pattern. When dataloading
this means you are constantly evaluating the result and the rest of the
rule for each propagation. When in realty what you want to do is load
the accumulate and execute the result and propagate at the end - however
trying to find a robust way of doing this isn't easy.
Mark
Anstis, Michael (M.) wrote:
> I wonder whether is's a "benefit" of truth maintenance? If a new fact is
> inserted into working memory that could cause an activation of a rule
> that contains an accumulate (or collect) to change then the whole
> accumulate (or collect) operator is executed again?!?
>
> -----Original Message-----
> From: rules-users-bounces at lists.jboss.org
> [mailto:rules-users-bounces at lists.jboss.org] On Behalf Of Fenderbosch,
> Eric
> Sent: 09 July 2008 14:41
> To: Rules Users List
> Subject: RE: [rules-users] Drools 4 poor performance scaling?
>
> FYI for the group. We seem to have solved our performance problem.
>
> I'll describe our problem space a bit some people have some context. We
> load up about 1200 Jobs with about 3000 Stops and about 1500 Vehicles
> with about 2000 Workers. We then calculate Scores for each Vehicle for
> each Job. Some combinations get excluded for various reasons, but we
> end up with 700k - 900k total facts. We do score totaling and sorting
> using accumulators.
>
> One of our teams members (nice find Dan) decided to try to isolate the
> accumulation rules until all our other facts are loaded. Those rules
> now have a "not ColdStarting()" condition and our startup code inserts a
> ColdStarting fact as the first fact and retracts it when all the Jobs
> and Workers have been loaded. This changed our startup time from over
> 50 minutes to under 5. There's some sort of strange propagation and
> looping going on with accumulation on the fly, at least with our facts
> and rules.
>
> I'll put an entry on the wiki as well.
>
> -----Original Message-----
> From: rules-users-bounces at lists.jboss.org
> [mailto:rules-users-bounces at lists.jboss.org] On Behalf Of Fenderbosch,
> Eric
> Sent: Monday, June 30, 2008 11:46 AM
> To: Rules Users List
> Subject: RE: [rules-users] Drools 4 poor performance scaling?
>
> We are having a similar problem, although our fact count is much higher.
> Performance seems pretty good and consistent until about 400k facts,
> then performance degrades significantly. Part of the degradation is
> from bigger and more frequent GCs, but not all of it.
>
> Time to load first 100k facts: ~1 min
> Time to load next 100k facts: ~1 min
> Time to load next 100k facts: ~2 min
> Time to load next 100k facts: ~4 min
>
> This trend continues, going from 600k to 700k facts takes over 7
> minutes. We're running 4.0.7 on a 4 CPU box with 12 GB, 64 bit RH Linux
> and 64 bit JRockit 5. We've allocated a 9 GB heap for the VM using
> large pages, so no memory paging is happening. JRockit is started w/
> the -XXagressive parameter, which enables large pages and the more
> efficient hash function in HashMap which was introduced in Java5 update
> 8.
>
> http://e-docs.bea.com/jrockit/jrdocs/refman/optionXX.html
>
> The end state is over 700k facts, with the possibility of nearly 1M
> facts in production. After end state is reached and we issue a few GC
> requests, if looks like our memory per fact is almost 9k, which seems
> quite high as most of the facts are very simple. Could that be due to
> our liberal use of insertLogical and TMS?
>
> We've tried performing a "commit" every few hundred fact insertions by
> issuing a fireAllRules periodically, and that seems to have helped
> marginally.
>
> I tried disabling shadow proxies and a few of our ~390 test cases fail
> and one loops indefinitely. I'm pretty sure we could fix those, but
> don't want to bother if this isn't a realistic solution.
>
> Any thoughts?
>
> Thanks
>
> Eric
>
> -----Original Message-----
> From: rules-users-bounces at lists.jboss.org
> [mailto:rules-users-bounces at lists.jboss.org] On Behalf Of Ron Kneusel
> Sent: Thursday, June 26, 2008 12:47 PM
> To: rules-users at lists.jboss.org
> Subject: [rules-users] Drools 4 poor performance scaling?
>
>
> I am testing Drools 4 for our application and while sequential mode is
> very fast I get very poor scaling when I increase the number of facts
> for stateful or stateless sessions. I want to make sure I'm not doing
> something foolish before deciding on whether or not to use Drools
> because from what I am reading online it should be fast with the number
> of facts I have.
>
> The scenario: I have 1000 rules in a DRL file. They are all of the
> form:
>
> rule rule0000
> when
> Data(type == 0, value> 0.185264);
> Data(type == 3, value < 0.198202);
> then
> insert(new AlarmRaised(0));
> warnings.setAlarm(0, true);
> end
>
> where the ranges checked on the values and the types are randomly
> generated. Then, I create a Stateful session and run in a loop timing
> how long it takes the engine to fire all rules as the number of inserted
> facts increases:
>
> // Run
> for(j=0; j < 100; j+=5) {
>
> if (j==0) {
> nfacts = 1;
> } else {
> nfacts = j;
> }
>
> System.out.println(nfacts + ":");
>
> // Get a working memory
> StatefulSession wm = ruleBase.newStatefulSession();
>
> // Global - output
> warnings = new Alarm();
> wm.setGlobal("warnings", warnings);
>
> // Add facts
> st = (new Date()).getTime();
> for(i=0; i < nfacts; i++) {
> wm.insert(new Data(rand.nextInt(4),
> rand.nextDouble()-0.5));
> }
> en = (new Date()).getTime();
> System.out.println(" facts = " + (en-st));
>
> // Now run the rules
> st = (new Date()).getTime();
> wm.fireAllRules();
> en = (new Date()).getTime();
> System.out.println(" rules = " + (en-st));
>
> // Clean up
> wm.dispose();
>
> System.out.println("\n");
> }
>
> This code is based on the HelloWorldExample.java code from the manual
> and the setup for the rule base is the same as in the manual. As the
> number of facts increases runtime increases dramatically:
>
> facts -- runtime (ms)
> 10 -- 168
> 20 -- 166
> 30 -- 344
> 40 -- 587
> 50 -- 1215
> 60 -- 1931
> 70 -- 2262
> 80 -- 3000
> 90 -- 4754
>
> with a maximum memory use of about 428 MB RAM. By contrast, if I use
> sequential stateless sessions, everything runs in about 1-5 ms.
>
> Is there something in my set up that would cause this, or is this how
> one would expect Drools to scale? I read about people using thousands
> of facts so I suspect I'm setting something up incorrectly.
>
> Any help appreciated!
>
> Ron
>
> _________________________________________________________________
> The other season of giving begins 6/24/08. Check out the i'm Talkathon.
> http://www.imtalkathon.com?source=TXT_EML_WLH_SeasonOfGiving
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
More information about the rules-users
mailing list