[rules-users] Shadow facts(was JRules\Drools benchmarking)

Mark Proctor mproctor at codehaus.org
Thu May 15 12:21:59 EDT 2008 

Hehl, Thomas wrote:
>
> I did some reading about shadow facts and I was thinking about turning 
> them off, but it seems to be that if I use stateless sessions 
> (session.execute()) that shadow facts are irrelevant. Is this true?
>
No, only if you turn on sequential mode.

Mark
>
>  
>
> ------------------------------------------------------------------------
>
> *From:* rules-users-bounces at lists.jboss.org 
> [mailto:rules-users-bounces at lists.jboss.org] *On Behalf Of *Edson Tirelli
> *Sent:* Thursday, May 15, 2008 10:00 AM
> *To:* Rules Users List
> *Subject:* Re: [rules-users] JRules\Drools benchmarking...
>
>  
>
>
>    It seems you are using a good strategy to do your tests. But still, 
> it is difficult to explain why one is slower than the other without 
> seeing the actual test code. This is because all the engines have 
> stronger and weaker spots. Just to mention one example, some engines 
> (not talking specifically about drools and jrules, but about all 
> engines) implement faster alpha evaluation, others implement faster 
> beta (join) evaluation, others implement good optimizations for not() 
> while others may focus on eval(), etc. It is up to the point that when 
> comparing 2 engines, one performs better in hardware with a bigger L2 
> cache while the other performs better in hardware with a smaller L2 cache.
>
>    So, best I can do without looking at the actual tests is provide 
> you some tips:
>
> 1. First of all, are you using Drools 4.0.7? It is very important that 
> you use this version over the previous ones.
>
> 2. Are you using stateful or stateless sessions? If you are using 
> stateful sessions are you calling dispose() after using the session? 
> If not, you are inflating your memory and certainly causing the engine 
> to run slower over time.
>
> 3. Are you sharing the rulebase among multiple requests? The drools 
> rulebase is designed to be shared and the compilation process is eager 
> and pretty heavy compared to session creation. So, it pays off to 
> create the rulebase once and share among requests.
>
> 4. Did you disabled shadow facts? Test cases usually use a really 
> small fact base, so would not be much affected by shadow facts, but 
> still, disabling them improves performance, but require some best 
> practices to be followed.
>
> 5. Do your rules follow best practices (similar to SQL writing best 
> practices), i.e., write the most constraining patterns first, write 
> the most constraining restrictions first, etc? Do you write patterns 
> in the same order among rules to maximize node sharing? I guess you 
> do, but worth mentioning anyway.
>
>    Anyway, just some tips.
>
>    Regarding the jrules blog, I know it, but I make a bet with you. 
> Download the manners benchmark to your machine, make sure the rules 
> are the correct ones (not cheated ones), run the test on both engines 
> and share the results. I pay you a beer if you get results similar to 
> those published in the blog. :)
>    My point is not that we are faster (what I know we are) or them are 
> faster. My point is that perf benchmarks for rules engines are a 
> really tricky matter, with lots of variables involved, that make every 
> test case configuration unique. Try to reproduce in a different 
> environment and you will get different performance rates between the 
> engines.
>
>    That is why, our recommendation is to always do what you are doing: 
> try your own use case. Now, whatever you are trying, I'm sure it is 
> possible to optimize if we see the test case, but is it worth it? Or 
> the perf as it is already meets your requirements?
>
>    Cheers,
>        Edson
>
> PS: I'm serious about the beer... ;) run and share the results with us...
>
> 2008/5/15 mmquelo massi <mmquelo at gmail.com <mailto:mmquelo at gmail.com>>:
>
> You r right...
>
> I have to tell you what I have done...
>
> I did not define a "stand-alone" benchmark like the "Manners" one.
>
> I benchmarked a real j2ee application.
>
> I have got jrules deployed with a resource adapter and drools deployed
> with simple jars libraries plus jbrms.
>
> Jrules uses a "bres" module which does the same trick jbrms does.
>
> Both of them are deployed on the same AS, in the same time, same
> machine (my laptop: dual core 2 duo 1.66, 2GB).
>
> Using the inversion of control pattern I found out how to "switch the
> rule engine" at run-time. So I can easily choose then rule engine to
> use between drools and jrules.
>
> Ofcourse thay have got two separate rule repositories but both of them
> persist the rules on the same DB which is Derby.
>
> The j2ee application I benchmarked sends a request object to the
> current rule engine and get back a reply from it. I just measured the
> elapsed time between the request and reply generation using drools
> first and the jrules.
>
> I did the measurements tens of times.
>
> Both rule engines implement the same rules and the Drools rules (which
> I personally implemented) are at least as optimized as the jrules
> ones. In the Jrules version of the rules there are a lot of
> "Eval(...)" blocks in the Drools version I did not use the "Eval()" at
> all ....but I just did pattern matching.
>
> If you want i can send you a more specific documentation but I hope
> this explanation will be enough to show you that the measurements I
> have done are not that bad.
>
> In any case I noticed that after a warming-up phase, the drools engine
> gives a reply back 3 times slower than the jrules engine.
>
> The link I have sent show you something related to it, It reports the
> manners execution time using drools and jrules. As you can see the
> difference is a 1,5x factor....so I was wrong... drools is not that
> slow. In anycase seems to be slower that jrules.
>
> Look at this:
>
>
> http://blogs.ilog.com/brms/wp-content/uploads/2007/10/jrules-perf-manners.png
>
> Massimiliano
>
>
>
>
> On 5/15/08, Edson Tirelli <tirelli at post.com <mailto:tirelli at post.com>> 
> wrote:
> >    The old recurring performance evaluation question... :)
> >
> >    You know that an explanation can only be made after having looked 
> at the
> > tests used in the benchmark, the actual rules used by both products,
> > hardware specs, etc... so, not quite sure what answer do you want?
> >
> >    For instance, there are a lot of people that think exactly the 
> contrary.
> > Just one example:
> > http://blog.athico.com/2007/08/drools-vs-jrules-performance-and-future.html
> >
> >    My preferred answer is still:
> >
> > "In 99% of the applications, the bottleneck is IO: databases, 
> network, etc.
> > So, test your use case with both products, make sure it performs well
> > enough, add to your analysis the products feature set, expressiveness 
> power,
> > product flexibility, cost, professionals availability, support 
> quality, etc,
> > and choose the one that best fits you."
> >
> >    That is because I'm sure, whatever your rules are, in whatever product
> > you try them, they can be further optimized by having a product expert
> > looking into them. But what is the point?
> >
> >    Cheers,
> >       Edson
> >
> >
> >
> > 2008/5/14 mmquelo massi <mmquelo at gmail.com <mailto:mmquelo at gmail.com>>:
> >
> >>
> >> Hi everybody,
> >>
> >> I did a benchmark on Drools\Jrules.
> >>
> >> I found out that drools is about 2,5-3 times slower than Jrules.
> >>
> >> How comes?
> >>
> >> The results I got are quite similar to the ones in:
> >>
> >>
> >> 
> http://images.google.com/imgres?imgurl=http://blogs.ilog.com/brms/wp-content/uploads/2007/10/jrules-perf-manners.png&imgrefurl=http://blogs.ilog.com/brms/category/jrules/&h=516&w=722&sz=19&hl=it&start=1&um=1&tbnid=YBqwC0nwaSLxwM:&tbnh=100&tbnw=140&prev=/images%3Fq%3Dbrms%2Bbencmark%26um%3D1%26hl%3Dit 
> <http://images.google.com/imgres?imgurl=http://blogs.ilog.com/brms/wp-content/uploads/2007/10/jrules-perf-manners.png&imgrefurl=http://blogs.ilog.com/brms/category/jrules/&h=516&w=722&sz=19&hl=it&start=1&um=1&tbnid=YBqwC0nwaSLxwM:&tbnh=100&tbnw=140&prev=/images%3Fq%3Dbrms%2Bbencmark%26um%3D1%26hl%3Dit>
> >>
> >> Any explanations?
> >>
> >> Thank you.
> >>
> >> Bye
> >>
> >> Massi
> >>
> >> _______________________________________________
> >> rules-users mailing list
> >> rules-users at lists.jboss.org <mailto:rules-users at lists.jboss.org>
> >> https://lists.jboss.org/mailman/listinfo/rules-users
> >>
> >>
> >
> >
> > --
> > Edson Tirelli
> > JBoss Drools Core Development
> > Office: +55 11 3529-6000
> > Mobile: +55 11 9287-5646
> > JBoss, a division of Red Hat @ www.jboss.com <http://www.jboss.com>
> >
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org <mailto:rules-users at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
>
>
> -- 
> Edson Tirelli
> JBoss Drools Core Development
> Office: +55 11 3529-6000
> Mobile: +55 11 9287-5646
> JBoss, a division of Red Hat @ www.jboss.com <http://www.jboss.com>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20080515/0c5b1f56/attachment.html

More information about the rules-users mailing list