Re: [rules-users] optimization on a lot of simple rules

From: nesta <nesta.fdb(a)163.com&gt; Subject: Re: [rules-users] optimization on a lot of simple rules To: rules-users(a)lists.jboss.org Date: Monday, July 20, 2009, 10:13 PM I execute above test codes, with 100 rules running 1000 times. In the same time, I profile the test codes. http://www.nabble.com/file/p24581025/profile.jpg The profile result shows that some methods execute 100,000 times. In test codes, Service has two states, 0 or 1. Subscriber has two states, 0 or 1. inserted facts: Product product = new Product(1, 1); Service service = new Service(1); Subscriber subscriber = new Subscriber(1); Service matches a half of total rules, namely 50. So is Subscriber. (50 + 50 ) * 1000 (running times) = 100,000 I am confused what drools does. Is there a way to optimize it? Greg Barton wrote: > > > 1) Yes, if you eliminate joins in rules, you will have no joins in the > rete.  This is self evident. > > 2) The way you have the rules structured, there is no relationship between > the joined objects.  This will cause what's called a "cartesian join" > where all combinations of all instances of each object type are > instantiated.  This can be very expensive, memory and CPU wise.  You've > stated that there are only one instance of each object type in working > memory, but are you absolutely sure of that?  Cartesian joins can easily > cause performance problems quite quickly. > > For instance, say you've got these objects in working memory: > > Subscriber(gender="male") > Subscriber(gender="female") > Service(name="ftp") > Service(name="http") > Product(id=1) > Product(id=2) > Product(id=3) > > After inserting a Decision into working memory, the rule will fire 2*2*3 > times.  (#Subscribers * #Services * #Products) This is by design.  Is this > what you want? > > 3) Do you really need the 'Subscriber(gender == "male" or "female")' term? > Why not just 'Subscriber()'?  Are you classifying transgendered or > nonhuman subscribers in your system? > > --- On Mon, 7/20/09, nesta <nesta.fdb(a)163.com&gt; wrote: > >> From: nesta <nesta.fdb(a)163.com&gt; >> Subject: Re: [rules-users] optimization on a lot of simple rules >> To: rules-users(a)lists.jboss.org >> Date: Monday, July 20, 2009, 10:22 AM >> >> I want to test the matching performance of drools. As I >> mentioned that there >> are a lot of rules and  the rule is like: >> rule 1 >>     when >>           Decision() >>          Subscriber(gender == >> "male" or "female") >>          Service(name == >> "ftp" or "http") >>          Product(id == 1) >>           ...... >>     then >> end >> >> After test, more condition elements under when, more time >> needs to execute >> the test. >> for example >> Location ( location == "home" or "office") >> and so on. >> So I worry about the matching performance with drools. >> >> I found that a lot of JoinNodes would be executed in >> runtime. I mean if >> there is 1000 rules, there will be a lot of JoinNodes >> (There are at least >> 1000 JoinNodes between Decision and Product ). And it >> exactly affects the >> execution performance. >> >> As you know, Decision, Product, Servcie and so on are plan >> Java classes. If >> I define all of attributes of above classes in one class >> named WholeFact, >> only one Java Type, there is no mentioned issue. >> >> With WholeFact class, the rule will be changed as follows: >> rule 1 >>     when >>           WholeFact( >> subscriberGender == "male" or "female", >>                 >>               serviceName >> == "ftp" or "http", >>                 >>             >>    productId == 1 or 2 or 3 ... >>           ) >>     then >> end >> >> >> Greg Barton wrote: >> > >> > >> > Now this finally rises to something that needs rules. >> :)  In all of the >> > previous examples you've given you could just have a >> > Map<ProductKey,Handler> where the Handler looks >> like this: >> > >> > interface Handler { >> >   void handle(Product product, Decision >> decision); >> > } >> > >> > ...and the ProductKey consists of properties that >> uniquely identify how >> > the Product is handled.  So, on it's own, that >> functionality did not >> > require rules. >> > >> > However, now that you've introduced more complex >> decisions, with varying >> > data, to affect the Decision for each Property type, >> rules are more >> > appropriate. >> > >> > Is there any reason why you only have one of each >> object type in memory at >> > one time?  Maybe if you state more of the problem >> requirements we can help >> > you better. >> > >> > --- On Mon, 7/20/09, nesta <nesta.fdb(a)163.com&gt; >> wrote: >> > >> >> From: nesta <nesta.fdb(a)163.com&gt; >> >> Subject: Re: [rules-users] optimization on a lot >> of simple rules >> >> To: rules-users(a)lists.jboss.org >> >> Date: Monday, July 20, 2009, 4:14 AM >> >> >> >> Thanks very much. >> >> But if for every rule, there is one algorithm or >> discount >> >> which means that >> >> result has nothing related with Product's id and >> usage. I >> >> can't merge all >> >> rules in one rule. At the same time, besides >> Product and >> >> Decision fact type, >> >> there are more fact types. >> >> For example: >> >> rule 1 >> >>     when >> >>          Decision() >> >>          Subscriber(gender == >> >> "male" or "female") >> >>          Service(name == >> >> "ftp" or "http") >> >>          Product(id == 1) >> >>          ...... >> >>    then >> >>          ...... >> >> end >> >> rule 2 >> >>     when >> >>          Decision() >> >>          Subscriber(gender == >> >> "male" or "female") >> >>          Service(name == >> >> "ftp" or "http") >> >>          Product(id == 2) >> >>          ...... >> >>    then >> >>          ...... >> >> end >> >> >> >> ..... >> >> ..... >> >> >> >> In this scenario, if there are 1000 rules,  there >> will >> >> be a lot of JoinNode. >> >> But in runtime, there is only one Decision >> instance, one >> >> Subscriber instance >> >> and Service instance. >> >> >> >> If I define all data in one fact type, I think >> that there >> >> are not a lot of >> >> JoinNodes. >> >> >> >> Is there any other method? >> >> >> >> >> >> >> >> Wolfgang Laun-2 wrote: >> >> > >> >> > Well, what is the realtion between id, usage >> and the >> >> result that's to be >> >> > stored in a Decision or a global? >> >> > >> >> > Typically, such rules could be written as >> >> > >> >> > rule x >> >> > no-loop true >> >> > when >> >> >     $d : Decision() >> >> >     $p :Product( id == 1, $usage : >> >> usage ) >> >> > then >> >> >     compute/store value, depending >> >> on the formula for id == 1 (using >> >> > usage) >> >> > end >> >> > // similar rule for id == 2,3,... >> >> > >> >> > If value is a straightforward function of id >> (and >> >> usage), then implement a >> >> > function compValue and use a single rule, >> e.g.: >> >> > >> >> > rule x >> >> > no-loop true >> >> > when >> >> >     $d : Decision() >> >> >     Product( $id : id, $usage : >> >> usage) >> >> > then >> >> >    modify $d value to compValue( $id, >> $usage >> >> ) >> >> > >> >> > Distinguishing all individual combinations of >> id and >> >> usage on the LHS >> >> > seems >> >> > excessive. >> >> > >> >> > The ordering of CEs also affects execution >> times. >> >> > >> >> > -W >> >> > >> >> > On 7/20/09, nesta <nesta.fdb(a)163.com&gt; >> >> wrote: >> >> >> >> >> >> >> >> >> In this scenario, there are 1000 >> products, >> >> different product has >> >> >> different >> >> >> price, besides this, the price is >> affected by >> >> usage. I want to use >> >> >> Product.id to match the rules. >> >> >> >> >> >> As you mentioned "The crude duplication >> of rules >> >> where only the constant >> >> >> to >> >> >> be matched with >> >> >> Product.id varies can, most likely, be >> avoided." >> >> >> >> >> >> How to avoid it in this scenario? >> >> >> >> >> >> >> >> >> Wolfgang Laun-2 wrote: >> >> >> > >> >> >> > It's difficult to suggest an >> optimized form >> >> for your rules 1-infinity, >> >> >> > since >> >> >> > we do not know what you want to >> achieve. >> >> >> > >> >> >> > The crude duplication of rules where >> only the >> >> constant to be matched >> >> >> with >> >> >> > Product.id varies can, most likely, >> be >> >> avoided. >> >> >> > >> >> >> > -W >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Sun, Jul 19, 2009 at 3:15 PM, >> nesta <nesta.fdb(a)163.com&gt; >> >> wrote: >> >> >> > >> >> >> >> >> >> >> >> Hi, >> >> >> >> >> >> >> >> I am a newbie in drools. There >> are a lot >> >> of simple rules in a >> >> >> scenario. >> >> >> >> For example >> >> >> >> rule 1 >> >> >> >>    when >> >> >> >>        Product( id >> >> ==1, usage == 1) >> >> >> >>        $decision : >> >> Decision() >> >> >> >>    then >> >> >> >>      >> >>    $decision.setValue(1); >> >> >> >> end >> >> >> >> >> >> >> >> rule 2 >> >> >> >> when Product( id ==2, usage == >> 1) >> >> >> >>  $decision : Decision() >> >> >> >> rule 3 >> >> >> >> when Product( id ==3, usage == >> 1) >> >> >> >>  $decision : Decision() >> >> >> >> rule 4 >> >> >> >> when Product( id ==4, usage == >> 1) >> >> >> >>  $decision : Decision() >> >> >> >> rule 5 >> >> >> >> when Product( id ==5, usage == >> 1) >> >> >> >>  $decision : Decision() >> >> >> >> ...... >> >> >> >> >> >> >> >> I have a Product fact whose id = >> 5 and >> >> usage = 1, in my first >> >> >> thinking, >> >> >> >> only >> >> >> >> rule 5 is matched, there should >> be not >> >> much more different between 1 >> >> >> rule >> >> >> >> and a lot of rules in runtime. >> >> >> >> >> >> >> >> But the result shows that they >> are >> >> different. More rules will cost >> >> >> more >> >> >> >> time. If there are 1 thousand >> rules, some >> >> Node and Sink will execute 1 >> >> >> >> thousand times. >> >> >> >> >> >> >> >> My question is how to optimize >> this >> >> scenario? >> >> >> >> -- >> >> >> >> View this message in context: >> >> >> >> >> >> >> >> >> >> http://www.nabble.com/optimization-on-a-lot-of-simple-rules-tp24556724p24... >> >> >> >> Sent from the drools - user >> mailing list >> >> archive at Nabble.com. >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> >> rules-users mailing list >> >> >> >> rules-users(a)lists.jboss.org >> >> >> >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> >> >> >> >> >> > >> >> >> > >> >> _______________________________________________ >> >> >> > rules-users mailing list >> >> >> > rules-users(a)lists.jboss.org >> >> >> > https://lists.jboss.org/mailman/listinfo/rules-users >> >> >> > >> >> >> > >> >> >> >> >> >> -- >> >> >> View this message in context: >> >> >> >> >> >> http://www.nabble.com/optimization-on-a-lot-of-simple-rules-tp24556724p24... >> >> >> Sent from the drools - user mailing list >> archive >> >> at Nabble.com. >> >> >> >> >> >> >> _______________________________________________ >> >> >> rules-users mailing list >> >> >> rules-users(a)lists.jboss.org >> >> >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> >> >> >> > >> >> > >> _______________________________________________ >> >> > rules-users mailing list >> >> > rules-users(a)lists.jboss.org >> >> > https://lists.jboss.org/mailman/listinfo/rules-users >> >> > >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/optimization-on-a-lot-of-simple-rules-tp24556724p24... >> >> Sent from the drools - user mailing list archive >> at >> >> Nabble.com. >> >> >> >> _______________________________________________ >> >> rules-users mailing list >> >> rules-users(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> >> > >> > >> >        >> > >> > _______________________________________________ >> > rules-users mailing list >> > rules-users(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/rules-users >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/optimization-on-a-lot-of-simple-rules-tp24556724p24... >> Sent from the drools - user mailing list archive at >> Nabble.com. >> >> >> _______________________________________________ >> rules-users mailing list >> rules-users(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/rules-users >> > > >        > > _______________________________________________ > rules-users mailing list > rules-users(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/rules-users > > -- View this message in context: http://www.nabble.com/optimization-on-a-lot-of-simple-rules-tp24556724p24... Sent from the drools - user mailing list archive at Nabble.com. _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users