[rules-users] Large number of rules in Drools
Michael Anstis
michael.anstis at gmail.com
Tue Jan 17 04:12:27 EST 2012
This is the JIRA to improve scalability:
https://issues.jboss.org/browse/GUVNOR-1635
Otherwise I would still recommend splitting your monolithic DRL file into
smaller ones and upload each individually.
Increasing your web-server time-out is also an option.
2012/1/16 Shweta1986 <shweta.a.thakkar at gmail.com>
> Can someone please look into this. Would really appreciate some help with
> these questions.
>
> 2012/1/11 Shweta Turakhia <[hidden email]<http://user/SendEmail.jtp?type=node&node=3664475&i=0>
> >
>
>> Thanks for your replies. Please see my response inline
>> My intention to use guvnor was to let non technical person go and change
>> the rules. But due to the size of the file and drl format it is difficult
>> for them to first search the rule and then edit it, save it and build it.
>>
>> Also since now my rules are in drl format there is no easy way to convert
>> them to excel format. Since that will make the file smaller with just 20K
>> rows which can be easily uploaded.
>> Please suggest.
>>
>>
>> "Thousands" of lines of DRL in a single file represent an issue not just
>> to Guvnor but to anything handling it.
>> >> Agreed. But if you provide a way to upload drl file as it is it
>> would not be that bad right?
>>
>> The recommendation would be to split the DRL into individual rules,
>> resulting of thousands of assets in Guvnor.
>> >> This takes more than 20 minutes and times out.
>>
>>
>> The next question would be if you have "thousands" of similar rules why
>> not use a singe Template or Decision Table?
>> Having said that, using a Template or Decision table in Guvnor with
>> "thousands" of rows could exhibit a problem at present.
>> Both of the editors in Guvnor do not scale well past a few thousand. This
>> is a known issue and we have an open item to address given time.
>> >> Can you please point me to the jira item for this ? I would like to
>> look into it.
>>
>> Using an Excel based decision table loaded into Guvnor would be workable,
>> however the compilation time to build a binary package may be an issue (I
>> don't know).
>> >> For me 20K rules got saved once but they never got built. I always get
>> "Network error" It times out since the browser is idle for more time.
>>
>>
>> 1) Guvnor needs more that 2.5GB Ram => Go to 64 bits java (we have 5GB
>> heap)
>> 2) Building the binary on guvnor on a virtual environment takes a few
>> minutes (4 minutes in our case)
>> >> For me build fails as mentioned earlier. It says "Network" error" I
>> never got this error when my rules were 100. With number of rules
>> increasing I think it takes more time to build and on the UI it times out.
>>
>> 3) For the runtime, the rulebase to build is less then 1 minute when
>> getting the rulebase from guvnor
>> 4) We are using decision tables updated from remote, so to view them the
>> only way is to download via rest the decision table xml et edit it.
>> >> Is there any documentation available as how to do this ?
>>
>> 5) Performance are excellent, for one session execution, it takes 250 ms
>> to apply 300 rules again in a virtual environement.
>> >> I agree for me it is less than 100 ms for each request. But since I am
>> using the Resource Change Scanner whenever my rules change the response
>> time increases to 40 seconds for 15 minutes and comes back to normal after
>> that. I also had to increase the heap size for my application to 3GB
>> otherwise it was having memory issues due to the number of rules in memory.
>>
>>
>> 2012/1/11 Nicolas Héron [via Drools] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3664475&i=1>
>> >
>>
>> Hello,
>>> I just worked on a drools project with many thousands rules (100k). The
>>> project is now in production.
>>> the issues are :
>>> 1) Guvnor needs more that 2.5GB Ram => Go to 64 bits java (we have 5GB
>>> heap)
>>> 2) Building the binary on guvnor on a virtual environment takes a few
>>> minutes (4 minutes in our case)
>>> 3) For the runtime, the rulebase to build is less then 1 minute when
>>> getting the rulebase from guvnor
>>> 4) We are using decision tables updated from remote, so to view them the
>>> only way is to download via rest the decision table xml et edit it.
>>> 5) Performance are excellent, for one session execution, it takes 250 ms
>>> to apply 300 rules again in a virtual environement.
>>> I would like to know which JIRA is related to the issues in guvnor when
>>> there are many lines ?
>>> Regards
>>> Nicolas Héron
>>>
>>> 2012/1/11 Michael Anstis <[hidden email]<http://user/SendEmail.jtp?type=node&node=3650237&i=0>
>>> >
>>>
>>> "Thousands" of lines of DRL in a single file represent an issue not just
>>>> to Guvnor but to anything handling it.
>>>>
>>>> The recommendation would be to split the DRL into individual rules,
>>>> resulting of thousands of assets in Guvnor.
>>>>
>>>> The next question would be if you have "thousands" of similar rules why
>>>> not use a singe Template or Decision Table?
>>>>
>>>> Having said that, using a Template or Decision table in Guvnor with
>>>> "thousands" of rows could exhibit a problem at present.
>>>>
>>>> Both of the editors in Guvnor do not scale well past a few thousand.
>>>> This is a known issue and we have an open item to address given time.
>>>>
>>>> Using an Excel based decision table loaded into Guvnor would be
>>>> workable, however the compilation time to build a binary package may be an
>>>> issue (I don't know).
>>>>
>>>> 2012/1/10 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3650237&i=1>
>>>> >
>>>>
>>>> I can look into the solution of writing my own Java application for
>>>>> this solution. But I would still like to understand, if the problem is just
>>>>> because of the 3 values being checked in the rules ? Do complex rules dont
>>>>> have more than 1 checks ?
>>>>> If yes then this case should be supported no matter how simple the
>>>>> rule is right ?
>>>>>
>>>>> Also for the UI issue I mentioned. Assume some other complex rules. Is
>>>>> copying the thousands of lines in the text area not a problem in guvnor ?
>>>>>
>>>>> On Tue, Jan 10, 2012 at 12:38 PM, laune [via Drools] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3649201&i=0>
>>>>> > wrote:
>>>>>
>>>>>> Assuming that the remaining 19998 rules are of the same kind, there
>>>>>> is a lot of data in these rules, and only very little logic. Ignoring the
>>>>>> issue of "Olympia" being one of the cities not being "Seattle", we have the
>>>>>> problem of matching a triplet of <field,state,city> from a Student fact so
>>>>>> that we can store one or more Subjects in Student.subjects.
>>>>>>
>>>>>> Imagine that you have objects of type Inscribe containing a triplet
>>>>>> of field, state, city and the pertaining List<Subject>. Then you could
>>>>>> write a single (!) rule
>>>>>>
>>>>>> rule register
>>>>>> when
>>>>>> $stu: Student ( $f: field, $s: state, $c: city )
>>>>>> $ins: Inscribe( field==$f, state==$s, city==$c )
>>>>>> then
>>>>>> $stu.setSubjects( $ins.getSubjects() );
>>>>>> end
>>>>>>
>>>>>> BTW: Why do you use equals with state? Using "==" should be possible
>>>>>> here, too.
>>>>>>
>>>>>> And, of course, matching ~20000 different triplets to facts of a
>>>>>> single type isn't really RBS fodder. You can design a simple Java program
>>>>>> that achieves this goal, assuming you can come up with a straightforward
>>>>>> representation of the <field,state,city;subject+> mapping.
>>>>>>
>>>>>> -W
>>>>>>
>>>>>>
>>>>>> 2012/1/10 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3648792&i=0>
>>>>>> >
>>>>>>
>>>>>> My rules are as follows.
>>>>>>>
>>>>>>> rule "rule1"
>>>>>>> salience -2
>>>>>>> dialect "mvel"
>>>>>>> when
>>>>>>> stud : Student ( field == "ARTS", state.equals("WA"))
>>>>>>> then
>>>>>>> java.util.List subList = new java.util.ArrayList();
>>>>>>> Subject subject = new Subject();
>>>>>>> subject.setName("ABC");
>>>>>>> subject.setFees(33454);
>>>>>>> subject.duration(4);
>>>>>>> subList.add(subject);
>>>>>>>
>>>>>>> subject = new Subject();
>>>>>>> subject.setName("XYZ");
>>>>>>> subject.setFees(33454);
>>>>>>> subject.duration(4);
>>>>>>> subList.add(subject);
>>>>>>>
>>>>>>> stud.setSubjects(subList);
>>>>>>> end
>>>>>>> rule "rule2"
>>>>>>> salience -100
>>>>>>> dialect "mvel"
>>>>>>> when
>>>>>>> stud : Student ( field == "ARTS", state.equals("WA") , city ==
>>>>>>> "SEATTLE"))
>>>>>>> then
>>>>>>> java.util.List subList = new java.util.ArrayList();
>>>>>>> Subject subject = new Subject();
>>>>>>> subject.setName("ABC");
>>>>>>> subject.setFees(33454);
>>>>>>> subject.duration(4);
>>>>>>> subList.add(subject);
>>>>>>>
>>>>>>> subject = new Subject();
>>>>>>> subject.setName("XYZ");
>>>>>>> subject.setFees(33454);
>>>>>>> subject.duration(4);
>>>>>>> subList.add(subject);
>>>>>>>
>>>>>>> stud.setSubjects(subList);
>>>>>>> end
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Student is the main object and it contains List of Subjects. here
>>>>>>> the way I am using the salience because
>>>>>>> if the input us shweta, ARTS, WA,Olympia it matched rule1
>>>>>>> but if the input is shweta, ARTS, WA, Seattle it matches 1 and 2 but
>>>>>>> i want the rule 2 to be executed so salience.
>>>>>>>
>>>>>>> I am using StatelessKnowledge Session and also the resource change
>>>>>>> scanner.
>>>>>>> I execute one rule at a time and the scanner poll interval is 1 hour.
>>>>>>> When I started with 100 rules I was using "PK" as my source in
>>>>>>> changeSet.xml but when the rules grew to 20K I was not able to build the
>>>>>>> rules in Guvnor UI and hence I started using "DRL" as source.
>>>>>>>
>>>>>>> The way I generate drl is using a script which converts the csv to
>>>>>>> drl format.
>>>>>>> I copy the drl to the text area in techincal rules and save it. (
>>>>>>> this copy is also paiful so I asked that if there is an upload drl file
>>>>>>> unlike the import drl file)
>>>>>>> In changeset I give the path to the url in webdav
>>>>>>> /guvnor/org.drools.guvnor.Guvnor/webdav/packages/StudentRules/StudentRules.drl
>>>>>>>
>>>>>>> I am not sure if I am doing it the right way. Please suggest.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Shweta
>>>>>>>
>>>>>>> On Mon, Jan 9, 2012 at 11:58 PM, laune [via Drools] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3648661&i=0>
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Salience is just one way to force the evaluation of a condition
>>>>>>>> before others. In fact, using salience throughout is just a way of
>>>>>>>> re-creating a sequence of plain old if statements using a rules engine.
>>>>>>>>
>>>>>>>> I'm not going to waste my time and yours by elaborating on possible
>>>>>>>> alternatives unless you come up with a detailed description of your rules,
>>>>>>>> and how you run your rules engine: fact insertion, calls to fire, etc.
>>>>>>>>
>>>>>>>> -W
>>>>>>>>
>>>>>>>> 2012/1/10 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3647015&i=0>
>>>>>>>> >
>>>>>>>>
>>>>>>>>> I am not sure I understand that. What is the other option for this
>>>>>>>>> case. Why would simple rules not work for 20 K rules. Am I missing
>>>>>>>>> something. I cannot use database to store my rules. Drools also gives me
>>>>>>>>> salience which I need for my usecase.
>>>>>>>>>
>>>>>>>>> On Thu, Jan 5, 2012 at 11:29 PM, laune [via Drools] <[hidden
>>>>>>>>> email] <http://user/SendEmail.jtp?type=node&node=3646449&i=0>>wrote:
>>>>>>>>>
>>>>>>>>>> 2012/1/6 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3637255&i=0>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>> And my rules are very simple it is like if string comparison
>>>>>>>>>>> then set that's it..
>>>>>>>>>>>
>>>>>>>>>> This doesn't really warrant the use of a rule based system. If
>>>>>>>>>> keeping this logic out of the application or an easy maintenance of these
>>>>>>>>>> if-then's are your main goals, there are other ways. I've already hinted at
>>>>>>>>>> this, with the idea of providing the information embedded in similar rules
>>>>>>>>>> as data.
>>>>>>>>>> -W
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> rules-users mailing list
>>>>>>>>>> [hidden email]<http://user/SendEmail.jtp?type=node&node=3637255&i=1>
>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>> If you reply to this email, your message will be added to the
>>>>>>>>>> discussion below:
>>>>>>>>>>
>>>>>>>>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3637255.html
>>>>>>>>>> To start a new topic under Drools: User forum, email [hidden
>>>>>>>>>> email] <http://user/SendEmail.jtp?type=node&node=3646449&i=1>
>>>>>>>>>> To unsubscribe from Drools: User forum, click here.
>>>>>>>>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>> View this message in context: Re: [rules-users] Large number of
>>>>>>>>> rules in Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3646449.html>
>>>>>>>>>
>>>>>>>>> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rules-users mailing list
>>>>>>>>> [hidden email]<http://user/SendEmail.jtp?type=node&node=3647015&i=1>
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rules-users mailing list
>>>>>>>> [hidden email]<http://user/SendEmail.jtp?type=node&node=3647015&i=2>
>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>> If you reply to this email, your message will be added to the
>>>>>>>> discussion below:
>>>>>>>>
>>>>>>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3647015.html
>>>>>>>> To start a new topic under Drools: User forum, email [hidden
>>>>>>>> email] <http://user/SendEmail.jtp?type=node&node=3648661&i=1>
>>>>>>>> To unsubscribe from Drools: User forum, click here.
>>>>>>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> View this message in context: Re: [rules-users] Large number of
>>>>>>> rules in Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3648661.html>
>>>>>>> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rules-users mailing list
>>>>>>> [hidden email]<http://user/SendEmail.jtp?type=node&node=3648792&i=1>
>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> rules-users mailing list
>>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3648792&i=2>
>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>>>> discussion below:
>>>>>>
>>>>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3648792.html
>>>>>> To start a new topic under Drools: User forum, email [hidden email]<http://user/SendEmail.jtp?type=node&node=3649201&i=1>
>>>>>> To unsubscribe from Drools: User forum, click here.
>>>>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: [rules-users] Large number of rules
>>>>> in Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3649201.html>
>>>>> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>>>>>
>>>>> _______________________________________________
>>>>> rules-users mailing list
>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3650237&i=2>
>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> rules-users mailing list
>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3650237&i=3>
>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>>>
>>>
>>> _______________________________________________
>>> rules-users mailing list
>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3650237&i=4>
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>> Nicolas Héron
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the
>>> discussion below:
>>>
>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3650237.html
>>> To start a new topic under Drools: User forum, email [hidden email]<http://user/SendEmail.jtp?type=node&node=3664475&i=2>
>>> To unsubscribe from Drools: User forum, click here.
>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>
> ------------------------------
> View this message in context: Re: [rules-users] Large number of rules in
> Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3664475.html>
> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20120117/3931879e/attachment.html
More information about the rules-users
mailing list