[rules-users] Large number of rules in Drools
Shweta1986
shweta.a.thakkar at gmail.com
Wed Jan 11 18:41:59 EST 2012
Thanks for your replies. Please see my response inline
My intention to use guvnor was to let non technical person go and change
the rules. But due to the size of the file and drl format it is difficult
for them to first search the rule and then edit it, save it and build it.
Also since now my rules are in drl format there is no easy way to convert
them to excel format. Since that will make the file smaller with just 20K
rows which can be easily uploaded.
Please suggest.
"Thousands" of lines of DRL in a single file represent an issue not just to
Guvnor but to anything handling it.
>> Agreed. But if you provide a way to upload drl file as it is it would
not be that bad right?
The recommendation would be to split the DRL into individual rules,
resulting of thousands of assets in Guvnor.
>> This takes more than 20 minutes and times out.
The next question would be if you have "thousands" of similar rules why not
use a singe Template or Decision Table?
Having said that, using a Template or Decision table in Guvnor with
"thousands" of rows could exhibit a problem at present.
Both of the editors in Guvnor do not scale well past a few thousand. This
is a known issue and we have an open item to address given time.
>> Can you please point me to the jira item for this ? I would like to look
into it.
Using an Excel based decision table loaded into Guvnor would be workable,
however the compilation time to build a binary package may be an issue (I
don't know).
>> For me 20K rules got saved once but they never got built. I always get
"Network error" It times out since the browser is idle for more time.
1) Guvnor needs more that 2.5GB Ram => Go to 64 bits java (we have 5GB heap)
2) Building the binary on guvnor on a virtual environment takes a few
minutes (4 minutes in our case)
>> For me build fails as mentioned earlier. It says "Network" error" I
never got this error when my rules were 100. With number of rules
increasing I think it takes more time to build and on the UI it times out.
3) For the runtime, the rulebase to build is less then 1 minute when
getting the rulebase from guvnor
4) We are using decision tables updated from remote, so to view them the
only way is to download via rest the decision table xml et edit it.
>> Is there any documentation available as how to do this ?
5) Performance are excellent, for one session execution, it takes 250 ms to
apply 300 rules again in a virtual environement.
>> I agree for me it is less than 100 ms for each request. But since I am
using the Resource Change Scanner whenever my rules change the response
time increases to 40 seconds for 15 minutes and comes back to normal after
that. I also had to increase the heap size for my application to 3GB
otherwise it was having memory issues due to the number of rules in memory.
2012/1/11 Nicolas Héron [via Drools] <
ml-node+s46999n3650237h85 at n3.nabble.com>
> Hello,
> I just worked on a drools project with many thousands rules (100k). The
> project is now in production.
> the issues are :
> 1) Guvnor needs more that 2.5GB Ram => Go to 64 bits java (we have 5GB
> heap)
> 2) Building the binary on guvnor on a virtual environment takes a few
> minutes (4 minutes in our case)
> 3) For the runtime, the rulebase to build is less then 1 minute when
> getting the rulebase from guvnor
> 4) We are using decision tables updated from remote, so to view them the
> only way is to download via rest the decision table xml et edit it.
> 5) Performance are excellent, for one session execution, it takes 250 ms
> to apply 300 rules again in a virtual environement.
> I would like to know which JIRA is related to the issues in guvnor when
> there are many lines ?
> Regards
> Nicolas Héron
>
> 2012/1/11 Michael Anstis <[hidden email]<http://user/SendEmail.jtp?type=node&node=3650237&i=0>
> >
>
> "Thousands" of lines of DRL in a single file represent an issue not just
>> to Guvnor but to anything handling it.
>>
>> The recommendation would be to split the DRL into individual rules,
>> resulting of thousands of assets in Guvnor.
>>
>> The next question would be if you have "thousands" of similar rules why
>> not use a singe Template or Decision Table?
>>
>> Having said that, using a Template or Decision table in Guvnor with
>> "thousands" of rows could exhibit a problem at present.
>>
>> Both of the editors in Guvnor do not scale well past a few thousand. This
>> is a known issue and we have an open item to address given time.
>>
>> Using an Excel based decision table loaded into Guvnor would be workable,
>> however the compilation time to build a binary package may be an issue (I
>> don't know).
>>
>> 2012/1/10 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3650237&i=1>
>> >
>>
>>> I can look into the solution of writing my own Java application for
>>> this solution. But I would still like to understand, if the problem is just
>>> because of the 3 values being checked in the rules ? Do complex rules dont
>>> have more than 1 checks ?
>>> If yes then this case should be supported no matter how simple the rule
>>> is right ?
>>>
>>> Also for the UI issue I mentioned. Assume some other complex rules. Is
>>> copying the thousands of lines in the text area not a problem in guvnor ?
>>>
>>> On Tue, Jan 10, 2012 at 12:38 PM, laune [via Drools] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3649201&i=0>
>>> > wrote:
>>>
>>>> Assuming that the remaining 19998 rules are of the same kind, there is
>>>> a lot of data in these rules, and only very little logic. Ignoring the
>>>> issue of "Olympia" being one of the cities not being "Seattle", we have the
>>>> problem of matching a triplet of <field,state,city> from a Student fact so
>>>> that we can store one or more Subjects in Student.subjects.
>>>>
>>>> Imagine that you have objects of type Inscribe containing a triplet of
>>>> field, state, city and the pertaining List<Subject>. Then you could write a
>>>> single (!) rule
>>>>
>>>> rule register
>>>> when
>>>> $stu: Student ( $f: field, $s: state, $c: city )
>>>> $ins: Inscribe( field==$f, state==$s, city==$c )
>>>> then
>>>> $stu.setSubjects( $ins.getSubjects() );
>>>> end
>>>>
>>>> BTW: Why do you use equals with state? Using "==" should be possible
>>>> here, too.
>>>>
>>>> And, of course, matching ~20000 different triplets to facts of a single
>>>> type isn't really RBS fodder. You can design a simple Java program that
>>>> achieves this goal, assuming you can come up with a straightforward
>>>> representation of the <field,state,city;subject+> mapping.
>>>>
>>>> -W
>>>>
>>>>
>>>> 2012/1/10 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3648792&i=0>
>>>> >
>>>>
>>>> My rules are as follows.
>>>>>
>>>>> rule "rule1"
>>>>> salience -2
>>>>> dialect "mvel"
>>>>> when
>>>>> stud : Student ( field == "ARTS", state.equals("WA"))
>>>>> then
>>>>> java.util.List subList = new java.util.ArrayList();
>>>>> Subject subject = new Subject();
>>>>> subject.setName("ABC");
>>>>> subject.setFees(33454);
>>>>> subject.duration(4);
>>>>> subList.add(subject);
>>>>>
>>>>> subject = new Subject();
>>>>> subject.setName("XYZ");
>>>>> subject.setFees(33454);
>>>>> subject.duration(4);
>>>>> subList.add(subject);
>>>>>
>>>>> stud.setSubjects(subList);
>>>>> end
>>>>> rule "rule2"
>>>>> salience -100
>>>>> dialect "mvel"
>>>>> when
>>>>> stud : Student ( field == "ARTS", state.equals("WA") , city ==
>>>>> "SEATTLE"))
>>>>> then
>>>>> java.util.List subList = new java.util.ArrayList();
>>>>> Subject subject = new Subject();
>>>>> subject.setName("ABC");
>>>>> subject.setFees(33454);
>>>>> subject.duration(4);
>>>>> subList.add(subject);
>>>>>
>>>>> subject = new Subject();
>>>>> subject.setName("XYZ");
>>>>> subject.setFees(33454);
>>>>> subject.duration(4);
>>>>> subList.add(subject);
>>>>>
>>>>> stud.setSubjects(subList);
>>>>> end
>>>>>
>>>>>
>>>>>
>>>>> Student is the main object and it contains List of Subjects. here the
>>>>> way I am using the salience because
>>>>> if the input us shweta, ARTS, WA,Olympia it matched rule1
>>>>> but if the input is shweta, ARTS, WA, Seattle it matches 1 and 2 but i
>>>>> want the rule 2 to be executed so salience.
>>>>>
>>>>> I am using StatelessKnowledge Session and also the resource change
>>>>> scanner.
>>>>> I execute one rule at a time and the scanner poll interval is 1 hour.
>>>>> When I started with 100 rules I was using "PK" as my source in
>>>>> changeSet.xml but when the rules grew to 20K I was not able to build the
>>>>> rules in Guvnor UI and hence I started using "DRL" as source.
>>>>>
>>>>> The way I generate drl is using a script which converts the csv to drl
>>>>> format.
>>>>> I copy the drl to the text area in techincal rules and save it. ( this
>>>>> copy is also paiful so I asked that if there is an upload drl file unlike
>>>>> the import drl file)
>>>>> In changeset I give the path to the url in webdav
>>>>> /guvnor/org.drools.guvnor.Guvnor/webdav/packages/StudentRules/StudentRules.drl
>>>>>
>>>>> I am not sure if I am doing it the right way. Please suggest.
>>>>>
>>>>> Thanks,
>>>>> Shweta
>>>>>
>>>>> On Mon, Jan 9, 2012 at 11:58 PM, laune [via Drools] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3648661&i=0>
>>>>> > wrote:
>>>>>
>>>>>> Salience is just one way to force the evaluation of a condition
>>>>>> before others. In fact, using salience throughout is just a way of
>>>>>> re-creating a sequence of plain old if statements using a rules engine.
>>>>>>
>>>>>> I'm not going to waste my time and yours by elaborating on possible
>>>>>> alternatives unless you come up with a detailed description of your rules,
>>>>>> and how you run your rules engine: fact insertion, calls to fire, etc.
>>>>>>
>>>>>> -W
>>>>>>
>>>>>> 2012/1/10 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3647015&i=0>
>>>>>> >
>>>>>>
>>>>>>> I am not sure I understand that. What is the other option for this
>>>>>>> case. Why would simple rules not work for 20 K rules. Am I missing
>>>>>>> something. I cannot use database to store my rules. Drools also gives me
>>>>>>> salience which I need for my usecase.
>>>>>>>
>>>>>>> On Thu, Jan 5, 2012 at 11:29 PM, laune [via Drools] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3646449&i=0>
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> 2012/1/6 Shweta1986 <[hidden email]<http://user/SendEmail.jtp?type=node&node=3637255&i=0>
>>>>>>>> >
>>>>>>>>
>>>>>>>>> And my rules are very simple it is like if string comparison then
>>>>>>>>> set that's it..
>>>>>>>>>
>>>>>>>> This doesn't really warrant the use of a rule based system. If
>>>>>>>> keeping this logic out of the application or an easy maintenance of these
>>>>>>>> if-then's are your main goals, there are other ways. I've already hinted at
>>>>>>>> this, with the idea of providing the information embedded in similar rules
>>>>>>>> as data.
>>>>>>>> -W
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rules-users mailing list
>>>>>>>> [hidden email]<http://user/SendEmail.jtp?type=node&node=3637255&i=1>
>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>> If you reply to this email, your message will be added to the
>>>>>>>> discussion below:
>>>>>>>>
>>>>>>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3637255.html
>>>>>>>> To start a new topic under Drools: User forum, email [hidden
>>>>>>>> email] <http://user/SendEmail.jtp?type=node&node=3646449&i=1>
>>>>>>>> To unsubscribe from Drools: User forum, click here.
>>>>>>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> View this message in context: Re: [rules-users] Large number of
>>>>>>> rules in Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3646449.html>
>>>>>>>
>>>>>>> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rules-users mailing list
>>>>>>> [hidden email]<http://user/SendEmail.jtp?type=node&node=3647015&i=1>
>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> rules-users mailing list
>>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3647015&i=2>
>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>>>> discussion below:
>>>>>>
>>>>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3647015.html
>>>>>> To start a new topic under Drools: User forum, email [hidden email]<http://user/SendEmail.jtp?type=node&node=3648661&i=1>
>>>>>> To unsubscribe from Drools: User forum, click here.
>>>>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: [rules-users] Large number of rules
>>>>> in Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3648661.html>
>>>>> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>>>>>
>>>>> _______________________________________________
>>>>> rules-users mailing list
>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3648792&i=1>
>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> rules-users mailing list
>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3648792&i=2>
>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>>
>>>> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3648792.html
>>>> To start a new topic under Drools: User forum, email [hidden email]<http://user/SendEmail.jtp?type=node&node=3649201&i=1>
>>>> To unsubscribe from Drools: User forum, click here.
>>>> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: [rules-users] Large number of rules
>>> in Drools<http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3649201.html>
>>> Sent from the Drools: User forum mailing list archive<http://drools.46999.n3.nabble.com/Drools-User-forum-f47000.html>at Nabble.com.
>>>
>>> _______________________________________________
>>> rules-users mailing list
>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3650237&i=2>
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>
>>>
>>
>> _______________________________________________
>> rules-users mailing list
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=3650237&i=3>
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>
> _______________________________________________
> rules-users mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=3650237&i=4>
> https://lists.jboss.org/mailman/listinfo/rules-users
> Nicolas Héron
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3650237.html
> To start a new topic under Drools: User forum, email
> ml-node+s46999n47000h38 at n3.nabble.com
> To unsubscribe from Drools: User forum, click here<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=47000&code=c2h3ZXRhLmEudGhha2thckBnbWFpbC5jb218NDcwMDB8LTE5NzY4Nzg0NzI=>
> .
> NAML<http://drools.46999.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://drools.46999.n3.nabble.com/Large-number-of-rules-in-Drools-tp3630385p3652218.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20120111/6fa98cef/attachment.html
More information about the rules-users
mailing list