RE: [rules-users] Re: Drools and BRMS (60,000+ rules)

Hi Michael, Yes, our biggest challenge is the design of our BRMS (rules, packages, categories etc.). I don't think compilation times of several minutes is a problem since we will probably compile new or modified rules/packages/categories in a verification/staging environment for testing before moving them to production. Maybe we can queue builds/tests for resource management (since it is already running in a verification environment) so the BRMS doesn't run out of resources. So far, I think the performance evaluation looks pretty good with no areas needing particular attention. We will probably know more once we have some real use cases. Thanks, Len ________________________________ From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On Behalf Of Michael Neale Sent: Monday, November 26, 2007 8:24 PM To: Rules Users List Subject: Re: [rules-users] Re: Drools and BRMS (60,000+ rules) Hi Len. Yes the main challenges I see are to do with the sheer numbers of rules you see yourself happening, and how to manage that in any sort of BRMS. ie are there 100's of packages, or a few uber packages etc. How do you break it up so its manageable? etc... even simple things, like if there is a drop down with rule names - that doesnt' scale to well to that number of rules (in a single package). We have categories which allow you to break down views into rules etc. The other side of the coin is compile time taken and memory used. Good to see you have done some profiling of it (I saw you PDF) - I haven't had a chance to load up your project yet but I did do some of my own profiling. Since 4.0.3 the impact of deserialization and rule packages has been partly addressed. I am worried that one rule change would require a 5 minute rebuild of an entire package (is 5 minutes too long? I really don't know how much your rules will change etc). On the BRMS server, we need to have some background resource management where we control how many builds/tests are going on so that we dont' starve the BRMS as resources (so in that sense it becomes a bit like a CI server, where it detects changes and then builds int he background). Regarding your profiling so far - nothing seems that surprising - were there any areas that you think needed particular attention? Michael. On Nov 25, 2007 6:43 AM, Carlsen, Len < len.carlsen(a)ubc.ca&gt; wrote: We would probably use the existing server side code and extent the current BRMS client UI, to satisfy our needs. By designing our BRMS I mostly meant designing the structure of all our rule domains, rules/packages and facts (courses/learning units). Len ________________________________ From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On Behalf Of Mark Proctor Sent: Saturday, November 24, 2007 1:02 AM To: Rules Users List Subject: Re: [rules-users] Re: Drools and BRMS (60,000+ rules) Carlsen, Len wrote: This is very cool and is exactly what we will need for our new open source SIS. Once we have designed our BRMS then we will look more closely at Drools Solver (especially after the ITC competition). Are you writting your own BRMS from scratch? Are there any particular issues with the BRMS at the moment that would warrant this? Maybe you could work on the BRMS we have now to extend it to satisfy your needs. Currently, we are using Ad Astra for our course timetabling and exam scheduling and it would be good if we could replace that with Solver. Thanks for your interest, Len -----Original Message----- From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On Behalf Of Geoffrey De Smet Sent: Friday, November 23, 2007 5:09 AM To: rules-users(a)lists.jboss.org Subject: [rules-users] Re: Drools and BRMS (60,000+ rules) Mark Proctor wrote: Carlsen, Len wrote: Other issues are the performance of the Drools Solver and having thousands of rules compiled, loaded and running. During registration we could have thousands of rules executing concurrently. The rules engine and the BRMS would be services as part of an SOA infrastructure (and an OSGi service environment). Anyone got any experience with Drools in an OSGi environment with respect to class loading issues when compiling rules and facts from other bundles since rules and facts may be in different bundles? Bundle buddy class loading? No but it is something that interests us, please give us your OSGi feedback. Solver is not currently a supported product, its very much R&D alphaware, but I'm sure geoffrey would love your feedback, good to see people are evaluating it. yes, feedback on solver is very welcome :) The good news is, I am now working on an example called Examination, which is an implementation for the itc2007 examination competition: http://www.cs.qub.ac.uk/itc2007/examtrack/exam_track_index_files/exameva luation.htm It's based on real-world data. The "starter" dataset contains 7883 students, 607 exams, 54 periods and 7 rooms. There are 5 hard constraints and 7 soft constraints: http://www.cs.qub.ac.uk/itc2007/examtrack/exam_track_index_files/exameva luation.htm The bad news is, it's not performing yet, because I need to make a scalable selector implementation, which I am working on. There's an example called lesson schedule (which schedules courses), but that's just a toy example to learn drools-solver, at it just has 2 hard constraints. One piece of advice though: separate the solver part from the rest. Once the solver starts calculating, you 'll only want to have score-related rules in the rulebase. The solver can easily do a million fireAllRules per minute, so every performance loss is blown up. With kind regards, Geoffrey De Smet _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users -- Michael D Neale home: www.michaelneale.net blog: michaelneale.blogspot.com