Re: [rules-users] Best model for planning? technicians, airplanes and shifts => insertLogical problems

Hey, Thank you James for your response. I manage to solve a 324 tasks for 71 candidate technicians in 11 periods in less than 10 minutes. I did not do much of optimization yet, as I need to look into that part. My knowledge about selectors and tabu-searches and heuristics is still far to limited. The main effort I made is enhancing the algorithm to give a weight to the planning-enitities (Tasks) and technicians. It did not seem to influence the solve-time that much, though it had major impact on the final score. I have no idea why solving sometimes stops before 3600 seconds if hard-score is less than 0. This is my termination config:   <termination>     <maximumSecondsSpend>3600</maximumSecondsSpend>     <scoreAttained>0hard/-999999soft</scoreAttained>   </termination> But as POC (proof of concept) I guess I achieved my goal. One unanswered question from my previous mail: While rules are running: why do I get planningentities with uninitialized planningvariables (null)? If "Task" is the planningentity and period is a planningvariable I need to put:     $task : Task(period != null, ... whatever) Else the rule is executed with period is null, which results in all sorts of trouble, especially when using this planning variable in 'not equals' comparison. Example of such rule which needs not-null checking: rule "tasksInSameJobMustBeInSamePeriod"     when         $taskA : Task(period!=null, $id : id, $jobId : jobId, $periodId : periodId )         $taskB : Task(period!=null, id > $id, jobId == $jobId, periodId != $periodId )     then         insertLogical(new IntConstraintOccurrence("tasksInSameJobMustBeInSamePeriod",  $taskA, $taskB)); end Without the (period!=null you get exceptions thrown complaining about constraints in excess. Any ideas on why this is needed and if it is possible to never get nulls as planningvariable in the rules? Regards, Michiel   ----------------- http://www.codessentials.com - Your essential software, for free! Follow us at http://twitter.com/#!/Codessentials ________________________________ From: James Owen - North Texas <jco(a)kbsc.com&gt; To: Michiel Vermandel <mvermand(a)yahoo.com>; Rules Users List <rules-users(a)lists.jboss.org&gt; Sent: Thursday, September 6, 2012 4:30 PM Subject: Re: [rules-users] Best model for planning? technicians, airplanes and shifts => insertLogical problems Greetings: I started at the beginning of this thread and followed along (skipping only a one or two emails) and I was surprised that along the way that the problem grew and grew.  There is an excellent discussion of this very situation in the seminal book on expert systems by Girratano and Riley (about chapter 5 or 6) on pattern matching wherein they discuss this very problem with matching multiple patterns of objects.  When pattern matching it does matter the order in which you match the objects and their attributes because of the combinatorial explosion.  You have only so much memory with which to work and, sooner or later, you will run out of memory.   You can't just throw all of the objects into a bucket willy-nilly and hope for the best results.  The returned error may or may not be the one that you thought that it would be.  500 planes or 1,000 planes matched against multiple tasks matched against 12 months matched against multiple weeks matched against multiple shifts matched against … Well, you get the idea.  As the list expands, the possibilities expand by the number of possibilities against which it can be matched.  The project has to be architected and broken up into proper "chunks" so that it can be digested properly.  You don't eat a steak nor an apple in one bite.  You take it one bite at a time and chew it slowly and carefully.  So it is with a large problem. Even virtual memory is limited and will die out, as I have seen it done with benchmarks and the WaltzDB benchmark when programmed poorly.  When one more item is added to the problem space you must remember that the memory required expands exponentially ever upwards so it is always best to keep the requirements small at all times and move from small sets to small sets, even when you have GigaBytes of RAM these days.  Even later when we have TeraBytes, good programming will still be required for good results. BTW, it seems that the God Father of benchmarks, Dr. Daniel Miranker (formerly of CMU and now at UT Austin) will be speaking at Intellifest 2012 this year.  Visit http://www.IntelliFest.org for more details as they develop.  They probably have a gathering of some of the best minds of AI this year if anyone would care to attend.  Also, if anyone is looking for employment, they are planning on having recruiters there as well. Shalom jco On Sep 3, 2012, at 7:31 AM, Michiel Vermandel wrote: Ok, estimate - at this time - how much time it will take me to build it correctly afterwards. is giving me such a hard to solve issue. Incremental score calculation isn't a simple concept nor easy to implement. complexity and take the heat there. incremental score calculation, simply the complexity to the user. > > >I had thought that - given the project is only a very few classes - it would be peanuts for you or any other expert to pinpoint what I'm doing wrong. I didn't have the time to read all the classes in detail, just glimpsed over them. > > >>Non the less... >>I tried to have a look again to a number of examples and I changed my rules, with a positive result! changes are valid. solutions away? when numbers grow and possible solutions shrink. >>So what I did is going from >>rule "tasksInSameJobMustBeInSamePeriod" >>    when >>        $taskA : MaintenanceTask($id : id, $jobId : jobId, $periodId : periodId ) IntConstraintOccurrence("tasksInSameJobMustBeInSamePeriod",  $taskA)); >>end >>to >>rule "tasksInSameJobMustBeInSamePeriod" >>    when >>        $taskA : MaintenanceTask($id : id, $jobId : jobId, $periodId : periodId ) $taskB: > >        MaintenanceTask(id > $id, Good, because if you count the combination task5-task7, you don't want to count the combination task7-task5 too. > >jobId == $jobId, periodId > $periodId )    //  <================ a > $a Bad, keep this on periodId != $periodId (or even period != $period) IntConstraintOccurrence("tasksInSameJobMustBeInSamePeriod",  $taskA)); Add $taskB too: >insertLogical(new IntConstraintOccurrence("tasksInSameJobMustBeInSamePeriod",  $taskA, $taskB)); > >end  >>Can you please tell me if this is THE solution or a dangerous move that works out now but will give issues when numbers grow? The id > $id change is good, the periodId != $periodId isn't. any exceptions in environmentMode DEBUG or TRACE? > >Keep looking at the examples: they work and they scale. >Keep providing feedback as to the pain points too of course. > >Hope that helps. > > >>Thanks a lot. >>Michiel >>  >>----------------- >>http://www.codessentials.com - Your essential software, for free! >>Follow us at http://twitter.com/#!/Codessentials >>________________________________ >> From: Geoffrey De Smet <ge0ffrey.spam(a)gmail.com&gt; >>To: Michiel Vermandel <mvermand(a)yahoo.com>; Rules Users List <rules-users(a)lists.jboss.org&gt; >>Sent: Monday, September 3, 2012 11:56 AM >>Subject: Re: Best model for planning? technicians, airplanes and shifts => insertLogical problems >> >>Op 03-09-12 11:30, Michiel Vermandel schreef: >>I did not really start on one example. I have scrolled through several to try to figure out how to do it, then started from scratch for the POJO's and Rules. >>>>The config file was copies from one of the examples because it contained too many things that were hard to get right in the beginning. >>>I know that is a risk but I needed to start somewhere. >>I am thinking how to standardize the getting started with planner experience. at the moment. example that's similar to the user's use case. similar and which is not. >>>>3) What does "The workingMemory has 2 ConstraintOccurrence(s) in excess:" really mean? >>>    - Are the constraints there more than once? No, it means that in a clean WorkingMemory, those 2 ConstraintOccurrences aren't there, automatically retracted by the rule engine, but for some reason, they are not. >>Read this section about incremental score calculation to understand why this complexity is needed: >>  http://docs.jboss.org/drools/release/5.4.0.Final/drools-planner-docs/html... >>If you just want to prove that a Planner POC works for now (especially if you're close to giving up), calculator for now: (but it should still be faster than anything you can invent yourself within reasonable time). toy problem and you can scale out to 100+ jobs, and follow the rest of this mail. >>    - has this something to do with the equals and hashcode (which I did implement (see below))? Likely. The equals/hashcode methods are used of all objects in the causes parameter. equals/hashcode() needs to be used for the ConstraintOccurrence's causes. >>Compensation action >>>>    About the compensation action: is it already available on 5.4.0 final? Should I try that? The plumbing is there in Drools Expert, but in Planner there are no decent examples, supporting code or even complex experiments yet. it done or you have more Planner experience :/ >>>>4) I have been looking to the equals and hashcode, though found many examples that implement solutionEquals and solutionHashcode instead. >>>    Currently I implemented them like this: >>>>    @Override >>>    public int hashCode() { >>>        return id.hashCode(); //(*) >>>    } >>>>    @Override >>>    public boolean equals(Object o) { >>>        if (this == o) { >>>            return true; >>>        } >>>        if (id == null || !(o instanceof MaintenanceTask)) { (MaintenanceTask) o; >>>            return id.equals(other.id); >>>        } >>>    } >>Looks good >>>>    (*) id is a String property which is passed into the entity object through the constructor and upon cloning it is passed from the clone source to the clone target: >>>        public MaintenanceTask clone() { >>>            System.out.println("Cloning task " + id); >>>            MaintenanceTask clone = new MaintenanceTask(job, id); >>>            clone.period = this.period; >>>            clone.technician = this.technician; >>>            return clone; >>>        } >Looks good. >>    >>>>>>    I am still confused about: >>>>    - Which ones do I need to implement (equals or solutionEquals, ...)? because of this code: objects (see drools expert manual on insertLogical) >>    - Should an entity and a cloned entity have the same result for both equals and hashcode? (I guess so) Yes, definitely. >>    - Should only the entity objects have such implementations? (Planning variables are never cloned, right?) >It looks good. Only the entity's are cloned indeed during cloneSolution(): they are the only instances that change during planning. >>   >>>>  >>>----------------- >>>http://www.codessentials.com - Your essential software, for free! >>>Follow us at http://twitter.com/#!/Codessentials >>>>>________________________________ >>> From: Geoffrey De Smet <ge0ffrey.spam(a)gmail.com&gt; >>>To: Rules Users List <rules-users(a)lists.jboss.org&gt; >>>Sent: Monday, September 3, 2012 10:44 AM >>>Subject: Re: [rules-users] Best model for planning? technicians, airplanes and shifts => insertLogical problems >>> >>>Op 03-09-12 10:21, Michiel Vermandel schreef: >>>> Hi Geoffrey, >>>>>> Thanks for the support so far. >>>> I understand that you do not provide full support on this level. really into it now for about 3 days be not obvious to find for a code. documentation ;-) ;-) >>>>Good point, the score corruption problem is often a beginner problem and >>>it's a PITA. I 'll write some more docs about. >>>>Do note that your 3 day implementation should be able to scale out to wrong example nqueens (if similar to this kind of in the docs: the nqueens too simple to copy from >>>for real world stuff :/ Feedback welcome. >>>>>>>>> Ok, >>>>>> 1) adding the $t2 results in the same sort of exception, only >>>> planningEntity seems different: >>>>>> with insertLogical(new >> UnweightedConstraintOccurrence("tasksInSameJobMustBeInSamePeriod", $t1, >>>> $t2)); >>>>>> Exception in thread "main" java.lang.IllegalStateException: Score the uncorruptedScore (0): ConstraintOccurrence(s) in excess: tasksInSameJobMustBeInSamePeriod/NEGATIVE_HARD:[Maintenance of of Boeing 737 - PJ23.I#2 >>>> 427578167] tasksInSameJobMustBeInSamePeriod/NEGATIVE_HARD:[Maintenance of Maintenance of Boeing 737 - PJ23.I#1 ConstraintOccurrences. Verify weight is correct. >>>>      at >>org.drools.planner.core.score.director.AbstractScoreDirector.assertWorkingScore(AbstractScoreDirector.java:101) >>>>      at >>org.drools.planner.core.constructionheuristic.greedyFit.decider.DefaultGreedyDecider.doMove(DefaultGreedyDecider.java:110) >>>>      at >>org.drools.planner.core.constructionheuristic.greedyFit.decider.DefaultGreedyDecider.decideNextStep(DefaultGreedyDecider.java:78) >>>>      at >>org.drools.planner.core.constructionheuristic.greedyFit.DefaultGreedyFitSolverPhase.solve(DefaultGreedyFitSolverPhase.java:63) >>>>      at >>org.drools.planner.core.solver.DefaultSolver.runSolverPhases(DefaultSolver.java:183) >>>>      at >>org.drools.planner.core.solver.DefaultSolver.solve(DefaultSolver.java:151) >>>>      at >>be.axi.planner.domain.MaintenancePlanning.main(MaintenancePlanning.java:27) >>>>>> with insertLogical(new >> UnweightedConstraintOccurrence("tasksInSameJobMustBeInSamePeriod", $t1)); >>>>>> Exception in thread "main" java.lang.IllegalStateException: Score the uncorruptedScore (0): ConstraintOccurrence(s) in excess: tasksInSameJobMustBeInSamePeriod/NEGATIVE_HARD:[Maintenance of >>>> Airbus A350 - XJ34.I#2 778813475] tasksInSameJobMustBeInSamePeriod/NEGATIVE_HARD:[Maintenance of ConstraintOccurrences. Verify weight is correct. >>>>      at >>org.drools.planner.core.score.director.AbstractScoreDirector.assertWorkingScore(AbstractScoreDirector.java:101) >>>>      at >>org.drools.planner.core.constructionheuristic.greedyFit.decider.DefaultGreedyDecider.doMove(DefaultGreedyDecider.java:110) >>>>      at >>org.drools.planner.core.constructionheuristic.greedyFit.decider.DefaultGreedyDecider.decideNextStep(DefaultGreedyDecider.java:78) >>>>      at >>org.drools.planner.core.constructionheuristic.greedyFit.DefaultGreedyFitSolverPhase.solve(DefaultGreedyFitSolverPhase.java:63) >>>>      at >>org.drools.planner.core.solver.DefaultSolver.runSolverPhases(DefaultSolver.java:183) >>>>      at >>org.drools.planner.core.solver.DefaultSolver.solve(DefaultSolver.java:151) >>>>      at >>be.axi.planner.domain.MaintenancePlanning.main(MaintenancePlanning.java:27) >>>>>>>> 2) You suggested to replace UnweightedConstraintOccurrence with the very basic Queens example >>>> though... >>>>Yep, my mistake. >>>>>>> 3) Where is the best place to read about what insertLogical and which should be passed? explanation about this? >>>> (http://docs.jboss.org/drools/release/5.4.0.Final/drools-planner-docs/html... >>>> doesn't really enlighten me on that part) >>>>Look for "insertLogical" in the Drools Expert guide: >>>>http://docs.jboss.org/drools/release/5.4.0.Final/drools-expert-docs/html_single/index.html >>>>When rules do an insertLogical of an object A, it's discarded if another (through equals() and ConstraintOccurrences need to be unique constraintType and causes are >>>used for equals()/hashcode(). >>>>Future work: "compensation action" >>>Recently, drools introduced something called "compensation action", insertLogical(ConstraintOccurrence) haven't got time yet to easy for users to use. >>>>It would allow us to do something like this in the then part of a rule: >>>  hardAndSoftScoreHelper.addHardScore(-5); >>>or >>>  hardAndSoftScoreHelper.addSoftScore(- $sum); >>>or >>>  simpleScoreHelper.addScore(-7); >>>>No need for causes, insertLogicals, no equals/hashcode() worries, much >>>more flexible, ... >>>>>4) Does your MaintenanceTask implement equals()/hashcode() other than >>>Object's original implementation? >>>>>>>> Thanks in advance. >>>>>>yw >>>>_______________________________________________ >>>rules-users mailing list &gt;&gt;&gt;rules-users(a)lists.jboss.org >>>https://lists.jboss.org/mailman/listinfo/rules-users >>>>>>>>_______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users >>_______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users