[rules-users] Needs advice to help Drools on a big problem (and avoid GC overhead)

[Raphael Jolivet]

Guys,

Splitting the problem into smaller ones and retracting facts totally solved
the memory problem and also speed up the process.
I would like to thank you for your responsiveness and expertise.

I am truly amazed by the activity of this community, and Drools is an
awesome piece of software !

Thanks,
Raphael



2014-03-20 21:57 GMT+01:00 Raphael Jolivet <raphael.jolivet at gmail.com>:

> Hmm, sure it makes sense. The operations and resources are independent. I
> think I can safely split the problem into smaller ones. I will try this
> approach.
>
> Thanks for the tip
> Le 20 mars 2014 20:51, "Wolfgang Laun" <wolfgang.laun at gmail.com> a écrit :
>
>  I may fail to understand the fundamentals of this problem, but it seems
>> to me that it doesn't have the dynamic requirements that would
>> warrant a full "blow up" of all Resources and Operations at the same time.
>>
>> We have the 300 rules, and I'm assuming that the pattern is the one
>> shown in OP's mail: a Resource subset times an Operation subset. So,
>> if you insert just one Resource (or Operation) fact together with
>> all Operation (or Resource) facts and fire all rules, you'll get all
>> Forbids
>> for the one Resource with all Operations (or vice versa). Then, you
>> retract the singleton and insert the next one, fire all rules, and so on.
>>
>> -W
>>
>>
>> On 20/03/2014, Raphael Jolivet <raphael.jolivet at gmail.com> wrote:
>> > Thanks for you quick answer.
>> >
>> > Not yet,
>> >
>> > For that I will need to switch to fact insertion instead of Java Set.
>> > Do you think to will make a difference in the way the RETE Tree will be
>> > built ?
>> >
>> >
>> >
>> >
>> > 2014-03-20 20:27 GMT+01:00 Mark Proctor <mproctor at codehaus.org>:
>> >
>> >> Did you try using the engine in equality mode? It will only ever allow
>> >> one
>> >> "equal" object to be inserted, others are just ignored.
>> >>
>> >> Mark
>> >> On 20 Mar 2014, at 18:54, Raphael <raphael.jolivet at gmail.com> wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > In our project, we use Drools to compute forbidden allocations as
>> input
>> >> for
>> >> > an allocation problem  (for which we do not use Drools BTW).
>> >> >
>> >> > The constraint of the problem (forbidden operation for given
>> resources)
>> >> are
>> >> > expressed as drools rules.
>> >> >
>> >> > We have about
>> >> > -200 resources
>> >> > -700 operations
>> >> >
>> >> > Those are the Drools facts.
>> >> > We have about 200 Drools rules describing the forbidden allocations
>> >> >
>> >> > The problem here is that many rules produce duplicates forbids.
>> >> > For instance :
>> >> > Rule 1 may forbid Op1 on Res1 & Res3
>> >> > Rule 2 may forbid Op1 on Res2 & res3
>> >> >
>> >> > Currently the RHS of rules just insert forbids into a pure Java Set
>> (no
>> >> fact
>> >> > insertion).
>> >> > The Set takes care of avoiding duplicates, but still, we feel we
>> could
>> >> help
>> >> > Drools by giving him a hint that
>> >> > all rules are actually producing pairs of <Resource, Allocation> and
>> >> that he
>> >> > should not try to produce them more than once.
>> >> >
>> >> > Currently the structure of each rule looks like that :
>> >> >
>> >> > when
>> >> > $resource : Resource(<someconditions>)
>> >> > $operation : Operation(<someconditions>)
>> >> > then
>> >> >  globalSet.add(Pair<resrouce, operation>)
>> >> > end
>> >> >
>> >> > With the size of the problem, we often end up with outOfMem/ GC
>> limit.
>> >> > When analyzing memory dump, we see about
>> >> > 300.000 drools left tuple object.
>> >> >
>> >> > I am not sure if we could help drools to reduce the Rete tree here.
>> >> > I mean, the allocation matrix is already : 200*700 = 140.000 cells.
>> >> > Drools only uses 2 nodes for each possible forbids, which seems
>> already
>> >> well
>> >> > optimized.
>> >> >
>> >> > We had several idea to improve that though, but we are not enough
>> >> > expert
>> >> in
>> >> > the internal of the RETE algorithm to decide if that's a good way to
>> go
>> >> > :
>> >> >
>> >> > 1) Insert forbids as facts in the RHS and check they are not already
>> >> there
>> >> > in the LHS :
>> >> > when
>> >> > $resource : Resource(<someconditions>)
>> >> > $operation : Operation(<someconditions>)
>> >> > not Forbid(res=$resource, op=$operation)
>> >> > then
>> >> >  insert(Forbid($resource, $operation))
>> >> > end
>> >> >
>> >> > 2) Merge all rules in one big rule with a single RHS
>> >> > Is it possible to have something like :
>> >> > when
>> >> > // Rule 1
>> >> > ($resource : Resource(<someconditions>)
>> >> > $operation : Operation(<someconditions>))
>> >> >  or
>> >> > // Rule 2
>> >> > ($resource : Resource(<someconditions>)
>> >> > $operation : Operation(<someconditions>))
>> >> > // Etc ...
>> >> > then
>> >> >  globalSet.add(Pair<resrouce, operation>)
>> >> > end
>> >> >
>> >> > 3) Define a common RHS somewhere and tell Drools all rules use this
>> >> > one.
>> >> > Is there a possibility in Drools to define reusable RHS ? Would it
>> help
>> >> > ?
>> >> >
>> >> > 4) Any other idea ?
>> >> >
>> >> >
>> >> > Thanks in advance for your help.
>> >> > Brgds,
>> >> >
>> >> > Raphael Jolivet
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > View this message in context:
>> >>
>> http://drools.46999.n3.nabble.com/Needs-advice-to-help-Drools-on-a-big-problem-and-avoid-GC-overhead-tp4028838.html
>> >> > Sent from the Drools: User forum mailing list archive at Nabble.com.
>> >> > _______________________________________________
>> >> > rules-users mailing list
>> >> > rules-users at lists.jboss.org
>> >> > https://lists.jboss.org/mailman/listinfo/rules-users
>> >>
>> >>
>> >> _______________________________________________
>> >> rules-users mailing list
>> >> rules-users at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/rules-users
>> >>
>> >
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20140321/8ec6cbb6/attachment.html