HI Mark,<br><br><blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" class="gmail_quote"> IEventTrigger ( text.onInput == onInput ) - is the text here the bound variable text, or a field on EventTrigger? the logic isn't very clear. What you have written in java would look like<br>
eventTrigger.getText().getOnInput().equals( eventTrigger)<br><br> Is that what you where expecting? that's why we often use the $ prefix to differentiate fields from variables.<br></blockquote><br><div bgcolor="#ffffff" text="#000000">
EventTrigger doesn't have a field "text", so I think it uses the bound variable instead and actually written as:<br><br><div style="margin-left: 40px;">text.getOnInput().equals( eventTrigger );<br></div><br>
I use variables very sparingly in my rules, but I didn't realise that using '$' as a prefix was optional. So using '$onInput' rather than 'onInput' would make no difference, except for readability?<br>
<br>Jevon<br></div><br><div class="gmail_quote">2010/7/22 Mark Proctor <span dir="ltr"><<a href="mailto:mproctor@codehaus.org">mproctor@codehaus.org</a>></span><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000"><div class="im">
On 22/07/2010 03:28, Jevon Wright wrote:
<blockquote type="cite">Hi Mark and Wolfgang,<br>
<br>
Thank you for your replies! Comments below.<br>
<br>
A bit of background: I am using Drools to take a given EMF model
instance, and insert new EObjects into the instance, according to
the given rules. I try to perform inference top-down, so there is
more than one iteration of insertion - as objects are inserted,
the rules need to be re-evaluated. If I understand correctly, this
means that I can't use a stateless session or the sequential
option, because the working memory is changing with inserted
facts.<br>
<br>
The rules don't appear to insert directly, because I insert new
objects into a queue instead (queue.add(object, drools)) - once
rule evaluation is complete, I insert the contents of the queue
into the existing working memory and fire all the rules again. I
try to prevent the rules modifying the working memory directly.
This is also why all the rules are of the format (x, ..., y, not z
=> insert z).<br>
<br>
This approach has a number of benefits. It finds inconsistencies
in the rules and means rules have no order, because inserted facts
don't effect the working memory immediately. It also allows me to
detect infinite loops, without restricting the number of times a
rule can fire. This was described in our 2010 paper [1].<br>
<br>
I don't think my implementation of this approach is causing the
memory problem, but I could be wrong.<br>
<br>
<blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" class="gmail_quote">
detail : DetailWire ( (from == source && to == target)
|| (from == target && to == source) )<br>
The above is turned effectively into an MVEL statement, you
might get better performance with a ConditionalElement 'or' as
lont as the<br>
two are mutually exclusive:<br>
<br>
( DetailWire (from == source, to == target ) or <br>
DetailWire (from == target, to == source) )<br>
</blockquote>
<div><br>
I thought this was the case. However in this case, you can't
bind the variable "detail" (the Drools compiler won't accept the
syntax), is this correct? I think one solution is to split the
rule into two separate rules for each "or" part (thus a DSL) - I
don't want to have to expand these rules by hand.<br>
</div>
</blockquote></div>
( $d : DetailWire (from == source, to == target ) or <br>
$d : DetailWire (from == target, to == source) )<br>
<br>
is valid<div class="im"><br>
<blockquote type="cite">
<div>
<br>
<blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" class="gmail_quote">And then i'm not sure what it is you are
doing in the second two rules, but it looks wrong.<br>
text : InputTextField ( eContainer == form, eval
(functions.getAutocompleteInputName(attribute).equals(name)) )
<br>
onInput : EventTrigger ( text.onInput == onInput )<br>
currentInput : Property ( text.currentInput == currentInput )<br>
</blockquote>
<br>
The point of this rule is to select something like the following
(from an EMF instance):<br>
<br>
<child name="form"><br>
<child xsi:type="InputTextField" name="..."><br>
<onInput xsi:type="EventTrigger" ... /><br>
<currentInput xsi:type="Property" ... /><br>
<events xsi:type="EventTrigger" ... /><br>
<properties xsi:type="Property" ... /><br>
</child><br>
</child><br>
</div>
</blockquote></div>
IEventTrigger ( text.onInput == onInput ) - is the text here the
bound variable text, or a field on EventTrigger? the logic isn't
very clear. What you have written in java would look like<br>
eventTrigger.getText().getOnInput().equals( eventTrigger)<br>
<br>
Is that what you where expecting? that's why we often use the $
prefix to differentiate fields from variables.<div><div></div><div class="h5"><br>
<blockquote type="cite">
<div><br>
I can't use use 'eContainer', because 'text' can also contain
EventTriggers in 'text.events'. These bound variables are then
supposed to be used later within the rule, either to select
other variables, or as part of the created element.<br>
<br>
I am going to try and remove unused bound variables, though. I
think I will try and write a script to analyse the exported XML
for the rules to analyse automatically (I have 264 rules written
by hand).<br>
<br>
Thanks<br>
Jevon<br>
<br>
[1]: J. Wright and J. Dietrich, "Non-Montonic Model Completion
in Web Application Engineering," in Proceedings of the <a href="http://aswec2010.massey.ac.nz/" target="_blank">21st
Australian Software Engineering Conference (ASWEC 2010)</a>,
Auckland, New Zealand, 2010. <a href="http://openiaml.org/#completion" target="_blank">http://openiaml.org/#completion</a><br>
<br>
</div>
<div class="gmail_quote">2010/7/16 Mark Proctor <span dir="ltr"><<a href="mailto:mproctor@codehaus.org" target="_blank">mproctor@codehaus.org</a>></span><br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">
<pre>detail : DetailWire ( (from == source && to == target) || (from == target && to == source) )
The above is turned effectively into an MVEL statement, you might get better performance with a ConditionalElement 'or' as lont as the
two are mutually exclusive:
( DetailWire (from == source, to == target ) or
DetailWire (from == target, to == source) )
I saw you did this:
not ( form : InputForm ( eContainer == container, name == <a href="http://iterator.name" target="_blank">iterator.name</a> ) )
The 'form' is not accessible outside the 'not', and that rule does not need it.
Is this not a bug. You bind "text". And then i'm not sure what it is you are doing in the second two rules, but it looks wrong.
text : InputTextField ( eContainer == form, eval (functions.getAutocompleteInputName(attribute).equals(name)) )
onInput : EventTrigger ( text.onInput == onInput
currentInput : Property ( text.currentInput == currentInput )
It doesn't look like you are updating the session with facts, i.e. it's a stateless session. See if this helps
KnowledgeBaseConfiguration kconf = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
kconf.setOption( SequentialOption.YES );
KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase( kconf );
final StatelessKnowledgeSession ksession = kbase.newStatelessKnowledgeSession();
ksession.execute(....);
In the execute you can provie it with a batch of commands to execute, or just a list of objects, up to you. see stateless session for
more details.
The SequentialOption may help memory, a small mount, if you aren't doing any working memory modifications (insert/modify/update/retract).
Mark
</pre>
<div> <br>
On 16/07/2010 04:16, Jevon Wright wrote: </div>
<div>
<div>
<blockquote type="cite">Hi again,<br>
<br>
By removing all of the simple eval()s from my rules, I
have cut heap usage by at least an order of magnitude.
However this still isn't enough.<br>
<br>
Since I am trying to reduce the cross-product size (as
in SQL), I recall that most SQL implementations have a
"DESCRIBE SELECT" query which provides real-time
information about the complexity of a given SQL query
- i.e. the size of the tables, indexes used, and so
on. Is there any such tool available for Drools? Are
there any tools which can provide clues as to which
rules are using the most memory?<br>
<br>
Alternatively, I am wondering what kind of benefit I
could expect from using materialized views to create
summary tables; that is, deriving and inserting
additional facts. This would allow Drools to rewrite
queries that currently use eval(), but would increase
the size of working memory, so would this actually
save heap size?<br>
<br>
To what extent does Drools rewrite queries? Is there
any documentation describing the approaches used?<br>
<br>
Any other ideas on how to reduce heap memory usage?
I'd appreciate any ideas :)<br>
<br>
Thanks<br>
Jevon<br>
<br>
<br>
<div class="gmail_quote">On Mon, Jul 12, 2010 at 5:56
PM, Jevon Wright <span dir="ltr"><<a href="mailto:jevon@jevon.org" target="_blank">jevon@jevon.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"> Hi Wolfgang and
Mark,<br>
<br>
Thank you for your replies! You were correct: my
eval() functions<br>
could generally be rewritten into Drools directly.<br>
<br>
I had one function "connectsDetail" that was
constraining<br>
unidirectional edges, and could be rewritten from:<br>
<div> detail : DetailWire ( )<br>
eval ( functions.connectsDetail(detail, source,
target) )<br>
<br>
</div>
to:<br>
detail : DetailWire ( from == source, to ==
target )<br>
<br>
Another function, "connects", was constraining
bidirectional edges,<br>
and could be rewritten from:<br>
sync : SyncWire( )<br>
eval ( functions.connects(sync, source, target) )<br>
<br>
to:<br>
sync : SyncWire( (from == source && to ==
target) || (from == target<br>
&& to == source) )<br>
<br>
Finally, the "veto" function could be rewritten
from:<br>
detail : DetailWire ( )<br>
eval ( handler.veto(detail) )<br>
<br>
to:<br>
detail : DetailWire ( overridden == false )<br>
<br>
I took each of these three changes, and evaluated
them separately [1].<br>
I found that:<br>
<br>
1. Inlining 'connectsDetail' made a huge
difference - 10-30% faster<br>
execution and 50-60% less allocated heap.<br>
2. Inlining 'connects' made very little difference
- 10-30% faster<br>
execution, but 0-20% more allocated heap.<br>
3. Inlining 'veto' made no difference - no
significant change in<br>
execution speed or allocated heap.<br>
<br>
I think I understand why inlining 'connects' would
improve heap usage<br>
- because the rules essentially have more
conditionals?<br>
<br>
I also understand why 'veto' made no difference -
for most of my test<br>
models, "overridden" was never true, so adding
this conditional was<br>
not making the cross product set any smaller.<br>
<br>
Finally, I also tested simply joining all of the
rules together into<br>
one file. This happily made no difference at all
(although made it<br>
more difficult to edit).<br>
<br>
So I think I can safely conclude that eval()
should be used as little<br>
as possible - however, this means that the final
rules are made more<br>
complicated and less human-readable, so a DSL may
be best for my<br>
common rule patterns in the future.<br>
<br>
Thanks again!<br>
Jevon<br>
<br>
[1]: <a href="http://www.jevon.org/wiki/Improving_Drools_Memory_Performance" target="_blank">http://www.jevon.org/wiki/Improving_Drools_Memory_Performance</a><br>
<div>
<div><br>
On Sat, Jul 10, 2010 at 12:28 AM, Wolfgang
Laun <<a href="mailto:wolfgang.laun@gmail.com" target="_blank">wolfgang.laun@gmail.com</a>>
wrote:<br>
> On 9 July 2010 14:14, Mark Proctor <<a href="mailto:mproctor@codehaus.org" target="_blank">mproctor@codehaus.org</a>>
wrote:<br>
>> You have many objects there that are
not constrained;<br>
><br>
> I have an inkling that the functions.*()
are hiding just these contraints,<br>
> It's certainly the wrong way, starting
with oodles of node pairs, just to<br>
> pick out connected ones by fishing for
the connecting edge. And this<br>
> is worsened by trying to find two such
pairs which meet at some<br>
> DomainSource<br>
><br>
> Guesswork, hopefully educated ;-)<br>
><br>
> -W<br>
><br>
><br>
>> if there are<br>
>> multiple versions of those objects
you are going to get massive amounts<br>
>> of cross products. Think in terms of
SQL, each pattern you add is like<br>
>> an SQL join.<br>
>><br>
>> Mark<br>
>> On 09/07/2010 09:20, Jevon Wright
wrote:<br>
>>> Hi everyone,<br>
>>><br>
>>> I am working on what appears to
be a fairly complex rule base based on<br>
>>> EMF. The rules aren't operating
over a huge number of facts (less than<br>
>>> 10,000 EObjects) and there aren't
too many rules (less than 300), but<br>
>>> I am having a problem with
running out of Java heap space (set at ~400<br>
>>> MB).<br>
>>><br>
>>> Through investigation, I came to
the conclusion that this is due to<br>
>>> the design of the rules, rather
than the number of facts. The engine<br>
>>> uses less memory inserting many
facts that use simple rules, compared<br>
>>> with inserting few facts that use
many rules.<br>
>>><br>
>>> Can anybody suggest some tips for
reducing heap memory usage in<br>
>>> Drools? I don't have a time
constraint, only a heap/memory constraint.<br>
>>> A sample rule in my project looks
like this:<br>
>>><br>
>>> rule "Create QueryParameter
for target container of DetailWire"<br>
>>> when<br>
>>> container : Frame( )<br>
>>> schema : DomainSchema ( )<br>
>>> domainSource :
DomainSource ( )<br>
>>> instance : DomainIterator(
)<br>
>>> selectEdge : SelectEdge (
eval (<br>
>>>
functions.connectsSelect(selectEdge, instance,
domainSource )) )<br>
>>> schemaEdge : SchemaEdge (
eval (<br>
>>>
functions.connectsSchema(schemaEdge,
domainSource, schema )) )<br>
>>> source : VisibleThing (
eContainer == container )<br>
>>> target : Frame ( )<br>
>>> instanceSet : SetWire (
eval(functions.connectsSet(instanceSet,<br>
>>> instance, source )) )<br>
>>> detail : DetailWire ( )<br>
>>> eval (
functions.connectsDetail(detail, source,
target ))<br>
>>> pk : DomainAttribute (
eContainer == schema, primaryKey == true )<br>
>>> not ( queryPk :
QueryParameter ( eContainer == target, name ==
<a href="http://pk.name" target="_blank">pk.name</a>
) )<br>
>>> eval ( handler.veto(
detail ))<br>
>>><br>
>>> then<br>
>>> QueryParameter qp =
handler.generatedQueryParameter(detail,
target);<br>
>>> handler.setName(qp,
pk.getName());<br>
>>> queue.add(qp, drools); //
wraps insert(...)<br>
>>><br>
>>> end<br>
>>><br>
>>> I try to order the select
statements in an order that will reduce the<br>
>>> size of the cross-product (in
theory), but I also try and keep the<br>
>>> rules fairly human readable. I
try to avoid comparison operators like<br>
>>> < and>. Analysing a heap
dump shows that most of the memory is being<br>
>>> used in
StatefulSession.nodeMemories>
PrimitiveLongMap.<br>
>>><br>
>>> I am using a StatefulSession; if
I understand correctly, I can't use a<br>
>>> StatelessSession with sequential
mode since I am inserting facts as<br>
>>> part of the rules. If I also
understand correctly, I'd like the Rete<br>
>>> graph to be tall, rather than
wide.<br>
>>><br>
>>> Some ideas I have thought of
include the following:<br>
>>> 1. Creating a separate
intermediary meta-model to split up the sizes<br>
>>> of the rules. e.g. instead of (if
A and B and C then insert D), using<br>
>>> (if A and B then insert E; if E
and C then insert D).<br>
>>> 2. Moving eval() statements
directly into the Type(...) selectors.<br>
>>> 3. Removing eval() statements.
Would this allow for better indexing by<br>
>>> the Rete algorithm?<br>
>>> 4. Reducing the height, or the
width, of the class hierarchy of the<br>
>>> facts. e.g. Removing interfaces
or abstract classes to reduce the<br>
>>> possible matches. Would this make
a difference?<br>
>>> 5. Conversely, increasing the
height, or the width, of the class<br>
>>> hierarchy. e.g. Adding interfaces
or abstract classes to reduce field<br>
>>> accessors.<br>
>>> 6. Instead of using
EObject.eContainer, creating an explicit<br>
>>> containment property in all of my
EObjects.<br>
>>> 7. Creating a DSL that is
human-readable, but allows for the<br>
>>> automation of some of these
approaches.<br>
>>> 8. Moving all rules into one rule
file, or splitting up rules into<br>
>>> smaller files.<br>
>>><br>
>>> Is there kind of profiler for
Drools that will let me see the size (or<br>
>>> the memory usage) of particular
rules, or of the memory used after<br>
>>> inference? Ideally I'd use this
to profile any changes.<br>
>>><br>
>>> Thanks for any thoughts or tips!
:-)<br>
>>><br>
>>> Jevon<br>
>>>
_______________________________________________<br>
>>> rules-users mailing list<br>
>>> <a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
>>> <a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
>>><br>
>>><br>
>><br>
>><br>
>>
_______________________________________________<br>
>> rules-users mailing list<br>
>> <a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
>> <a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
>><br>
><br>
>
_______________________________________________<br>
> rules-users mailing list<br>
> <a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
><br>
</div>
</div>
</blockquote>
</div>
<br>
<pre><fieldset></fieldset>
_______________________________________________
rules-users mailing list
<a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br>
</blockquote>
</div>
<br>
<pre><fieldset></fieldset>
_______________________________________________
rules-users mailing list
<a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br>