<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
On 22/07/2010 04:05, Jevon Wright wrote:
<blockquote
cite="mid:AANLkTinZ2__ZG_1qrmC85MKyCoNySFIkp8HENgAUxkCn@mail.gmail.com"
type="cite">HI Mark,<br>
<br>
<blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px
solid rgb(204, 204, 204); padding-left: 1ex;"
class="gmail_quote"> IEventTrigger ( text.onInput == onInput
) - is the text here the bound variable text, or a field on
EventTrigger? the logic isn't very clear. What you have written
in java would look like<br>
eventTrigger.getText().getOnInput().equals( eventTrigger)<br>
<br>
Is that what you where expecting? that's why we often use
the $ prefix to differentiate fields from variables.<br>
</blockquote>
<br>
<div bgcolor="#ffffff" text="#000000">
EventTrigger doesn't have a field "text", so I think it uses the
bound variable instead and actually written as:<br>
<br>
<div style="margin-left: 40px;">text.getOnInput().equals(
eventTrigger );<br>
</div>
<br>
I use variables very sparingly in my rules, but I didn't realise
that using '$' as a prefix was optional. So using '$onInput'
rather than 'onInput' would make no difference, except for
readability?<br>
</div>
</blockquote>
I'm surprised that parses, it's almost definitely wrong then and
should throw up an error; please open a jira for that so it can be
fixed later. You should wrap it with an eval:<br>
onInput : IEventTrigger ( eval( text.onInput == onInput ) ) <br>
Or better still write it like this:<br>
IEventTrigger( this == text.onInput )<br>
<br>
Mark<br>
<blockquote
cite="mid:AANLkTinZ2__ZG_1qrmC85MKyCoNySFIkp8HENgAUxkCn@mail.gmail.com"
type="cite">
<div bgcolor="#ffffff" text="#000000">
<br>
Jevon<br>
</div>
<br>
<div class="gmail_quote">2010/7/22 Mark Proctor <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:mproctor@codehaus.org">mproctor@codehaus.org</a>></span><br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">
<div class="im"> On 22/07/2010 03:28, Jevon Wright wrote:
<blockquote type="cite">Hi Mark and Wolfgang,<br>
<br>
Thank you for your replies! Comments below.<br>
<br>
A bit of background: I am using Drools to take a given
EMF model instance, and insert new EObjects into the
instance, according to the given rules. I try to perform
inference top-down, so there is more than one iteration
of insertion - as objects are inserted, the rules need
to be re-evaluated. If I understand correctly, this
means that I can't use a stateless session or the
sequential option, because the working memory is
changing with inserted facts.<br>
<br>
The rules don't appear to insert directly, because I
insert new objects into a queue instead
(queue.add(object, drools)) - once rule evaluation is
complete, I insert the contents of the queue into the
existing working memory and fire all the rules again. I
try to prevent the rules modifying the working memory
directly. This is also why all the rules are of the
format (x, ..., y, not z => insert z).<br>
<br>
This approach has a number of benefits. It finds
inconsistencies in the rules and means rules have no
order, because inserted facts don't effect the working
memory immediately. It also allows me to detect infinite
loops, without restricting the number of times a rule
can fire. This was described in our 2010 paper [1].<br>
<br>
I don't think my implementation of this approach is
causing the memory problem, but I could be wrong.<br>
<br>
<blockquote style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;" class="gmail_quote"> detail :
DetailWire ( (from == source && to == target)
|| (from == target && to == source) )<br>
The above is turned effectively into an MVEL
statement, you might get better performance with a
ConditionalElement 'or' as lont as the<br>
two are mutually exclusive:<br>
<br>
( DetailWire (from == source, to == target ) or <br>
DetailWire (from == target, to == source) )<br>
</blockquote>
<div><br>
I thought this was the case. However in this case, you
can't bind the variable "detail" (the Drools compiler
won't accept the syntax), is this correct? I think one
solution is to split the rule into two separate rules
for each "or" part (thus a DSL) - I don't want to have
to expand these rules by hand.<br>
</div>
</blockquote>
</div>
( $d : DetailWire (from == source, to == target ) or <br>
$d : DetailWire (from == target, to == source) )<br>
<br>
is valid
<div class="im"><br>
<blockquote type="cite">
<div> <br>
<blockquote style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;" class="gmail_quote">And then i'm
not sure what it is you are doing in the second two
rules, but it looks wrong.<br>
text : InputTextField ( eContainer == form, eval
(functions.getAutocompleteInputName(attribute).equals(name))
) <br>
onInput : EventTrigger ( text.onInput == onInput )<br>
currentInput : Property ( text.currentInput ==
currentInput )<br>
</blockquote>
<br>
The point of this rule is to select something like the
following (from an EMF instance):<br>
<br>
<child name="form"><br>
<child xsi:type="InputTextField" name="..."><br>
<onInput xsi:type="EventTrigger" ... /><br>
<currentInput xsi:type="Property" ... /><br>
<events xsi:type="EventTrigger" ... /><br>
<properties xsi:type="Property" ... /><br>
</child><br>
</child><br>
</div>
</blockquote>
</div>
IEventTrigger ( text.onInput == onInput ) - is the text here
the bound variable text, or a field on EventTrigger? the
logic isn't very clear. What you have written in java would
look like<br>
eventTrigger.getText().getOnInput().equals( eventTrigger)<br>
<br>
Is that what you where expecting? that's why we often use
the $ prefix to differentiate fields from variables.
<div>
<div class="h5"><br>
<blockquote type="cite">
<div><br>
I can't use use 'eContainer', because 'text' can
also contain EventTriggers in 'text.events'. These
bound variables are then supposed to be used later
within the rule, either to select other variables,
or as part of the created element.<br>
<br>
I am going to try and remove unused bound variables,
though. I think I will try and write a script to
analyse the exported XML for the rules to analyse
automatically (I have 264 rules written by hand).<br>
<br>
Thanks<br>
Jevon<br>
<br>
[1]: J. Wright and J. Dietrich, "Non-Montonic Model
Completion in Web Application Engineering," in
Proceedings of the <a moz-do-not-send="true"
href="http://aswec2010.massey.ac.nz/"
target="_blank">21st Australian Software
Engineering Conference (ASWEC 2010)</a>, Auckland,
New Zealand, 2010. <a moz-do-not-send="true"
href="http://openiaml.org/#completion"
target="_blank">http://openiaml.org/#completion</a><br>
<br>
</div>
<div class="gmail_quote">2010/7/16 Mark Proctor <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:mproctor@codehaus.org"
target="_blank">mproctor@codehaus.org</a>></span><br>
<blockquote class="gmail_quote" style="margin: 0pt
0pt 0pt 0.8ex; border-left: 1px solid rgb(204,
204, 204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">
<pre>detail : DetailWire ( (from == source && to == target) || (from == target && to == source) )
The above is turned effectively into an MVEL statement, you might get better performance with a ConditionalElement 'or' as lont as the
two are mutually exclusive:
( DetailWire (from == source, to == target ) or
DetailWire (from == target, to == source) )
I saw you did this:
not ( form : InputForm ( eContainer == container, name == <a moz-do-not-send="true" href="http://iterator.name" target="_blank">iterator.name</a> ) )
The 'form' is not accessible outside the 'not', and that rule does not need it.
Is this not a bug. You bind "text". And then i'm not sure what it is you are doing in the second two rules, but it looks wrong.
text : InputTextField ( eContainer == form, eval (functions.getAutocompleteInputName(attribute).equals(name)) )
onInput : EventTrigger ( text.onInput == onInput
currentInput : Property ( text.currentInput == currentInput )
It doesn't look like you are updating the session with facts, i.e. it's a stateless session. See if this helps
KnowledgeBaseConfiguration kconf = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
kconf.setOption( SequentialOption.YES );
KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase( kconf );
final StatelessKnowledgeSession ksession = kbase.newStatelessKnowledgeSession();
ksession.execute(....);
In the execute you can provie it with a batch of commands to execute, or just a list of objects, up to you. see stateless session for
more details.
The SequentialOption may help memory, a small mount, if you aren't doing any working memory modifications (insert/modify/update/retract).
Mark
</pre>
<div> <br>
On 16/07/2010 04:16, Jevon Wright wrote: </div>
<div>
<div>
<blockquote type="cite">Hi again,<br>
<br>
By removing all of the simple eval()s from
my rules, I have cut heap usage by at
least an order of magnitude. However this
still isn't enough.<br>
<br>
Since I am trying to reduce the
cross-product size (as in SQL), I recall
that most SQL implementations have a
"DESCRIBE SELECT" query which provides
real-time information about the complexity
of a given SQL query - i.e. the size of
the tables, indexes used, and so on. Is
there any such tool available for Drools?
Are there any tools which can provide
clues as to which rules are using the most
memory?<br>
<br>
Alternatively, I am wondering what kind of
benefit I could expect from using
materialized views to create summary
tables; that is, deriving and inserting
additional facts. This would allow Drools
to rewrite queries that currently use
eval(), but would increase the size of
working memory, so would this actually
save heap size?<br>
<br>
To what extent does Drools rewrite
queries? Is there any documentation
describing the approaches used?<br>
<br>
Any other ideas on how to reduce heap
memory usage? I'd appreciate any ideas :)<br>
<br>
Thanks<br>
Jevon<br>
<br>
<br>
<div class="gmail_quote">On Mon, Jul 12,
2010 at 5:56 PM, Jevon Wright <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:jevon@jevon.org"
target="_blank">jevon@jevon.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;"> Hi Wolfgang
and Mark,<br>
<br>
Thank you for your replies! You were
correct: my eval() functions<br>
could generally be rewritten into
Drools directly.<br>
<br>
I had one function "connectsDetail"
that was constraining<br>
unidirectional edges, and could be
rewritten from:<br>
<div> detail : DetailWire ( )<br>
eval (
functions.connectsDetail(detail,
source, target) )<br>
<br>
</div>
to:<br>
detail : DetailWire ( from == source,
to == target )<br>
<br>
Another function, "connects", was
constraining bidirectional edges,<br>
and could be rewritten from:<br>
sync : SyncWire( )<br>
eval ( functions.connects(sync,
source, target) )<br>
<br>
to:<br>
sync : SyncWire( (from == source
&& to == target) || (from ==
target<br>
&& to == source) )<br>
<br>
Finally, the "veto" function could be
rewritten from:<br>
detail : DetailWire ( )<br>
eval ( handler.veto(detail) )<br>
<br>
to:<br>
detail : DetailWire ( overridden ==
false )<br>
<br>
I took each of these three changes,
and evaluated them separately [1].<br>
I found that:<br>
<br>
1. Inlining 'connectsDetail' made a
huge difference - 10-30% faster<br>
execution and 50-60% less allocated
heap.<br>
2. Inlining 'connects' made very
little difference - 10-30% faster<br>
execution, but 0-20% more allocated
heap.<br>
3. Inlining 'veto' made no difference
- no significant change in<br>
execution speed or allocated heap.<br>
<br>
I think I understand why inlining
'connects' would improve heap usage<br>
- because the rules essentially have
more conditionals?<br>
<br>
I also understand why 'veto' made no
difference - for most of my test<br>
models, "overridden" was never true,
so adding this conditional was<br>
not making the cross product set any
smaller.<br>
<br>
Finally, I also tested simply joining
all of the rules together into<br>
one file. This happily made no
difference at all (although made it<br>
more difficult to edit).<br>
<br>
So I think I can safely conclude that
eval() should be used as little<br>
as possible - however, this means that
the final rules are made more<br>
complicated and less human-readable,
so a DSL may be best for my<br>
common rule patterns in the future.<br>
<br>
Thanks again!<br>
Jevon<br>
<br>
[1]: <a moz-do-not-send="true"
href="http://www.jevon.org/wiki/Improving_Drools_Memory_Performance"
target="_blank">http://www.jevon.org/wiki/Improving_Drools_Memory_Performance</a><br>
<div>
<div><br>
On Sat, Jul 10, 2010 at 12:28 AM,
Wolfgang Laun <<a
moz-do-not-send="true"
href="mailto:wolfgang.laun@gmail.com"
target="_blank">wolfgang.laun@gmail.com</a>>
wrote:<br>
> On 9 July 2010 14:14, Mark
Proctor <<a
moz-do-not-send="true"
href="mailto:mproctor@codehaus.org"
target="_blank">mproctor@codehaus.org</a>>
wrote:<br>
>> You have many objects
there that are not constrained;<br>
><br>
> I have an inkling that the
functions.*() are hiding just
these contraints,<br>
> It's certainly the wrong way,
starting with oodles of node
pairs, just to<br>
> pick out connected ones by
fishing for the connecting edge.
And this<br>
> is worsened by trying to find
two such pairs which meet at some<br>
> DomainSource<br>
><br>
> Guesswork, hopefully educated
;-)<br>
><br>
> -W<br>
><br>
><br>
>> if there are<br>
>> multiple versions of
those objects you are going to get
massive amounts<br>
>> of cross products. Think
in terms of SQL, each pattern you
add is like<br>
>> an SQL join.<br>
>><br>
>> Mark<br>
>> On 09/07/2010 09:20,
Jevon Wright wrote:<br>
>>> Hi everyone,<br>
>>><br>
>>> I am working on what
appears to be a fairly complex
rule base based on<br>
>>> EMF. The rules aren't
operating over a huge number of
facts (less than<br>
>>> 10,000 EObjects) and
there aren't too many rules (less
than 300), but<br>
>>> I am having a problem
with running out of Java heap
space (set at ~400<br>
>>> MB).<br>
>>><br>
>>> Through
investigation, I came to the
conclusion that this is due to<br>
>>> the design of the
rules, rather than the number of
facts. The engine<br>
>>> uses less memory
inserting many facts that use
simple rules, compared<br>
>>> with inserting few
facts that use many rules.<br>
>>><br>
>>> Can anybody suggest
some tips for reducing heap memory
usage in<br>
>>> Drools? I don't have
a time constraint, only a
heap/memory constraint.<br>
>>> A sample rule in my
project looks like this:<br>
>>><br>
>>> rule "Create
QueryParameter for target
container of DetailWire"<br>
>>> when<br>
>>> container :
Frame( )<br>
>>> schema :
DomainSchema ( )<br>
>>> domainSource :
DomainSource ( )<br>
>>> instance :
DomainIterator( )<br>
>>> selectEdge :
SelectEdge ( eval (<br>
>>>
functions.connectsSelect(selectEdge,
instance, domainSource )) )<br>
>>> schemaEdge :
SchemaEdge ( eval (<br>
>>>
functions.connectsSchema(schemaEdge,
domainSource, schema )) )<br>
>>> source :
VisibleThing ( eContainer ==
container )<br>
>>> target : Frame
( )<br>
>>> instanceSet :
SetWire (
eval(functions.connectsSet(instanceSet,<br>
>>> instance, source )) )<br>
>>> detail :
DetailWire ( )<br>
>>> eval (
functions.connectsDetail(detail,
source, target ))<br>
>>> pk :
DomainAttribute ( eContainer ==
schema, primaryKey == true )<br>
>>> not ( queryPk
: QueryParameter ( eContainer ==
target, name == <a
moz-do-not-send="true"
href="http://pk.name"
target="_blank">pk.name</a> ) )<br>
>>> eval (
handler.veto( detail ))<br>
>>><br>
>>> then<br>
>>> QueryParameter
qp =
handler.generatedQueryParameter(detail,
target);<br>
>>>
handler.setName(qp,
pk.getName());<br>
>>> queue.add(qp,
drools); // wraps insert(...)<br>
>>><br>
>>> end<br>
>>><br>
>>> I try to order the
select statements in an order that
will reduce the<br>
>>> size of the
cross-product (in theory), but I
also try and keep the<br>
>>> rules fairly human
readable. I try to avoid
comparison operators like<br>
>>> < and>.
Analysing a heap dump shows that
most of the memory is being<br>
>>> used in
StatefulSession.nodeMemories>
PrimitiveLongMap.<br>
>>><br>
>>> I am using a
StatefulSession; if I understand
correctly, I can't use a<br>
>>> StatelessSession with
sequential mode since I am
inserting facts as<br>
>>> part of the rules. If
I also understand correctly, I'd
like the Rete<br>
>>> graph to be tall,
rather than wide.<br>
>>><br>
>>> Some ideas I have
thought of include the following:<br>
>>> 1. Creating a
separate intermediary meta-model
to split up the sizes<br>
>>> of the rules. e.g.
instead of (if A and B and C then
insert D), using<br>
>>> (if A and B then
insert E; if E and C then insert
D).<br>
>>> 2. Moving eval()
statements directly into the
Type(...) selectors.<br>
>>> 3. Removing eval()
statements. Would this allow for
better indexing by<br>
>>> the Rete algorithm?<br>
>>> 4. Reducing the
height, or the width, of the class
hierarchy of the<br>
>>> facts. e.g. Removing
interfaces or abstract classes to
reduce the<br>
>>> possible matches.
Would this make a difference?<br>
>>> 5. Conversely,
increasing the height, or the
width, of the class<br>
>>> hierarchy. e.g.
Adding interfaces or abstract
classes to reduce field<br>
>>> accessors.<br>
>>> 6. Instead of using
EObject.eContainer, creating an
explicit<br>
>>> containment property
in all of my EObjects.<br>
>>> 7. Creating a DSL
that is human-readable, but allows
for the<br>
>>> automation of some of
these approaches.<br>
>>> 8. Moving all rules
into one rule file, or splitting
up rules into<br>
>>> smaller files.<br>
>>><br>
>>> Is there kind of
profiler for Drools that will let
me see the size (or<br>
>>> the memory usage) of
particular rules, or of the memory
used after<br>
>>> inference? Ideally
I'd use this to profile any
changes.<br>
>>><br>
>>> Thanks for any
thoughts or tips! :-)<br>
>>><br>
>>> Jevon<br>
>>>
_______________________________________________<br>
>>> rules-users mailing
list<br>
>>> <a
moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org"
target="_blank">rules-users@lists.jboss.org</a><br>
>>> <a
moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
>>><br>
>>><br>
>><br>
>><br>
>>
_______________________________________________<br>
>> rules-users mailing list<br>
>> <a
moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org"
target="_blank">rules-users@lists.jboss.org</a><br>
>> <a
moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
>><br>
><br>
>
_______________________________________________<br>
> rules-users mailing list<br>
> <a moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org"
target="_blank">rules-users@lists.jboss.org</a><br>
> <a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
><br>
</div>
</div>
</blockquote>
</div>
<br>
<pre><fieldset></fieldset>
_______________________________________________
rules-users mailing list
<a moz-do-not-send="true" href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a>
<a moz-do-not-send="true" href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
rules-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org"
target="_blank">rules-users@lists.jboss.org</a><br>
<a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br>
</blockquote>
</div>
<br>
<pre><fieldset></fieldset>
_______________________________________________
rules-users mailing list
<a moz-do-not-send="true" href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a>
<a moz-do-not-send="true" href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
rules-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br>
</blockquote>
</div>
<br>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
rules-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/rules-users">https://lists.jboss.org/mailman/listinfo/rules-users</a>
</pre>
</blockquote>
<br>
</body>
</html>