<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
On 16/07/2010 04:16, Jevon Wright wrote:
<blockquote
cite="mid:AANLkTikfKMqEauQSHcS8Twxmlgk_9f6XudDoMXy61u45@mail.gmail.com"
type="cite">Hi again,<br>
<br>
By removing all of the simple eval()s from my rules, I have cut
heap usage by at least an order of magnitude. However this still
isn't enough.<br>
<br>
Since I am trying to reduce the cross-product size (as in SQL), I
recall that most SQL implementations have a "DESCRIBE SELECT"
query which provides real-time information about the complexity of
a given SQL query - i.e. the size of the tables, indexes used, and
so on. Is there any such tool available for Drools? Are there any
tools which can provide clues as to which rules are using the most
memory?<br>
</blockquote>
no, but sounds like a nice side project :)<br>
<blockquote
cite="mid:AANLkTikfKMqEauQSHcS8Twxmlgk_9f6XudDoMXy61u45@mail.gmail.com"
type="cite">
<br>
Alternatively, I am wondering what kind of benefit I could expect
from using materialized views to create summary tables; that is,
deriving and inserting additional facts. This would allow Drools
to rewrite queries that currently use eval(), but would increase
the size of working memory, so would this actually save heap size?<br>
<br>
To what extent does Drools rewrite queries? Is there any
documentation describing the approaches used?<br>
</blockquote>
The only rewriting we do is on literal constraints, moving == ones
to the front, that's it. Also make sure you are using trunk, which
is more memory efficient for modifies:<br>
<a class="moz-txt-link-freetext" href="https://hudson.jboss.org/hudson/job/drools/lastSuccessfulBuild/artifact/trunk/target/">https://hudson.jboss.org/hudson/job/drools/lastSuccessfulBuild/artifact/trunk/target/</a><br>
<br>
Mark<br>
<blockquote
cite="mid:AANLkTikfKMqEauQSHcS8Twxmlgk_9f6XudDoMXy61u45@mail.gmail.com"
type="cite"><br>
Any other ideas on how to reduce heap memory usage? I'd appreciate
any ideas :)<br>
<br>
Thanks<br>
Jevon<br>
<br>
<br>
<div class="gmail_quote">On Mon, Jul 12, 2010 at 5:56 PM, Jevon
Wright <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jevon@jevon.org">jevon@jevon.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
Hi Wolfgang and Mark,<br>
<br>
Thank you for your replies! You were correct: my eval()
functions<br>
could generally be rewritten into Drools directly.<br>
<br>
I had one function "connectsDetail" that was constraining<br>
unidirectional edges, and could be rewritten from:<br>
<div class="im"> detail : DetailWire ( )<br>
eval ( functions.connectsDetail(detail, source, target) )<br>
<br>
</div>
to:<br>
detail : DetailWire ( from == source, to == target )<br>
<br>
Another function, "connects", was constraining bidirectional
edges,<br>
and could be rewritten from:<br>
sync : SyncWire( )<br>
eval ( functions.connects(sync, source, target) )<br>
<br>
to:<br>
sync : SyncWire( (from == source && to == target) ||
(from == target<br>
&& to == source) )<br>
<br>
Finally, the "veto" function could be rewritten from:<br>
detail : DetailWire ( )<br>
eval ( handler.veto(detail) )<br>
<br>
to:<br>
detail : DetailWire ( overridden == false )<br>
<br>
I took each of these three changes, and evaluated them
separately [1].<br>
I found that:<br>
<br>
1. Inlining 'connectsDetail' made a huge difference - 10-30%
faster<br>
execution and 50-60% less allocated heap.<br>
2. Inlining 'connects' made very little difference - 10-30%
faster<br>
execution, but 0-20% more allocated heap.<br>
3. Inlining 'veto' made no difference - no significant change
in<br>
execution speed or allocated heap.<br>
<br>
I think I understand why inlining 'connects' would improve
heap usage<br>
- because the rules essentially have more conditionals?<br>
<br>
I also understand why 'veto' made no difference - for most of
my test<br>
models, "overridden" was never true, so adding this
conditional was<br>
not making the cross product set any smaller.<br>
<br>
Finally, I also tested simply joining all of the rules
together into<br>
one file. This happily made no difference at all (although
made it<br>
more difficult to edit).<br>
<br>
So I think I can safely conclude that eval() should be used as
little<br>
as possible - however, this means that the final rules are
made more<br>
complicated and less human-readable, so a DSL may be best for
my<br>
common rule patterns in the future.<br>
<br>
Thanks again!<br>
Jevon<br>
<br>
[1]: <a moz-do-not-send="true"
href="http://www.jevon.org/wiki/Improving_Drools_Memory_Performance"
target="_blank">http://www.jevon.org/wiki/Improving_Drools_Memory_Performance</a><br>
<div>
<div class="h5"><br>
On Sat, Jul 10, 2010 at 12:28 AM, Wolfgang Laun <<a
moz-do-not-send="true"
href="mailto:wolfgang.laun@gmail.com">wolfgang.laun@gmail.com</a>>
wrote:<br>
> On 9 July 2010 14:14, Mark Proctor <<a
moz-do-not-send="true"
href="mailto:mproctor@codehaus.org">mproctor@codehaus.org</a>>
wrote:<br>
>> You have many objects there that are not
constrained;<br>
><br>
> I have an inkling that the functions.*() are hiding
just these contraints,<br>
> It's certainly the wrong way, starting with oodles of
node pairs, just to<br>
> pick out connected ones by fishing for the connecting
edge. And this<br>
> is worsened by trying to find two such pairs which
meet at some<br>
> DomainSource<br>
><br>
> Guesswork, hopefully educated ;-)<br>
><br>
> -W<br>
><br>
><br>
>> if there are<br>
>> multiple versions of those objects you are going
to get massive amounts<br>
>> of cross products. Think in terms of SQL, each
pattern you add is like<br>
>> an SQL join.<br>
>><br>
>> Mark<br>
>> On 09/07/2010 09:20, Jevon Wright wrote:<br>
>>> Hi everyone,<br>
>>><br>
>>> I am working on what appears to be a fairly
complex rule base based on<br>
>>> EMF. The rules aren't operating over a huge
number of facts (less than<br>
>>> 10,000 EObjects) and there aren't too many
rules (less than 300), but<br>
>>> I am having a problem with running out of
Java heap space (set at ~400<br>
>>> MB).<br>
>>><br>
>>> Through investigation, I came to the
conclusion that this is due to<br>
>>> the design of the rules, rather than the
number of facts. The engine<br>
>>> uses less memory inserting many facts that
use simple rules, compared<br>
>>> with inserting few facts that use many rules.<br>
>>><br>
>>> Can anybody suggest some tips for reducing
heap memory usage in<br>
>>> Drools? I don't have a time constraint, only
a heap/memory constraint.<br>
>>> A sample rule in my project looks like this:<br>
>>><br>
>>> rule "Create QueryParameter for target
container of DetailWire"<br>
>>> when<br>
>>> container : Frame( )<br>
>>> schema : DomainSchema ( )<br>
>>> domainSource : DomainSource ( )<br>
>>> instance : DomainIterator( )<br>
>>> selectEdge : SelectEdge ( eval (<br>
>>> functions.connectsSelect(selectEdge,
instance, domainSource )) )<br>
>>> schemaEdge : SchemaEdge ( eval (<br>
>>> functions.connectsSchema(schemaEdge,
domainSource, schema )) )<br>
>>> source : VisibleThing ( eContainer ==
container )<br>
>>> target : Frame ( )<br>
>>> instanceSet : SetWire (
eval(functions.connectsSet(instanceSet,<br>
>>> instance, source )) )<br>
>>> detail : DetailWire ( )<br>
>>> eval (
functions.connectsDetail(detail, source, target ))<br>
>>> pk : DomainAttribute ( eContainer ==
schema, primaryKey == true )<br>
>>> not ( queryPk : QueryParameter (
eContainer == target, name == <a moz-do-not-send="true"
href="http://pk.name" target="_blank">pk.name</a> ) )<br>
>>> eval ( handler.veto( detail ))<br>
>>><br>
>>> then<br>
>>> QueryParameter qp =
handler.generatedQueryParameter(detail, target);<br>
>>> handler.setName(qp, pk.getName());<br>
>>> queue.add(qp, drools); // wraps
insert(...)<br>
>>><br>
>>> end<br>
>>><br>
>>> I try to order the select statements in an
order that will reduce the<br>
>>> size of the cross-product (in theory), but I
also try and keep the<br>
>>> rules fairly human readable. I try to avoid
comparison operators like<br>
>>> < and>. Analysing a heap dump shows
that most of the memory is being<br>
>>> used in StatefulSession.nodeMemories>
PrimitiveLongMap.<br>
>>><br>
>>> I am using a StatefulSession; if I understand
correctly, I can't use a<br>
>>> StatelessSession with sequential mode since I
am inserting facts as<br>
>>> part of the rules. If I also understand
correctly, I'd like the Rete<br>
>>> graph to be tall, rather than wide.<br>
>>><br>
>>> Some ideas I have thought of include the
following:<br>
>>> 1. Creating a separate intermediary
meta-model to split up the sizes<br>
>>> of the rules. e.g. instead of (if A and B and
C then insert D), using<br>
>>> (if A and B then insert E; if E and C then
insert D).<br>
>>> 2. Moving eval() statements directly into the
Type(...) selectors.<br>
>>> 3. Removing eval() statements. Would this
allow for better indexing by<br>
>>> the Rete algorithm?<br>
>>> 4. Reducing the height, or the width, of the
class hierarchy of the<br>
>>> facts. e.g. Removing interfaces or abstract
classes to reduce the<br>
>>> possible matches. Would this make a
difference?<br>
>>> 5. Conversely, increasing the height, or the
width, of the class<br>
>>> hierarchy. e.g. Adding interfaces or abstract
classes to reduce field<br>
>>> accessors.<br>
>>> 6. Instead of using EObject.eContainer,
creating an explicit<br>
>>> containment property in all of my EObjects.<br>
>>> 7. Creating a DSL that is human-readable, but
allows for the<br>
>>> automation of some of these approaches.<br>
>>> 8. Moving all rules into one rule file, or
splitting up rules into<br>
>>> smaller files.<br>
>>><br>
>>> Is there kind of profiler for Drools that
will let me see the size (or<br>
>>> the memory usage) of particular rules, or of
the memory used after<br>
>>> inference? Ideally I'd use this to profile
any changes.<br>
>>><br>
>>> Thanks for any thoughts or tips! :-)<br>
>>><br>
>>> Jevon<br>
>>>
_______________________________________________<br>
>>> rules-users mailing list<br>
>>> <a moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
>>> <a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
>>><br>
>>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> rules-users mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
>> <a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
>><br>
><br>
> _______________________________________________<br>
> rules-users mailing list<br>
> <a moz-do-not-send="true"
href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
> <a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/rules-users"
target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
><br>
</div>
</div>
</blockquote>
</div>
<br>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
rules-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/rules-users">https://lists.jboss.org/mailman/listinfo/rules-users</a>
</pre>
</blockquote>
<br>
</body>
</html>