<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
There is a recent post on "poor performance from a simple join" that
highligths almost the same questions : because "insert" trigger RETE
propagation, time to insert depends on rules complexity.<br>
<br>
May be you can start by looking at your rules to optimise them (see
the previous post for some tips).<br>
<br>
If it is still too long, may be you can cut your data in smaller
groups. The main problem here is to be able to cut the data into
pertinent groups according to rules (problems can happend if you
have some accumulates, or exists, or not .... If you only have
simple filters, you can cut your data where your want. If your rules
are reasonning with global existence or lack for a fact, then you
must ensure that, for example, a "not MyFact()" is true because the
fact does not exists at all, and not only because it is not part of
the chunk ...).<br>
<br>
<br>
Le 20/12/2011 14:27, Mark Proctor a écrit :
<blockquote cite="mid:4EF08D3A.4030805@codehaus.org" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
On 20/12/2011 13:09, Zhuo Li wrote:
<blockquote cite="mid:006c01ccbf18$9c040050$d40c00f0$@com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:\5B8B\4F53;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:\5B8B\4F53;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@\5B8B\4F53";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
text-indent:21.0pt;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:547034152;
mso-list-type:hybrid;
mso-list-template-ids:1018445210 -909993680 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:18.0pt;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
-->
</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi, folks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I recently did a
benchmark on Drools 5.1.2 and noticed that data insert
into a stateful session is very time consuming. It took me
about 30 minutes to insert 10,000 data rows on a 512M
heapsize JVM. Hence I have to keep inserting data rows
when I receive them and keep them in working memory,
rather than loading them in a batch at a given time. This
is not a friendly way for disaster recovery and I have two
questions here to see if anybody has any thoughts:</span></p>
</div>
</blockquote>
10K rows? is that 10K bean insertions? 30 minutes sounds bad. We
know people doing far more than that much quicker.<br>
<blockquote cite="mid:006c01ccbf18$9c040050$d40c00f0$@com"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="margin-left: 18pt;
text-indent: -18pt;"><!--[if !supportLists]--><span
lang="EN-US"><span style="mso-list:Ignore">1.<span
style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US">Is
there any better way to improve the performance of data
insert into a stateful session;</span></p>
</div>
</blockquote>
There is nothing faster than "insert".<br>
If you don't need inference, you can try turning on "sequential"
mode, but in general the performance gain is < 5%.<br>
<blockquote cite="mid:006c01ccbf18$9c040050$d40c00f0$@com"
type="cite">
<div class="WordSection1">
<p class="MsoListParagraph"
style="margin-left:18.0pt;text-indent:-18.0pt; mso-list:l0
level1 lfo1"><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left: 18pt;
text-indent: -18pt;"><!--[if !supportLists]--><span
lang="EN-US"><span style="mso-list:Ignore">2.<span
style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US">I
noticed that there is a method called BatchExecution() for
a stateless session. Did not get a chance to test it yet
but is this a better way to load data in a batch and then
run rules?</span></p>
</div>
</blockquote>
That is related to scripting an engine, it uses command objects to
call the inert() method - so definitely not faster.<br>
<blockquote cite="mid:006c01ccbf18$9c040050$d40c00f0$@com"
type="cite">
<div class="WordSection1">
<p class="MsoListParagraph"
style="margin-left:18.0pt;text-indent:-18.0pt; mso-list:l0
level1 lfo1"><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">My requirement is I
need to load a batch of data once by end of the day, and
then run the rules to filter out matched data against
unmatched data. I have a 3-hour processing window to
complete this loading and matching process, and the data I
need to load is about 1 million to 2 millions. My JVM
heapsize can be set up to 1024 M.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Best regards<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Abe<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</blockquote>
</blockquote>
<br>
</body>
</html>