<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=gb2312">
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri","sans-serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
text-indent:21.0pt;
font-size:10.5pt;
font-family:"Calibri","sans-serif";
color:black;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:547034152;
mso-list-type:hybrid;
mso-list-template-ids:1018445210 -909993680 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:18.0pt;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor=white lang=ZH-CN link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Thanks Vincent. Slicing
the data is definitely an option but I guess stateless session is a better fit
for this solution? Stateful session means you will have to keep your slices
into different working memory slots which will not be efficient intuitionally. <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Best<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Abe<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal align=left style='text-align:left'><b><span
style='font-size:10.0pt;font-family:SimSun;color:windowtext'>发件人<span
lang=EN-US>:</span></span></b><span lang=EN-US style='font-size:10.0pt;
font-family:SimSun;color:windowtext'> rules-users-bounces@lists.jboss.org
[mailto:rules-users-bounces@lists.jboss.org] </span><b><span style='font-size:
10.0pt;font-family:SimSun;color:windowtext'>代表 </span></b><span
lang=EN-US style='font-size:10.0pt;font-family:SimSun;color:windowtext'>Vincent
Legendre<br>
</span><b><span style='font-size:10.0pt;font-family:SimSun;color:windowtext'>发送时间<span
lang=EN-US>:</span></span></b><span lang=EN-US style='font-size:10.0pt;
font-family:SimSun;color:windowtext'> 2011</span><span style='font-size:10.0pt;
font-family:SimSun;color:windowtext'>年<span lang=EN-US>12</span>月<span
lang=EN-US>20</span>日<span lang=EN-US> 22:01<br>
</span><b>收件人<span lang=EN-US>:</span></b><span
lang=EN-US> Rules Users List<br>
</span><b>主题<span lang=EN-US>:</span></b><span lang=EN-US> Re:
[rules-users] Working memory batch insert performance<o:p></o:p></span></span></p>
</div>
</div>
<p class=MsoNormal align=left style='text-align:left'><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>There is a recent post on "poor
performance from a simple join" that highligths almost the same questions
: because "insert" trigger RETE propagation, time to insert depends
on rules complexity.<br>
<br>
May be you can start by looking at your rules to optimise them (see the
previous post for some tips).<br>
<br>
If it is still too long, may be you can cut your data in smaller groups. The
main problem here is to be able to cut the data into pertinent groups according
to rules (problems can happend if you have some accumulates, or exists, or not
.... If you only have simple filters, you can cut your data where your want. If
your rules are reasonning with global existence or lack for a fact, then you
must ensure that, for example, a "not MyFact()" is true because the
fact does not exists at all, and not only because it is not part of the chunk
...).<br>
<br>
<br>
Le 20/12/2011 14:27, Mark Proctor a écrit : <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>On 20/12/2011 13:09, Zhuo Li wrote: <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Hi, folks,<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>I recently did a benchmark on Drools 5.1.2
and noticed that data insert into a stateful session is very time consuming. It
took me about 30 minutes to insert 10,000 data rows on a 512M heapsize JVM.
Hence I have to keep inserting data rows when I receive them and keep them in
working memory, rather than loading them in a batch at a given time. This is
not a friendly way for disaster recovery and I have two questions here to see
if anybody has any thoughts:<o:p></o:p></span></p>
<p class=MsoNormal align=left style='text-align:left'><span lang=EN-US
style='font-size:12.0pt;font-family:"Times New Roman","serif"'>10K rows? is
that 10K bean insertions? 30 minutes sounds bad. We know people doing far more
than that much quicker.<br>
<br>
<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoListParagraph style='margin-left:18.0pt;text-indent:-18.0pt'><span
lang=EN-US>Is there any better way to improve the performance of data insert
into a stateful session;<o:p></o:p></span></p>
<p class=MsoNormal align=left style='text-align:left'><span lang=EN-US
style='font-size:12.0pt;font-family:"Times New Roman","serif"'>There is nothing
faster than "insert".<br>
If you don't need inference, you can try turning on "sequential"
mode, but in general the performance gain is < 5%.<br>
<br>
<o:p></o:p></span></p>
<p class=MsoListParagraph style='margin-left:18.0pt;text-indent:-18.0pt'><span
lang=EN-US>I noticed that there is a method called BatchExecution() for a
stateless session. Did not get a chance to test it yet but is this a better way
to load data in a batch and then run rules?<o:p></o:p></span></p>
<p class=MsoNormal align=left style='text-align:left'><span lang=EN-US
style='font-size:12.0pt;font-family:"Times New Roman","serif"'>That is related
to scripting an engine, it uses command objects to call the inert() method - so
definitely not faster.<br>
<br>
<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>My requirement is I need to load a batch of
data once by end of the day, and then run the rules to filter out matched data
against unmatched data. I have a 3-hour processing window to complete this
loading and matching process, and the data I need to load is about 1 million to
2 millions. My JVM heapsize can be set up to 1024 M.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Best regards<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Abe<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal align=left style='text-align:left'><span lang=EN-US
style='font-size:12.0pt;font-family:"Times New Roman","serif"'><o:p> </o:p></span></p>
</div>
</body>
</html>