Hi Tim,<div><br></div><div>I don't believe Drools internals provides multi-threading under "usual" operation.</div><div><br></div><div>If you start to use Timers and Calendars in your rules I have a strong suspicion that it will; however this is an edge case.</div>
<div><br></div><div>CEP has the benefit of automatically garbage collecting facts that can no longer match patterns; so if your rules reason over a sliding window some of your 100K's facts may have been purged from WM keeping resource usage lower than perhaps a stateless session. You could use your "timestamp" field as the "@timestamp" meta annotation. </div>
<div><br></div><div>I can't say whether this would deliver performance improvements as I feel any benefit could depend upon the shape or your data; e.g. is the aggregation over a relatively small range of timestamps vs the whole range of the batch (i.e. batch represents 6 hours but your rules look for groups within 5 minute windows). Ordering the facts by timestamp before insertion would theoretically preclude the need to have them all in WM at once.</div>
<div><br></div><div>Edson is the CEP expert however he's at Rules Fest at the moment so I doubt able to answer until he comes back.</div><div><br></div><div>Possibly helpful, possibly the rant of somebody miss-informed.</div>
<div><br></div><div>With kind regards,</div><div><br></div><div>Mike</div><div><br></div><div><div class="gmail_quote">2010/10/13 Tim Jones <span dir="ltr"><<a href="mailto:jones.tim36@gmail.com">jones.tim36@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>Interesting suggestions. Couple of questions:</div>
<div> </div>
<div>Is drools not internally able to multithread a single execution of fireallrules?</div>
<div> </div>
<div>Does CEP offer performance benefits, or is it just a different way of structuring the problem?</div>
<div> </div>
<div> </div>
<div>Cheers,</div>
<div>Tim<br><br></div>
<div class="gmail_quote">2010/10/12 Michael Anstis <span dir="ltr"><<a href="mailto:michael.anstis@gmail.com" target="_blank">michael.anstis@gmail.com</a>></span><br>
<blockquote style="border-left:#ccc 1px solid;margin:0px 0px 0px 0.8ex;padding-left:1ex" class="gmail_quote">Can the aggregation or timestamp range be used to partion your data?<br><br>e.g. if you're looking for a data pattern where a fact matches X and Y and Z can the most course constraint, say X, not be used to partion?<br>
<br>So you may have pre-processing (to partion the data) before hitting other finer grained rules?<br><br>Would CEP in stream mode provide an opportunity either?<br><br>2010/10/12 Wolfgang Laun <span dir="ltr"><<a href="mailto:wolfgang.laun@gmail.com" target="_blank">wolfgang.laun@gmail.com</a>></span>
<div>
<div></div>
<div><br>
<div class="gmail_quote">
<blockquote style="border-left:rgb(204,204,204) 1px solid;margin:0pt 0pt 0pt 0.8ex;padding-left:1ex" class="gmail_quote">If you have to use a stateful session, with new objects being generated in RHS code and triggering more rules, then you've had it (since there is no way to split the 100k facts).<br>
<br>If you don't create new facts in RHS code, you should investigate stateless sessions. It should be more efficient.<br><br>Also, think about what you RHS code does. Can this processing be delegated to another thread? After a rule has fired, the "fate" of this activation is firmly established; rather than executing (timeconsuming i/o?) operations inline, queue the collected date to a processor thread and let this one crunch it.<br>
<br>(I've only thought about this for a few minutes, so there may be other options.)<br><br>-W<br><br>
<div class="gmail_quote">2010/10/12 Tim Jones <span dir="ltr"><<a href="mailto:jones.tim36@gmail.com" target="_blank">jones.tim36@gmail.com</a>></span><br>
<blockquote style="border-left:rgb(204,204,204) 1px solid;margin:0pt 0pt 0pt 0.8ex;padding-left:1ex" class="gmail_quote">
<div>
<div></div>
<div>
<p class="MsoNormal">Hello,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I’m working on a project that needs a high performance rules system for processing batches of objects. Typically I’ll have a dozen or so rules, the most complex of which will aggregate several objects based on timestamps and specified data patterns. The objects will come in batches of a few 100ks. The system is reset back to the starting point after each batch is processed.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">My guess at doing this with Drools is that you load up all the rules and enter all the objects as “facts”. You then hit fireallrules and sit back and wait. Doing this, I only get so much performance and I can see that its only using a single thread. Is there a way to process the whole lot in a parallel, or multithreaded way? Unfortunately there's no natural way to partition the objects that would make things easier.<br>
</p>
<p class="MsoNormal"><br></p>
<p class="MsoNormal"><br></p>
<p class="MsoNormal">Cheers,</p>
<p class="MsoNormal">Tim<br></p><br></div></div>_______________________________________________<br>rules-users mailing list<br><a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br><br></blockquote></div><br><br>_______________________________________________<br>
rules-users mailing list<br><a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br><a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br></div></div><br>_______________________________________________<br>rules-users mailing list<br><a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br><a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br>
<br>_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br></div>