well, fixed now :-)<br>the benchmark (part of a junit) looks like this:<br><br>protected void init() {<br> KnowledgeBuilderConfiguration bConf = KnowledgeBuilderFactory.newKnowledgeBuilderConfiguration();<br> KnowledgeBuilder builder = KnowledgeBuilderFactory.newKnowledgeBuilder(bConf);<br>
builder.add(...);<br><br> KnowledgeBaseConfiguration kbConf = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();<br> kbConf.setOption( EventProcessingOption.STREAM );<br> //todo - turn this on when <a href="https://issues.jboss.org/browse/JBRULES-2845">https://issues.jboss.org/browse/JBRULES-2845</a> is fixed<br>
//kbConf.setOption( MultithreadEvaluationOption.YES );<br> kb = KnowledgeBaseFactory.newKnowledgeBase(kbConf);<br> kb.addKnowledgePackages(builder.getKnowledgePackages());<br><br> KnowledgeSessionConfiguration sConf = KnowledgeBaseFactory.newKnowledgeSessionConfiguration();<br>
sConf.setOption(ClockTypeOption.get("pseudo"));<br> Environment env = EnvironmentFactory.newEnvironment();<br> s = kb.newStatefulKnowledgeSession(sConf ,env);<br><br> clock = (SessionPseudoClock) s.getSessionClock();<br>
<br> backupStream = s.getWorkingMemoryEntryPoint("Backup Stream");<br>}<br><br>followed by a test method that looks like this:<br><br>public void testLargeBatches() {<br><br> Node node1 = new Node("node 1");<br>
node1.setId(1L);<br> s.insert(node1);<br> s.fireAllRules();<br><br> clock.advanceTime(1L, TimeUnit.DAYS);<br> s.fireAllRules();<br><br> int batchSize = 40000;<br> int daysToRun = 1;<br>
ArrayList<Backup> batch = new ArrayList<Backup>(batchSize);<br> ArrayList<Long> nodeIds = new ArrayList<Long>();<br> nodeIds.add(1L);<br> DateTime engineTime;<br><br> long start;<br>
long accumulated = 0;<br><br> System.err.println("days: "+daysToRun+". batch: "+batchSize+". rules: "+ruleFile);<br><br> for (int i=0; i<daysToRun; i++) {<br> engineTime = new DateTime(clock.getCurrentTime(), DateTimeZone.UTC);<br>
batch.clear();<br> //generate random<br> BackupGenerationUtil.fillBackupBatch(batch, nodeIds, engineTime.minusHours(23).minusMinutes(59).minusSeconds(59), engineTime.minusSeconds(1), batchSize);<br>
<br> start = System.currentTimeMillis();<br><br> //feed the batch in<br> for (Backup b : batch) {<br> backupStream.insert(b);<br> }<br><br> //advance a day and run all rules<br>
s.fireAllRules();<br> clock.advanceTime(1L, TimeUnit.DAYS);<br> s.fireAllRules();<br><br> accumulated += (System.currentTimeMillis() - start);<br> }<br><br> long averageBatchProcessingTime = (long)(accumulated/(double)daysToRun);<br>
double perBackupEventTime = ((double)accumulated)/(batchSize*daysToRun);<br> int eventsPerSecond = (int)(1000D/perBackupEventTime);<br><br> System.err.println("average batch processing time is "+averageBatchProcessingTime+" which is "+perBackupEventTime+"/event or "+eventsPerSecond+" events/second");<br>
}<br><br>running this with or without the 2nd "trimming" rules makes a huge difference. on my machine i can run the non-trimming version in batches of 300K @~2500 events/second @1GB RAM. with the trimming rule i cant get anywhere near that - the above value (40K) is already enough to slow it down to ~1800 events/second, and i dont understand whats so "heavy" about my attempt to save memory.<br>
<br>any help/thoughts/clues would be most welcome.<br><br><br><div class="gmail_quote">2010/12/23 Mauricio Salatino <span dir="ltr"><<a href="mailto:salaboy@gmail.com">salaboy@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
hehe yes.. the same happens to "me" :)<br><br><div class="gmail_quote">2010/12/23 Wolfgang Laun <span dir="ltr"><<a href="mailto:wolfgang.laun@gmail.com" target="_blank">wolfgang.laun@gmail.com</a>></span><div>
<div></div><div class="h5"><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
(OT: your nick "me" is confusing other gmail users. I see your mail as if I had sent it to myself. I was almost binning it.)<br><br>The delay you see may depend on the way you run the test with 40K events. Please describe the sequence of insertions and calls to fire...(), the session setup (pseudo-clock, etc.)<br>
<br>-W<br><br><br><div class="gmail_quote">2010/12/23 me <span dir="ltr"><<a href="mailto:radai.rosenblatt@gmail.com" target="_blank">radai.rosenblatt@gmail.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div>
Hi.<br><br>im trying to demo drools-fusion for a system that processes backup events. <br>i have the following 2 CEP rules:<br><br>rule "Backup Not Succeeded For At Least 3 Days"<br>@ruleId(1)<br>when<br> Node($id : id)<br>
not ( Backup(clientId == $id, $state: state == BackupStateEnum.FINISHED) over window:time( 3d ) from entry-point "Backup Stream" )<br>then<br> //nothing for now<br>end<br><br>rule "Prune Previous Successful Backups"<br>
@ruleId(2)<br>when<br> $prevBackup : Backup($id : clientId, state == BackupStateEnum.FINISHED) from entry-point "Backup Stream"<br> $newerBackup : Backup(clientId == $id, state == BackupStateEnum.FINISHED, this after $prevBackup) from entry-point "Backup Stream"<br>
then<br> drools.retract($prevBackup);<br>end<br><br>rule #2 is my attempt to cut down on memory usage. the problem is that rule #2 slows down the processing very very considerably (2 orders of magnitude slower when simulating batches of 40K events/day).<br>
<br>what would be my next step in trying to find the reason for it? my intuition tells me that with 2 events in the system at any given time instead of 120K (single node, 40K events/day 3 day window) i should have seen either a speedup or reduced memory consumption (if not both) and yet the results im seeing are the exact opposite.<br>
is there anything im missing ? i could try adding some sort of LastSuccessfulBackup "marker" object into main memory and updating it using events from the backup stream, but then how would i express rule #1 ?<br>
<br>any help/clues/suggestions would be greatly appreciated,<br><br> radai.<br>
<br></div></div>_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br>
<br>_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org" target="_blank">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div></div></div><font color="#888888"><br><br clear="all"><br>-- <br> - CTO @ <a href="http://www.plugtree.com" target="_blank">http://www.plugtree.com</a> <br> - MyJourney @ <a href="http://salaboy.wordpress.com" target="_blank">http://salaboy.wordpress.com</a><br>
- Co-Founder @ <a href="http://www.jbug.com.ar" target="_blank">http://www.jbug.com.ar</a><br> <br> - Salatino "Salaboy" Mauricio -<br>
</font><br>_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
<br></blockquote></div><br>