Re: [rules-users] help with diagnosing CEP performance issue

well, fixed now :-) the benchmark (part of a junit) looks like this: protected void init() { KnowledgeBuilderConfiguration bConf = KnowledgeBuilderFactory.newKnowledgeBuilderConfiguration(); KnowledgeBuilder builder = KnowledgeBuilderFactory.newKnowledgeBuilder(bConf); builder.add(...); KnowledgeBaseConfiguration kbConf = KnowledgeBaseFactory.newKnowledgeBaseConfiguration(); kbConf.setOption( EventProcessingOption.STREAM ); //todo - turn this on when https://issues.jboss.org/browse/JBRULES-2845 is fixed //kbConf.setOption( MultithreadEvaluationOption.YES ); kb = KnowledgeBaseFactory.newKnowledgeBase(kbConf); kb.addKnowledgePackages(builder.getKnowledgePackages()); KnowledgeSessionConfiguration sConf = KnowledgeBaseFactory.newKnowledgeSessionConfiguration(); sConf.setOption(ClockTypeOption.get("pseudo")); Environment env = EnvironmentFactory.newEnvironment(); s = kb.newStatefulKnowledgeSession(sConf ,env); clock = (SessionPseudoClock) s.getSessionClock(); backupStream = s.getWorkingMemoryEntryPoint("Backup Stream"); } followed by a test method that looks like this: public void testLargeBatches() { Node node1 = new Node("node 1"); node1.setId(1L); s.insert(node1); s.fireAllRules(); clock.advanceTime(1L, TimeUnit.DAYS); s.fireAllRules(); int batchSize = 40000; int daysToRun = 1; ArrayList<Backup> batch = new ArrayList<Backup>(batchSize); ArrayList<Long> nodeIds = new ArrayList<Long>(); nodeIds.add(1L); DateTime engineTime; long start; long accumulated = 0; System.err.println("days: "+daysToRun+". batch: "+batchSize+". rules: "+ruleFile); for (int i=0; i<daysToRun; i++) { engineTime = new DateTime(clock.getCurrentTime(), DateTimeZone.UTC); batch.clear(); //generate random BackupGenerationUtil.fillBackupBatch(batch, nodeIds, engineTime.minusHours(23).minusMinutes(59).minusSeconds(59), engineTime.minusSeconds(1), batchSize); start = System.currentTimeMillis(); //feed the batch in for (Backup b : batch) { backupStream.insert(b); } //advance a day and run all rules s.fireAllRules(); clock.advanceTime(1L, TimeUnit.DAYS); s.fireAllRules(); accumulated += (System.currentTimeMillis() - start); } long averageBatchProcessingTime = (long)(accumulated/(double)daysToRun); double perBackupEventTime = ((double)accumulated)/(batchSize*daysToRun); int eventsPerSecond = (int)(1000D/perBackupEventTime); System.err.println("average batch processing time is "+averageBatchProcessingTime+" which is "+perBackupEventTime+"/event or "+eventsPerSecond+" events/second"); } running this with or without the 2nd "trimming" rules makes a huge difference. on my machine i can run the non-trimming version in batches of 300K @~2500 events/second @1GB RAM. with the trimming rule i cant get anywhere near that - the above value (40K) is already enough to slow it down to ~1800 events/second, and i dont understand whats so "heavy" about my attempt to save memory. any help/thoughts/clues would be most welcome. 2010/12/23 Mauricio Salatino <salaboy(a)gmail.com&gt;