Re: [rules-users] Drools Fusion Dropping Events?

Wolfgang, the code to reproduce is below. I'm hoping to process between 20k and 50k events through Drools per second thus the extreme high-throughput testing. I could settle for a single Drools node handling only say 5K per second IFF I could cluster Drools but I've not yet found a way to distribute workload across an active-active Drools cluster (seems there is no such thing?). Since you're recommendation I've shifted to using Drools 5.3 just FYI: ### Average.java ### package drools53fusioneval; import java.io.IOException; import java.util.Random; import org.drools.KnowledgeBase; import org.drools.KnowledgeBaseConfiguration; import org.drools.KnowledgeBaseFactory; import org.drools.builder.KnowledgeBuilder; import org.drools.builder.KnowledgeBuilderFactory; import org.drools.builder.ResourceType; import org.drools.conf.EventProcessingOption; import org.drools.io.ResourceFactory; import org.drools.runtime.StatefulKnowledgeSession; import org.drools.runtime.rule.WorkingMemoryEntryPoint; class AvgDFEChannel implements org.drools.runtime.Channel { @Override public void send(Object o) { System.err.println("Recieved channel message: " + o); } } public class Average { public static void main(String[] args) throws InterruptedException, IOException { KnowledgeBaseConfiguration kbconfig = KnowledgeBaseFactory.newKnowledgeBaseConfiguration(); kbconfig.setOption(EventProcessingOption.STREAM); KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase(kbconfig); KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder(); kbuilder.add(ResourceFactory.newClassPathResource("drools53fusioneval/basic.drl"), ResourceType.DRL); if (kbuilder.hasErrors()) { System.err.println(kbuilder.getErrors().toString()); } kbase.addKnowledgePackages(kbuilder.getKnowledgePackages()); final StatefulKnowledgeSession session = kbase.newStatefulKnowledgeSession(); session.registerChannel("heartbeat", new AvgDFEChannel()); WorkingMemoryEntryPoint ep01 = session.getWorkingMemoryEntryPoint("ep01"); new Thread() { public void run() { session.fireUntilHalt(); } }.start(); Thread.sleep(5000); // give the engine time to get setup Server hiwaesdk = new Server("hiwaesdk"); session.insert(hiwaesdk); long LIMIT = 10000; long sentCount = 0; int batchSize = 10000; Random rnd = new Random(System.nanoTime()); int temp = 0; long startTS = System.nanoTime(); while (sentCount < LIMIT) { for (int i = 0; i < batchSize; i++) { temp = rnd.nextInt(212); IntEvent evt = new IntEvent (temp); ep01.insert(evt); sentCount++; } Thread.sleep (0x1); } double duration = (System.nanoTime() - startTS)/1000000.0; System.out.println(LIMIT +" events generated in "+ duration +" milliseconds"); System.out.println("Last temperature submitted: "+ temp); for (int i = 0; i < 5; i++) { System.out.println ("Sec "+ i +": "+ hiwaesdk.currentTemp +", "+ hiwaesdk.readingCount); Thread.sleep (1000); } System.exit(0); } } ### basic.drl ### package drools53fusioneval declare IntEvent @role ( event ) end rule "number rule" when $e : IntEvent () from entry-point ep01 $s : Server (hostname == "hiwaesdk") then $s.currentTemp = $e.data; $s.readingCount++; end On Thu, May 16, 2013 at 4:25 AM, Wolfgang Laun <wolfgang.laun(a)gmail.com <mailto:wolfgang.laun@gmail.com>> wrote: What you describe is definitely not "expected behaviour" - it's more like that race condition (or another one) already being in 5.3. Posting your code might be a good idea ;-) -W On 16/05/2013, Jason Barto <jason.p.barto(a)gmail.com <mailto:jason.p.barto@gmail.com>> wrote: > I've been working to load test Drools 5.3 to determine if its a fit for a > project. As part of the test I programmatically insert events as rapidly > as possible; an example, my earlier test inserted 10k events in about > 300ms. There is currently a single rule which reads the event and stores > the event's value into the only fact in Drools. I'm very happy to report, > and I'm sure it will be no surprise to anyone, that the engine processes > all the events in roughly 1 sec. However I have noticed that any large > number of events (~>1000) usually sees that a small number of events don't > get processed. I think after 10k events as many as 7 appear to have gone > unprocessed. If 100 events are inserted, rather than 10k, no events get > disregarded. Being new to Drools I can easily accept that my java code or > DRL code is off or unoptimized in some way. However not knowing how Drools > does its magic I'm currently inclined to think that Drools gets swamped > (10k in 300ms is a lot) and a few events get dropped so Drools can keep > operating. Is this a known or expected behavior from Drools? If not I am > happy to post my code, it is similar to the other code sets I've posted in > the last few days. I'm still new to Drools and trying to learn its behavior > so any insight or enlightenment is greatly appreciated. > > Sincerely, > Jason > _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org <mailto:rules-users@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/rules-users _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users