[rules-users] Drools Fusion Dropping Events?
Jason Barto
jason.p.barto at gmail.com
Thu May 16 02:12:51 EDT 2013
Wolfgang,
the code to reproduce is below. I'm hoping to process between 20k and 50k
events through Drools per second thus the extreme high-throughput testing.
I could settle for a single Drools node handling only say 5K per second IFF
I could cluster Drools but I've not yet found a way to distribute workload
across an active-active Drools cluster (seems there is no such thing?).
Since you're recommendation I've shifted to using Drools 5.3 just FYI:
### Average.java ###
package drools53fusioneval;
import java.io.IOException;
import java.util.Random;
import org.drools.KnowledgeBase;
import org.drools.KnowledgeBaseConfiguration;
import org.drools.KnowledgeBaseFactory;
import org.drools.builder.KnowledgeBuilder;
import org.drools.builder.KnowledgeBuilderFactory;
import org.drools.builder.ResourceType;
import org.drools.conf.EventProcessingOption;
import org.drools.io.ResourceFactory;
import org.drools.runtime.StatefulKnowledgeSession;
import org.drools.runtime.rule.WorkingMemoryEntryPoint;
class AvgDFEChannel implements org.drools.runtime.Channel {
@Override
public void send(Object o) {
System.err.println("Recieved channel message: " + o);
}
}
public class Average {
public static void main(String[] args) throws InterruptedException,
IOException {
KnowledgeBaseConfiguration kbconfig =
KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
kbconfig.setOption(EventProcessingOption.STREAM);
KnowledgeBase kbase =
KnowledgeBaseFactory.newKnowledgeBase(kbconfig);
KnowledgeBuilder kbuilder =
KnowledgeBuilderFactory.newKnowledgeBuilder();
kbuilder.add(ResourceFactory.newClassPathResource("drools53fusioneval/basic.drl"),
ResourceType.DRL);
if (kbuilder.hasErrors()) {
System.err.println(kbuilder.getErrors().toString());
}
kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
final StatefulKnowledgeSession session =
kbase.newStatefulKnowledgeSession();
session.registerChannel("heartbeat", new AvgDFEChannel());
WorkingMemoryEntryPoint ep01 =
session.getWorkingMemoryEntryPoint("ep01");
new Thread() {
public void run() {
session.fireUntilHalt();
}
}.start();
Thread.sleep(5000); // give the engine time to get setup
Server hiwaesdk = new Server("hiwaesdk");
session.insert(hiwaesdk);
long LIMIT = 10000;
long sentCount = 0;
int batchSize = 10000;
Random rnd = new Random(System.nanoTime());
int temp = 0;
long startTS = System.nanoTime();
while (sentCount < LIMIT) {
for (int i = 0; i < batchSize; i++) {
temp = rnd.nextInt(212);
IntEvent evt = new IntEvent (temp);
ep01.insert(evt);
sentCount++;
}
Thread.sleep (0x1);
}
double duration = (System.nanoTime() - startTS)/1000000.0;
System.out.println(LIMIT +" events generated in "+ duration +"
milliseconds");
System.out.println("Last temperature submitted: "+ temp);
for (int i = 0; i < 5; i++) {
System.out.println ("Sec "+ i +": "+ hiwaesdk.currentTemp +",
"+ hiwaesdk.readingCount);
Thread.sleep (1000);
}
System.exit(0);
}
}
### basic.drl ###
package drools53fusioneval
declare IntEvent
@role ( event )
end
rule "number rule"
when
$e : IntEvent () from entry-point ep01
$s : Server (hostname == "hiwaesdk")
then
$s.currentTemp = $e.data;
$s.readingCount++;
end
On Thu, May 16, 2013 at 4:25 AM, Wolfgang Laun <wolfgang.laun at gmail.com>wrote:
> What you describe is definitely not "expected behaviour" - it's more
> like that race condition (or another one) already being in 5.3. Posting
> your code might be a good idea ;-)
> -W
>
> On 16/05/2013, Jason Barto <jason.p.barto at gmail.com> wrote:
> > I've been working to load test Drools 5.3 to determine if its a fit for a
> > project. As part of the test I programmatically insert events as rapidly
> > as possible; an example, my earlier test inserted 10k events in about
> > 300ms. There is currently a single rule which reads the event and stores
> > the event's value into the only fact in Drools. I'm very happy to report,
> > and I'm sure it will be no surprise to anyone, that the engine processes
> > all the events in roughly 1 sec. However I have noticed that any large
> > number of events (~>1000) usually sees that a small number of events
> don't
> > get processed. I think after 10k events as many as 7 appear to have gone
> > unprocessed. If 100 events are inserted, rather than 10k, no events get
> > disregarded. Being new to Drools I can easily accept that my java code
> or
> > DRL code is off or unoptimized in some way. However not knowing how
> Drools
> > does its magic I'm currently inclined to think that Drools gets swamped
> > (10k in 300ms is a lot) and a few events get dropped so Drools can keep
> > operating. Is this a known or expected behavior from Drools? If not I am
> > happy to post my code, it is similar to the other code sets I've posted
> in
> > the last few days. I'm still new to Drools and trying to learn its
> behavior
> > so any insight or enlightenment is greatly appreciated.
> >
> > Sincerely,
> > Jason
> >
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20130516/3b9b4d2a/attachment.html
More information about the rules-users
mailing list