Last week I found and fixed a very bad bug that seems to fit your case
https://issues.jboss.org/browse/DROOLS-131

This code should work in 5.6.0-SNAPSHOT and later versions.

I have seen DROOLS-138, but won't be able to work on it before the weekend

Davide

On 05/15/2013 11:12 PM, Jason Barto wrote:
Wolfgang,
the code to reproduce is below.  I'm hoping to process between 20k and 50k events through Drools per second thus the extreme high-throughput testing.  I could settle for a single Drools node handling only say 5K per second IFF I could cluster Drools but I've not yet found a way to distribute workload across an active-active Drools cluster (seems there is no such thing?). 

Since you're recommendation I've shifted to using Drools 5.3 just FYI:


### Average.java ###
package drools53fusioneval;

import java.io.IOException;
import java.util.Random;
import org.drools.KnowledgeBase;
import org.drools.KnowledgeBaseConfiguration;
import org.drools.KnowledgeBaseFactory;
import org.drools.builder.KnowledgeBuilder;
import org.drools.builder.KnowledgeBuilderFactory;
import org.drools.builder.ResourceType;
import org.drools.conf.EventProcessingOption;
import org.drools.io.ResourceFactory;
import org.drools.runtime.StatefulKnowledgeSession;
import org.drools.runtime.rule.WorkingMemoryEntryPoint;

class AvgDFEChannel implements org.drools.runtime.Channel {

    @Override
    public void send(Object o) {
        System.err.println("Recieved channel message: " + o);
    }
}

public class Average {

    public static void main(String[] args) throws InterruptedException, IOException {
        KnowledgeBaseConfiguration kbconfig = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
        kbconfig.setOption(EventProcessingOption.STREAM);

        KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase(kbconfig);

        KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();
        kbuilder.add(ResourceFactory.newClassPathResource("drools53fusioneval/basic.drl"), ResourceType.DRL);
        if (kbuilder.hasErrors()) {
            System.err.println(kbuilder.getErrors().toString());
        }
        kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());

        final StatefulKnowledgeSession session = kbase.newStatefulKnowledgeSession();
        session.registerChannel("heartbeat", new AvgDFEChannel());
        WorkingMemoryEntryPoint ep01 = session.getWorkingMemoryEntryPoint("ep01");

        new Thread() {
            public void run() {
                session.fireUntilHalt();
            }
        }.start();

        Thread.sleep(5000); // give the engine time to get setup

        Server hiwaesdk = new Server("hiwaesdk");
        session.insert(hiwaesdk);
       
        long LIMIT = 10000;
        long sentCount = 0;
        int batchSize = 10000;
        Random rnd = new Random(System.nanoTime());
        int temp = 0;
        long startTS = System.nanoTime();
        while (sentCount < LIMIT) {
            for (int i = 0; i < batchSize; i++) {
                temp = rnd.nextInt(212);
                IntEvent evt = new IntEvent (temp);
                ep01.insert(evt);
                sentCount++;
            }
            Thread.sleep (0x1);
        }
        double duration = (System.nanoTime() - startTS)/1000000.0;
        System.out.println(LIMIT +" events generated in "+ duration +" milliseconds");
        System.out.println("Last temperature submitted: "+ temp);
        for (int i = 0; i < 5; i++) {
            System.out.println ("Sec "+ i +": "+ hiwaesdk.currentTemp +", "+ hiwaesdk.readingCount);
            Thread.sleep (1000);
        }
        System.exit(0);
    }
}


### basic.drl ###
package drools53fusioneval

declare IntEvent
    @role ( event )
end

rule "number rule"
when
    $e : IntEvent () from entry-point ep01
    $s : Server (hostname == "hiwaesdk")
then
    $s.currentTemp = $e.data;
    $s.readingCount++;
end




On Thu, May 16, 2013 at 4:25 AM, Wolfgang Laun <wolfgang.laun@gmail.com> wrote:
What you describe is definitely not "expected behaviour" - it's more
like that race condition (or another one) already being in 5.3. Posting
your code might be a good idea ;-)
-W

On 16/05/2013, Jason Barto <jason.p.barto@gmail.com> wrote:
> I've been working to load test Drools 5.3 to determine if its a fit for a
> project.  As part of the test I programmatically insert events as rapidly
> as possible; an example, my earlier test inserted 10k events in about
> 300ms.  There is currently a single rule which reads the event and stores
> the event's value into the only fact in Drools. I'm very happy to report,
> and I'm sure it will be no surprise to anyone, that the engine processes
> all the events in roughly 1 sec. However I have noticed that any large
> number of events (~>1000) usually sees that a small number of events don't
> get processed. I think after 10k events as many as 7 appear to have gone
> unprocessed. If 100 events are inserted, rather than 10k, no events get
> disregarded.  Being new to Drools I can easily accept that my java code or
> DRL code is off or unoptimized in some way. However not knowing how Drools
> does its magic I'm currently inclined to think that Drools gets swamped
> (10k in 300ms is a lot) and a few events get dropped so Drools can keep
> operating. Is this a known or expected behavior from Drools? If not I am
> happy to post my code, it is similar to the other code sets I've posted in
> the last few days. I'm still new to Drools and trying to learn its behavior
> so any insight or enlightenment is greatly appreciated.
>
> Sincerely,
> Jason
>
_______________________________________________
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users