<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>Hello everyone,</div>
<div> </div>
<div>i'm having some design problems with Drools Fusion and CEP. Maybe you can guide me into the right direction.<br/>
(Drools 5.6)</div>
<div> </div>
<div> </div>
<div> </div>
<div>Problem:<br/>
I want to use Fusion CEP to handle a quite large amount of events (25-50 thousand per minute as a start). The events don't have to live very long so I plan to expire them very soon.<br/>
My rule is pretty simple:<br/>
There is only one event type (which has a string value field in it).<br/>
The rule should fire when there are X events with the same value Y in time window Z.<br/>
The effect of the rule should be something like an Alarm which will be extracted from outside the rule engine.</div>
<div> </div>
<div><br/>
I found some similar examples (and also studies the documentation) but the race condition (see down below) destroys all my plans.</div>
<div> </div>
<div> </div>
<div> </div>
<div>ATM it looks like this:<br/>
declare MyEvent<br/>
@role(event)<br/>
@expires( 30s )<br/>
end</div>
<div> </div>
<div>declare Alarm<br/>
@role(event)<br/>
end</div>
<div> </div>
<div> </div>
<div>rule "RULE 1"<br/>
when<br/>
$e : MyEvent( ) from entry-point entryone<br/>
// not Alarm(eventValue == $e.getEventValue())<br/>
Number( $count : intValue, $count > 30) from accumulate( $i : (MyEvent ( eventValue == $e.getEventValue() ) over window:time(15s) from entry-point entryone), count( $i ) )<br/>
then<br/>
// insert( new Alarm($e.getEventValue(), $count));<br/>
System.out.println("rule 1. value="+$e.getEventValue() +" count=" + $count);<br/>
end</div>
<div> </div>
<div>-> When there are more than 30 "same" events (same value) in the last 15 seconds.</div>
<div>MyEvent and Alarm are simple POJOs. Only fields, getter, setter, ... (value is a String, count is a Integer)</div>
<div> </div>
<div> </div>
<div> </div>
<div> </div>
<div>The rule like this will fire for EVERY event -> when there are 30 similar events then the log message is printed 30 times.<br/>
I'm unhappy with this situation because I actually only need the RHS triggered only once for each value. Secondly I thought that the performance would be better when firing only once.</div>
<div>So I introduced the Alarm object which is created after the rule triggers the first time for the given value. (the two commented lines)<br/>
When the rule triggers the first time for value X, I create an Alarm object for this value. Then the "not Alarm(...)" should prevent the execution.</div>
<div>This works for the first few thousand events. Then I receive a race condition exception. Occurs usually between event 2000 - 10000.</div>
<div>Exception in thread "main" org.drools.RuntimeDroolsException: Unexpected exception executing action org.drools.reteoo.PropagationQueuingNode$PropagateAction@61f0b700<br/>
at org.drools.common.AbstractWorkingMemory.executeQueuedActions(AbstractWorkingMemory.java:1047)<br/>
at org.drools.reteoo.WindowNode.assertObject(WindowNode.java:182)<br/>
at org.drools.reteoo.CompositeObjectSinkAdapter.doPropagateAssertObject(CompositeObjectSinkAdapter.java:497)<br/>
at org.drools.reteoo.CompositeObjectSinkAdapter.propagateAssertObject(CompositeObjectSinkAdapter.java:382)<br/>
at org.drools.reteoo.ObjectTypeNode.assertObject(ObjectTypeNode.java:302)<br/>
at org.drools.reteoo.EntryPointNode.assertObject(EntryPointNode.java:254)<br/>
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:366)<br/>
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:327)<br/>
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:130)<br/>
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:58)<br/>
at ........myClass.main(...)<br/>
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br/>
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)<br/>
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br/>
at java.lang.reflect.Method.invoke(Method.java:597)<br/>
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)<br/>
Caused by: java.lang.NullPointerException<br/>
at org.drools.reteoo.ReteooWorkingMemory$EvaluateResultConstraints.execute(ReteooWorkingMemory.java:593)<br/>
at org.drools.common.PropagationContextImpl.evaluateActionQueue(PropagationContextImpl.java:364)<br/>
at org.drools.reteoo.PropagationQueuingNode$AssertAction.execute(PropagationQueuingNode.java:449)<br/>
at org.drools.reteoo.PropagationQueuingNode.propagateActions(PropagationQueuingNode.java:289)<br/>
at org.drools.reteoo.PropagationQueuingNode$PropagateAction.execute(PropagationQueuingNode.java:611)<br/>
at org.drools.common.AbstractWorkingMemory.executeQueuedActions(AbstractWorkingMemory.java:1045)<br/>
... 16 more</div>
<div> </div>
<div> </div>
<div>I'm adding a few thousand events into the knowledge session. Then let the thread sleep for a few seconds for drools to catch up. I tried this with various combinations and always received the exception.<br/>
It works when trying for a few hundred events. (rule triggers as planed)</div>
<div> </div>
<div> </div>
<div>for the sake of completeness:<br/>
My Java class that starts the knowledge session and inserts the events:<br/>
KnowledgeBaseConfiguration config = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();<br/>
config.setOption(EventProcessingOption.STREAM);<br/>
kbase = KnowledgeBaseFactory.newKnowledgeBase(config);<br/>
kbase.addKnowledgePackages(pkgs);</div>
<div> KnowledgeSessionConfiguration conf = KnowledgeBaseFactory.newKnowledgeSessionConfiguration();<br/>
ksession = kbase.newStatefulKnowledgeSession(conf, null);<br/>
entryPoint1 = ksession.getWorkingMemoryEntryPoint("entryone");<br/>
new Thread() {<br/>
@Override<br/>
public void run() {<br/>
ksession.fireUntilHalt();<br/>
}<br/>
}.start();</div>
<div><br/>
...<br/>
reader = new BufferedReader(new FileReader("50k.log"));<br/>
String line = reader.readLine();<br/>
long i = 0;<br/>
while (line != null) {<br/>
MyEvent entry = parser.parse(line.trim());<br/>
if ( entry != null ) {<br/>
entryPoint1.insert(entry);<br/>
}<br/>
line = reader.readLine();</div>
<div> if ( ++i % 5000 == 0 ) {<br/>
System.out.println("Inserter: " + i + " lines inserted. - " + sdf.format(new Date()));<br/>
Thread.sleep(5000);<br/>
}<br/>
}</div>
<div><br/>
</div>
<div>My main question: <br/>
Apart from the race condition problem, are there any other ways I could solve my problem?</div>
<div> </div>
<div>E.g.<br/>
* Do you think that I shouldn't care about "not Alarm(eventValue == $e.getEventValue())" and just live with the result that the program, that uses the knowledge base, has to deal with lots of identical Alarm events? (And has to perform the accumulate more often)<br/>
* Is there an easier way to count the events?<br/>
* Is Drools overburdened with the number of events?<br/>
* Is Drools Fusion/CEP not designed for my problem and other tools are more useful instead? Which ones?</div>
<div> </div>
<div>I would appreciate any ideas.</div>
<div> </div>
<div>Sincerely,<br/>
Martin</div>
<div> </div>
<div> </div>
</div></div></body></html>