<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">we’ve being auditing the code here, and we’ve found an issue on what happens when a rule is re-matched and the timer updated. Mario is addressing this now, and also re-viewing dropping of output. He’ll post with an update soon.<div><br></div><div>Mark<br><div><div>On 25 Mar 2014, at 16:09, Vieri <<a href="mailto:vieri.emiliani@gmail.com">vieri.emiliani@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">First of all, thanks for the support.<div>Mario, I confirm that the case you provided works fine also at my side. Still, have you tried my first example (using the cron directly to the counting rule)? Can you confirm it is not working, since this was my first concern?</div>
<div>As I said in the previous post, using the CronTrigger pattern greatly improves stability (I managed to run it at 500 eps), but it is not resolutive.</div><div>Adding a few rules, it's enough to go back to instability. </div>
<div><br></div><div><br></div><div>Here's the complete test case. </div><div>Basically, a SynthEvent is created with the meta list populated randomly with up to 4 values picked from a list of 10 strings ("one" to "ten").</div>
<div>For each meta value an Entity object is created (for enumeration purposes).</div><div>Then, rules generate metrics for both events (every 10s) and meta (every 60s). </div><div>Test is ran at 300 events per second +/- 20% (our real use case is closer to some hundreds events per second than some tens).</div>
<div><br></div><div>First of all, the full SynthEvent class:</div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><font face="courier new, monospace">package it.intext.unity.test;</font></div>
</div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">import java.util.Arrays;</font></div><div><div><font face="courier new, monospace">import java.util.Date;</font></div>
</div><div><font face="courier new, monospace">import java.util.List;</font></div><div><br></div><div><font face="courier new, monospace">public class SynthEvent {</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>long id;</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>Date timestamp;</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>List<String> meta;</font></div><div><div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public long getId() {</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>return id;</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public void setId(long id) {</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span><a href="http://this.id/">this.id</a> = id;</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div>
</div><div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public Date getTimestamp() {</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>return timestamp;</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public void setTimestamp(Date timestamp) {</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>this.timestamp = timestamp;</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public List<String> getMeta() {</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>return meta;</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public void setMeta(List<String> meta) {</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>this.meta = meta;</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div>
</div><div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public SynthEvent() {</font></div>
</div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>super();</font></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div>
</div><div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div><div><span style="font-family:'courier new',monospace;white-space:pre"> </span><font face="courier new, monospace"><span style="white-space:pre">@Override
public String toString() {
StringBuilder builder = new StringBuilder();
builder.append("SynthEvent [id=");
builder.append(id);
builder.append(", timestamp=");
builder.append(timestamp);
builder.append(", meta=");
builder.append(meta);
builder.append("]");
return builder.toString();
}</span></font></div><div><font face="courier new, monospace"><span style="white-space:pre"> public String toString() {</span></font></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="courier new, monospace"><span style="white-space:pre"> StringBuilder builder = new StringBuilder();</span></font></div><div><font face="courier new, monospace"><span style="white-space:pre"> builder.append("SynthEvent [id=");</span></font></div>
<div><font face="courier new, monospace"><span style="white-space:pre"> builder.append(id);</span></font></div><div><font face="courier new, monospace"><span style="white-space:pre"> builder.append(", timestamp=");</span></font></div>
<div><font face="courier new, monospace"><span style="white-space:pre"> builder.append(timestamp);</span></font></div><div><font face="courier new, monospace"><span style="white-space:pre"> builder.append(", meta=");</span></font></div>
<div><font face="courier new, monospace"><span style="white-space:pre"> builder.append(meta);</span></font></div><div><font face="courier new, monospace"><span style="white-space:pre"> builder.append("]");</span></font></div>
<div><font face="courier new, monospace"><span style="white-space:pre"> return builder.toString();</span></font></div><div><font face="courier new, monospace"><span style="white-space:pre"> }</span></font> </div><div>
<span style="font-family:'courier new',monospace;white-space:pre"> </span></div><div><font face="courier new, monospace">}</font></div></blockquote><div><br></div><div>Here's the feeding part of the test class:</div>
<div><br></div><div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>private void process(final KieSession session) {</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>new Thread(){</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public void run() {</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>session.fireUntilHalt();</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>};</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}.start();</font></div><div>
<font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>private void feed(final KieSession session) {</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>new Thread(){</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>public void run() {</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>try {</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>int counter = 0;</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>while(true) { </font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>counter++;</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>session.getEntryPoint("synth").insert(createEvent());</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>Thread.sleep(getSleepRate());</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>if ((counter % 1000) == 0) {</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>logger.debug("Total events: {}", counter);</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>} catch (InterruptedException e) {</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>logger.warn("{}", e);</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>};</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}.start();</font></div><div>
<font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>static final int eventThroughput = 300;</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>static final double variance = 0.2;</font></div><div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>protected long getSleepRate() {</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>return (long) (1000.0 / (eventThroughput * ((1.0 - variance) + Math.random() * variance * 2)));</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>protected SynthEvent createEvent() {</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>SynthEvent ret = new SynthEvent();</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ret.setId(System.currentTimeMillis());</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ret.setTimestamp(new Date());</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ret.setMeta(createMeta());</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>return ret;</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>static final List<String> metas = Arrays.asList("one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten");</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>static final int maxMeta = 4;</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>static final int metaSize = metas.size();</font></div>
<div><span class="" style="white-space:pre"><font face="courier new, monospace"> </font></span></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>protected List<String> createMeta() {</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>List<String> ret = new ArrayList<String>();</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>int nofItems = (int) (Math.random() * maxMeta);</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>for (int i=0; i<nofItems; i++) {</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ret.add(metas.get((int) (Math.random() * metaSize)));</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>return ret;</font></div><div>
<font face="courier new, monospace"><span class="" style="white-space:pre"> </span>}</font></div></div><div><br></div><div><br></div><div>And here's the drl, using the CronTrigger pattern:</div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="courier new, monospace">package it.intext.unity.test</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">import it.intext.unity.test.SynthEvent;</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">import java.util.Date;</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">global org.slf4j.Logger logger;</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">declare SynthEvent</font></div><div><font face="courier new, monospace"> @role( event )</font></div><div><font face="courier new, monospace"> @timestamp( timestamp )</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">declare EventCounter</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>@role( event )</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>@timestamp( timestamp )</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>id <span class="" style="white-space:pre"> </span>: long</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>key<span class="" style="white-space:pre"> </span>: String</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>timestamp <span class="" style="white-space:pre"> </span>: Date</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">declare CronTrigger</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>@role( event )</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>@timestamp( timestamp )</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>interval<span class="" style="white-space:pre"> </span>: String</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>timestamp <span class="" style="white-space:pre"> </span>: Date</font></div><div><font face="courier new, monospace">end</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">declare MetricRequest</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>metric<span class="" style="white-space:pre"> </span>: String</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">declare Subscription</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>key<span class="" style="white-space:pre"> </span>: String</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>interval<span class="" style="white-space:pre"> </span>: String</font></div><div><font face="courier new, monospace">end</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><div><br></div><div>declare Entity</div><div><span class="" style="white-space:pre"> </span>key<span class="" style="white-space:pre"> </span>: String</div>
<div>end</div><div><br></div></font></div><div><font face="courier new, monospace">// Setup rules</font></div><div><font face="courier new, monospace">rule "Create subscriptions"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>insert( new Subscription("epm", "10s") );</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Create meta subscriptions 60s"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>Entity( $key : key )</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>insert( new Subscription($key, "60s") );</font></div><div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace">// Cron management</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Cron trigger 10s"</font></div>
<div><font face="courier new, monospace"> timer ( cron: 0/10 * * * * ? )</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>entryPoints["triggers"].insert( new CronTrigger( "10s", new Date() ) );</font></div><div><font face="courier new, monospace">end</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Cron trigger 60s"</font></div><div><font face="courier new, monospace"> timer ( cron: 0/60 * * * * ? )</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>entryPoints["triggers"].insert( new CronTrigger( "60s", new Date() ) );</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Subscription"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>CronTrigger( $interval := interval ) from entry-point "triggers"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>Subscription( $interval := interval, $key : key )</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>entryPoints["requests"].insert( new MetricRequest($key) );</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">// Business rules</font></div><div><font face="courier new, monospace">rule "Create counter"</font></div>
<div><font face="courier new, monospace"> when</font></div><div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>$e : SynthEvent() from entry-point "synth"</font></div><div>
<font face="courier new, monospace"> then</font></div><div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>entryPoints["counters"].insert( new EventCounter( $e.getId(), "event", $e.getTimestamp() ) );</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Create meta counter"</font></div><div><font face="courier new, monospace"> when</font></div>
<div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>$e : SynthEvent() from entry-point "synth"</font></div><div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>$meta : String( ) from $e.getMeta() </font></div>
<div><font face="courier new, monospace"> then</font></div><div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>entryPoints["counters"].insert( new EventCounter( $e.getId(), $meta, $e.getTimestamp() ) );</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Create meta"</font></div><div><font face="courier new, monospace"> when</font></div>
<div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>$e : SynthEvent() from entry-point "synth"</font></div><div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>$meta : String( ) from $e.getMeta() </font></div>
<div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>not( Entity( key == $meta ) ) </font></div><div><font face="courier new, monospace"> then</font></div><div><font face="courier new, monospace"> <span class="" style="white-space:pre"> </span>insert( new Entity($meta) );</font></div>
<div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"> </font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Count epm"</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>$req : MetricRequest( metric == "epm" ) from entry-point "requests"</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> Number( $count : intValue ) from accumulate(</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> EventCounter( key == "event" ) over window:time( 60s ) from entry-point "counters", count(1) )</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> logger.debug("epm = {}", $count );</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> retract( $req );</font></div><div><font face="courier new, monospace">end</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">rule "Count meta"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>when</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>$req : MetricRequest( $key := metric ) from entry-point "requests"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> Number( $count : intValue ) from accumulate(</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> EventCounter( $key := key ) over window:time( 60s ) from entry-point "counters", count(1) )</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>then</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> logger.debug("{} = {}", $key, $count );</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span> retract( $req );</font></div>
<div><font face="courier new, monospace">end</font></div><div><br></div></blockquote><div><br></div><div><br></div><div>Here's the log of a quick-failing run:</div><div><br></div><div><div>[DEBUG] 2014-03-25 16:28:04.769 (StreamTester.java:run:68) Total events: 1000</div>
<div>[DEBUG] 2014-03-25 16:28:07.774 (StreamTester.java:run:68) Total events: 2000</div><div>[DEBUG] 2014-03-25 16:28:10.100 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 2769</div><div>[DEBUG] 2014-03-25 16:28:10.723 (StreamTester.java:run:68) Total events: 3000</div>
<div>[DEBUG] 2014-03-25 16:28:13.681 (StreamTester.java:run:68) Total events: 4000</div><div>[DEBUG] 2014-03-25 16:28:16.668 (StreamTester.java:run:68) Total events: 5000</div><div>[DEBUG] 2014-03-25 16:28:19.541 (StreamTester.java:run:68) Total events: 6000</div>
<div>[DEBUG] 2014-03-25 16:28:20.054 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 6164</div><div>[DEBUG] 2014-03-25 16:28:22.421 (StreamTester.java:run:68) Total events: 7000</div><div>[DEBUG] 2014-03-25 16:28:25.336 (StreamTester.java:run:68) Total events: 8000</div>
<div>[DEBUG] 2014-03-25 16:28:28.217 (StreamTester.java:run:68) Total events: 9000</div><div>[DEBUG] 2014-03-25 16:28:30.030 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 9627</div><div>[DEBUG] 2014-03-25 16:28:31.088 (StreamTester.java:run:68) Total events: 10000</div>
<div>[DEBUG] 2014-03-25 16:28:33.968 (StreamTester.java:run:68) Total events: 11000</div><div>[DEBUG] 2014-03-25 16:28:36.904 (StreamTester.java:run:68) Total events: 12000</div><div>[DEBUG] 2014-03-25 16:28:39.782 (StreamTester.java:run:68) Total events: 13000</div>
<div>[DEBUG] 2014-03-25 16:28:40.024 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 13077</div><div>[DEBUG] 2014-03-25 16:28:42.736 (StreamTester.java:run:68) Total events: 14000</div><div>[DEBUG] 2014-03-25 16:28:45.615 (StreamTester.java:run:68) Total events: 15000</div>
<div>[DEBUG] 2014-03-25 16:28:48.459 (StreamTester.java:run:68) Total events: 16000</div><div>[DEBUG] 2014-03-25 16:28:50.030 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 16538</div><div>[DEBUG] 2014-03-25 16:28:51.412 (StreamTester.java:run:68) Total events: 17000</div>
<div>[DEBUG] 2014-03-25 16:28:54.293 (StreamTester.java:run:68) Total events: 18000</div><div>[DEBUG] 2014-03-25 16:28:57.549 (StreamTester.java:run:68) Total events: 19000</div><div>[DEBUG] 2014-03-25 16:29:00.021 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 19860</div>
<div>[DEBUG] 2014-03-25 16:29:00.211 (Rule_Count_meta1429890510.java:defaultConsequence:14) two = 2972</div><div>[DEBUG] 2014-03-25 16:29:00.214 (Rule_Count_meta1429890510.java:defaultConsequence:14) eight = 3099</div><div>
[DEBUG] 2014-03-25 16:29:00.215 (Rule_Count_meta1429890510.java:defaultConsequence:14) ten = 3008</div><div>[DEBUG] 2014-03-25 16:29:00.215 (Rule_Count_meta1429890510.java:defaultConsequence:14) three = 2982</div><div>[DEBUG] 2014-03-25 16:29:00.216 (Rule_Count_meta1429890510.java:defaultConsequence:14) six = 3006</div>
<div>[DEBUG] 2014-03-25 16:29:00.216 (Rule_Count_meta1429890510.java:defaultConsequence:14) five = 2959</div><div>[DEBUG] 2014-03-25 16:29:00.217 (Rule_Count_meta1429890510.java:defaultConsequence:14) one = 2934</div><div>
[DEBUG] 2014-03-25 16:29:00.217 (Rule_Count_meta1429890510.java:defaultConsequence:14) seven = 3013</div><div>[DEBUG] 2014-03-25 16:29:00.218 (Rule_Count_meta1429890510.java:defaultConsequence:14) four = 2973</div><div>[DEBUG] 2014-03-25 16:29:00.218 (Rule_Count_meta1429890510.java:defaultConsequence:14) nine = 3012</div>
<div>[DEBUG] 2014-03-25 16:29:00.510 (StreamTester.java:run:68) Total events: 20000</div><div>[DEBUG] 2014-03-25 16:29:03.671 (StreamTester.java:run:68) Total events: 21000</div><div>[DEBUG] 2014-03-25 16:29:06.798 (StreamTester.java:run:68) Total events: 22000</div>
<div>[DEBUG] 2014-03-25 16:29:09.969 (StreamTester.java:run:68) Total events: 23000</div><div>[DEBUG] 2014-03-25 16:29:10.045 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 20242</div><div>[DEBUG] 2014-03-25 16:29:13.362 (StreamTester.java:run:68) Total events: 24000</div>
<div>[DEBUG] 2014-03-25 16:29:16.352 (StreamTester.java:run:68) Total events: 25000</div><div>[DEBUG] 2014-03-25 16:29:19.400 (StreamTester.java:run:68) Total events: 26000</div><div>[DEBUG] 2014-03-25 16:29:20.849 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 20029</div>
<div>[DEBUG] 2014-03-25 16:29:22.795 (StreamTester.java:run:68) Total events: 27000</div><div>[DEBUG] 2014-03-25 16:29:25.785 (StreamTester.java:run:68) Total events: 28000</div><div>[DEBUG] 2014-03-25 16:29:28.855 (StreamTester.java:run:68) Total events: 29000</div>
<div><b>[DEBUG] 2014-03-25 16:29:30.018 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 18647</b></div><div>[DEBUG] 2014-03-25 16:29:31.858 (StreamTester.java:run:68) Total events: 30000</div><div>[DEBUG] 2014-03-25 16:29:34.983 (StreamTester.java:run:68) Total events: 31000</div>
<div>[DEBUG] 2014-03-25 16:29:37.947 (StreamTester.java:run:68) Total events: 32000</div><div>[DEBUG] 2014-03-25 16:29:40.011 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 15201</div><div>[DEBUG] 2014-03-25 16:29:40.928 (StreamTester.java:run:68) Total events: 33000</div>
<div>[DEBUG] 2014-03-25 16:29:43.920 (StreamTester.java:run:68) Total events: 34000</div><div>[DEBUG] 2014-03-25 16:29:46.911 (StreamTester.java:run:68) Total events: 35000</div><div>[DEBUG] 2014-03-25 16:29:50.011 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 11740</div>
<div>[DEBUG] 2014-03-25 16:29:50.023 (StreamTester.java:run:68) Total events: 36000</div><div>[DEBUG] 2014-03-25 16:29:52.998 (StreamTester.java:run:68) Total events: 37000</div><div>[DEBUG] 2014-03-25 16:29:55.925 (StreamTester.java:run:68) Total events: 38000</div>
<div>[DEBUG] 2014-03-25 16:29:58.845 (StreamTester.java:run:68) Total events: 39000</div><div>[DEBUG] 2014-03-25 16:30:00.009 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 8417</div><div>[DEBUG] 2014-03-25 16:30:00.038 (Rule_Count_meta1429890510.java:defaultConsequence:14) two = 2606</div>
<div>[DEBUG] 2014-03-25 16:30:00.039 (Rule_Count_meta1429890510.java:defaultConsequence:14) eight = 2534</div><div>[DEBUG] 2014-03-25 16:30:00.039 (Rule_Count_meta1429890510.java:defaultConsequence:14) ten = 2763</div><div>
[DEBUG] 2014-03-25 16:30:00.040 (Rule_Count_meta1429890510.java:defaultConsequence:14) three = 2623</div><div>[DEBUG] 2014-03-25 16:30:00.040 (Rule_Count_meta1429890510.java:defaultConsequence:14) six = 2641</div><div>[DEBUG] 2014-03-25 16:30:00.040 (Rule_Count_meta1429890510.java:defaultConsequence:14) five = 2579</div>
<div>[DEBUG] 2014-03-25 16:30:00.040 (Rule_Count_meta1429890510.java:defaultConsequence:14) one = 2690</div><div>[DEBUG] 2014-03-25 16:30:00.041 (Rule_Count_meta1429890510.java:defaultConsequence:14) seven = 2698</div><div>
[DEBUG] 2014-03-25 16:30:00.041 (Rule_Count_meta1429890510.java:defaultConsequence:14) four = 2658</div><div>[DEBUG] 2014-03-25 16:30:00.041 (Rule_Count_meta1429890510.java:defaultConsequence:14) nine = 2669</div><div>[DEBUG] 2014-03-25 16:30:01.735 (StreamTester.java:run:68) Total events: 40000</div>
<div>[DEBUG] 2014-03-25 16:30:04.682 (StreamTester.java:run:68) Total events: 41000</div><div>[DEBUG] 2014-03-25 16:30:07.604 (StreamTester.java:run:68) Total events: 42000</div><div>[DEBUG] 2014-03-25 16:30:10.011 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 5266</div>
<div>[DEBUG] 2014-03-25 16:30:10.622 (StreamTester.java:run:68) Total events: 43000</div><div>[DEBUG] 2014-03-25 16:30:13.493 (StreamTester.java:run:68) Total events: 44000</div><div>[DEBUG] 2014-03-25 16:30:16.383 (StreamTester.java:run:68) Total events: 45000</div>
<div>[DEBUG] 2014-03-25 16:30:19.263 (StreamTester.java:run:68) Total events: 46000</div><div>[DEBUG] 2014-03-25 16:30:20.012 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 2078</div><div>[DEBUG] 2014-03-25 16:30:22.146 (StreamTester.java:run:68) Total events: 47000</div>
<div>[DEBUG] 2014-03-25 16:30:25.036 (StreamTester.java:run:68) Total events: 48000</div><div>[DEBUG] 2014-03-25 16:30:27.911 (StreamTester.java:run:68) Total events: 49000</div><div>[DEBUG] 2014-03-25 16:30:30.003 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 0</div>
<div>[DEBUG] 2014-03-25 16:30:30.800 (StreamTester.java:run:68) Total events: 50000</div></div><div><br></div><div>At 250 eps it took about 56K events (3'30") to hang (short log below)</div><div><br></div><div><div>
[DEBUG] 2014-03-25 16:37:20.013 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 16084</div><div>[DEBUG] 2014-03-25 16:37:23.222 (StreamTester.java:run:68) Total events: 52000</div><div>[DEBUG] 2014-03-25 16:37:26.751 (StreamTester.java:run:68) Total events: 53000</div>
<div>[DEBUG] 2014-03-25 16:37:30.069 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 16085</div><div>[DEBUG] 2014-03-25 16:37:30.510 (StreamTester.java:run:68) Total events: 54000</div><div>[DEBUG] 2014-03-25 16:37:34.099 (StreamTester.java:run:68) Total events: 55000</div>
<div>[DEBUG] 2014-03-25 16:37:37.701 (StreamTester.java:run:68) Total events: 56000</div><div><b>[DEBUG] 2014-03-25 16:37:40.014 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 15381</b></div><div>[DEBUG] 2014-03-25 16:37:41.381 (StreamTester.java:run:68) Total events: 57000</div>
<div>[DEBUG] 2014-03-25 16:37:44.977 (StreamTester.java:run:68) Total events: 58000</div><div>[DEBUG] 2014-03-25 16:37:48.632 (StreamTester.java:run:68) Total events: 59000</div><div>[DEBUG] 2014-03-25 16:37:50.011 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 12677</div>
<div>[DEBUG] 2014-03-25 16:37:52.343 (StreamTester.java:run:68) Total events: 60000</div><div>[DEBUG] 2014-03-25 16:37:55.958 (StreamTester.java:run:68) Total events: 61000</div><div>[DEBUG] 2014-03-25 16:37:59.546 (StreamTester.java:run:68) Total events: 62000</div>
<div>[DEBUG] 2014-03-25 16:38:00.009 (Rule_Count_epm701941061.java:defaultConsequence:14) epm = 10033</div></div><div><br></div><div>At 200 eps the system hanged after 185K events (more than 16 minutes). </div><div><br></div>
<div>More realish examples (e.g. counting how many distinct tweets get at least one retweet [<i>far less than 10%, BTW</i>]) hang in less than 10 minutes even at 20 events per second or less.</div><div><br></div><div>Hope this helps. Please guys, ask me for more details at any moment. </div>
<div>I would be definitely glad to solve this issue and to go on with our Drools Fusion roadmap ;)</div><div>Thanks in advance,</div><div>Vieri</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On 25 March 2014 14:09, Mario Fusco <span dir="ltr"><<a href="mailto:mario.fusco@gmail.com" target="_blank">mario.fusco@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
I am trying to reproduce the issue you reported, but so far I couldn't. This<br>
is drl I am using:<br>
<br>
import org.drools.test.SynthEvent<br>
import java.util.Date<br>
<br>
declare SynthEvent<br>
@role( event )<br>
@timestamp( timestamp )<br>
end<br>
<br>
declare EventCounter<br>
@role( event )<br>
@timestamp( timestamp )<br>
id : long<br>
key : String<br>
timestamp : Date<br>
end<br>
<br>
declare CronTrigger end<br>
<br>
rule "Trigger Metric"<br>
timer ( cron: 0/10 * * * * ? )\n<br>
when<br>
then<br>
entryPoints["triggers"].insert( new CronTrigger() );<br>
end<br>
<br>
rule "Create counter"<br>
when<br>
$e : SynthEvent() from entry-point "synth"<br>
then<br>
entryPoints["counters"].insert(new EventCounter( $e.getId(), "event",<br>
$e.getTimestamp() ) );<br>
end<br>
<br>
rule "Count epm"<br>
when<br>
$trigger : CronTrigger() from entry-point "triggers"<br>
Number( $count : intValue ) from accumulate(<br>
EventCounter( key == "event" ) over window:time( 60s )from entry-point<br>
"counters", count(1) )<br>
then<br>
System.out.println("[" + new Date() + "] epm = " + $count );<br>
retract($trigger);<br>
end<br>
<br>
and this is how I am using it:<br>
<br>
final KieSession ksession = kbase.newKieSession();<br>
EntryPoint synthEP = ksession.getEntryPoint("synth");<br>
<br>
new Thread(){<br>
public void run() {<br>
ksession.fireUntilHalt();<br>
};<br>
}.start();<br>
<br>
long counter = 0;<br>
while(true) {<br>
counter++;<br>
synthEP.insert(new SynthEvent(counter));<br>
try {<br>
Thread.sleep(20L);<br>
} catch (InterruptedException e) {<br>
throw new RuntimeException(e);<br>
}<br>
if ((counter % 1000) == 0) {<br>
System.out.println("Total events: " + counter);<br>
}<br>
}<br>
<br>
Note that since you mentioned you expected around 3K events per minute I am<br>
doing a sleep of 20 msec between the insertion of 2 consecutive events. Is<br>
that correct?<br>
<br>
I let this run for 10 minutes and I got the output I am pasting at the end<br>
of this email, so I think that it is working as expected. Please let me know<br>
if you'll give a try at my test case and if it works correctly also for you.<br>
Also if you find any further problem feel free to send a reproducer so I<br>
could give it a try.<br>
<br>
Regards,<br>
Mario<br>
<br>
[Tue Mar 25 13:37:30 CET 2014] epm = 318<br>
[Tue Mar 25 13:37:40 CET 2014] epm = 774<br>
Total events: 1000<br>
[Tue Mar 25 13:37:50 CET 2014] epm = 1259<br>
[Tue Mar 25 13:38:00 CET 2014] epm = 1746<br>
Total events: 2000<br>
[Tue Mar 25 13:38:10 CET 2014] epm = 2233<br>
[Tue Mar 25 13:38:20 CET 2014] epm = 2720<br>
Total events: 3000<br>
[Tue Mar 25 13:38:30 CET 2014] epm = 2890<br>
[Tue Mar 25 13:38:40 CET 2014] epm = 2922<br>
Total events: 4000<br>
[Tue Mar 25 13:38:50 CET 2014] epm = 2926<br>
[Tue Mar 25 13:39:00 CET 2014] epm = 2929<br>
Total events: 5000<br>
[Tue Mar 25 13:39:10 CET 2014] epm = 2933<br>
[Tue Mar 25 13:39:20 CET 2014] epm = 2938<br>
Total events: 6000<br>
[Tue Mar 25 13:39:30 CET 2014] epm = 2944<br>
[Tue Mar 25 13:39:40 CET 2014] epm = 2948<br>
Total events: 7000<br>
[Tue Mar 25 13:39:50 CET 2014] epm = 2952<br>
[Tue Mar 25 13:40:00 CET 2014] epm = 2956<br>
Total events: 8000<br>
[Tue Mar 25 13:40:10 CET 2014] epm = 2958<br>
[Tue Mar 25 13:40:20 CET 2014] epm = 2959<br>
Total events: 9000<br>
[Tue Mar 25 13:40:30 CET 2014] epm = 2959<br>
[Tue Mar 25 13:40:40 CET 2014] epm = 2960<br>
Total events: 10000<br>
[Tue Mar 25 13:40:50 CET 2014] epm = 2960<br>
[Tue Mar 25 13:41:00 CET 2014] epm = 2960<br>
Total events: 11000<br>
[Tue Mar 25 13:41:10 CET 2014] epm = 2961<br>
[Tue Mar 25 13:41:20 CET 2014] epm = 2963<br>
Total events: 12000<br>
[Tue Mar 25 13:41:30 CET 2014] epm = 2964<br>
[Tue Mar 25 13:41:40 CET 2014] epm = 2964<br>
Total events: 13000<br>
[Tue Mar 25 13:41:50 CET 2014] epm = 2966<br>
[Tue Mar 25 13:42:00 CET 2014] epm = 2966<br>
Total events: 14000<br>
[Tue Mar 25 13:42:10 CET 2014] epm = 2967<br>
[Tue Mar 25 13:42:20 CET 2014] epm = 2964<br>
Total events: 15000<br>
[Tue Mar 25 13:42:30 CET 2014] epm = 2965<br>
[Tue Mar 25 13:42:40 CET 2014] epm = 2966<br>
Total events: 16000<br>
[Tue Mar 25 13:42:50 CET 2014] epm = 2965<br>
[Tue Mar 25 13:43:00 CET 2014] epm = 2967<br>
Total events: 17000<br>
[Tue Mar 25 13:43:10 CET 2014] epm = 2966<br>
[Tue Mar 25 13:43:20 CET 2014] epm = 2969<br>
Total events: 18000<br>
[Tue Mar 25 13:43:30 CET 2014] epm = 2969<br>
[Tue Mar 25 13:43:40 CET 2014] epm = 2969<br>
[Tue Mar 25 13:43:50 CET 2014] epm = 2970<br>
Total events: 19000<br>
[Tue Mar 25 13:44:00 CET 2014] epm = 2969<br>
[Tue Mar 25 13:44:10 CET 2014] epm = 2970<br>
Total events: 20000<br>
[Tue Mar 25 13:44:20 CET 2014] epm = 2969<br>
[Tue Mar 25 13:44:30 CET 2014] epm = 2969<br>
Total events: 21000<br>
[Tue Mar 25 13:44:40 CET 2014] epm = 2969<br>
[Tue Mar 25 13:44:50 CET 2014] epm = 2969<br>
Total events: 22000<br>
[Tue Mar 25 13:45:00 CET 2014] epm = 2969<br>
[Tue Mar 25 13:45:10 CET 2014] epm = 2969<br>
Total events: 23000<br>
[Tue Mar 25 13:45:20 CET 2014] epm = 2969<br>
[Tue Mar 25 13:45:30 CET 2014] epm = 2968<br>
Total events: 24000<br>
[Tue Mar 25 13:45:40 CET 2014] epm = 2969<br>
[Tue Mar 25 13:45:50 CET 2014] epm = 2969<br>
Total events: 25000<br>
[Tue Mar 25 13:46:00 CET 2014] epm = 2969<br>
[Tue Mar 25 13:46:10 CET 2014] epm = 2969<br>
Total events: 26000<br>
[Tue Mar 25 13:46:20 CET 2014] epm = 2968<br>
[Tue Mar 25 13:46:30 CET 2014] epm = 2969<br>
Total events: 27000<br>
[Tue Mar 25 13:46:40 CET 2014] epm = 2968<br>
[Tue Mar 25 13:46:50 CET 2014] epm = 2969<br>
Total events: 28000<br>
[Tue Mar 25 13:47:00 CET 2014] epm = 2969<br>
[Tue Mar 25 13:47:10 CET 2014] epm = 2969<br>
Total events: 29000<br>
[Tue Mar 25 13:47:20 CET 2014] epm = 2969<br>
<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href="http://drools.46999.n3.nabble.com/rules-users-Drools-Fusion-inconsistencies-at-increasing-event-throughputs-tp4028947p4028955.html" target="_blank">http://drools.46999.n3.nabble.com/rules-users-Drools-Fusion-inconsistencies-at-increasing-event-throughputs-tp4028947p4028955.html</a><br>
Sent from the Drools: User forum mailing list archive at <a href="http://Nabble.com">Nabble.com</a>.<br>
_______________________________________________<br>
rules-users mailing list<br>
<a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/rules-users" target="_blank">https://lists.jboss.org/mailman/listinfo/rules-users</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Vieri Emiliani<br><a href="mailto:vieri.emiliani@gmail.com">vieri.emiliani@gmail.com</a><br>+39 335 1293649
</div>
_______________________________________________<br>rules-users mailing list<br><a href="mailto:rules-users@lists.jboss.org">rules-users@lists.jboss.org</a><br>https://lists.jboss.org/mailman/listinfo/rules-users</blockquote></div><br></div></body></html>