[rules-users] Sliding Windows - Error

[Edson Tirelli]

   Hi Paul,

   Unfortunately, the bottom line is that there is no magic. Time windows
are always based on the concept of "current time" and to be able to join
different streams of events Drools took the decision of having a
session-scoped clock.

   The major problem of a fact timestamp be in the "past" is that it might
miss the time window. But since we are taking decisions in near real time,
the decisions for that window are already taken anyway, so not much that can
be done if that happens.

   You asked for usual approaches. There are 2 general approaches that I
know and the variations of them:

1. is to synchronize the incoming event streams. This is possible and common
on playback systems, like telco rating/billing just to mention one example.
This is not possible on live monitoring systems.

2. is to use a controlled clock based on the arriving of events. In this
case, the clock is advanced based on the events arriving. Example: the clock
could be advanced based on the most recent timestamp among all events that
arrived, or based on the timestamp of a specific type of event that is
handled like a heartbeat event.

   In the end, it obviously depends on the use case.

   Others might have additional suggestions, though.

   Edson

2010/3/27 Paul R. <reverselogic at gmail.com>

> Hi Edson.
>
> Many thanks for the response. Firing the rules before advancing the clock
> fixes the problem. However,
> I'm now looking for suggestions on how I can use time windows - when I have
> events arriving from multiple
> unsynchronized streams :)
>
> For example, I have two events StreamOne and StreamTwo, which have their
> own timestamp properties,
> both of these events are inserted into the a single session.
>
> StreamOne Data.
>
> timestamp = 2010-03-27 04:20:05, x = 0, y = 0
> timestamp = 2010-03-27 04:20:10, x = 1, y = 0
> timestamp = 2010-03-27 04:20:15, x = 1, y = 0
> timestamp = 2010-03-27 04:20:20, x = 1, y = 0
> timestamp = 2010-03-27 04:20:25, x = 0, y = 0
>
> StreamTwo Data.
>
> timestamp = 2010-03-27 04:20:10, x = 0, y = 0
> timestamp = 2010-03-27 04:20:15, x = 1, y = 1
> timestamp = 2010-03-27 04:20:20, x = 1, y = 1
> timestamp = 2010-03-27 04:20:25, x = 0, y = 1
> timestamp = 2010-03-27 04:20:30, x = 0, y = 1
>
> I would like to use simple rules like the following to capture particular
> sequences of data.
>
> Number(intValue > 2) from accumulate
>         $f : StreamOne(x == 1)
>             over window:time ( 15s )
>
>             from entry-point ChannelDataStream,
>         count($f))
>
>
> Number(intValue > 2) from accumulate
>         $f : StreamOne(y == 1)
>             over window:time ( 15s )
>
>             from entry-point ChannelDataStream,
>         count($f))
>
> Because the timestamps are in the past, and because of the fact that
> multiple instances of the
> same event type (different timestamps) may arrive at the same time, due to
> network latency, etc.
> This rules out the use of the real-time clock.
>
> I also can't see how I can reliably use the pseudo-clock because of the
> lack of synchronization
> of the timestamps between the two streams?
>
> Any suggestions greatly appreciated!
>
> Thanks and Regards,
>
> Paul
>
>
>
>
>
>
>
>
> 2010/3/26 Edson Tirelli <ed.tirelli at gmail.com>
>
>
>>    Did you tried firing the rules before advancing the time clock?
>>
>>    This is a very tricky situation because you are working with the edge
>> limits of the time window. Events are expired from the time window when the
>> clock advances. You are inserting events every 5 seconds. So, lets say you
>> insert events on T0, T5, T10, T15, T20, T25... at this point in time the
>> rule activates, but before it fires, you advance the clock to T30, when T0
>> is expired from the window, making the condition intValue > 5 false and
>> deactivating the rule.
>>
>>     []s
>>     Edson
>>
>> 2010/3/25 Paul R. <reverselogic at gmail.com>
>>
>>>  Hi,
>>>
>>> I'm using an accumulate on a sliding window to count the number of
>>> events, which occur in a 30 second window;
>>> the events are inserted every 5 seconds - so with a 30 second window, I
>>> would expect 6 events to occur.
>>>
>>> The rule below never fires, though from the log output, it looks as
>>> though the condition has matched successfully?
>>>
>>> declare Foo
>>>     @role ( event )
>>>     @expires ( 60s )
>>>     @timestamp ( date )
>>> end
>>>
>>> rule "Count Foo"
>>> when
>>>     c : Number(intValue > 5) from accumulate
>>>         $f : Foo()
>>>             over window:time ( 30s )
>>>             from entry-point ChannelDataStream,
>>>         count($f))
>>> then
>>>     System.out.println("Count [30s] = " + c);
>>> end
>>>
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at c52200 factId: 1
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at 128edf2 factId: 3
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at 1dddba factId: 4
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at c7e8a7 factId: 5
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at 7b4703 factId: 6
>>> ACTIVATION CREATED rule:Count Foo activationId:Count Foo [2, 0]
>>> declarations: c=6(2)
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at 1732ed2 factId: 7
>>> ACTIVATION CANCELLED rule:Count Foo activationId:Count Foo [2, 0]
>>> declarations: c=6(2)
>>> ACTIVATION CREATED rule:Count Foo activationId:Count Foo [2, 0]
>>> declarations: c=6(2)
>>> OBJECT ASSERTED value:com.test.DroolsTest$Foo at be76c7 factId: 8
>>> ACTIVATION CANCELLED rule:Count Foo activationId:Count Foo [2, 0]
>>> declarations: c=6(2)
>>> ACTIVATION CREATED rule:Count Foo activationId:Count Foo [2, 0]
>>> declarations: c=6(2)
>>>
>>>
>>> Can anybody explain why this is happening? I've attached a test case,
>>> which demonstrates the problem.
>>>
>>> Thanks in advance.
>>>
>>> - Paul
>>>
>>> _______________________________________________
>>> rules-users mailing list
>>> rules-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>
>>>
>>
>>
>> --
>>  Edson Tirelli
>>  JBoss Drools Core Development
>>  JBoss by Red Hat @ www.jboss.com
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>


-- 
 Edson Tirelli
 JBoss Drools Core Development
 JBoss by Red Hat @ www.jboss.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20100330/9cc18dda/attachment.html