[rules-users] Drools and event based decisions, i.e. streaming market quotes

[Henry Canterburry]

Mark, at this point I have a semi-working drools integration utilizing 
the stateful session. With each event coming in, I insert the event into 
the session and call fireAllRules. I have also added memory managment 
rules to retract events past a certain time threshold or number of 
instances in memory.

I have a few thoughts however:
Events come in completely asynchronously and rapidly. What happens if a 
new fact enters the engine before all rules for the previous fact have 
finished evaluating/firing? Since I always have a certain number of past 
events kept in WM for history checking purposes, my key event processing 
rules are specifically written to target the "latest" instance of an 
event. Given the rapid speed at which new events enter the engine, might 
I encounter situations where rules later in an inference chain for a 
prior event are evaluating derived data partially from the previous 
event and from the latest event depending on what new facts have been 
established? (sort of a race condition). Or...will the inference chain 
for the previous fact always be allowed to finish before a new inference 
chain for the latest event is begun?

Thanks
HC

Mark Proctor wrote:
> Henry Canterburry wrote:
>> Mark,
>>
>> Thanks for the reply. My timeframe matches the one you mentioned 
>> although I would naturally be looking for working and production 
>> ready functionality. In the meantime, did any of the approaches I 
>> mentioned make any sense or would you say that drools is not the 
>> right tool at this point? I do have helper classes and other 
>> intermediary data caching approaches at this time that permit me to 
>> query past data and an entire streaming architecture to feed data.
>>
>> Thanks
>> HC
>>
>>
>> Mark Proctor wrote:
>>> What's your time scale for this? We are currently implementing CEP 
>>> extendions to the language and engine which will do all of this for 
>>> you, we'll have a milestone release out in february when you can 
>>> first play with this and aiming to get a full release end of Q1 
>>> start of Q2.
>>>
>>> Mark
>>> Henry Canterburry wrote:
>>>> What would be the best approach using drools to handle event driven 
>>>> decisions based on streaming data? In my case I am looking for a 
>>>> typical stock market scenario. Ticker quotes usually come in at 
>>>> second increments and depending on how many ticker symbols you 
>>>> subscribe to at any one time, there can be a lot of data coming and 
>>>> and changing every second. However, there probably isn't a need to 
>>>> keep large quantities of historic data in memory...maybe the last 
>>>> 200-500 ticks. The outputs are if a stock should be sold or bought, 
>>>> at what quantity and what price. Once the decision has been made, 
>>>> we need to make sure it does not persist past the point of being 
>>>> valid given the state of that data.
>>>>
>>>> If I have rules that are meant to derive/calculate info and 
>>>> decisions from the streaming data, what is the best integration 
>>>> architecture for the rule engine with the rest of the application?
>>>>
>>>> Stateful session which constantly updates the ticks in working 
>>>> memory and queries the memory for results on an ongoing basis? In 
>>>> this case, the session would be kept alive for as long as the data 
>>>> stream is going (i.e. hours)? This approach would require very 
>>>> rigorous working memory management and all the objects in it.
>>>>
>>>> Or...loop constantly over a stateless session for each tick? This 
>>>> would reduce the need to manage the number of objects in working 
>>>> memory since only the amount needed would be inserted in the first 
>>>> place and read back the results? Sounds inefficient and with lots 
>>>> of overhead.
> really don't know. Try the stateful approach first and see how that 
> pans out.
>>>>
>>>> Also, what about multi-threaded environments? Any potential for 
>>>> conflicts between concurrent session instances?
> events are normally stateless, so you should have no problem there. 
> Mutable objects in different threads being updated is of course a 
> problem, shadow proxies can help a little there - but you still need 
> to be careful.
>>>>
>>>> Thanks
>>>> HC
>>>>
>>>>
>>>> _______________________________________________
>>>> rules-users mailing list
>>>> rules-users at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>>
>>>
>>
>>
>
>