[rules-users] Drools and event based decisions, i.e. streaming market quotes

[Mark Proctor]

Henry Canterburry wrote:
> Mark, at this point I have a semi-working drools integration utilizing 
> the stateful session. With each event coming in, I insert the event 
> into the session and call fireAllRules. I have also added memory 
> managment rules to retract events past a certain time threshold or 
> number of instances in memory.
>
> I have a few thoughts however:
> Events come in completely asynchronously and rapidly. What happens if 
> a new fact enters the engine before all rules for the previous fact 
> have finished evaluating/firing? Since I always have a certain number 
> of past events kept in WM for history checking purposes, my key event 
> processing rules are specifically written to target the "latest" 
> instance of an event. Given the rapid speed at which new events enter 
> the engine, might I encounter situations where rules later in an 
> inference chain for a prior event are evaluating derived data 
> partially from the previous event and from the latest event depending 
> on what new facts have been established? (sort of a race condition). 
> Or...will the inference chain for the previous fact always be allowed 
> to finish before a new inference chain for the latest event is begun?
If that new fact enters it will match in the same way any other facts 
have. It's resulting activations will be executed based on the conflict 
resolution strategy, which is salienec+depth based - i.e. activations 
with new facts fire first. So I'm guessing this would break your model 
you could try a different conflict resolution strategy, it is pluggable.
>
> Thanks
> HC
>
> Mark Proctor wrote:
>> Henry Canterburry wrote:
>>> Mark,
>>>
>>> Thanks for the reply. My timeframe matches the one you mentioned 
>>> although I would naturally be looking for working and production 
>>> ready functionality. In the meantime, did any of the approaches I 
>>> mentioned make any sense or would you say that drools is not the 
>>> right tool at this point? I do have helper classes and other 
>>> intermediary data caching approaches at this time that permit me to 
>>> query past data and an entire streaming architecture to feed data.
>>>
>>> Thanks
>>> HC
>>>
>>>
>>> Mark Proctor wrote:
>>>> What's your time scale for this? We are currently implementing CEP 
>>>> extendions to the language and engine which will do all of this for 
>>>> you, we'll have a milestone release out in february when you can 
>>>> first play with this and aiming to get a full release end of Q1 
>>>> start of Q2.
>>>>
>>>> Mark
>>>> Henry Canterburry wrote:
>>>>> What would be the best approach using drools to handle event 
>>>>> driven decisions based on streaming data? In my case I am looking 
>>>>> for a typical stock market scenario. Ticker quotes usually come in 
>>>>> at second increments and depending on how many ticker symbols you 
>>>>> subscribe to at any one time, there can be a lot of data coming 
>>>>> and and changing every second. However, there probably isn't a 
>>>>> need to keep large quantities of historic data in memory...maybe 
>>>>> the last 200-500 ticks. The outputs are if a stock should be sold 
>>>>> or bought, at what quantity and what price. Once the decision has 
>>>>> been made, we need to make sure it does not persist past the point 
>>>>> of being valid given the state of that data.
>>>>>
>>>>> If I have rules that are meant to derive/calculate info and 
>>>>> decisions from the streaming data, what is the best integration 
>>>>> architecture for the rule engine with the rest of the application?
>>>>>
>>>>> Stateful session which constantly updates the ticks in working 
>>>>> memory and queries the memory for results on an ongoing basis? In 
>>>>> this case, the session would be kept alive for as long as the data 
>>>>> stream is going (i.e. hours)? This approach would require very 
>>>>> rigorous working memory management and all the objects in it.
>>>>>
>>>>> Or...loop constantly over a stateless session for each tick? This 
>>>>> would reduce the need to manage the number of objects in working 
>>>>> memory since only the amount needed would be inserted in the first 
>>>>> place and read back the results? Sounds inefficient and with lots 
>>>>> of overhead.
>> really don't know. Try the stateful approach first and see how that 
>> pans out.
>>>>>
>>>>> Also, what about multi-threaded environments? Any potential for 
>>>>> conflicts between concurrent session instances?
>> events are normally stateless, so you should have no problem there. 
>> Mutable objects in different threads being updated is of course a 
>> problem, shadow proxies can help a little there - but you still need 
>> to be careful.
>>>>>
>>>>> Thanks
>>>>> HC
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> rules-users mailing list
>>>>> rules-users at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>