[rules-users] StatefulKnowledgeSession and multi-threaded processing

[Richard Calmbach]

When you are dealing with events, the processing works a bit differently.
First, we have to clarify: Are you inserting events or plain facts? If it
has a timestamp, it has to be handled as an event. The distinction matters
because in an event-driven scenario, only events would be inserted "from
the outside" (i.e., from outside the session), and then plain facts
(representing the system state) get inserted/updated/deleted in response to
these incoming events (via rule consequences). The only time you *might*
insert plain facts "from the outside" is at the beginning, before the start
of event processing, to define an initial state of the system. (If anyone
can think of a good reason to insert plain facts *during* event processing
other than in response to events, please let us know - always willing to
learn something new :-).)

In your example, rather than inserting an object (that represents system
state) again with modified state, you would insert a "StateChangeEvent"
into the session, and a rule would update the object (fact) in response to
this event - that's what is meant by event-driven processing or
event-driven architecture. Think of your system state as getting updated in
response to external events, and the rule engine is in charge of performing
the updates.

So, if your objects (object[ABC]) are events, then the situation of
inserting the same event (as in reference identity: ref1 == ref2) twice
doesn't make sense. Honestly, I don't know what the rule engine would do in
this case, but I'd say you have to consider the behavior undefined. Each
state change should be represented by a separate, new event.

I strongly recommend calling fireAllRules() after every single event
insertion (event != fact, event is a special kind of fact). It may be
possible to do batched inserts of events, but the overhead of calling
fireAllRules() after every single event insertion should be minimal. Keep
in mind, either way the rule engine has to process each event in turn in
order to maintain the event-driven semantics. In an event-driven setting,
you would not find yourself doing batch inserts like this, anyway. Events
are processed as they arrive. If you have multiple event streams and
inevitable delays due to messaging, then it is on you as the rule author to
ensure that your rules don't break due to unpredictable event ordering. I
also recommend synchronizing on the session in case you are inserting
events from multiple threads (I have empirical evidence that the
StatefulKnowledgeSession is *not* thread-safe). In a single-event-stream
scenario, where events arrive in chronological order (and even if not), you
want to insert and fire immediately upon event arrival (and, yes, events
could come from an asynchronous messaging queue).

At least this approach has worked well for me. Happy to hear feedback from
others.

-Richard

On Thu, Dec 22, 2011 at 10:45 AM, Skiddlebop <lmalson at cisco.com> wrote:

> Thanks for the link :) 'Twas an interesting view! A few complexities still
> remain. I may be ill-informed about this matter so please correct me if I'm
> totally off-base in any of these statements.
>
> Alexandre states that the KnowledgeSession is thread safe, which is
> contrary
> to the documentation (if he is is referring to a StatefulKnowledgeSession).
> However, it seems like he is actually using a StatelessKnowledgeSession
> pulling his facts from Redis. Our requirements are different, we are
> dealing
> with real-time data, not historical data, and this real-time data will all
> be stored in a long-running StatefulKnowledgeSession. So every fact in that
> StatefulKnowledgeSession will be an active real-time representation of an
> object's state. However, we do share a similar approach in that we are also
> "going async" using queues. The reasoning for choosing the queue for us is
> based on a variety of considerations, the most relevant for this
> conversation is the fact that, based on the un-thread-safe nature of the
> StatefulKnowledgeSession, we anticipate processing one record at a time
> (one
> call to fireAllRules() per event). Does this seem like a reasonable
> approach?
>
> I have considered inserting multiple events (as facts) per call to
> FireAllRules(), but am uncertain of how the conflict resolution should work
> if say for instance two of those facts correspond to the same field of the
> same object. Here's a pseudo-code example outlining my concern:
>
> Key assumption: It is NOT possible to concurrently access a
> StatefulKnowledgeSession from multiple threads or separate parallel REST
> service calls.
>
> //Assume KnowledgeBase has already been built and sks is our long-running
> global singleton //StatefulKnowledgeSession which has continuity between
> individual REST web-service sessions.
>
> //Begin Pseudo-Code Snippet.
>
> //objectA with Data Member "Integer A" Value 1
> sks.insert(objectA);
> sks.insert(objectB);
> sks.insert(objectC);
> //objectA with Data Member "Integer A" Value 2 (this is an update to the
> same object which has a higher //index in the chronologically sorted
> queue).
> sks.insert(objectA);
> sks.fireAllRules();
>
> //End Pseudo-Code Snippet.
>
> As you can see, we enter multiple facts into the session for a single call
> to FireAllRules(), and objectA contains a conflict because there are two
> separate events relating to an update to the same object (objectA) and the
> same field (integerA).
>
> Does this seem like a reasonable or feasible approach? What would happen in
> this scenario? Should we just try to work around it by pre-determining such
> conflicts?
>
> Which of the two is advised? Single Event per SKS fireAllRules() call or
> Multiple Events per fireAllRules() call? Again, performance is important,
> but we must temper such desires with our considerations of data-integrity
> :)
>
> I REALLY REALLY Appreciate the help you've provided thus far.
>
> Thanks for all the shoes,
> Lucas (Skiddlebop)
>
> --
> View this message in context:
> http://drools.46999.n3.nabble.com/StatefulKnowledgeSession-and-multi-threaded-processing-tp3599689p3607250.html
> Sent from the Drools: User forum mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20111222/241a4b2b/attachment.html