[rules-users] Activation Firing slow on many events with timer attribute in rule (Drools Fusion)

Mark Proctor mproctor at codehaus.org
Tue Jan 31 13:50:26 EST 2012 

Every transaction boundary is a full working memory snapshot, it is not 
recommended for wm's with large number of facts or timers when high 
performance is needed.

Mark
On 31/01/2012 00:20, Philipp Herzig wrote:
> Thanks Wolfgang for testing.
>
> Well I found something. I am using the JPAKnowledgeService and when
> inserting the bunch of events within a JTA (with BTM) the discussed
> behavior occurs.
> If I am removing the JPA stuff it performs as expected.
>
> Well, of course, now my question is, how are the events are persisted
> when is timer is present and fetched (lazy, eager?) once the timer
> expires? Unfortunately, I would need session persistence because the
> timers can be long and need to be recreated after an outage.
>
> Thanks again for your time :)
>
> Philipp
>
> 2012/1/30 Wolfgang Laun<wolfgang.laun at gmail.com>:
>> Hmm, I'm using 5.3.0 and I can't confirm your "slow" results.
>>
>> -W
>>
>>
>> On 30/01/2012, Philipp Herzig<pherzig at googlemail.com>  wrote:
>>> Thanks again,
>>>
>>> yes I call fireAllRules() after insertion.
>>>
>>> My understanding is that the join on mrid reduces the search space
>>> (actually 2000) (actually canceled du to not) and the remainder can be
>>> processed as leftover from the outer join.
>>>
>>> Anyhow, when I remove the "not join" clause the behavior is exactly the
>>> same.
>>>
>>> (1)  8000 create events are inserted and objects are asserted and
>>> activations created!
>>> (2) after 20s the activations are actually fired, but with a
>>> noticeable delay (30min)
>>>
>>> Now, when I simply remove the timer, all 8000 activations are fired
>>> super fast (5s !!!)
>>>
>>> Or am I overlooking sth?
>>>
>>> Thanks for your patience,
>>>
>>> Philipp
>>>
>>>
>>> 2012/1/30 Wolfgang Laun<wolfgang.laun at gmail.com>:
>>>>
>>>> On 30 January 2012 17:18, Philipp Herzig<pherzig at googlemail.com>  wrote:
>>>>> Thanks Wolfgang for your reply.
>>>>>
>>>>> doSomething simply persists something with JPA, no events are
>>>>> retracted or updated. The behavior does also not change when there is
>>>>> a simple println in the then clause.
>>>>>
>>>> OK.
>>>>
>>>>
>>>>> I thought events are automatically retracted once no rule applied
>>>>> anymore.  The EventObject class with the map is used only for
>>>>> abstraction within the architecture (actually a property pattern). No
>>>>> intention to speed up something.
>>>>
>>>> There's no information in the (only) rule that would permit an automatic
>>>> retraction.
>>>> Mostly this is possible when temporal operators are used.
>>>>
>>>> Searching for EventObject/create without matching EventObject/delete is
>>>> causing a lot of work, since each insertion requires an exhaustive linear
>>>> search for the absence of a delete or the presence of a create.
>>>>
>>>> What do you call after the bunch insert? fireAllRules() ?
>>>>
>>>> -W
>>>>
>>>>
>>>>> As I said, the activation firing works fine when no timer is present
>>>>> and, of course, deletes are already inserted before the creates,
>>>>> otherwise the rule won't fire.
>>>>>
>>>>>
>>>>> Do you know how the Scheduler works? According to the observed
>>>>> behavior I would guess that there is a background job invoked every
>>>>> 100ms checking if there are timer delayed activations which has to be
>>>>> fired.
>>>>> If this is the case, I wonder why only some activations are fired and
>>>>> not all. Due to the upper time bound?
>>>>>
>>>>> Thanks again,
>>>>>
>>>>> Philipp
>>>>>
>>>>>
>>>>> 2012/1/30 Wolfgang Laun<wolfgang.laun at gmail.com>:
>>>>>> Does the doSomething() update or retract any EventObject facts and
>>>>>> notify
>>>>>> the Drools engine accordingly?
>>>>>>
>>>>>> (So far: Neither using the same class for the "create" and the "delete"
>>>>>> events nor
>>>>>> using a map (i.e., "data") for all properties is helping w.r.t. speed.)
>>>>>>
>>>>>> -W
>>>>>>
>>>>>>
>>>>>> On 30 January 2012 15:07, Philipp Herzig<pherzig at googlemail.com>
>>>>>> wrote:
>>>>>>> Sure, here it is. Sorry for any inconvienience!
>>>>>>>
>>>>>>> rule "new_intent"
>>>>>>> timer (20s)
>>>>>>> when
>>>>>>>       $evt : EventObject(data['type']=='create') from entry-point
>>>>>>> eventstream
>>>>>>>       not ( EventObject(data['type']=='delete',
>>>>>>> data['mrid']==$evt.data['mrid'], data['userid']==$evt.data['userid'])
>>>>>>> from entry-point eventstream)
>>>>>>> then
>>>>>>>   SomeAPI.getInstance().doSomething();
>>>>>>> end
>>>>>>>
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Philipp
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2012/1/30 Michael Anstis<michael.anstis at gmail.com>
>>>>>>>> It's going to help (probably) if you include the definition of your
>>>>>>>> rule
>>>>>>>> (or rules).
>>>>>>>>
>>>>>>>> 2012/1/30 Philipp Herzig<pherzig at googlemail.com>
>>>>>>>>> Dear Community,
>>>>>>>>>
>>>>>>>>> Drools is pretty fast regarding all my use cases. However, today I
>>>>>>>>> have
>>>>>>>>> found a problem where I cannot find any solution. Hopefully someone
>>>>>>>>> of you
>>>>>>>>> can help.
>>>>>>>>>
>>>>>>>>> 1. I have a rule with a @timer(10s) attribute (should be 24h later
>>>>>>>>> on
>>>>>>>>> but doesn't matter). This rule is activated when a "create" event
>>>>>>>>> occurs and
>>>>>>>>> invalidated once a "delete" event occurs within the timeframe of
>>>>>>>>> @timer.
>>>>>>>>>
>>>>>>>>> 2. I have approx. 9000 "create" events which are bulk loaded into
>>>>>>>>> the
>>>>>>>>> working memory and creating activations for the rule above.
>>>>>>>>>
>>>>>>>>> 3. I have approx. 2000 "delete" events which are bulk loaded into
>>>>>>>>> my
>>>>>>>>> entry-point cancelling the respective activations from step (2)
>>>>>>>>>
>>>>>>>>> 4. After the timer expired, the first activation is fired
>>>>>>>>> correctly.
>>>>>>>>> However, all other activations are fired with some noticeable delay
>>>>>>>>> (actually it needs 20-30minutes until all activations are fired).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Do you have an idea what the problem with the timer might be?
>>>>>>>>> Unfortunately, I have neither an idea how the scheduler in the
>>>>>>>>> background
>>>>>>>>> works nor which class I should start looking at.
>>>>>>>>>
>>>>>>>>> BTW: For testing purpose I switched step (2)&  (3), that is,
>>>>>>>>> "delete"
>>>>>>>>> events are inserted before the "create" events and removed the
>>>>>>>>> timer
>>>>>>>>> attribute which is obviously the same logic. It performs lightning
>>>>>>>>> fast in
>>>>>>>>> this case... (all remaining activations are fired within 5
>>>>>>>>> seconds). However, insertinging my "delete" events before the
>>>>>>>>> "create"
>>>>>>>>> events is ok for testing but not feasible in practice.
>>>>>>>>>
>>>>>>>>> It would be great if some of you has an idea or point to start
>>>>>>>>> within
>>>>>>>>> the code.
>>>>>>>>>
>>>>>>>>> Thanks in advance,
>>>>>>>>>
>>>>>>>>> Philipp
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rules-users mailing list
>>>>>>>>> rules-users at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rules-users mailing list
>>>>>>>> rules-users at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> --------------------------------------------
>>>>>>> Philipp Herzig, M.Sc.
>>>>>>>
>>>>>>> Mail: pherzig at googlemail.com
>>>>>>> Cell: 0178 - 6156244
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rules-users mailing list
>>>>>>> rules-users at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> rules-users mailing list
>>>>>> rules-users at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --------------------------------------------
>>>>> Philipp Herzig, M.Sc.
>>>>>
>>>>> Mail: pherzig at googlemail.com
>>>>> Cell: 0178 - 6156244
>>>>>
>>>>> _______________________________________________
>>>>> rules-users mailing list
>>>>> rules-users at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>>>
>>>> _______________________________________________
>>>> rules-users mailing list
>>>> rules-users at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>>
>>>
>>> --
>>> --------------------------------------------
>>> Philipp Herzig, M.Sc.
>>>
>>> Mail: pherzig at googlemail.com
>>> Cell: 0178 - 6156244
>>>
>>> _______________________________________________
>>> rules-users mailing list
>>> rules-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>
>

More information about the rules-users mailing list