[rules-users] Workitems doesn't get persisted when completing a task after rehydrating a knowledge session is some circumstances.

Mauricio Salatino salaboy at gmail.com
Tue Jun 26 06:39:41 EDT 2012 

Hi Alberto,
I'm not upset, kind the opposite. I'm sorry if my comments sounds harsh. I
was making some assumptions based on my previous experience.
My main point was, let's try to be concrete and let's work on code and
failing tests. I know that is not trivial, but if we want to make the
project better for everyone I think that's the only way to go. I'm keeping
my mind open and I think that we all here are open to discussions, but lets
discuss based on concrete proposals. If we don't go that way, this
conversation will become cyclic and we will all loose time instead of being
fixing bugs and adding new features :)


On Tue, Jun 26, 2012 at 5:43 AM, Alberto R. Galdo <argaldo at gmail.com> wrote:

> Mauricio, seems to me that you're upset. I'm really sorry, I didn't mean
> it. I didn't mean this thread to become a fud or some kind of rant.
>
> Comments inline:
>
> On Mon, Jun 25, 2012 at 1:36 PM, Mauricio Salatino <salaboy at gmail.com>wrote:
>
>> What I've noticed in the past, doing consulting is that people wants to
>> migrate from jBPM3 that is almost stateless to jBPM5 and have everything
>> inside a Stateful session with a richer context and expect that everything
>> will work in the same way.
>> If you run each of your process instances in different stateful sessions
>> (with local ht) you will have something similar to what jBPM3 does,
>> extremely reduced and isolated context. Now if you want to add Rules and
>> Events into the mix you will need to learn how Rules and Events works and
>> how they are mixed with processes inside the stateful session. You cannot
>> expect that all those features and the mix works in the same way as jBPM3
>> (just a stateless process engine) works, right?
>>
>
> That's offensive :(. You're making uninformed assumptions about our
> experiences with JBPM & Drools, both isolated and mixed, and our
> expectatives for the migration of our system from JBPM v3 to v5.
>
> We obviously were expecting some changes and some bugs. We were definetly
> not expecting such, IMHO, hard issues with the execution of long-running
> processes when persistence configured just because how the approach for
> mixing Drools & JBPM solution for persistence was done. This makes the
> system not fault tolerant, at least not without some pain and I agree in
> certain ( but not rare  ) configurations.
>
>
>>
>> I've also notice that this is a step-by-step learning process, once you
>> master BPMN2 and how process works inside the process engine you can move
>> to Rules and then to Events, learning in the middle the technical and
>> logical requirements of each of them.
>>
>>
>
>> Most of the time the "solution" is understanding how the components
>> interact and can be mixed. I know that this is difficult sometime, because
>> of the diversity of the technologies that are being mixed here.
>>
>> If you can create a test that shows the problems that you are mentioning
>> here, we can discuss why or why not this is a good or a wrong approach and
>> find bugs in case that you find one. If you are in a hurry and you think
>> that what you are trying to solve are problems, good luck with finding the
>> tricks.
>>
>>
>
> OK, let's call this a bug.
>
> We believe in open source, that's why we chose Drools & JBPM in favor of
> other privative solutions. Hey!, At least here we have the chance to hack
> the code for dirty tricks! ;-).  We also believe in an open and honest
> discussion of issues like this in this kind of projects.
>
> As you may know making a test case that reflects the situation mentioned
> in this thread takes time, is far from trivial, we really are in a hurry
> and deadlines are aproaching.
>
> I personally assigned resources in my team for making such tests and will
> create issues in Jira when available.
>
>
>
>>
>> Cheers
>>
>
> Let me finish quoting with one of your previous messages: "Keep your mind
> open, because there is no single solution for all the problems", which I
> agree.
>
>
>
>
>
>
>
>
>>
>>
>> On Mon, Jun 25, 2012 at 4:42 AM, Alberto R. Galdo <argaldo at gmail.com>wrote:
>>
>>>   We're in a hurry now to make our system work, unfortunately seems that
>>> we will be doing dirty tricks as this one for some time ... we'll open an
>>> issue whenever a test can be produced ...
>>>
>>>   We were running our system using JBPM 3 and both the integration and
>>> the persistence there were seamsly done. Our system has high availability
>>> constraints that forces us to be fault tolerant ( that includes running the
>>> human task server and process manager in different machines ) and when
>>> migrating to JBPM 5 we began to face ugly race conditions and rare
>>> transactional problems ... we honestly thought that must be our fault,
>>> that's why we opened this thread, just to check if someone had this
>>> problems and make ourselves wrong or found another "solution".
>>>
>>>
>>>
>>> On Fri, Jun 22, 2012 at 3:03 PM, Mauricio Salatino <salaboy at gmail.com>wrote:
>>>
>>>> So, can you create an isolated test where you reproduce:
>>>>
>>>>     "We are unable to complete a human task after rehydrating a Drools
>>>> knowledge session because in some circunstances the generated Drools'
>>>> workitems don't get persisted in the database after the completion of a
>>>> previous task"
>>>>
>>>> And I can take a look on that.. Please create Jira issue for that.
>>>>
>>> Without a concrete situation it's very difficult to analyze.. Did you
>>>> check your transactions not being rolledback.. That's the only situation
>>>> where I think that the workItem information will not be persisted.
>>>>
>>>> Cheers
>>>>
>>>> On Fri, Jun 22, 2012 at 9:55 AM, Alberto R. Galdo <argaldo at gmail.com>wrote:
>>>>
>>>>>
>>>>> Sure, WorkItemHandlers are never persisted. I re-register those
>>>>> handlers before staring the session, just because I want my tasks to be
>>>>> properly executed.
>>>>>
>>>>> :(
>>>>>
>>>>>
>>>>> Alberto R. Galdo
>>>>> argaldo at gmail.com
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jun 22, 2012 at 2:46 PM, Mauricio Salatino <salaboy at gmail.com>wrote:
>>>>>
>>>>>> There are two concepts here:
>>>>>> 1) WorkItem -> Persist the state of the activity
>>>>>> 2) WorkItemHandlers -> Never Persisted
>>>>>>
>>>>>> Are you re-registering the WorkItemHandlers at rehydratation?
>>>>>> WorkItemHandlers are part of the runtime status and don't get
>>>>>> persisted.
>>>>>> Cheers
>>>>>>
>>>>>> On Fri, Jun 22, 2012 at 9:39 AM, Alberto R. Galdo <argaldo at gmail.com>wrote:
>>>>>>
>>>>>>> No, I'm not registering pending workitems at rehydration. That's why
>>>>>>> I'm using Drools & JBPM persistence ;-). I don't want to write my own state
>>>>>>> persistence, as I am a mere user of JBPM & Drools services.
>>>>>>>
>>>>>>> >> "They are never persisted"
>>>>>>>
>>>>>>> This several methods in
>>>>>>> org.drools.persitence.jpa.JPAPersistenceContext seem to say just the
>>>>>>> opposite:
>>>>>>>
>>>>>>> public void persist(WorkItemInfo workItemInfo)
>>>>>>> public void remove(WorkItemInfo workItemInfo)
>>>>>>> public WorkItemInfo merge(WorkItemInfo workItemInfo)
>>>>>>>
>>>>>>> The fact that lots of workitems get created, persisted, merged and
>>>>>>> finally removed during the life of the process doesn't hide the fact that
>>>>>>> they're in fact, .... well, persisted.
>>>>>>>
>>>>>>> If you take a look at the changes in the database whenever a human
>>>>>>> task is involved in a BPMN process that is executed inside a Drools & JBPM
>>>>>>> JPA persisted environment you will realize that indeed the human task are
>>>>>>> *persisted* and like so, rehydrated when loading the session in Drools.  In
>>>>>>> fact, those human task related workitems are never removed from the
>>>>>>> database, but that's another bug ... :(
>>>>>>>
>>>>>>> Any insight?
>>>>>>>
>>>>>>> Alberto R. Galdo
>>>>>>> argaldo at gmail.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 22, 2012 at 2:15 PM, Mauricio Salatino <
>>>>>>> salaboy at gmail.com> wrote:
>>>>>>>
>>>>>>>>   "We are unable to complete a human task after rehydrating a
>>>>>>>> Drools knowledge session because in some circunstances the generated
>>>>>>>> Drools' workitems don't get persisted in the database after the completion
>>>>>>>> of a previous task"
>>>>>>>>
>>>>>>>> They are never persisted, they are runtime information that you
>>>>>>>> must re-register after rehydrating the session. Are you doing that?
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>> On Fri, Jun 22, 2012 at 7:34 AM, Alberto R. Galdo <
>>>>>>>> argaldo at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>>    We have a fairly large BPMN process running inside a JPA
>>>>>>>>> persisted StatefulKnowledgeSession using Drools 5.4 & JBPM 5.3. Our process
>>>>>>>>> involves timers, automated tasks, human tasks .... most of them are
>>>>>>>>> long-running processes, so a fault-tolerant scenario is a must.
>>>>>>>>>
>>>>>>>>>     We've found what seems to be a weird, weird bug in JBPM-Drools
>>>>>>>>> regarding the execution of BPMN processes. This is by best to summarize the
>>>>>>>>> problem:
>>>>>>>>>
>>>>>>>>>      "We are unable to complete a human task after rehydrating a
>>>>>>>>> Drools knowledge session because in some circunstances the generated
>>>>>>>>> Drools' workitems don't get persisted in the database after the completion
>>>>>>>>> of a previous task"
>>>>>>>>>
>>>>>>>>>     So, as the workitem is not in the database, when a human task
>>>>>>>>> client completes a task that is related to that non-existent workitem, the
>>>>>>>>> process doesn't get restarted. And the process fails.
>>>>>>>>>
>>>>>>>>>     ¿Why does this happens? Lets see:
>>>>>>>>>
>>>>>>>>>      When the processs is executed, different workitems get
>>>>>>>>> created, updated and eventually deleted during the execution of a process
>>>>>>>>> up until a human task is created ( in our process ). When living in a
>>>>>>>>> persistet knowledge session, the transaction that is associated to Drools'
>>>>>>>>> thread is commited right after the human task is created in the human task
>>>>>>>>> server ... as it is a "safe point". Nothing here. Everithing is consistent,
>>>>>>>>> if you look at the database you will see your session instance, your
>>>>>>>>> process instance, and the final human task workitem as it is the only
>>>>>>>>> workitem survivor after the execution ( whatever hadler-managed automated
>>>>>>>>> task that were executed before the human task are deleted and the human
>>>>>>>>> task workitem needs to survive as it's completion depends on asyncronous
>>>>>>>>> client interaction ).
>>>>>>>>>
>>>>>>>>>      Now, if you connect to the human task server and complete
>>>>>>>>> that human task, a message is sent to the Drools session to update the
>>>>>>>>> state of the work item. The workitem gets updated, the process get
>>>>>>>>> restarted and the flow continues ... maybe generating a new human task (
>>>>>>>>> which is our case ). At this very moment, if you take a look at the
>>>>>>>>> database, there are no automated-handled-task workitems ( as expected ) but
>>>>>>>>> there isn't any human task related work item, even worse, the task at the
>>>>>>>>> human task server is created, persisted and has a reference to the
>>>>>>>>> non-existant workitem.
>>>>>>>>>
>>>>>>>>>     Days of debugging led us to what we think is the source of the
>>>>>>>>> problem:
>>>>>>>>>
>>>>>>>>>     We found that the execution of the process after completing a
>>>>>>>>> task is being executed in the same thread as the one that receives the mina
>>>>>>>>> message that the human task server sends whenever a task is completed. This
>>>>>>>>> thread is not the same thread that executes the knowledgesession ( where
>>>>>>>>> the reteoo lives ) and so it doesn't have a transaction. By the way, we
>>>>>>>>> found that for  workitem persistence the JPAWorkitemManager never joins an
>>>>>>>>> active transaction. :(
>>>>>>>>>
>>>>>>>>>     That's why invoking the persistence of a workitem as a
>>>>>>>>> consequence of restarting the execution of a process inside the thread that
>>>>>>>>> receives the mina messages makes the database inconsistent, and so
>>>>>>>>> invalidating all means to make JBPM fault tolerant by making Drools session
>>>>>>>>> persistent.
>>>>>>>>>
>>>>>>>>>     We found a way to circunvent this problem, making all our
>>>>>>>>> human task nodes be followed by a event timer. That way, when the timer
>>>>>>>>> gets completed we force the execution of the process to live in the same
>>>>>>>>> thread that the reteoo session lives where a transaction is available and
>>>>>>>>> things get back to normal. But this is really dirty and wrong.
>>>>>>>>>
>>>>>>>>>     Any thoughts?
>>>>>>>>>
>>>>>>>>>     We are really eager to be wrong whith this. :'(
>>>>>>>>>
>>>>>>>>> Greets,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Alberto R. Galdo
>>>>>>>>> argaldo at gmail.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rules-users mailing list
>>>>>>>>> rules-users at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>  - MyJourney @ http://salaboy.wordpress.com
>>>>>>>>  - Co-Founder @ http://www.jugargentina.org
>>>>>>>>  - Co-Founder @ http://www.jbug.com.ar
>>>>>>>>
>>>>>>>>  - Salatino "Salaboy" Mauricio -
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rules-users mailing list
>>>>>>>> rules-users at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rules-users mailing list
>>>>>>> rules-users at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>  - MyJourney @ http://salaboy.wordpress.com
>>>>>>  - Co-Founder @ http://www.jugargentina.org
>>>>>>  - Co-Founder @ http://www.jbug.com.ar
>>>>>>
>>>>>>  - Salatino "Salaboy" Mauricio -
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> rules-users mailing list
>>>>>> rules-users at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> rules-users mailing list
>>>>> rules-users at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>  - MyJourney @ http://salaboy.wordpress.com
>>>>  - Co-Founder @ http://www.jugargentina.org
>>>>  - Co-Founder @ http://www.jbug.com.ar
>>>>
>>>>  - Salatino "Salaboy" Mauricio -
>>>>
>>>>
>>>> _______________________________________________
>>>> rules-users mailing list
>>>> rules-users at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>>
>>>>
>>>
>>> _______________________________________________
>>> rules-users mailing list
>>> rules-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/rules-users
>>>
>>>
>>
>>
>> --
>>  - MyJourney @ http://salaboy.wordpress.com
>>  - Co-Founder @ http://www.jugargentina.org
>>  - Co-Founder @ http://www.jbug.com.ar
>>
>>  - Salatino "Salaboy" Mauricio -
>>
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>


-- 
 - MyJourney @ http://salaboy.wordpress.com
 - Co-Founder @ http://www.jugargentina.org
 - Co-Founder @ http://www.jbug.com.ar

 - Salatino "Salaboy" Mauricio -
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20120626/8778b618/attachment-0001.html

More information about the rules-users mailing list