Re: [rules-users] Workitems doesn't get persisted when completing a task after rehydrating a knowledge session is some circumstances.

Mauricio, seems to me that you're upset. I'm really sorry, I didn't mean it. I didn't mean this thread to become a fud or some kind of rant. Comments inline: On Mon, Jun 25, 2012 at 1:36 PM, Mauricio Salatino <salaboy(a)gmail.com&gt;wrote: > What I've noticed in the past, doing consulting is that people wants to > migrate from jBPM3 that is almost stateless to jBPM5 and have everything > inside a Stateful session with a richer context and expect that everything > will work in the same way. > If you run each of your process instances in different stateful sessions > (with local ht) you will have something similar to what jBPM3 does, > extremely reduced and isolated context. Now if you want to add Rules and > Events into the mix you will need to learn how Rules and Events works and > how they are mixed with processes inside the stateful session. You cannot > expect that all those features and the mix works in the same way as jBPM3 > (just a stateless process engine) works, right? > That's offensive :(. You're making uninformed assumptions about our experiences with JBPM & Drools, both isolated and mixed, and our expectatives for the migration of our system from JBPM v3 to v5. We obviously were expecting some changes and some bugs. We were definetly not expecting such, IMHO, hard issues with the execution of long-running processes when persistence configured just because how the approach for mixing Drools & JBPM solution for persistence was done. This makes the system not fault tolerant, at least not without some pain and I agree in certain ( but not rare ) configurations. > > I've also notice that this is a step-by-step learning process, once you > master BPMN2 and how process works inside the process engine you can move > to Rules and then to Events, learning in the middle the technical and > logical requirements of each of them. > > > Most of the time the "solution" is understanding how the components > interact and can be mixed. I know that this is difficult sometime, because > of the diversity of the technologies that are being mixed here. > > If you can create a test that shows the problems that you are mentioning > here, we can discuss why or why not this is a good or a wrong approach and > find bugs in case that you find one. If you are in a hurry and you think > that what you are trying to solve are problems, good luck with finding the > tricks. > > OK, let's call this a bug. We believe in open source, that's why we chose Drools & JBPM in favor of other privative solutions. Hey!, At least here we have the chance to hack the code for dirty tricks! ;-). We also believe in an open and honest discussion of issues like this in this kind of projects. As you may know making a test case that reflects the situation mentioned in this thread takes time, is far from trivial, we really are in a hurry and deadlines are aproaching. I personally assigned resources in my team for making such tests and will create issues in Jira when available. > > Cheers > Let me finish quoting with one of your previous messages: "Keep your mind open, because there is no single solution for all the problems", which I agree. > > > On Mon, Jun 25, 2012 at 4:42 AM, Alberto R. Galdo <argaldo(a)gmail.com&gt;wrote: > >> We're in a hurry now to make our system work, unfortunately seems that >> we will be doing dirty tricks as this one for some time ... we'll open an >> issue whenever a test can be produced ... >> >> We were running our system using JBPM 3 and both the integration and >> the persistence there were seamsly done. Our system has high availability >> constraints that forces us to be fault tolerant ( that includes running the >> human task server and process manager in different machines ) and when >> migrating to JBPM 5 we began to face ugly race conditions and rare >> transactional problems ... we honestly thought that must be our fault, >> that's why we opened this thread, just to check if someone had this >> problems and make ourselves wrong or found another "solution". >> >> >> >> On Fri, Jun 22, 2012 at 3:03 PM, Mauricio Salatino <salaboy(a)gmail.com&gt;wrote: >> >>> So, can you create an isolated test where you reproduce: >>> >>> "We are unable to complete a human task after rehydrating a Drools >>> knowledge session because in some circunstances the generated Drools' >>> workitems don't get persisted in the database after the completion of a >>> previous task" >>> >>> And I can take a look on that.. Please create Jira issue for that. >>> >> Without a concrete situation it's very difficult to analyze.. Did you >>> check your transactions not being rolledback.. That's the only situation >>> where I think that the workItem information will not be persisted. >>> >>> Cheers >>> >>> On Fri, Jun 22, 2012 at 9:55 AM, Alberto R. Galdo <argaldo(a)gmail.com&gt;wrote: >>> >>>> >>>> Sure, WorkItemHandlers are never persisted. I re-register those >>>> handlers before staring the session, just because I want my tasks to be >>>> properly executed. >>>> >>>> :( >>>> >>>> >>>> Alberto R. Galdo >>>> argaldo(a)gmail.com >>>> >>>> >>>> >>>> On Fri, Jun 22, 2012 at 2:46 PM, Mauricio Salatino <salaboy(a)gmail.com&gt;wrote: >>>> >>>>> There are two concepts here: >>>>> 1) WorkItem -> Persist the state of the activity >>>>> 2) WorkItemHandlers -> Never Persisted >>>>> >>>>> Are you re-registering the WorkItemHandlers at rehydratation? >>>>> WorkItemHandlers are part of the runtime status and don't get >>>>> persisted. >>>>> Cheers >>>>> >>>>> On Fri, Jun 22, 2012 at 9:39 AM, Alberto R. Galdo <argaldo(a)gmail.com&gt;wrote: >>>>> >>>>>> No, I'm not registering pending workitems at rehydration. That's why >>>>>> I'm using Drools & JBPM persistence ;-). I don't want to write my own state >>>>>> persistence, as I am a mere user of JBPM & Drools services. >>>>>> >>>>>> >> "They are never persisted" >>>>>> >>>>>> This several methods in >>>>>> org.drools.persitence.jpa.JPAPersistenceContext seem to say just the >>>>>> opposite: >>>>>> >>>>>> public void persist(WorkItemInfo workItemInfo) >>>>>> public void remove(WorkItemInfo workItemInfo) >>>>>> public WorkItemInfo merge(WorkItemInfo workItemInfo) >>>>>> >>>>>> The fact that lots of workitems get created, persisted, merged and >>>>>> finally removed during the life of the process doesn't hide the fact that >>>>>> they're in fact, .... well, persisted. >>>>>> >>>>>> If you take a look at the changes in the database whenever a human >>>>>> task is involved in a BPMN process that is executed inside a Drools & JBPM >>>>>> JPA persisted environment you will realize that indeed the human task are >>>>>> *persisted* and like so, rehydrated when loading the session in Drools. In >>>>>> fact, those human task related workitems are never removed from the >>>>>> database, but that's another bug ... :( >>>>>> >>>>>> Any insight? >>>>>> >>>>>> Alberto R. Galdo >>>>>> argaldo(a)gmail.com >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Jun 22, 2012 at 2:15 PM, Mauricio Salatino < >>>>>> salaboy(a)gmail.com&gt; wrote: >>>>>> >>>>>>> "We are unable to complete a human task after rehydrating a >>>>>>> Drools knowledge session because in some circunstances the generated >>>>>>> Drools' workitems don't get persisted in the database after the completion >>>>>>> of a previous task" >>>>>>> >>>>>>> They are never persisted, they are runtime information that you >>>>>>> must re-register after rehydrating the session. Are you doing that? >>>>>>> Cheers >>>>>>> >>>>>>> On Fri, Jun 22, 2012 at 7:34 AM, Alberto R. Galdo < >>>>>>> argaldo(a)gmail.com&gt; wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> We have a fairly large BPMN process running inside a JPA >>>>>>>> persisted StatefulKnowledgeSession using Drools 5.4 & JBPM 5.3. Our process >>>>>>>> involves timers, automated tasks, human tasks .... most of them are >>>>>>>> long-running processes, so a fault-tolerant scenario is a must. >>>>>>>> >>>>>>>> We've found what seems to be a weird, weird bug in JBPM-Drools >>>>>>>> regarding the execution of BPMN processes. This is by best to summarize the >>>>>>>> problem: >>>>>>>> >>>>>>>> "We are unable to complete a human task after rehydrating a >>>>>>>> Drools knowledge session because in some circunstances the generated >>>>>>>> Drools' workitems don't get persisted in the database after the completion >>>>>>>> of a previous task" >>>>>>>> >>>>>>>> So, as the workitem is not in the database, when a human task >>>>>>>> client completes a task that is related to that non-existent workitem, the >>>>>>>> process doesn't get restarted. And the process fails. >>>>>>>> >>>>>>>> ¿Why does this happens? Lets see: >>>>>>>> >>>>>>>> When the processs is executed, different workitems get >>>>>>>> created, updated and eventually deleted during the execution of a process >>>>>>>> up until a human task is created ( in our process ). When living in a >>>>>>>> persistet knowledge session, the transaction that is associated to Drools' >>>>>>>> thread is commited right after the human task is created in the human task >>>>>>>> server ... as it is a "safe point". Nothing here. Everithing is consistent, >>>>>>>> if you look at the database you will see your session instance, your >>>>>>>> process instance, and the final human task workitem as it is the only >>>>>>>> workitem survivor after the execution ( whatever hadler-managed automated >>>>>>>> task that were executed before the human task are deleted and the human >>>>>>>> task workitem needs to survive as it's completion depends on asyncronous >>>>>>>> client interaction ). >>>>>>>> >>>>>>>> Now, if you connect to the human task server and complete >>>>>>>> that human task, a message is sent to the Drools session to update the >>>>>>>> state of the work item. The workitem gets updated, the process get >>>>>>>> restarted and the flow continues ... maybe generating a new human task ( >>>>>>>> which is our case ). At this very moment, if you take a look at the >>>>>>>> database, there are no automated-handled-task workitems ( as expected ) but >>>>>>>> there isn't any human task related work item, even worse, the task at the >>>>>>>> human task server is created, persisted and has a reference to the >>>>>>>> non-existant workitem. >>>>>>>> >>>>>>>> Days of debugging led us to what we think is the source of the >>>>>>>> problem: >>>>>>>> >>>>>>>> We found that the execution of the process after completing a >>>>>>>> task is being executed in the same thread as the one that receives the mina >>>>>>>> message that the human task server sends whenever a task is completed. This >>>>>>>> thread is not the same thread that executes the knowledgesession ( where >>>>>>>> the reteoo lives ) and so it doesn't have a transaction. By the way, we >>>>>>>> found that for workitem persistence the JPAWorkitemManager never joins an >>>>>>>> active transaction. :( >>>>>>>> >>>>>>>> That's why invoking the persistence of a workitem as a >>>>>>>> consequence of restarting the execution of a process inside the thread that >>>>>>>> receives the mina messages makes the database inconsistent, and so >>>>>>>> invalidating all means to make JBPM fault tolerant by making Drools session >>>>>>>> persistent. >>>>>>>> >>>>>>>> We found a way to circunvent this problem, making all our >>>>>>>> human task nodes be followed by a event timer. That way, when the timer >>>>>>>> gets completed we force the execution of the process to live in the same >>>>>>>> thread that the reteoo session lives where a transaction is available and >>>>>>>> things get back to normal. But this is really dirty and wrong. >>>>>>>> >>>>>>>> Any thoughts? >>>>>>>> >>>>>>>> We are really eager to be wrong whith this. :'( >>>>>>>> >>>>>>>> Greets, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Alberto R. Galdo >>>>>>>> argaldo(a)gmail.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> rules-users mailing list >>>>>>>> rules-users(a)lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> - MyJourney @ http://salaboy.wordpress.com >>>>>>> - Co-Founder @ http://www.jugargentina.org >>>>>>> - Co-Founder @ http://www.jbug.com.ar >>>>>>> >>>>>>> - Salatino "Salaboy" Mauricio - >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> rules-users mailing list >>>>>>> rules-users(a)lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> rules-users mailing list >>>>>> rules-users(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> - MyJourney @ http://salaboy.wordpress.com >>>>> - Co-Founder @ http://www.jugargentina.org >>>>> - Co-Founder @ http://www.jbug.com.ar >>>>> >>>>> - Salatino "Salaboy" Mauricio - >>>>> >>>>> >>>>> _______________________________________________ >>>>> rules-users mailing list >>>>> rules-users(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> rules-users mailing list >>>> rules-users(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>> >>>> >>> >>> >>> -- >>> - MyJourney @ http://salaboy.wordpress.com >>> - Co-Founder @ http://www.jugargentina.org >>> - Co-Founder @ http://www.jbug.com.ar >>> >>> - Salatino "Salaboy" Mauricio - >>> >>> >>> _______________________________________________ >>> rules-users mailing list >>> rules-users(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/rules-users >>> >>> >> >> _______________________________________________ >> rules-users mailing list >> rules-users(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> > > > -- > - MyJourney @ http://salaboy.wordpress.com > - Co-Founder @ http://www.jugargentina.org > - Co-Founder @ http://www.jbug.com.ar > > - Salatino "Salaboy" Mauricio - > > > _______________________________________________ > rules-users mailing list > rules-users(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/rules-users > > _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users