Mauricio Salatino [https://community.jboss.org/people/salaboy21] created the discussion "Re: jBPM5 - Can't Scale, Won't Scale" To view the discussion, visit: https://community.jboss.org/message/771738#771738 -------------------------------------------------------------- Hi Arnold, Look below for some comments: 1. Everytime jBPM5 executes a process the database transaction updates the process instance info AND the session info, therefore: 1a. it's not scalable to use a single session to execute all processes, or you would suffer contention on the session info. *Why do you want to use a single session?* 1b, it's not scalable to use a single session to execute all processes in a cluster, or the updated session info would have to be continually synchronised across the cluster. *Yes, its madness to think that you can do that in a simple way. At this point having multiple sessions is a must if you want to divide where the execution can happen.* 2. When using BPMN2 events, jBPM5 only allows you to send events to the process instances within a single session at a time.  You need to maintain a list of all the sessions which have incompleted process instances(*), and loop through them all to send events.  Therefore: 2a. you should execute all processes in as few sessions as possible, to lessen the number of iterations through this loop. *Or you can use a ESB or an lightweight integration framework to do that job like for example apache camel or Switchyard.* 3. jBPM5 persists BPMN2 timer info in the session info, but the session must be active (ie. loaded from persistence) in order for the timers to activate.  Therefore: 3a. when your application starts, you must load all sessions that have active process instances that have timers(**). *You can create a set of session with process that contains timers and a different set of sessions with the rest of the processes. I have an alternative solution for this kind of issues, which is to delegate those timers to an external component.* 3b. you must not have the same session active in two different nodes of a cluster, or the same timers will expire around the same time *Why do you want to have the same session in multiple nodes?* 3c. when a node crashes, your application must detect this and reload the sessions that were active in the crashed node *This is extremely related with your application, and yes you need to do your housekeeping* 4. If you start a process instance in a session, that process instance must always be executed in that session. 4a. when a node wishes to resume a process instance that was persisted, it must first (due to 3b) ask all other nodes if they have the session active, and if so instruct them to dispose it.  It can then load the session, load and resume the process.  All while preventing race conditions. *Why do you want to have the session in multiple nodes? you should think in a different scheme where a retry mechanisms kicks in if something goes wrong. That retrying mechanisms can be an external component that keeps track of the sessions and know where and how to load them.* 4b. when a node receives an event it must (due to 2) carry out all the processing in 4a for each session with active process instances. *same as before* *Hope it helps! * -------------------------------------------------------------- Reply to this message by going to Community [https://community.jboss.org/message/771738#771738] Start a new discussion in jBPM at Community [https://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Brian Amundson [https://community.jboss.org/people/amundb] created the discussion "Re: jBPM5 - Can't Scale, Won't Scale" To view the discussion, visit: https://community.jboss.org/message/779354#779354 -------------------------------------------------------------- Good show Arnold. Came here to basically find answers to your same questions. From what I have gathered and expierenced, you are correct. There is no practical way to deploy this in a cluster, as I have been trying for 4 weeks now. Whenever faced with this question, I have found the developers here give the same vauge answers with no real solutions. -------------------------------------------------------------- Reply to this message by going to Community [https://community.jboss.org/message/779354#779354] Start a new discussion in jBPM at Community [https://community.jboss.org/choose-container!input.jspa?contentType=1&...]

Kris Verlaenen [https://community.jboss.org/people/KrisVerlaenen] created the discussion "Re: jBPM5 - Can't Scale, Won't Scale" To view the discussion, visit: https://community.jboss.org/message/801189#801189 -------------------------------------------------------------- 1. Everytime jBPM5 executes a process the database transaction updates the process instance info AND the session info *Correct* therefore: 1a. it's not scalable to use a single session to execute all processes, or you would suffer contention on the session info *If you only have one session, there would be no contention as the ksession would solve this* *But if you have multiple sessions, they should have a different session id, yes, as otherwise they would be contending * 1b, it's not scalable to use a single session to execute all processes in a cluster, or the updated session info would have to be continually synchronised across the cluster *Correct, in a cluster, each node in the cluster would have a different session, you should avoid having multiple sessions with the same id active at the same time (within or across multiple nodes of a cluster) to avoid this contention* 2. When using BPMN2 events, jBPM5 only allows you to send events to the process instances within a single session at a time. *There are two methods, one to signal one process instance and one to signal the ksession (all process instances), but the ksession can load process instances on the fly.  Actually, by default, there are no process instances in a session, they are all loaded on demand.  For example, if you complete some task, the session will be notified, it will automatically load the process instance in question and signal it.  The same is done for other features like signaling, timers, etc.* You need to maintain a list of all the sessions which have incompleted process instances(*), and loop through them all to send events. *When signaling the session, the engine will figure out which process instances might be waiting for them and load them on the fly.* Therefore: 2a. you should execute all processes in as few sessions as possible, to lessen the number of iterations through this loop. *While I believe this isn't really necessary based on the previous argument (as a session can load process instances on the fly), this isn't per se a bad strategy (it's probably a good idea to not more sessions than you need active at the same time).* 3. jBPM5 persists BPMN2 timer info in the session info, but the session must be active (ie. loaded from persistence) in order for the timers to activate. *Correct* Therefore: 3a. when your application starts, you must load all sessions that have active process instances that have timers(**). *Correct, we are working on a strategy that allows you to dynamically reload sessions on the fly for timers, which will be available in the next release.  If you can limit the amount of active sessions that might contain timers though, this doesn't necessarily have to be a problem.* 3b. you must not have the same session active in two different nodes of a cluster, or the same timers will expire around the same time *Yes, see 1* 3c. when a node crashes, your application must detect this and reload the sessions that were active in the crashed node *If those sessions were holding timers, yes.  This is actually a common approach: have N active sessions distributed over M nodes, if a node goes down, sessions are redistributed amongst the available nodes.* 4. If you start a process instance in a session, that process instance must always be executed in that session. *No, process instances can easily continue execution in a different session than it was created in* 4a. when a node wishes to resume a process instance that was persisted, it must first (due to 3b) ask all other nodes if they have the session active, and if so instruct them to dispose it.  It can then load the session, load and resume the process.  All while preventing race conditions. *No, you can have the same session active on multiple nodes if you want.  This means however that there is a risk of conflicting commits.  If the risk on conflicting commits is high, this should be avoided.  But there are situations where this would be totally acceptable.  For example, consider a situation where you use a specific session per process instance.  The chance of having multiple requests coming in for the same process instance is generally low, and therefore instantiating and disposing sessions on request on whatever node the requests comes from would be possible.* 4b. when a node receives an event it must (due to 2) carry out all the processing in 4a for each session with active process instances. *Not sure what you mean by this* (*) I don't think it's possible to know if a session has incompleted process instances..? *Depends on what you mean.  As I explained in 2, as session doesn't keep all active process instances in memory, but they are loaded on demand, only the process instances that are currently executing are kept in the session.  At any point, you can ask the session which process instances it is executing.  If you want to konw which process instances are active, we recommend using the history log for this (as depending on your situation this could be a very large number).* (**) I don't think it's possible to know if a session and timers..? *You can ask a session its idle time, which means the time it expects to be idle (typically until the next timer would fire).  You can also ask the timer manager which timers it has if you need more detail* *Note that for 6.0, we will be providing a lot more advanced session management out-of-the-box.  You'll be able to use these session managers (in different configurations, like one singleton session or session per request etc.), and the jbpm-console (which currently uses one singleton session, or when used in a cluster one session per node) will also support these.* Kris -------------------------------------------------------------- Reply to this message by going to Community [https://community.jboss.org/message/801189#801189] Start a new discussion in jBPM at Community [https://community.jboss.org/choose-container!input.jspa?contentType=1&...]