[jbpm-issues] [JBoss JIRA] Created: (JBPM-2531) adding thread level priorities to job execution

adding thread level priorities to job execution ----------------------------------------------- Key: JBPM-2531 URL: https://jira.jboss.org/jira/browse/JBPM-2531 Project: jBPM Issue Type: Feature Request Security Level: Public (Everyone can see) Components: Runtime Engine Reporter: Tiese Barrell Fix For: jBPM 4.x Below is a description of a number of use cases this can be used for. Comment by Tom Baeyens (email): all these use cases boil down to one technical solution: job prioritization on the thread level. potentially in combination with async continuations which we already have. Use Case: Use priority in a system with multiple process definitions of different complexity to enable rapid responses for instances of simple process definitions without human tasks. Description: Our system provides different interface which consume input data and start new process instances to process these data. The data determine the process definition used to create a new process instance. The system currently supports about 20 different process definitions with varying complexity. The processing of some data sets is quite time consuming due to expensive calculations on large data sets (different data sets starting at 100 kB, mostly around 1 MB to 3 MB, sometimes even more than 10 MB) and complex process definitions involving multiple human tasks. Other data sets are smaller requiring much less calculation and the process definition is designed for rapid (a few seconds) responses without any human tasks. We would like to use priorities handled by jBPM in order to implement a mechanism that provides some limited control on the processing order. We do not expect that jBPM stops processing any process instance that it is currently processing just because a new process instance with higher priority was created. However, we would like to see that jBPM pays attention to priority when choosing a new process instance for processing, e.g. when a lot of process instances wait for processing, start with those having the highest priority. The priority values can be configurable values for the different process definitions or submitted by the originator together with the request which starts a new process instance. Use Case: Process priority needed under heavy load Description: We handle messages in different processes for several customers, using a single jBPM instance. The bulk work of the processes (all for one customer) takes a relatively long time to process (there are external systems involved) whereas some processes (for other customers) are more rare and short term processes, but of greater importance so they need to be handled quickly. jBPM offers no way to distinguish between process based on some sort of priority or to define how many processes may be run in parallel. Current solution: Using pooled executors (external lib) with an internal message queue that assign work to jBPM. Each executor has a number of threads it may use towards jBPM. Which executor is used (messages all arrive from a single JMS queue) depends on some characteristic of the message, which indicates the priority. The priority queue has a certain amount of dedicated threads to work with and the chance short running processes get through quicker is therefore greater, although it's not guaranteed. This solution is far from ideal since it requires an extra library and more importantly, management of the thread pool with all related problems (blocked threads, etc.) Use Case: Customer must indicate whether a process should be handled with priority Description: A flag needs to be set by the customer when creating a process that this should be handled with priority. This basically means that if there are two processes on offer to start (arriving on JMS queues), the priority one should be taken first by jBPM. Logically, this involves throttling throughput of work in a general sense since a workload must be maintained for some time to be able to compare priorities. If there is no other work, low priority processes can be started, but if there are high priority processes, these should go first. To ensure this works with a certain tolerance, you would have to postpone immediate execution (or think of some other solution). Current solution: still in development, but we are leaning towards the one described above. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira