[esb-issues] [JBoss JIRA] Created: (JBESB-2606) ESB aware listeners have single thread popping messages from queue

[Aaron Pestel (JIRA)]

ESB aware listeners have single thread popping messages from queue
------------------------------------------------------------------

                 Key: JBESB-2606
                 URL: https://jira.jboss.org/jira/browse/JBESB-2606
             Project: JBoss ESB
          Issue Type: Feature Request
      Security Level: Public (Everyone can see)
          Components: Transports
         Environment: SOA-P 4.3 CP01
            Reporter: Aaron Pestel
            Priority: Minor


When investigating a certain case where changing the number of threads configured for an ESB aware listener did not change throughput, the following was discovered:

-------------------------------------------------------------------------------
The class that services the ESB-aware queue is the MessageAwareListener.  It uses a single thread to read messages from the queue, then uses a thread pool to allocate a thread to the processing of each message.  The effect of this is:

1) No matter how fast and optimized the pipeline is, the rate at which messages can be processed is limited by the maximum retrieval rate of the queue.
2) If the pipeline is very fast the processing of the message finishes almost as quickly as the next message can be retrieved.
3) The overhead of the thread pool contributes to the significant overall overhead of the call.
4) This probably explains why topics are faster than queues, since they are typically less linear.

Since the pipeline processing finishes quickly there is rarely any need to use more than one thread from the pool, which explains why setting maxThreads to a higher number doesn't affect the outcome.  ...

Note also that this class is used to retrieve messages from the InVM queue as well.

There are two solutions to this problem:

1) Allow the number of threads that retrieve messages from the queue to be configurable.  If order is important than one thread should be used.  If order is not important the number can be set to a much higher value allowing messages to be retrieved from the queue at a much higher rate.  ... The thread pool for processing the message can still be used if message processing is very long.

2) Use a MessageListener.  This lets the JMS implementation "give" messages to the processor rather than having the processor "retrieve" them.  The performance of this method depends heavily on the quality of the underlying JMS implementation.  If the implementation uses a single queue that retrieves messages from the queue, then passes the message to the listener, the performance will be the same as the current ESB design.

Or allow either method using configuration. 
-------------------------------------------------------------------------------

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira