[jboss-dev-forums] [Design of Messaging on JBoss (Messaging/JBoss)] - Re: JBM 2.0 Thread Model

[david.lloyd@jboss.com]

"jmesnil" wrote : All JMS connections and their children (session, producer, consumer) to the same JMS server use a single MINA NioSocketConnector (i.e. one single TCP socket).

This is very similar to what I'm doing with Remoting 3 ("R3"), and I've run into the same issues as you.

In R3, the protocol consists of message datagrams tunneled over a single TCP connection.  We have the additional issue that certain messages must be well-ordered with respect to certain other messages (though this isn't always the case; some messages can be processed in any order).  In order to maximize throughput, we want to process messages in parallel whenever it's OK to do so.

To achieve this, I've done two things.  First, every incoming message is assigned to an Executor.  Second, I've got a simple class called OrderedExecutorFactory which can produce Executor instances which execute in order with respect to that Executor, but in parallel with respect to other ordered Executors and unordered tasks.  (see http://tinyurl.com/33b4gq)

As an aside, by using Executors rather than a ThreadFactory, I can push the responsibility of maintaining a thread pool off to someone else; yet a simple implementation is still available in the standalone case by doing Executors.newCachedThreadPool() for example.

Since the ordering of messages is a detail of the protocol, the messages have to be at least partially decoded before they can be assigned to an Executor.  Therefore I don't bother with IoFilters or anything like that - I've found that it's simpler to just have an IoHandler and do all the message decoding and Executor delegation within the handler.

I also hope to add support for SCTP in the future, which allows multiple independent streams within a single connection (among other cool features), reducing this "head-of-line" contention issue.  However, while I've heard that there is an implementation in the works at Sun, it's still probably a ways off yet (unless one wants to use APR, which adds other dependencies as well).

One thing I was thinking about as a possible performance enhancement would be to have more than one connection - not one per session necessarily, but maybe like 2-4 connections, using some type of load-balancing among them.  This should give SCTP-like behavior (well, to a limited extent) with reduced head-of-line contention, but without a ridiculous explosion of connections.

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4140594#4140594

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4140594