[jboss-dev-forums] [Design of Messaging on JBoss (Messaging/JBoss)] - Re: Remoting - Server-side Thread-per-connection Model

[timfox]

There is also another issue here:

Currently the channels are architected according to a SEDAish style.

All the message adding and delivery operations for a particular channel are handled by the same thread, this means we reduce context switches because there aren't multiple threads blocking and unblocking to do operations on the channel.

Each channel has it's own "event queue" where delivery, or message handling operations can be deposited for execution on the queues thread. (Actually different queues can share the same thread but that is not relevant to the discussion).

Currently we only have a partial SEDAish approach since remoting does not allow us to write back to the socket using a different thread.

E.g. ideally we want to do something like this (using NIO):

Have a selector loop which responds to NIO events on the NIO channels, then depending which queue the event is destined for (the event could be a send, or an acknowledge or a cancel for instance), the selector loop thread would lookup the correct queue and deposit the event on the queues event queue.

For those events which can be asynchronous (e.g. ack, cancel) the request eventually gets processed by on the event queue thread and that is the end of that.

For synchronous operations e.g. send, the event queue thread would write back the response (non blocking) to the channel when the send was complete.

This gives a very nice usage of threads with a minimal amount of forced context switches - the only ones are in transferring the event from the selector thread to the event queue thread.

Currently since we're using blocking IO, and remoting needs to write the response back on the same thread as the request, in the case of send the following happens:

The server socket thread unblocks as the invocation arrives, the event is added to the queues event queue, then the server socket thread needs to block again until the send has been processed on the event queue (it uses a future for that). Therefore we have an extra context switch.

If we have a lot of requests then we have a lot of threads blocking/unblocking.

I suspect this would have performance implications.

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3979343#3979343

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=3979343