Netty performance question

[Trustin Lee]

Hi Matt,

On Sun, Apr 26, 2009 at 4:53 AM, omsdw <mgiedt at gmail.com> wrote:
>
> Hi -- I have a question on how best to configure Netty for maximum message
> throughput.
>
> I've created a new 'AggregatedDelimiterBasedFrameDecoder' that is
> essentially identical to the DelimiterBasedFrameDecoder except that it
> returns a List<ChannelBuffer> as opposed to a single ChannelBuffer on each
> call to decode.
>
>    @Override
>    protected Object decode( ChannelHandlerContext ctx,
>                             Channel channel,
>                             ChannelBuffer buffer )
>        throws Exception
>    {
>        final List<ChannelBuffer> frames = new ArrayList();
>        ChannelBuffer frame = null;
>
>        while( ( frame = nextFrame( ctx, channel, buffer ) ) != null )
>        {
>            frames.add( frame );
>        }
>
>        return frames.size() == 0 ? null : frames;
>    }
>
>    private final ChannelBuffer nextFrame( ChannelHandlerContext ctx,
>                                           Channel channel,
>                                           ChannelBuffer buffer )
>        throws Exception
>    {
>       // this is an identical copy of the original decode from
> DelimiterBasedFrameDecoder...
>    }
>
> And then my AggregatedStringDecoder looks like...
>
>    public void handleUpstream( ChannelHandlerContext context,
>                                ChannelEvent evt )
>        throws Exception
>    {
>        if(! ( evt instanceof MessageEvent ) )
>        {
>            context.sendUpstream( evt );
>            return;
>        }
>
>        final MessageEvent e = (MessageEvent) evt;
>        if(!(e.getMessage() instanceof List))
>        {
>            context.sendUpstream(evt);
>            return;
>        }
>
>        final List<ChannelBuffer> frames =
> (List<ChannelBuffer>)e.getMessage();
>
>        for( int i=0; i<frames.size(); i++ )
>        {
>            fireMessageReceived( context, e.getChannel(),
>                frames.get( i ).toString( charsetName ) );
>        }
>    }
>
> So far so good? Would you recommend any changes? Curious if you think in the
> long run this is better than just one at a time? (Again, the goal here is to
> improve throughput -- I'm also happy to contribute the code...)

There's a chance where an ArrayList is created and unused.  I would
create it right after decoding the first message is done.

If you are using Netty 3.1, FrameDecoder has an option called 'unfold'
which allows you to return a Collection or an array and then
FrameDecoder will unfold it into multiple messageReceived events just
like your AggregatedStringDecoder does.  Same feature has been added
to ReplayingDecoder.

> Now I have a ExecutionHandler in between the AggregatedStringDecoder and my
> BusinessLogicHandler in the pipeline factory.
>
>    public ChannelPipeline getPipeline()
>        throws Exception
>    {
>        final ChannelPipeline pipeline = super.getPipeline();
>
>        pipeline.addLast( "framer",
>            new AggregatedDelimiterBasedFrameDecoder(
>                BUFFER, Delimiters.lineDelimiter() ) );
>
>        pipeline.addLast( "decoder", new AggregatedStringDecoder( "UTF-8" )
> );
>        pipeline.addLast( "encoder", new StringEncoder( "UTF-8" ) );
>
>        pipeline.addLast( "executor", new ExecutionHandler(
>            new OrderedMemoryAwareThreadPoolExecutor(
>                THREADS,
>                MAX_CHANNEL_MEMORY_SIZE,
>                MAX_TOTAL_MEMORY_SIZE ) ) );
>
>        pipeline.addLast( "handler", handler );
>
>        return pipeline;
>    }

You need to share the same ExecutionHandler instance for all channels
in the same service.  Otherwise, OrderedMemoryAwareThreadPoolExecutor
will be created per channel, which is very inefficient.  Perhaps you
will get an IllegalStateException during performance test because I
put some misconfiguration detection code so that it doesn't allow too
many active executor instances.

> Shouldn't each call to 'fireMessageReceived' be handled in a separate thread
> by the Executor? (I'm not seeing this) I also don't think I *really*
> understand the difference between the ordered and non-ordered executors.

All events for the handlers after the ExecutionHandler will be handled
in a separate thread.  If not, it is a bug.  Perhaps you could print
some stack trace and current thread name
(Thread.currentThread().getName()) to see if it's the case.

Non-ordered executors can mix the event orders and they can even be
executed simultaneously.  For example, if you decoded three messages
and the handler after the ExecutionHandler with a non-ordered executor
performs something that takes long time, your handler will be handling
the three messages at the same time unless your handler is
synchronized.  Also, channelConnected event can reach before
channelOpen event although it doesn't occur very often, so you can't
rely on event order to build a state machine at all.

HTH,

— Trustin Lee, http://gleamynode.net/