Maximum Performance: Netty with Protocol Buffers

[Replicator]

We are going through evaluation of Netty and ZeroMQ to see which one makes
more sense for our task:

    1. Mostly a one way communication => 99.999% of messages are coming from
A (C++) to B (Java) through the socket.

    2. A single message is roughly 120 bytes
 
    3. Need to consume 200,000 messages a second ( given a Gigabit network )

    4. Will use google protobufs to set a transparent protocol between a C++
producer and a Java consumer

Questions for the Netty community:

    1. A consumer side, don't really need to send messages to the producer
=> it just needs to consume. What would be the best way to simulate this use
case? An Echo server (but then a consumer constantly sending messages)? A
streaming server, some other setup?

    2. When using protocol buffers [ we did play with netty's LocalTime
example ], does it make sense to set FixedReceiveBufferSizePredictor /
AdaptiveReceiveBufferSizePredictor to change the buffer size, or since a
protocol would dictate a message length, the above parameters would not make
sense?

    3. We tried an example posted here on a mailing list: http://d.pr/SLYr
and we only see a steady 35,000 messages per second rate, and that is
without protocol buffers, or any major message parsing [ just getting first
byte of the message on arrival ]. It probably relates to the question #1,
but is there a way to improve this example?

    4. Since we are after the maximum throughput, would it make sense to use
a chunking functionality of Netty ( e.g. ChunkedWriteHandler, ChunkedInput,
etc.. ). If yes, are there any examples on how it may be applicable to
transferring large chunks, and then parsing messages out? Is there a way to
fetch uneven sized chunks that e.g. would stretch not to chop the last
message?

Thank you!

--
View this message in context: http://netty-forums-and-mailing-lists.685743.n2.nabble.com/Maximum-Performance-Netty-with-Protocol-Buffers-tp6750165p6750165.html
Sent from the Netty User Group mailing list archive at Nabble.com.