How to reliable and efficiently implement a string protocol

[Sébastien Pierre]

Hi !

I've been using Netty to implement a very simple text-based protocol for a
log service, where messages are UTF-8 strings like this:

<LOG_FILE_PATH>:<JSON_ENCODED_MESSAGE_DATA>\n

For instance

hello/world.log:"Hello, World"\n

would append "Hello, World" to the hello/world.log file.

The problem I have is that it seems that some messages get interwoven,
basically just as if the server considered two messages as being part of the
same one, and intermixing both. My problem is then: how do I ensure that
each message is reliably sent, and that the server does not "interweaves"
received messages.

The server is implemented using a ChannelPipelineCoverage("one")
SimpleChannelUpstreamHandler. The message aggregation happens in
messageReceived, which works like this:

 if message contains "\n"
   write(buffer)
else
  buffer append (message)
end

with a pipeline setup like this

        p.addLast("decoder", new StringDecoder(CharsetUtil.UTF_8));

and the client is implemented with the following decoders

        e.getChannel().getPipeline().addLast("frameDecoder", new
DelimiterBasedFrameDecoder(65535, Delimiters.lineDelimiter()(0),
Delimiters.lineDelimiter()(1)))
        e.getChannel().getPipeline().addLast("encoder",      new
StringEncoder(CharsetUtil.UTF_8));

I suppose the problem might be related to the "frameDecoder", which, to be
frank, I don't remember why I used it. So what would be the best setup for a
high-performance (10+K messages/second) client/server for this simple text
protocol ? I could use 0MQ for that, but Netty being pure Java, it's easier
to deploy.

Thanks,

 -- Sébastien
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/netty-users/attachments/20101102/c0790307/attachment.html