How to properly handle decoder chain in netty?
Jiang Bian
timobj at gmail.com
Wed Jun 3 08:47:32 EDT 2009
Frederic -
Thank you for the help!
I have talked to trustin yesterday on IRC, and he pointed out exactly
the same thing that the context is shared between handler. When I read
the doc, I thought, it's unique for each channel, but I didn't realize
it's unique for each handler. Thanks for clear that up.
And thank you for point out the System.exit(-1). I was trying to solve
another problem, so I put in System.exit(-1) to see the exception (too
many data coming through, it's hard to catch up with the log...).
About the problem, probably you have a better idea than me. In my own
protocol, each communication should start with a magic number
(integer) followed by another integer to indicate the actual
operation. Also, I am transferring pretty big files from the client to
the server, so I did a customized protocol which split the file into
chunks (16384 bytes) each, and join them on the server side. However,
when I am sending a big file (20M not so big, actually), there are
always a couple (around 20) chunks get corrupted.
I can understand if I am sending the file through a real network,
there might be packet lost. However, I am doing the test on one
computer (loopback). Is this a normally behavior?
Thanks again,
Jiang
On Wed, Jun 3, 2009 at 12:45 AM, Frederic Bregier (via Nabble)
<ml-user+63180-750184808 at n2.nabble.com> wrote:
> Hi Jiang,
>
> I will only focus on some points, hoping this can help you.
>
> First, on ChannelHandlerContext, if you refer to the API (see
> ChannelHandlerContext.html),
> it says that a context is unique for each couple channel/handler, which
> means
> the two decoders will not share the same context.
>
> According to the API, even several instance of the same handler class will
> result in different context.
> And even several entries of the same handler object in the same pipeline
> will result in different context.
>
> As you rely on the context between the two decoder handlers, and as they
> should not share
> any context (they are different object even if they are in the same
> pipeline), and thus the context attachment should be different.
> You could use ChannelLocal for this or passing to the next decoder an object
> that have
> both the JigDFSCommand and the buffer as attributes, then getting this
> object
> in the second decoder directly.
>
> Also another point, is it ok for your program to have a "System.exit(-1)" in
> the first decoder?
> Your server will immediately stop, even if they are other clients connected.
> If this is the behaviour you wanted, that's fine. However I would suggest
> you to cleanly
> close the connection in order to not have "opened" port on your server...
>
> If this is not what you wanted, perhaps what you need is to close the
> channel
> immediately and not passing to the next decoder (for instance, returning
> null
> from the decoder method could prevent the next steps in the next decoder in
> the pipeline).
> Another way (probably better) is to throw an exception and to get it in the
> last handler and to close the
> connection there.
>
> HTH,
> Frederic
>
> Jiang Bian wrote:
> I am just wondering whether I am doing the right thing. Don't want to walk
> on the wrong path too far...
>
> I am doing a customized binary protocol. Here is my pipeline factory
>
> @Override
> public ChannelPipeline getPipeline() throws Exception {
> ChannelPipeline p = pipeline();
> p.addLast("logger", new LoggingHandler(true));
> p.addLast("jigdfsDecoder", new JigDFSCommandDecoder());
> p.addLast("chunkDecoder", new FileChunkDecoder());
> p.addLast("handler", new NodeServerHandler(this.node));
> return p;
> }
>
> As you might have guessed, every message in my system starts with a magic
> number, followed by another integer indicates the command. So I have a
> JigDFSCommandDecoder() (extends OneToOneDecoder) upfront to make sure every
> message is generated by my protocol.
>
> I am also sending large files across the network. So I have a
> FileChunkDecoder (extends FrameDecoder) to decoder the ChannelBuffer to a
> Chunk object. It also do some other crazy things, verifying md5, tracking
> segment index, etc., so I didn't used LengthBasedFrameDecoder directly.
>
> Here comes the question: In the JigDFSCommandDecoder, I still return the
> Channel, and set the context attachment to be the JigDFSCommand.
>
> @Override
> protected Object decode(ChannelHandlerContext ctx, Channel channel,
> Object msg) throws Exception {
> if (!(msg instanceof ChannelBuffer)) {
> return msg;
> } else {
> ChannelBuffer buffer = (ChannelBuffer) msg;
>
> if (buffer.readableBytes() < 8) {
> return null;
> }
>
> int magicNumber = buffer.readInt();
> if (magicNumber != JigDFSCommand.JIGDFS_MAGIC_NUMBER) {
> logger.info("It's not a JigDFS message! magic number: "
> + magicNumber);
> logger.info(buffer.toString());
> System.exit(-1);
> return 0;
> }
>
> int command = buffer.readInt();
>
> ctx.setAttachment(new JigDFSCommand(command));
> return buffer;
> }
>
> }
>
> Then in the FileChunkDecoder, I am checking to see if it's ChunkData
> command.
>
> @Override
> protected Object decode(ChannelHandlerContext ctx, Channel channel,
> ChannelBuffer buffer) throws Exception {
>
> if(!(ctx.getAttachment() instanceof JigDFSCommand)){
> return null;
> }
>
> JigDFSCommand cmd = (JigDFSCommand) ctx.getAttachment();
>
> if(cmd.getJigDFSCommand() != JigDFSCommand.CHUNK_DATA){
> return null;
> }
>
> // Make sure if the segment, md5 & length fields were received.
> if (buffer.readableBytes() < 24) {
> // The length field was not received yet - return null.
> // This method will be invoked again when more packets are
> // received and appended to the buffer.
> return null;
> }
>
> ... the rest is irrelevant...
>
> }
>
> Am I doing the right thing? Or is there a better/correct way to do this?
>
> Thanks a bunch in advance,
>
> Jiang
>
>
>
>
> Hardware/Software Architect
>
> ________________________________
> This email is a reply to your post @
> http://n2.nabble.com/How-to-properly-handle-decoder-chain-in-netty--tp3015408p3016523.html
> You can reply by email or by visting the link above.
>
>
--
View this message in context: http://n2.nabble.com/How-to-properly-handle-decoder-chain-in-netty--tp3015408p3017984.html
Sent from the Netty User Group mailing list archive at Nabble.com.
More information about the netty-users
mailing list