Progressively decode a message (and other questions)

["이희승 (Trustin Lee)"]

Hi Ashwin,

On 11/15/2010 04:16 AM, Ashwin Jayaprakash wrote:
> Hi, I'm a Netty noob and I have some basic questions. I'm still
> exploring the API so be gentle :)
> 
> Is there a way to incrementally decode a message from a ChannelBuffer?
> Like this:
> 
>    1. Read some key integer fields
>    2. If not interested, discard message - do not decode the rest of the
>       message. Just discard it. How do I cancel further downstream codecs?

You need to maintain additional state to discard the remainder.
LengthFieldBasedFrameDecoder does exactly what you want:


http://docs.jboss.org/netty/3.2/xref/org/jboss/netty/handler/codec/frame/LengthFieldBasedFrameDecoder.html

You can optimize it even further by not extending FrameDecoder if you
really want, but I think it will perform just well enough.

> There are some other restrictions:
> 
>     * I do not want to do a getBytes(xx) on the message and make a copy
>       of the remaining message from the buffer to a byte[] . I know
>       people would suggest this because it's the easiest workaround by
>       doing the decode on my own from a byte[]. I want to avoid copying
>       the bytes again and again from layer to layer

>     * If I used slice() I suppose it would help avoid a copy?

Yes.

>     * Are these channelbuffers pooled? How long can I hang on to the
>       MessageEvent, Channel or Channelbuffer instance? Would I be
>       starving some other component/connection?
>           o FYI, I had asked a similar question on the Google Protobuf
>             forum -
>             http://groups.google.com/group/protobuf/browse_thread/thread/4563998c85d01882/1859fe2958485da2

ChannelBuffers are not pooled.  By default, Netty relies on JVM's GC and
it seems to scale pretty well.  However, under some constraints, you
could write a very efficient buffer pool which might beat JVM, but I'm
not sure it will be a generic enough solution, and that is why Netty
does not ship a buffer pool at the moment.  If relatively generic
solution is found, I'd love to ship it so that many applications reap
the benefit.

>     * If I want to re-route my server request from 1 server to another
>       server - read some bits and then decide to "forward" the message
>       to another server, can I just do a zero-copy of the original bytes
>       and send them to another server? You see - like a Hibernate
>       disconnected session or a SQL Disconnected row set?
>           o The server now becomes a client to another server, so can I
>             just use the same buffers and "transfer" them to the other
>             handler that is talking to the final server?

Reading some bits involve copying them from kernel socket buffer to user
heap, so there's no way to do a zero copy.  What you can do is always
'minimize' the number of copies, and you can avoid unnecessary memory
copy as much as you can with Netty as you write your own NIO application.

> Confused? :)

Good luck! :-)

> Super clean APIs though, I'm impressed.

Thanks.  Any suggestion for improvements?

-- 
Trustin Lee, http://gleamynode.net/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 294 bytes
Desc: OpenPGP digital signature
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20110110/cead120a/attachment.bin