DelimiterBasedFrameDecoder Help

[Trustin Lee (이희승)]

Hi David,

If there is an extra bad byte in the frame, I usually raise an
exception in the decoder.  Netty will trigger an exceptionCaught
event, then you can handle it to close the connection immediately.
Depending on your preference, you can just close the connection
immediately instead of raising an exception.

FixedLengthFrameDecoder simply generates a fixed length frame, and
therefore it does not know anything about the semantics of the
received data.  Therefore bad bytes are not skipped at all.  If you
want to implement such functionality, you need to implement a decoder
by yourself.  In my opinion, generally, you should not implement such
functionality to detect bad data because 1) TCP/IP already has a
mechanism for error detection and 2) misbehaving (or malicious) client
should be disconnected immediately to avoid possible misinterpretation
of packets.  Bad data could accidently change the semantic of the
message, bypassing the validation layer of your server and leading to
a security issue.

HTH

— Trustin Lee, http://gleamynode.net/

On Fri, Sep 25, 2009 at 5:09 PM, Hoyt, David <hoyt6 at llnl.gov> wrote:
> Thanks for your help - it's very much appreciated! If we connected to a malicious server and it injected an extra byte somewhere (even though all subsequent packets would be good - say packet 1 is fine, packet 2 contains 1 extra byte, but packets 3 through 1 million are all fine), is FixedLengthFrameDecoder resilient enough to catch it and skip a frame until we get the next valid packet? Or would packets 3 through 1 million all be off? Is it easy enough to add this functionality? I think I understand how I could do it - by returning null until I see the first byte of my header and then discarding everything in the meantime.
>
> How do you discard bytes in the frame decoder? I don't want incoming bytes to accumulate if I don't care about them. If I got 100 bad packets in a row and then 1 good one, I don't want to buffer 100 bad packets - I'd want to discard all of them as they come in and keep only the good one.
>
>
>
> On another note, btw -- excellent work w/ netty! We're using it in other places, and it's outperforming every other implementation we've ever used! Very impressive...
>
> -----Original Message-----
> From: netty-users-bounces at lists.jboss.org [mailto:netty-users-bounces at lists.jboss.org] On Behalf Of Trustin Lee (???)
> Sent: Thursday, September 24, 2009 5:14 PM
> To: Netty Users
> Subject: Re: DelimiterBasedFrameDecoder Help
>
> Hi David,
>
> Actually, your message got through the mailing list.  It just took a
> while for me to answer your question.  Thanks for your patience.
>
> On Fri, Sep 25, 2009 at 2:33 AM, Hoyt, David <hoyt6 at llnl.gov> wrote:
>> I'm not sure if my original post made it or not because I had a few failed attempts - but I'm hoping this one makes it through...
>>
>> I’m still learning netty and I need to find the best classes to do what I need or find out if I need to code it myself – we have our own protocol that’s very, very simple – a header used to synchronize where we’re at in the stream and then some int’s, long’s, etc. in a known order. The entire length of the packet is known ahead of time (it's a fixed-length packet). We intend to maintain this connection for days, weeks, months, and years. So an example of a stream is (just the bytes -- "Packet #", "Header", and "Payload" don't actually appear in the data):
>>
>> Packet 1
>> Header: 0xdeadbeef
>> Payload: 0x00000000, 0x00000001, 0x00000002
>>
>> Packet 2
>> Header: 0xdeadbeef
>> Payload: 0x00000000, 0x00000001, 0x00000002
>
> If the length of the packet is fixed, it is dead simple - use
> FixedLengthFrameDecoder.  You never need DelimiterBasedFrameDecoder.
>
>> We use the header (delimiter) in case (for whatever reason – a particle from space flips a bit on the server producing these messages) we lose where we’re at and need to synchronize with the next packet. So say we got the following sequence:
>>
>> Packet 1: 0xdeadbeef 0x0x00000000 0x00000000 0x00000000
>> Packet 2: 0xabcdabcd 0x0x00000000 0x00000000 0x00000000
>> Packet 3: 0xdeadbeef 0x0x00000000 0x00000000 0x00000000
>>
>> We would want to skip packet 2 and continue on with packet 3. We would like to know if we found a problem with packet 2, but it's not a requirement.
>
> Once the FixedLengthFrameDecoder splits the incoming data as you want,
> you can inspect the header of the message to determine if the message
> is what you were expecting.
>
>> We also need to know if there's a significant delay between messages (e.g. a read timeout of, say, 30 seconds or more). We only need to ever read and never write to this stream.
>
> You could use ReadTimeoutHandler or IdleStateHandler (in
> org.jboss.netty.handler.timeout).
>
>> Does DelimiterBasedFrameDecoder allow me to recover from a bad packet? Can I recover without having to catch an exception? I'm not sure where to begin. I know how I would do it using old IO, but I'd prefer to take advantage of netty's power and expressiveness and I'm sure netty can handle this. I just don't want to reinvent the wheel.
>
> DelimiterBasedFrameDecoder is used only when there is an explicit
> delimiter defined in the protocol.  For example, a pipe (|), NUL
> (0x00), or newline character is the case - mostly text protocols.
> Your protocol does not have a delimiter, so DelimiterBasedFrameDecoder
> cannot be used.
>
> HTH
>
> — Trustin Lee, http://*gleamynode.net/
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://*lists.jboss.org/mailman/listinfo/netty-users
>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>