Content ChannelBuffer Factory for HTTP Codec?

[Trustin Lee]

Hi Wade,

On Wed, Dec 03, 2008 at 01:00:16PM +0900, Wade Poziombka wrote:
> 
> You guys are doing great work. We are very keen to use Netty in our product. 
> We've done a lot of testing of the various async frameworks and netty has
> come out on top.  I was delighted to see the HTTP codec recently added as
> that is of primary interest along with a few other protocols such as MLLP.  

Thank you!  I appreciate that you shared your idea about Netty, and
please keep us updated with your feed back. :)

> We have some special needs as we integrate with some native components that
> have a preexisting buffering mechanism (chains of buffers from pools in form
> of native (shared memory) heap).
> 
> I do not want to use these buffers for all IO (but it would be ideal if the
> HTTP content could be populated into these buffers without first buffering
> the entire message then copying.  

The current HTTP decoder is implemented to read the whole content.  It
is true that it can consume a lot of memory.  As an alternative, chunked
encoding could be used to send and receive the content split by multiple
chunks.  Unfortunately, the support for chunked encoding has not been
implemented yet - it should be supported before Netty 3.1 enters release
candidate phase though.  Once implemented, HTTP message decoder will
generate a decoded object per chunk, avoiding excessive memory
consumption.

> It seems that if there were some mechanism for me to supply a ChannelBuffer
> to the Codec when decoding the content portion of the message that I might
> be able to achieve the desired affect.  
> 
> What I want to avoid is buffering the entire content of a request into
> ChannelBuffer(s) and then have to copy it to other buffers, especially given
> that these might be large (megabytes).
> 
> Any thoughts on this idea, or perhaps I'm missing a way to achieve this
> already?

If the chunked encoding is not an option for you, I think
HttpMessageDecoder could be modified to 'emulate' the chunked encoding,
which means the decoder could generate the decoded object for every 8192
bytes for example.  Please let me know if there's a better idea.

-- 
Trustin Lee, Principal Software Engineer, JBoss, a division of Red Hat
--
what we call human nature in actuality is human habit
--
http://gleamynode.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20081203/79494ebb/attachment.bin