Content ChannelBuffer Factory for HTTP Codec?

[Wade Poziombka]

Thanks for your prompt reply!

 

We do need chunked encoding but my issue is more that ultimately I need the
content in buffer(s) of my own.  

 

My suggestion is to allow me to provide a ChannelBuffer that wraps my own
buffer(s) for the content portion of the message.  In order to do so I would
need to provide some kind of a factory to the decoder.  This way as the
decoder is decoding message content (chunked or otherwise) it could be put
into my own buffer.  For example if a method on HttpMessageDecoder I could
supply an object that has method similar to:

 

public interface ContentChannelBufferFactory

{

    ChannelBuffer createContentBuffer(ChannelContext ctx);

}

 

Then I could provide a buffer that will receive the content from the decoder
into the buffers without additional copies (or buffering the entire
message).

 

I could modify the encoder myself but under LGPL I would then have to
publish my code.  Looking at the code it seems quite easy to do technically.
However, this is difficult legally.  

 

Wade

 

  _____  

From: Trustin Lee-2 (via Nabble)
[mailto:ml-user+63047-729345966 at n2.nabble.com] 
Sent: Wednesday, December 03, 2008 12:22 AM
To: Wade Poziombka
Subject: Re: Content ChannelBuffer Factory for HTTP Codec?

 

Hi Wade, 

On Wed, Dec 03, 2008 at 01:00:16PM +0900, Wade Poziombka wrote: 
> 
> You guys are doing great work. We are very keen to use Netty in our
product. 
> We've done a lot of testing of the various async frameworks and netty has 
> come out on top.  I was delighted to see the HTTP codec recently added as 
> that is of primary interest along with a few other protocols such as MLLP.


Thank you!  I appreciate that you shared your idea about Netty, and 
please keep us updated with your feed back. :) 

> We have some special needs as we integrate with some native components
that 
> have a preexisting buffering mechanism (chains of buffers from pools in
form 
> of native (shared memory) heap). 
> 
> I do not want to use these buffers for all IO (but it would be ideal if
the 
> HTTP content could be populated into these buffers without first buffering

> the entire message then copying.   

The current HTTP decoder is implemented to read the whole content.  It 
is true that it can consume a lot of memory.  As an alternative, chunked 
encoding could be used to send and receive the content split by multiple 
chunks.  Unfortunately, the support for chunked encoding has not been 
implemented yet - it should be supported before Netty 3.1 enters release 
candidate phase though.  Once implemented, HTTP message decoder will 
generate a decoded object per chunk, avoiding excessive memory 
consumption. 


> It seems that if there were some mechanism for me to supply a
ChannelBuffer 
> to the Codec when decoding the content portion of the message that I might

> be able to achieve the desired affect.   
> 
> What I want to avoid is buffering the entire content of a request into 
> ChannelBuffer(s) and then have to copy it to other buffers, especially
given 
> that these might be large (megabytes). 
> 
> Any thoughts on this idea, or perhaps I'm missing a way to achieve this 
> already? 

If the chunked encoding is not an option for you, I think 
HttpMessageDecoder could be modified to 'emulate' the chunked encoding, 
which means the decoder could generate the decoded object for every 8192 
bytes for example.  Please let me know if there's a better idea. 

-- 
Trustin Lee, Principal Software Engineer, JBoss, a division of Red Hat 
-- 
what we call human nature in actuality is human habit 
-- 
http://gleamynode.net/


_______________________________________________ 
netty-users mailing list 
netty-users at ...
<http://n2.nabble.com/user/SendEmail.jtp?type=node&node=1607761&i=0>  
https://lists.jboss.org/mailman/listinfo/netty-users


  <http://n2.nabble.com/images/icon_attachment.gif> attachment0 (204 bytes)
Download <http://n2.nabble.com/attachment/1607761/0/attachment0>  Attachment

 

  _____  

This email is a reply to your post @
http://n2.nabble.com/Content-ChannelBuffer-Factory-for-HTTP-Codec--tp1607490
p1607761.html
You can reply by email or by visting the link above.

 


-- 
View this message in context: http://n2.nabble.com/Content-ChannelBuffer-Factory-for-HTTP-Codec--tp1607490p1609102.html
Sent from the Netty User Group mailing list archive at Nabble.com.