New ChannelBuffer type proposed

[이희승 (Trustin Lee)]

Hi Frederic,

Now I've got some time to think about this new buffer type finally. :)

If I understood correctly, you want to improve the performance of
ChannelBuffer.discardReadBytes().  Am I right?

If so, I think we can achieve this by either:

1) Relaxing the definition of ChannelBuffer.discardReadBytes() so that
you can remove the reference to the discardable component instead of
expensive memory copy:

  ChannelBuffer composite = ...;
  .. read some ..
  composite.discardReadBytes(); // Release the reference to the
                                // discardable bytes - capacity is
                                // decreased.

2) Optimizing the implementation of CompositeChannelBuffer.slice() so
that it returns the slice that does not refer to unused components:

  ChannelBuffer composite = ...;
  .. read some ..
  composite = composite.slice(); // Unreadable components are
                                 // released.

The first change needs more thoughts but the second one can be applied
as we don't need to break backward compatibility at all.

Both changes could be made as they are complimentary, and once done, we
don't need AggregatedChannelBuffer, or am I missing out something?

BTW I hope my suggestion did not discourage you.  I just want to make
sure to find the most efficient way with least effort and least change.

Trustin

On Sun, 23 Aug 2009 06:34:17 +0900 (KST)
Frederic Bregier <fredbregier at free.fr> wrote:
> 
> Hi Trustin and all,
> 
> I've playing a bit with the Http codec and I found that having an
> aggregate version of the channelBuffer, almost like the
> CompositeChannelBuffer one, but that differs in such way that:
> 
> - when an aggregation is done (at construction for instance), only
> non empty buffer are kept.
> - when some aggregate buffer is passed at constructor argument, they
> are parsed from their slices too.
> 
> The main idea is to keep the advantage of the Composite but:
> - the memory consumption is lower by discarding unsuseful
> channelBuffer
> - the speed where no copy is necessary is the same than in Composite
> - only one level of buffer most of the time is keeped (in fact 2:
> each slice is a Slice ChannelBuffer which points to the real previous
> ChannelBuffer)
> 
> Therefore, it is quite efficient to do:
> 
> ChannelBuffer mybuffer = ChannelBuffers.EMPTY_BUFFER;
> mybuffer = AggregateChannelBuffer.wrappedCheckedBuffer(mybuffer,
> otherbuffer, ...);
> while (true) {
>     consumeSomeBytes(mybuffer);
>     mybuffer = AggregateChannelBuffer.wrappedCheckedBuffer(mybuffer,
> newlyProduceBuffer);
> }
> 
> since there is a chance that the number of buffers in the mybuffer
> will stay low in memory since there are consumed.
> In the HttpDataDecoder, the FileUpload is one good example.
> 
> Not all methods are already optimzed. For instance, it is possible to
> optimize:
> - writeBytes(ChannelBuffer, ...) methods
> I try to implement in the joined implementation the method
> writeBytes(ChannelBuffer, int, int) (the others will use it).
> 
> Note that AggregateCB has no write capability by default (due to the
> slice command used to add CB from constructor), like with CompositeCB
> too (at least, if I understand well). 
> However, getting such function could help the previous code as the
> following (simpler) but could be dizzy if the user is not warned
> enough -he could believe that other write operations are possible- :
> 
> ChannelBuffer mybuffer = ChannelBuffers.EMPTY_BUFFER;
> mybuffer = AggregateChannelBuffer.wrappedCheckedBuffer(mybuffer,
> otherbuffer, ...);
> while (true) {
>     consumeSomeBytes(mybuffer);
>     mybuffer.writeBytes(newlyProduceBuffer);
> }
> 
> Perhaps better would be to create a new method addBytes(ChannelBuffer)
> instead of writeBytes, which is more exact.
> 
>     mybuffer.addBytes(newlyProduceBuffer);
> 
> http://n2.nabble.com/file/n3496606/AggregateChannelBuffer.java
> AggregateChannelBuffer.java 
> 
> However, this is a proposition, perhaps I'm wrong...
> Of course any idea or enhancement... ;-)
> 
> Cheers,
> Frederic
> 
> -----
> Hardware/Software Architect



-- 
Trustin Lee, http://gleamynode.net/