HTTP method for POST query argument

Tue Sep 1 14:57:20 EDT 2009

Hi Trustin,

No problem... I'm still currently improving it (same idea, same API but more
efficient on memory aspect).

Also I try to start a simple encoder, but it is no so simple since I need to
make the header too (due to multipart) and to take care of specific cases
(multiple files send from one form parameter which implies multipart with
mixed multipart).

However, I will not publish it until I've come to something clear enough and
also until we make some general ideas on how to do this, perhaps even on
decoder, in order to not make this thread full of iterative versions... ;-)
In this case, less version is better (at least, what I feel ;-)

Cheers,
Frederic

Trustin Lee wrote:
> 
> Hi Frederic,
> 
> This sounds pretty neat although I didn't review it yet.  I think I
> need to review the new buffer type you proposed first and then get back
> here.
> 
> Thanks and cheers,
> Trustin
> 
> On Sat, 29 Aug 2009 22:28:48 +0900 (KST)
> Frederic Bregier <fredbregier at free.fr> wrote:
>> 
>> Hi,
>> 
>> First, I thank you all for your advices!
>> 
>> I try to follow your ideas. I hope I did well. I join the zip that
>> includes all files.
>> As I said, I create a fork of http codec into http2 while
>> implementing and testing, but of course, it should replaced once ok
>> the current codec. In the zip, I join too the AggregateChannelBuffer
>> since my implementation uses it to minimize memory usage with still
>> no copy.
>> 
>> I retain the following ideas:
>> 
>> - I rename HttpDataDecoder to HttpBodyRequestDecoder, which seems to
>> me more accurate.
>> 
>> - Each time a chunk is offered, the decoder starts to decode what it
>> can already decode. The interest is it moves from the current
>> ChannelBuffer to HttpData objects as soon as possible, so, using
>> AggregateCB, minimize the memory usage compares to if we retain all
>> ChannelBuffer until all data will be decoded (doubles data, one in CB
>> and one in HttpData). I take your proposition of decoder.offer(chunk).
>>   It throws ErrorDataDecoderException if an error occurs while
>> decoding (bad charset, bad IO, ...).
>> 
>>   Note that Attribute can be huge (TEXTAREA in a FORM has no real
>> limit and can be huge, for instance copying the content of a Text
>> file into it) but it will not be stored in a File since it is not a
>> FileUpload but a simple parameter. So I like the idea to try to
>> prevent OOME as much as possible. Perhaps more optimization can be
>> done, like implementing a virtual attribute that could be stored into
>> a temporary file (like DiskFileUpload but for Attribute) if the size
>> becomes too high. WDYT?
>> 
>> - Of course it is compatible with both chunked and chunked body. In
>> not chunked body, from the constructor, all HttpData are decoded.
>> 
>> - No finish() is necessary since it will find by itself that the last
>> chunk is the last one.
>> 
>> - Two kinds of access to HttpData (either Attribute or FileUpload) co
>> exist since I feel like one is simpler (yours) but the other one
>> allows some specific optimizations if wanted by the user.
>> 
>> 1) Access to HttpData once all chunks are received (throw
>> NotEnoughDataDecoderException if the last chunk is still not
>> received).
>>   - List<HttpData> getBodyHttpDatas(); // all HttpData
>>   - List<HttpData> getBodyHttpDatas(String name); // all HttpData
>> associated with name
>>   - HttpData getBodyHttpData(String name); // first HttpData
>> associated with name
>> 
>> 2) Access to HttpData along offer(chunk) calls:
>>   - boolean hasNext() throws EndOfDataDecoderException 
>>     //which says if there is at least one available HttpData in order
>> (no back but could be implemented)
>>   - HttpData next() throws EndOfDataDecoderException // gives next
>> available HttpData in order
>> 
>>     EndOfDataDecoderException is throwed if the decoder arrives at
>> the very end (last chunk was received and the iteration is over).
>> This is necessary to make the difference between no data are
>> available but could be after next call to offer(chunk), or no data
>> are available and they will never be others (last chunk is already
>> received).
>> 
>> In the snoop example, I show both usages.
>> 
>> - I remove all references to Cookie and Header since you're right they
>> already have their own decoder and its fine. However I add some
>> convenient methods to HttpMessage that I feel they were missing:
>>    - Map<String, List<String>> getHeaders();
>>    - Set<Cookie> getCookies();
>>    - void addCookie(Cookie cookie, boolean isServer);
>>    - void setCookies(Set<Cookie> cookies, boolean isServer);
>> 
>>    For Cookies, I admit this is not efficient since I create a
>> encoder or decoder at each call. I would prefer at least to store the
>> Set result of decoder. For the encoder, as once encode() method is
>> called, it is purged, except rewriting a bit the encoder, I saw no
>> efficient implementations. I'm not so happy with this, but I feel
>> like at least the getHeaders() and getCookies() should be kept.
>>    setCookies() is so simple that it could be kept too.
>>    But addCookie could perhaps be forgotten since it is really not
>> efficient (better to manage itself the encoder).
>>    WDYT?
>> 
>> - I keep this as a Decoder, not a handler, in order to enable mixed
>> usage by end users (implementing for instance something else than a
>> pure HTTP server).
>> 
>> - I change the DefaultFileUpload to MemoryFileUpload name.
>> 
>> - I did not changed yet the FileUpload name, but as I wrote
>> previously in this post, if Attribute could have also a Disk
>> Implementation (or other), perhaps I can try to make a common part
>> for Memory, Disk and Mixte implementation, and then extend them into
>> Attribute or FileUpload. WDYT?
>>    
>>    By the way, I believe Attribute and FileUpload still need to be
>> different types since they don't have the same properties (FileUpload
>> must have a Content-Type for instance, while Attribute does not have
>> any).
>> 
>> - I agree with Jean-Francois and you that having
>> HttpServletRequest/Response could be really interesting. But I don't
>> feel right now that I've got enough knowledge to begin that work.
>> Perhaps later on, once I learn a bit more about under the wood of
>> servlet and take the time to read carefuly the Grizzly
>> implementation...
>> 
>> - For the moment, I remove the encoder from the zip, since I have
>> different options and I don't know which one could be better.
>> The question is basically:
>> Should the encoder encode the full response or just passing the
>> necessary body part?
>> The problem is that in Multipart mode or not, the header contains a
>> different Content-Type (multipart/form-data and
>> application/x-www-form-urlencoded respectively).
>> So should the encoder take care also about the header part encoding
>> (calling other encoders itself)?
>> If so, then it should take care too about the other headers and
>> Cookies... WDYT?
>> 
>> Well, I think it is a good start to stop here... ;-)
>> 
>> Cheers,
>> Frederic
>> 
>> http://n2.nabble.com/file/n3542051/http-decode4.zip http-decode4.zip 
>> 
>> 
>> -----
>> Hardware/Software Architect
> 
> 
> 
> -- 
> Trustin Lee, http://gleamynode.net/
> _______________________________________________
> netty-dev mailing list
> netty-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-dev
> 
> 

-----
Hardware/Software Architect
-- 
View this message in context: http://n2.nabble.com/HTTP-method-for-POST-query-argument-tp3457351p3562393.html
Sent from the Netty Developer Group mailing list archive at Nabble.com.