Announcing LittleProxy 0.1 based on Netty

[Adam Fisk]

Hi Trustin- Thanks for the detailed response. I already fixed this
independently in LittleProxy running with Netty 3.1.5 through creating
a new message, so it's not an issue there. I *think* there's still a
problem in Netty in the following case, however.

Leaving the proxy case aside, the straight HttpMessageDecoder will
switch to chunked encoding any time the response body length is
greater than maxChunkSize. It does this regardless of protocol
version, so it will produce HTTP 1.0 request and responses that use
Transfer-Encoding: chunked.

It seems like it's always wrong whenever that happens, no? It could
potentially only convert to chunked encoding if the version is HTTP
1.1, but I'm pretty sure the current implementation will break things
other than LittleProxy.

Thanks again.

-Adam


On Wed, Nov 11, 2009 at 1:41 AM, Trustin Lee (이희승) <trustin at gmail.com> wrote:
> I guess I found a problem with upgrading the protocol version automatically.
>
> If a client sends a request in HTTP/1.0 (i.e. no chunked encoding
> support on the client side), the server (Netty HTTP) should not
> respond with chunked content.  If the decoder upgrades the protocol
> version, there's no way for an HTTP request handler to determine if
> the client actually supports HTTP/1.1 or not.
>
> Therefore, the original protocol version needs to be retained instead
> of being simply overridden by the decoder.  Actually, I don't see a
> problem with retaining the original version because a broken client
> implementation can send such a request and the request handler should
> be able to handle it anyway.
>
> Because HttpMessageDecoder should not modify the original protocol
> version, LittleProxy will have to handle protocol version upgrade and
> downgrade by itself like the following:
>
> For the HTTP proxy to send the chunked content converted by
> HttpRequestDecoder to the actual HTTP server, the proxy can either:
>
> 1) modify the outbound HTTP request by calling
> HttpMessage.setProtocolVersion() and store the actual client protocol
> version somewhere (assuming NETTY-247[1] has been resolved), or
> 2) create a new outbound HTTP request object with upgraded protocol version.
>
> If the proxy receives the chunked content converted by
> HttpResponseDecoder from the actual server, the same modification
> needs to be performed against the received HttpResponse.  If the
> client sent HTTP/1.0 request, the received chunks have to be merged
> back because the client will probably not understand chunked encoding.
>
> Another tricky case would be the case where the actual server does not
> understand chunked encoding.  In such a case, the actual server will
> respond with "HTTP/1.0 406 Not Acceptable" and the proxy server should
> retry after merging the chunks and downgrading the protocol version.
>
> Please let me know if I am missing something.  Otherwise, I will
> retain the current behavior while adding setter methods to
> HttpMessage.
>
> Thanks
>
> — Trustin Lee, http://gleamynode.net/
>
> [1] NETTY-247: https://jira.jboss.org/jira/browse/NETTY-247
>
> On Wed, Nov 4, 2009 at 3:34 PM, Adam Fisk <a at littleshoot.org> wrote:
>> OK - finally knocked this off, and it's working beautifully in FireFox and
>> Safari. All the credit really goes to Stephen - I'm just implementing his
>> fix here.
>> I agree with some of the earlier comments that since HttpMessage and its
>> subclasses are already mutable, it's not a big loss to add another mutable
>> field. That's the approach I took with the patch. I also think it makes
>> sense to keep HttpChunk immutable unless there's a concrete need to make it
>> mutable. I also don't think this is indicative of a design flaw.
>> Feel free to change the style, approach, or comments as you see fit,
>> Trustin. Thanks again Stephen. I like that handy netcat trick, by the way!
>> -Adam
>>
>> On Tue, Nov 3, 2009 at 8:47 PM, Stephen Haberman <stephen at exigencecorp.com>
>> wrote:
>>>
>>> > I'm also happy to give you commit permissions if you like. This in
>>> > particular is clearly a Netty and not a LittleProxy issue, but I can
>>> > give you access to the repo if you need to make any changes going
>>> > forward.
>>>
>>> That's fine--I enjoyed tracking the bug down and also reading through
>>> the LittleProxy code.
>>>
>>> My proxy is more of a weekend (or two or three) hobby project--I
>>> wouldn't be surprised if its missing several boundary cases that
>>> LittleProxy handles more robustly.
>>>
>>> But I did finally get it into github:
>>>
>>> http://github.com/stephenh/fakehttp
>>>
>>> Being a hobby project, I also wrote it in Scala and, purposefully
>>> trying to keep it small, have kept it to ~450 LOC so far.
>>>
>>> - Stephen
>>>
>>> _______________________________________________
>>> netty-users mailing list
>>> netty-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>>
>> --
>> Adam Fisk
>> http://www.littleshoot.org | http://adamfisk.wordpress.com |
>> http://twitter.com/adamfisk
>>
>> _______________________________________________
>> netty-users mailing list
>> netty-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>



-- 
Adam Fisk
http://www.littleshoot.org | http://adamfisk.wordpress.com |
http://twitter.com/adamfisk