Netty has problem to deal with the "Connection: close" message ... We cannot read the content.
huican ping
pinghuican at gmail.com
Thu Jun 4 01:17:18 EDT 2009
Hello Trustin,
Using the HttpClient example, I did see that " } END OF CHUNKED
CONTENT " for that "connection: close" response.
While in my other code base I am working on, I didn't get the last
(3rd) messageReceived() call. It is strange.
Do you have any idea about it?
# 99th
inside messageReceived()
First chunk. Setting readingChunks to true
inside messageReceived()
another chunk chunksize = 242
inside messageReceived()
another chunk chunksize = 0
is last chunk... yes
# 100th
inside messageReceived()
First chunk. Setting readingChunks to true
inside messageReceived()
inside chunk chunksize = 242
On Wed, Jun 3, 2009 at 11:36 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
> In revision 1320, I updated the HTTP example to reproduce the problem.
> With the HttpServer and HttpClient, I was able to receive the last
> HttpChunk successfully for a response without Content-Length. Could you
> try to run the HttpClient and let me know if you see any differences.
> The following is an example output:
>
> STATUS: 200 OK
> VERSION: HTTP/1.1
>
> HEADER: Content-Type = text/plain; charset=UTF-8
> HEADER: Set-Cookie = another-cookie=bar;my-cookie=foo
> HEADER: Transfer-Encoding = chunked
>
> CHUNKED CONTENT {
> WELCOME TO THE WILD WILD WEB SERVER
> ===================================
> VERSION: HTTP/1.1
> HOSTNAME: localhost
> REQUEST_URI: http://localhost:8080/hello
>
> HEADER: Connection = close
> HEADER: Cookie = another-cookie=bar;my-cookie=foo
> HEADER: Host = localhost
>
> } END OF CHUNKED CONTENT <-- isLast() returned true
>
> Thanks in advance,
> Trustin
>
> On 04-Jun-2009 13:16, huican ping wrote:
>> FYI, I synced to rev 1320, and "mvn clean install" generated the new jar file.
>>
>> 2009/6/3 huican ping <pinghuican at gmail.com>:
>>> Hello Trustin,
>>>
>>> Surprisingly, I got the same things as my very last time which is:
>>> I saw the first 2 messageReceived() calls, but I didn't see the last
>>> messageReceived() call which will tells me chunk.isLast().
>>>
>>> Is there anything still missing.
>>>
>>> Thanks
>>> Huican
>>>
>>>
>>> On Wed, Jun 3, 2009 at 10:11 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>> The same fix should have been applied to
>>>> ReplayingDecoderBuffer.readable() but it was mistakenly forgotten.
>>>> Revision 1317 should now work fine.
>>>>
>>>> Thanks,
>>>> Trustin
>>>>
>>>> On 04-Jun-2009 04:20, huican ping wrote:
>>>>> Hello Trustin,
>>>>>
>>>>> Debugged more, I noticed the 1316 is better than before BETA3, but the
>>>>> problem now is:
>>>>>
>>>>> For a normal request which content is smaller than MAXCHUNKSIZE, there
>>>>> are 3 messageReceived() calls.
>>>>> The first one is just for Headers, no content; The second one is
>>>>> for the content; The last is for chunk.isLast() which satisfies the
>>>>> check "if (chunk.isLast())". This "if(chunk.isLast())" is same as the
>>>>> http sample code.
>>>>>
>>>>> Now at rev 1316, I saw the first 2 messageReceived() calls (sorry, I
>>>>> said no 2nd time, it was not right), but I didn't see the last
>>>>> messageReceived() call which will tells me chunk.isLast(). I need that
>>>>> check to do some essential steps at that condition, but now I didn't
>>>>> get that condition.
>>>>>
>>>>> You wont see the problem at http example since the sample code does
>>>>> nothing really important at that condition except set readingChunks =
>>>>> false.
>>>>>
>>>>> Thanks
>>>>> huican
>>>>>
>>>>>
>>>>> On Wed, Jun 3, 2009 at 10:36 AM, Huican Ping <pinghuican at gmail.com> wrote:
>>>>>>
>>>>>> Sorry Trustin,
>>>>>>
>>>>>> I synced to Rev 1316, and got the same result: the last response's content
>>>>>> chunk was not received, e.g, no messageReceived() called second time.
>>>>>>
>>>>>> For the 100th response like below (FYI, the body is NOT wrapped with chunked
>>>>>> byte size, I don't know whether it will cause problem since netty THINKs
>>>>>> this response is chunked encoding message but as you see the body is not
>>>>>> wrapped the content size as normally chunked message does):
>>>>>> HTTP/1.1 200 OK
>>>>>> Server: Apache-Coyote/1.1
>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>> Connection: close
>>>>>>
>>>>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
>>>>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><ns1:echo
>>>>>> xmlns:ns1="http://example1.org/example1"><Text>echo:
>>>>>> hello</Text></ns1:echo></soapenv:Body></soapenv:Envelope>
>>>>>>
>>>>>>
>>>>>> In my httpResponseHandler, the code goes into messageReceived() first time,
>>>>>> and the check "respStatus.getCode() == 200 && resp.isChunked()" is true,
>>>>>> then I just mark "readingChunk = true". Later on, there is no 2nd
>>>>>> messageReceived() call.
>>>>>>
>>>>>>
>>>>>> The messageReceived method is same as the HttpResponseHandler example
>>>>>> logically, such as:
>>>>>> if ( !readingChunks )
>>>>>> {
>>>>>> HttpResponse resp = (HttpResponse) event.getMessage();
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> if (respStatus.getCode() == 200 && resp.isChunked() )
>>>>>> {
>>>>>> // First one is always 0 length, just set readingCHunks to true
>>>>>> readingChunks = true;
>>>>>> ....
>>>>>> }
>>>>>> else
>>>>>> {
>>>>>> // Here is the case of - content-length < 'maxChunkSize'
>>>>>> // we need get the content and send resp.
>>>>>> final ChannelBuffer buffer = resp.getContent();
>>>>>> ....
>>>>>> }
>>>>>> }
>>>>>> // for the left chunks
>>>>>> else
>>>>>> {
>>>>>> HttpChunk chunk = (HttpChunk) event.getMessage();
>>>>>> ...
>>>>>>
>>>>>> if (chunk.isLast())
>>>>>> {
>>>>>> readingChunks = false;
>>>>>> // Send the response to the workflow
>>>>>> ....
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Huican Ping wrote:
>>>>>>>
>>>>>>>
>>>>>>> Thanks Trustin,
>>>>>>>
>>>>>>> Thank you for the quick fix. I will have a try then.
>>>>>>> I will let you know in cast if I still see the problem.
>>>>>>>
>>>>>>> Huican
>>>>>>>
>>>>>>>
>>>>>>> Trustin Lee wrote:
>>>>>>>>
>>>>>>>> Uh, I've just succeeded to reproduce the problem you described by
>>>>>>>> myself. There was a regression in ReplayingDecoder when I fixed
>>>>>>>> NETTY-142. I've just checked in the fix - revision 1316. Here's the
>>>>>>>> filed JIRA issue:
>>>>>>>>
>>>>>>>> * https://jira.jboss.org/jira/browse/NETTY-164
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Trustin
>>>>>>>>
>>>>>>>> On 03-Jun-2009 17:21, "이희승 (Trustin Lee) wrote:
>>>>>>>>> 1) Netty HTTP decoder never closes the connection. If the connection is
>>>>>>>>> closed, it's the remote peer (server in your case) or your handler
>>>>>>>>> implementation. Please let me know if you still think Netty HTTP decoder
>>>>>>>>> closes the connection.
>>>>>>>>>
>>>>>>>>> 2) If there's no way to know the length of the content, Netty transforms
>>>>>>>>> a non-chunked HTTP message into a chunked HTTP message.
>>>>>>>>>
>>>>>>>>> In case of last response, the length of content is only determined when
>>>>>>>>> the connection is closed. Because it is not safe to buffer the whole
>>>>>>>>> content until the connection is closed, Netty generates HttpMessage with
>>>>>>>>> no content first and then HttpChunks.
>>>>>>>>>
>>>>>>>>> Your handler should be able to handle this case because any HTTP server
>>>>>>>>> can send you a chunked HTTP message. Please make sure that you handle
>>>>>>>>> HttpChunk message properly. The HttpClient example already handles such
>>>>>>>>> a case, so you could refer to the example.
>>>>>>>>>
>>>>>>>>> If you do not like this behavior but just want to limit the content
>>>>>>>>> length, you can insert HttpChunkAggregator right after
>>>>>>>>> Http(Request|Response)Decoder in the pipeline. Then you will never
>>>>>>>>> receive HttpChunk, and an exception will be raised if the content length
>>>>>>>>> is too large.
>>>>>>>>>
>>>>>>>>> Please let me know if HttpChunk is not received at all for the last
>>>>>>>>> response - it might be a bug then.
>>>>>>>>>
>>>>>>>>> HTH,
>>>>>>>>> Trustin
>>>>>>>>>
>>>>>>>>> On 03-Jun-2009 04:02, Huican Ping wrote:
>>>>>>>>>>
>>>>>>>>>> More information:
>>>>>>>>>>
>>>>>>>>>> The messageReceived() message is same logic as the ones at http example
>>>>>>>>>> where it does
>>>>>>>>>> check "respStatus.getCode() == 200&& resp.isChunked() "
>>>>>>>>>>
>>>>>>>>>> I noticed for the headers at 100th response:
>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>>> Connection: close
>>>>>>>>>>
>>>>>>>>>> That resp.isChunked() returns true !! While stepping into the code (at
>>>>>>>>>> DefaultHttpMessage.java isChunked() method), and I dumpped out
>>>>>>>>>> headers, it
>>>>>>>>>> has Transfer-Encoding (which I don't know who added it mysteriously).
>>>>>>>>>> {Connection=[close], Content-Type=[text/xml;charset=UTF-8], Date=[Tue,
>>>>>>>>>> 02
>>>>>>>>>> Jun 2009 18:29:59 GMT], Server=[Apache-Coyote/1.1],
>>>>>>>>>> Transfer-Encoding=[chunked]}
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>>
>>>>>>>>>>> I am using the 3.1.0.Beta3.
>>>>>>>>>>>
>>>>>>>>>>> That 100th message has no "content-length" header, but has
>>>>>>>>>>> "connection:
>>>>>>>>>>> close" inside. It is still a valid message.
>>>>>>>>>>>
>>>>>>>>>>> Any suggestion or workaround?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> netty-users mailing list
>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> netty-users mailing list
>>>>>>>> netty-users at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context: http://n2.nabble.com/Netty-has-problem-to-deal-with-the-%22Connection%3A-close%22-message-...-We-cannot-read-the-content.-tp3013392p3019015.html
>>>>>> Sent from the Netty User Group mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> netty-users mailing list
>>>>>> netty-users at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> netty-users mailing list
>>>>> netty-users at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>
>>>>
>>>> --
>>>> -- Trustin Lee, http://gleamynode.net/
>>>>
>>>>
>>>> _______________________________________________
>>>> netty-users mailing list
>>>> netty-users at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> netty-users mailing list
>> netty-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-users
>
>
> --
> -- Trustin Lee, http://gleamynode.net/
>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>
>
More information about the netty-users
mailing list