Netty has problem to deal with the "Connection: close" message ... We cannot read the content.
"이희승 (Trustin Lee)"
trustin at gmail.com
Thu Jun 4 01:19:47 EDT 2009
Did you set maxChunkSize to 242?
On 04-Jun-2009 14:17, huican ping wrote:
> Hello Trustin,
>
> Using the HttpClient example, I did see that " } END OF CHUNKED
> CONTENT " for that "connection: close" response.
> While in my other code base I am working on, I didn't get the last
> (3rd) messageReceived() call. It is strange.
>
> Do you have any idea about it?
>
> # 99th
> inside messageReceived()
> First chunk. Setting readingChunks to true
> inside messageReceived()
> another chunk chunksize = 242
> inside messageReceived()
> another chunk chunksize = 0
> is last chunk... yes
>
> # 100th
> inside messageReceived()
> First chunk. Setting readingChunks to true
> inside messageReceived()
> inside chunk chunksize = 242
>
>
> On Wed, Jun 3, 2009 at 11:36 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>> In revision 1320, I updated the HTTP example to reproduce the problem.
>> With the HttpServer and HttpClient, I was able to receive the last
>> HttpChunk successfully for a response without Content-Length. Could you
>> try to run the HttpClient and let me know if you see any differences.
>> The following is an example output:
>>
>> STATUS: 200 OK
>> VERSION: HTTP/1.1
>>
>> HEADER: Content-Type = text/plain; charset=UTF-8
>> HEADER: Set-Cookie = another-cookie=bar;my-cookie=foo
>> HEADER: Transfer-Encoding = chunked
>>
>> CHUNKED CONTENT {
>> WELCOME TO THE WILD WILD WEB SERVER
>> ===================================
>> VERSION: HTTP/1.1
>> HOSTNAME: localhost
>> REQUEST_URI: http://localhost:8080/hello
>>
>> HEADER: Connection = close
>> HEADER: Cookie = another-cookie=bar;my-cookie=foo
>> HEADER: Host = localhost
>>
>> } END OF CHUNKED CONTENT <-- isLast() returned true
>>
>> Thanks in advance,
>> Trustin
>>
>> On 04-Jun-2009 13:16, huican ping wrote:
>>> FYI, I synced to rev 1320, and "mvn clean install" generated the new jar file.
>>>
>>> 2009/6/3 huican ping <pinghuican at gmail.com>:
>>>> Hello Trustin,
>>>>
>>>> Surprisingly, I got the same things as my very last time which is:
>>>> I saw the first 2 messageReceived() calls, but I didn't see the last
>>>> messageReceived() call which will tells me chunk.isLast().
>>>>
>>>> Is there anything still missing.
>>>>
>>>> Thanks
>>>> Huican
>>>>
>>>>
>>>> On Wed, Jun 3, 2009 at 10:11 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>>> The same fix should have been applied to
>>>>> ReplayingDecoderBuffer.readable() but it was mistakenly forgotten.
>>>>> Revision 1317 should now work fine.
>>>>>
>>>>> Thanks,
>>>>> Trustin
>>>>>
>>>>> On 04-Jun-2009 04:20, huican ping wrote:
>>>>>> Hello Trustin,
>>>>>>
>>>>>> Debugged more, I noticed the 1316 is better than before BETA3, but the
>>>>>> problem now is:
>>>>>>
>>>>>> For a normal request which content is smaller than MAXCHUNKSIZE, there
>>>>>> are 3 messageReceived() calls.
>>>>>> The first one is just for Headers, no content; The second one is
>>>>>> for the content; The last is for chunk.isLast() which satisfies the
>>>>>> check "if (chunk.isLast())". This "if(chunk.isLast())" is same as the
>>>>>> http sample code.
>>>>>>
>>>>>> Now at rev 1316, I saw the first 2 messageReceived() calls (sorry, I
>>>>>> said no 2nd time, it was not right), but I didn't see the last
>>>>>> messageReceived() call which will tells me chunk.isLast(). I need that
>>>>>> check to do some essential steps at that condition, but now I didn't
>>>>>> get that condition.
>>>>>>
>>>>>> You wont see the problem at http example since the sample code does
>>>>>> nothing really important at that condition except set readingChunks =
>>>>>> false.
>>>>>>
>>>>>> Thanks
>>>>>> huican
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 3, 2009 at 10:36 AM, Huican Ping <pinghuican at gmail.com> wrote:
>>>>>>>
>>>>>>> Sorry Trustin,
>>>>>>>
>>>>>>> I synced to Rev 1316, and got the same result: the last response's content
>>>>>>> chunk was not received, e.g, no messageReceived() called second time.
>>>>>>>
>>>>>>> For the 100th response like below (FYI, the body is NOT wrapped with chunked
>>>>>>> byte size, I don't know whether it will cause problem since netty THINKs
>>>>>>> this response is chunked encoding message but as you see the body is not
>>>>>>> wrapped the content size as normally chunked message does):
>>>>>>> HTTP/1.1 200 OK
>>>>>>> Server: Apache-Coyote/1.1
>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>> Connection: close
>>>>>>>
>>>>>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
>>>>>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><ns1:echo
>>>>>>> xmlns:ns1="http://example1.org/example1"><Text>echo:
>>>>>>> hello</Text></ns1:echo></soapenv:Body></soapenv:Envelope>
>>>>>>>
>>>>>>>
>>>>>>> In my httpResponseHandler, the code goes into messageReceived() first time,
>>>>>>> and the check "respStatus.getCode() == 200 && resp.isChunked()" is true,
>>>>>>> then I just mark "readingChunk = true". Later on, there is no 2nd
>>>>>>> messageReceived() call.
>>>>>>>
>>>>>>>
>>>>>>> The messageReceived method is same as the HttpResponseHandler example
>>>>>>> logically, such as:
>>>>>>> if ( !readingChunks )
>>>>>>> {
>>>>>>> HttpResponse resp = (HttpResponse) event.getMessage();
>>>>>>>
>>>>>>> ...
>>>>>>>
>>>>>>> if (respStatus.getCode() == 200 && resp.isChunked() )
>>>>>>> {
>>>>>>> // First one is always 0 length, just set readingCHunks to true
>>>>>>> readingChunks = true;
>>>>>>> ....
>>>>>>> }
>>>>>>> else
>>>>>>> {
>>>>>>> // Here is the case of - content-length < 'maxChunkSize'
>>>>>>> // we need get the content and send resp.
>>>>>>> final ChannelBuffer buffer = resp.getContent();
>>>>>>> ....
>>>>>>> }
>>>>>>> }
>>>>>>> // for the left chunks
>>>>>>> else
>>>>>>> {
>>>>>>> HttpChunk chunk = (HttpChunk) event.getMessage();
>>>>>>> ...
>>>>>>>
>>>>>>> if (chunk.isLast())
>>>>>>> {
>>>>>>> readingChunks = false;
>>>>>>> // Send the response to the workflow
>>>>>>> ....
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Huican Ping wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks Trustin,
>>>>>>>>
>>>>>>>> Thank you for the quick fix. I will have a try then.
>>>>>>>> I will let you know in cast if I still see the problem.
>>>>>>>>
>>>>>>>> Huican
>>>>>>>>
>>>>>>>>
>>>>>>>> Trustin Lee wrote:
>>>>>>>>>
>>>>>>>>> Uh, I've just succeeded to reproduce the problem you described by
>>>>>>>>> myself. There was a regression in ReplayingDecoder when I fixed
>>>>>>>>> NETTY-142. I've just checked in the fix - revision 1316. Here's the
>>>>>>>>> filed JIRA issue:
>>>>>>>>>
>>>>>>>>> * https://jira.jboss.org/jira/browse/NETTY-164
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Trustin
>>>>>>>>>
>>>>>>>>> On 03-Jun-2009 17:21, "이희승 (Trustin Lee) wrote:
>>>>>>>>>> 1) Netty HTTP decoder never closes the connection. If the connection is
>>>>>>>>>> closed, it's the remote peer (server in your case) or your handler
>>>>>>>>>> implementation. Please let me know if you still think Netty HTTP decoder
>>>>>>>>>> closes the connection.
>>>>>>>>>>
>>>>>>>>>> 2) If there's no way to know the length of the content, Netty transforms
>>>>>>>>>> a non-chunked HTTP message into a chunked HTTP message.
>>>>>>>>>>
>>>>>>>>>> In case of last response, the length of content is only determined when
>>>>>>>>>> the connection is closed. Because it is not safe to buffer the whole
>>>>>>>>>> content until the connection is closed, Netty generates HttpMessage with
>>>>>>>>>> no content first and then HttpChunks.
>>>>>>>>>>
>>>>>>>>>> Your handler should be able to handle this case because any HTTP server
>>>>>>>>>> can send you a chunked HTTP message. Please make sure that you handle
>>>>>>>>>> HttpChunk message properly. The HttpClient example already handles such
>>>>>>>>>> a case, so you could refer to the example.
>>>>>>>>>>
>>>>>>>>>> If you do not like this behavior but just want to limit the content
>>>>>>>>>> length, you can insert HttpChunkAggregator right after
>>>>>>>>>> Http(Request|Response)Decoder in the pipeline. Then you will never
>>>>>>>>>> receive HttpChunk, and an exception will be raised if the content length
>>>>>>>>>> is too large.
>>>>>>>>>>
>>>>>>>>>> Please let me know if HttpChunk is not received at all for the last
>>>>>>>>>> response - it might be a bug then.
>>>>>>>>>>
>>>>>>>>>> HTH,
>>>>>>>>>> Trustin
>>>>>>>>>>
>>>>>>>>>> On 03-Jun-2009 04:02, Huican Ping wrote:
>>>>>>>>>>>
>>>>>>>>>>> More information:
>>>>>>>>>>>
>>>>>>>>>>> The messageReceived() message is same logic as the ones at http example
>>>>>>>>>>> where it does
>>>>>>>>>>> check "respStatus.getCode() == 200&& resp.isChunked() "
>>>>>>>>>>>
>>>>>>>>>>> I noticed for the headers at 100th response:
>>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>>>> Connection: close
>>>>>>>>>>>
>>>>>>>>>>> That resp.isChunked() returns true !! While stepping into the code (at
>>>>>>>>>>> DefaultHttpMessage.java isChunked() method), and I dumpped out
>>>>>>>>>>> headers, it
>>>>>>>>>>> has Transfer-Encoding (which I don't know who added it mysteriously).
>>>>>>>>>>> {Connection=[close], Content-Type=[text/xml;charset=UTF-8], Date=[Tue,
>>>>>>>>>>> 02
>>>>>>>>>>> Jun 2009 18:29:59 GMT], Server=[Apache-Coyote/1.1],
>>>>>>>>>>> Transfer-Encoding=[chunked]}
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I am using the 3.1.0.Beta3.
>>>>>>>>>>>>
>>>>>>>>>>>> That 100th message has no "content-length" header, but has
>>>>>>>>>>>> "connection:
>>>>>>>>>>>> close" inside. It is still a valid message.
>>>>>>>>>>>>
>>>>>>>>>>>> Any suggestion or workaround?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> netty-users mailing list
>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> netty-users mailing list
>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
--
— Trustin Lee, http://gleamynode.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url : http://lists.jboss.org/pipermail/netty-users/attachments/20090604/f436aff5/attachment.bin
More information about the netty-users
mailing list