Netty has problem to deal with the "Connection: close" message ... We cannot read the content.
huican ping
pinghuican at gmail.com
Thu Jun 4 01:22:17 EDT 2009
no, the 242 is the response size which I got from tomcat server.
The maxChunkSize should be 8192 which is default.
On Thu, Jun 4, 2009 at 12:19 AM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
> Did you set maxChunkSize to 242?
>
> On 04-Jun-2009 14:17, huican ping wrote:
>> Hello Trustin,
>>
>> Using the HttpClient example, I did see that " } END OF CHUNKED
>> CONTENT " for that "connection: close" response.
>> While in my other code base I am working on, I didn't get the last
>> (3rd) messageReceived() call. It is strange.
>>
>> Do you have any idea about it?
>>
>> # 99th
>> inside messageReceived()
>> First chunk. Setting readingChunks to true
>> inside messageReceived()
>> another chunk chunksize = 242
>> inside messageReceived()
>> another chunk chunksize = 0
>> is last chunk... yes
>>
>> # 100th
>> inside messageReceived()
>> First chunk. Setting readingChunks to true
>> inside messageReceived()
>> inside chunk chunksize = 242
>>
>>
>> On Wed, Jun 3, 2009 at 11:36 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>> In revision 1320, I updated the HTTP example to reproduce the problem.
>>> With the HttpServer and HttpClient, I was able to receive the last
>>> HttpChunk successfully for a response without Content-Length. Could you
>>> try to run the HttpClient and let me know if you see any differences.
>>> The following is an example output:
>>>
>>> STATUS: 200 OK
>>> VERSION: HTTP/1.1
>>>
>>> HEADER: Content-Type = text/plain; charset=UTF-8
>>> HEADER: Set-Cookie = another-cookie=bar;my-cookie=foo
>>> HEADER: Transfer-Encoding = chunked
>>>
>>> CHUNKED CONTENT {
>>> WELCOME TO THE WILD WILD WEB SERVER
>>> ===================================
>>> VERSION: HTTP/1.1
>>> HOSTNAME: localhost
>>> REQUEST_URI: http://localhost:8080/hello
>>>
>>> HEADER: Connection = close
>>> HEADER: Cookie = another-cookie=bar;my-cookie=foo
>>> HEADER: Host = localhost
>>>
>>> } END OF CHUNKED CONTENT <-- isLast() returned true
>>>
>>> Thanks in advance,
>>> Trustin
>>>
>>> On 04-Jun-2009 13:16, huican ping wrote:
>>>> FYI, I synced to rev 1320, and "mvn clean install" generated the new jar file.
>>>>
>>>> 2009/6/3 huican ping <pinghuican at gmail.com>:
>>>>> Hello Trustin,
>>>>>
>>>>> Surprisingly, I got the same things as my very last time which is:
>>>>> I saw the first 2 messageReceived() calls, but I didn't see the last
>>>>> messageReceived() call which will tells me chunk.isLast().
>>>>>
>>>>> Is there anything still missing.
>>>>>
>>>>> Thanks
>>>>> Huican
>>>>>
>>>>>
>>>>> On Wed, Jun 3, 2009 at 10:11 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>>>> The same fix should have been applied to
>>>>>> ReplayingDecoderBuffer.readable() but it was mistakenly forgotten.
>>>>>> Revision 1317 should now work fine.
>>>>>>
>>>>>> Thanks,
>>>>>> Trustin
>>>>>>
>>>>>> On 04-Jun-2009 04:20, huican ping wrote:
>>>>>>> Hello Trustin,
>>>>>>>
>>>>>>> Debugged more, I noticed the 1316 is better than before BETA3, but the
>>>>>>> problem now is:
>>>>>>>
>>>>>>> For a normal request which content is smaller than MAXCHUNKSIZE, there
>>>>>>> are 3 messageReceived() calls.
>>>>>>> The first one is just for Headers, no content; The second one is
>>>>>>> for the content; The last is for chunk.isLast() which satisfies the
>>>>>>> check "if (chunk.isLast())". This "if(chunk.isLast())" is same as the
>>>>>>> http sample code.
>>>>>>>
>>>>>>> Now at rev 1316, I saw the first 2 messageReceived() calls (sorry, I
>>>>>>> said no 2nd time, it was not right), but I didn't see the last
>>>>>>> messageReceived() call which will tells me chunk.isLast(). I need that
>>>>>>> check to do some essential steps at that condition, but now I didn't
>>>>>>> get that condition.
>>>>>>>
>>>>>>> You wont see the problem at http example since the sample code does
>>>>>>> nothing really important at that condition except set readingChunks =
>>>>>>> false.
>>>>>>>
>>>>>>> Thanks
>>>>>>> huican
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 3, 2009 at 10:36 AM, Huican Ping <pinghuican at gmail.com> wrote:
>>>>>>>>
>>>>>>>> Sorry Trustin,
>>>>>>>>
>>>>>>>> I synced to Rev 1316, and got the same result: the last response's content
>>>>>>>> chunk was not received, e.g, no messageReceived() called second time.
>>>>>>>>
>>>>>>>> For the 100th response like below (FYI, the body is NOT wrapped with chunked
>>>>>>>> byte size, I don't know whether it will cause problem since netty THINKs
>>>>>>>> this response is chunked encoding message but as you see the body is not
>>>>>>>> wrapped the content size as normally chunked message does):
>>>>>>>> HTTP/1.1 200 OK
>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>> Connection: close
>>>>>>>>
>>>>>>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
>>>>>>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><ns1:echo
>>>>>>>> xmlns:ns1="http://example1.org/example1"><Text>echo:
>>>>>>>> hello</Text></ns1:echo></soapenv:Body></soapenv:Envelope>
>>>>>>>>
>>>>>>>>
>>>>>>>> In my httpResponseHandler, the code goes into messageReceived() first time,
>>>>>>>> and the check "respStatus.getCode() == 200 && resp.isChunked()" is true,
>>>>>>>> then I just mark "readingChunk = true". Later on, there is no 2nd
>>>>>>>> messageReceived() call.
>>>>>>>>
>>>>>>>>
>>>>>>>> The messageReceived method is same as the HttpResponseHandler example
>>>>>>>> logically, such as:
>>>>>>>> if ( !readingChunks )
>>>>>>>> {
>>>>>>>> HttpResponse resp = (HttpResponse) event.getMessage();
>>>>>>>>
>>>>>>>> ...
>>>>>>>>
>>>>>>>> if (respStatus.getCode() == 200 && resp.isChunked() )
>>>>>>>> {
>>>>>>>> // First one is always 0 length, just set readingCHunks to true
>>>>>>>> readingChunks = true;
>>>>>>>> ....
>>>>>>>> }
>>>>>>>> else
>>>>>>>> {
>>>>>>>> // Here is the case of - content-length < 'maxChunkSize'
>>>>>>>> // we need get the content and send resp.
>>>>>>>> final ChannelBuffer buffer = resp.getContent();
>>>>>>>> ....
>>>>>>>> }
>>>>>>>> }
>>>>>>>> // for the left chunks
>>>>>>>> else
>>>>>>>> {
>>>>>>>> HttpChunk chunk = (HttpChunk) event.getMessage();
>>>>>>>> ...
>>>>>>>>
>>>>>>>> if (chunk.isLast())
>>>>>>>> {
>>>>>>>> readingChunks = false;
>>>>>>>> // Send the response to the workflow
>>>>>>>> ....
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> Huican Ping wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks Trustin,
>>>>>>>>>
>>>>>>>>> Thank you for the quick fix. I will have a try then.
>>>>>>>>> I will let you know in cast if I still see the problem.
>>>>>>>>>
>>>>>>>>> Huican
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Trustin Lee wrote:
>>>>>>>>>>
>>>>>>>>>> Uh, I've just succeeded to reproduce the problem you described by
>>>>>>>>>> myself. There was a regression in ReplayingDecoder when I fixed
>>>>>>>>>> NETTY-142. I've just checked in the fix - revision 1316. Here's the
>>>>>>>>>> filed JIRA issue:
>>>>>>>>>>
>>>>>>>>>> * https://jira.jboss.org/jira/browse/NETTY-164
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>> Trustin
>>>>>>>>>>
>>>>>>>>>> On 03-Jun-2009 17:21, "이희승 (Trustin Lee) wrote:
>>>>>>>>>>> 1) Netty HTTP decoder never closes the connection. If the connection is
>>>>>>>>>>> closed, it's the remote peer (server in your case) or your handler
>>>>>>>>>>> implementation. Please let me know if you still think Netty HTTP decoder
>>>>>>>>>>> closes the connection.
>>>>>>>>>>>
>>>>>>>>>>> 2) If there's no way to know the length of the content, Netty transforms
>>>>>>>>>>> a non-chunked HTTP message into a chunked HTTP message.
>>>>>>>>>>>
>>>>>>>>>>> In case of last response, the length of content is only determined when
>>>>>>>>>>> the connection is closed. Because it is not safe to buffer the whole
>>>>>>>>>>> content until the connection is closed, Netty generates HttpMessage with
>>>>>>>>>>> no content first and then HttpChunks.
>>>>>>>>>>>
>>>>>>>>>>> Your handler should be able to handle this case because any HTTP server
>>>>>>>>>>> can send you a chunked HTTP message. Please make sure that you handle
>>>>>>>>>>> HttpChunk message properly. The HttpClient example already handles such
>>>>>>>>>>> a case, so you could refer to the example.
>>>>>>>>>>>
>>>>>>>>>>> If you do not like this behavior but just want to limit the content
>>>>>>>>>>> length, you can insert HttpChunkAggregator right after
>>>>>>>>>>> Http(Request|Response)Decoder in the pipeline. Then you will never
>>>>>>>>>>> receive HttpChunk, and an exception will be raised if the content length
>>>>>>>>>>> is too large.
>>>>>>>>>>>
>>>>>>>>>>> Please let me know if HttpChunk is not received at all for the last
>>>>>>>>>>> response - it might be a bug then.
>>>>>>>>>>>
>>>>>>>>>>> HTH,
>>>>>>>>>>> Trustin
>>>>>>>>>>>
>>>>>>>>>>> On 03-Jun-2009 04:02, Huican Ping wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> More information:
>>>>>>>>>>>>
>>>>>>>>>>>> The messageReceived() message is same logic as the ones at http example
>>>>>>>>>>>> where it does
>>>>>>>>>>>> check "respStatus.getCode() == 200&& resp.isChunked() "
>>>>>>>>>>>>
>>>>>>>>>>>> I noticed for the headers at 100th response:
>>>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>>>>> Connection: close
>>>>>>>>>>>>
>>>>>>>>>>>> That resp.isChunked() returns true !! While stepping into the code (at
>>>>>>>>>>>> DefaultHttpMessage.java isChunked() method), and I dumpped out
>>>>>>>>>>>> headers, it
>>>>>>>>>>>> has Transfer-Encoding (which I don't know who added it mysteriously).
>>>>>>>>>>>> {Connection=[close], Content-Type=[text/xml;charset=UTF-8], Date=[Tue,
>>>>>>>>>>>> 02
>>>>>>>>>>>> Jun 2009 18:29:59 GMT], Server=[Apache-Coyote/1.1],
>>>>>>>>>>>> Transfer-Encoding=[chunked]}
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am using the 3.1.0.Beta3.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That 100th message has no "content-length" header, but has
>>>>>>>>>>>>> "connection:
>>>>>>>>>>>>> close" inside. It is still a valid message.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any suggestion or workaround?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> netty-users mailing list
>>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> netty-users mailing list
>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
> --
> -- Trustin Lee, http://gleamynode.net/
>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>
>
More information about the netty-users
mailing list