Netty has problem to deal with the "Connection: close" message ... We cannot read the content.
huican ping
pinghuican at gmail.com
Thu Jun 4 02:01:33 EDT 2009
Hello Trustin,
Good news.
I rebuilt my whole source code base, and it worked fine now. It is
strange that why it is not working by only replacing the the netty lib
file.
Thanks a lot.
Huican
2009/6/4 huican ping <pinghuican at gmail.com>:
> no, the 242 is the response size which I got from tomcat server.
> The maxChunkSize should be 8192 which is default.
>
> On Thu, Jun 4, 2009 at 12:19 AM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>> Did you set maxChunkSize to 242?
>>
>> On 04-Jun-2009 14:17, huican ping wrote:
>>> Hello Trustin,
>>>
>>> Using the HttpClient example, I did see that " } END OF CHUNKED
>>> CONTENT " for that "connection: close" response.
>>> While in my other code base I am working on, I didn't get the last
>>> (3rd) messageReceived() call. It is strange.
>>>
>>> Do you have any idea about it?
>>>
>>> # 99th
>>> inside messageReceived()
>>> First chunk. Setting readingChunks to true
>>> inside messageReceived()
>>> another chunk chunksize = 242
>>> inside messageReceived()
>>> another chunk chunksize = 0
>>> is last chunk... yes
>>>
>>> # 100th
>>> inside messageReceived()
>>> First chunk. Setting readingChunks to true
>>> inside messageReceived()
>>> inside chunk chunksize = 242
>>>
>>>
>>> On Wed, Jun 3, 2009 at 11:36 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>> In revision 1320, I updated the HTTP example to reproduce the problem.
>>>> With the HttpServer and HttpClient, I was able to receive the last
>>>> HttpChunk successfully for a response without Content-Length. Could you
>>>> try to run the HttpClient and let me know if you see any differences.
>>>> The following is an example output:
>>>>
>>>> STATUS: 200 OK
>>>> VERSION: HTTP/1.1
>>>>
>>>> HEADER: Content-Type = text/plain; charset=UTF-8
>>>> HEADER: Set-Cookie = another-cookie=bar;my-cookie=foo
>>>> HEADER: Transfer-Encoding = chunked
>>>>
>>>> CHUNKED CONTENT {
>>>> WELCOME TO THE WILD WILD WEB SERVER
>>>> ===================================
>>>> VERSION: HTTP/1.1
>>>> HOSTNAME: localhost
>>>> REQUEST_URI: http://localhost:8080/hello
>>>>
>>>> HEADER: Connection = close
>>>> HEADER: Cookie = another-cookie=bar;my-cookie=foo
>>>> HEADER: Host = localhost
>>>>
>>>> } END OF CHUNKED CONTENT <-- isLast() returned true
>>>>
>>>> Thanks in advance,
>>>> Trustin
>>>>
>>>> On 04-Jun-2009 13:16, huican ping wrote:
>>>>> FYI, I synced to rev 1320, and "mvn clean install" generated the new jar file.
>>>>>
>>>>> 2009/6/3 huican ping <pinghuican at gmail.com>:
>>>>>> Hello Trustin,
>>>>>>
>>>>>> Surprisingly, I got the same things as my very last time which is:
>>>>>> I saw the first 2 messageReceived() calls, but I didn't see the last
>>>>>> messageReceived() call which will tells me chunk.isLast().
>>>>>>
>>>>>> Is there anything still missing.
>>>>>>
>>>>>> Thanks
>>>>>> Huican
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 3, 2009 at 10:11 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>>>>> The same fix should have been applied to
>>>>>>> ReplayingDecoderBuffer.readable() but it was mistakenly forgotten.
>>>>>>> Revision 1317 should now work fine.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Trustin
>>>>>>>
>>>>>>> On 04-Jun-2009 04:20, huican ping wrote:
>>>>>>>> Hello Trustin,
>>>>>>>>
>>>>>>>> Debugged more, I noticed the 1316 is better than before BETA3, but the
>>>>>>>> problem now is:
>>>>>>>>
>>>>>>>> For a normal request which content is smaller than MAXCHUNKSIZE, there
>>>>>>>> are 3 messageReceived() calls.
>>>>>>>> The first one is just for Headers, no content; The second one is
>>>>>>>> for the content; The last is for chunk.isLast() which satisfies the
>>>>>>>> check "if (chunk.isLast())". This "if(chunk.isLast())" is same as the
>>>>>>>> http sample code.
>>>>>>>>
>>>>>>>> Now at rev 1316, I saw the first 2 messageReceived() calls (sorry, I
>>>>>>>> said no 2nd time, it was not right), but I didn't see the last
>>>>>>>> messageReceived() call which will tells me chunk.isLast(). I need that
>>>>>>>> check to do some essential steps at that condition, but now I didn't
>>>>>>>> get that condition.
>>>>>>>>
>>>>>>>> You wont see the problem at http example since the sample code does
>>>>>>>> nothing really important at that condition except set readingChunks =
>>>>>>>> false.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> huican
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 3, 2009 at 10:36 AM, Huican Ping <pinghuican at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Sorry Trustin,
>>>>>>>>>
>>>>>>>>> I synced to Rev 1316, and got the same result: the last response's content
>>>>>>>>> chunk was not received, e.g, no messageReceived() called second time.
>>>>>>>>>
>>>>>>>>> For the 100th response like below (FYI, the body is NOT wrapped with chunked
>>>>>>>>> byte size, I don't know whether it will cause problem since netty THINKs
>>>>>>>>> this response is chunked encoding message but as you see the body is not
>>>>>>>>> wrapped the content size as normally chunked message does):
>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>> Connection: close
>>>>>>>>>
>>>>>>>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
>>>>>>>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><ns1:echo
>>>>>>>>> xmlns:ns1="http://example1.org/example1"><Text>echo:
>>>>>>>>> hello</Text></ns1:echo></soapenv:Body></soapenv:Envelope>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In my httpResponseHandler, the code goes into messageReceived() first time,
>>>>>>>>> and the check "respStatus.getCode() == 200 && resp.isChunked()" is true,
>>>>>>>>> then I just mark "readingChunk = true". Later on, there is no 2nd
>>>>>>>>> messageReceived() call.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The messageReceived method is same as the HttpResponseHandler example
>>>>>>>>> logically, such as:
>>>>>>>>> if ( !readingChunks )
>>>>>>>>> {
>>>>>>>>> HttpResponse resp = (HttpResponse) event.getMessage();
>>>>>>>>>
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> if (respStatus.getCode() == 200 && resp.isChunked() )
>>>>>>>>> {
>>>>>>>>> // First one is always 0 length, just set readingCHunks to true
>>>>>>>>> readingChunks = true;
>>>>>>>>> ....
>>>>>>>>> }
>>>>>>>>> else
>>>>>>>>> {
>>>>>>>>> // Here is the case of - content-length < 'maxChunkSize'
>>>>>>>>> // we need get the content and send resp.
>>>>>>>>> final ChannelBuffer buffer = resp.getContent();
>>>>>>>>> ....
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>> // for the left chunks
>>>>>>>>> else
>>>>>>>>> {
>>>>>>>>> HttpChunk chunk = (HttpChunk) event.getMessage();
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> if (chunk.isLast())
>>>>>>>>> {
>>>>>>>>> readingChunks = false;
>>>>>>>>> // Send the response to the workflow
>>>>>>>>> ....
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks Trustin,
>>>>>>>>>>
>>>>>>>>>> Thank you for the quick fix. I will have a try then.
>>>>>>>>>> I will let you know in cast if I still see the problem.
>>>>>>>>>>
>>>>>>>>>> Huican
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Trustin Lee wrote:
>>>>>>>>>>>
>>>>>>>>>>> Uh, I've just succeeded to reproduce the problem you described by
>>>>>>>>>>> myself. There was a regression in ReplayingDecoder when I fixed
>>>>>>>>>>> NETTY-142. I've just checked in the fix - revision 1316. Here's the
>>>>>>>>>>> filed JIRA issue:
>>>>>>>>>>>
>>>>>>>>>>> * https://jira.jboss.org/jira/browse/NETTY-164
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>> Trustin
>>>>>>>>>>>
>>>>>>>>>>> On 03-Jun-2009 17:21, "이희승 (Trustin Lee) wrote:
>>>>>>>>>>>> 1) Netty HTTP decoder never closes the connection. If the connection is
>>>>>>>>>>>> closed, it's the remote peer (server in your case) or your handler
>>>>>>>>>>>> implementation. Please let me know if you still think Netty HTTP decoder
>>>>>>>>>>>> closes the connection.
>>>>>>>>>>>>
>>>>>>>>>>>> 2) If there's no way to know the length of the content, Netty transforms
>>>>>>>>>>>> a non-chunked HTTP message into a chunked HTTP message.
>>>>>>>>>>>>
>>>>>>>>>>>> In case of last response, the length of content is only determined when
>>>>>>>>>>>> the connection is closed. Because it is not safe to buffer the whole
>>>>>>>>>>>> content until the connection is closed, Netty generates HttpMessage with
>>>>>>>>>>>> no content first and then HttpChunks.
>>>>>>>>>>>>
>>>>>>>>>>>> Your handler should be able to handle this case because any HTTP server
>>>>>>>>>>>> can send you a chunked HTTP message. Please make sure that you handle
>>>>>>>>>>>> HttpChunk message properly. The HttpClient example already handles such
>>>>>>>>>>>> a case, so you could refer to the example.
>>>>>>>>>>>>
>>>>>>>>>>>> If you do not like this behavior but just want to limit the content
>>>>>>>>>>>> length, you can insert HttpChunkAggregator right after
>>>>>>>>>>>> Http(Request|Response)Decoder in the pipeline. Then you will never
>>>>>>>>>>>> receive HttpChunk, and an exception will be raised if the content length
>>>>>>>>>>>> is too large.
>>>>>>>>>>>>
>>>>>>>>>>>> Please let me know if HttpChunk is not received at all for the last
>>>>>>>>>>>> response - it might be a bug then.
>>>>>>>>>>>>
>>>>>>>>>>>> HTH,
>>>>>>>>>>>> Trustin
>>>>>>>>>>>>
>>>>>>>>>>>> On 03-Jun-2009 04:02, Huican Ping wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> More information:
>>>>>>>>>>>>>
>>>>>>>>>>>>> The messageReceived() message is same logic as the ones at http example
>>>>>>>>>>>>> where it does
>>>>>>>>>>>>> check "respStatus.getCode() == 200&& resp.isChunked() "
>>>>>>>>>>>>>
>>>>>>>>>>>>> I noticed for the headers at 100th response:
>>>>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>>>>>> Connection: close
>>>>>>>>>>>>>
>>>>>>>>>>>>> That resp.isChunked() returns true !! While stepping into the code (at
>>>>>>>>>>>>> DefaultHttpMessage.java isChunked() method), and I dumpped out
>>>>>>>>>>>>> headers, it
>>>>>>>>>>>>> has Transfer-Encoding (which I don't know who added it mysteriously).
>>>>>>>>>>>>> {Connection=[close], Content-Type=[text/xml;charset=UTF-8], Date=[Tue,
>>>>>>>>>>>>> 02
>>>>>>>>>>>>> Jun 2009 18:29:59 GMT], Server=[Apache-Coyote/1.1],
>>>>>>>>>>>>> Transfer-Encoding=[chunked]}
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am using the 3.1.0.Beta3.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That 100th message has no "content-length" header, but has
>>>>>>>>>>>>>> "connection:
>>>>>>>>>>>>>> close" inside. It is still a valid message.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any suggestion or workaround?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> netty-users mailing list
>>>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> netty-users mailing list
>>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>> --
>> -- Trustin Lee, http://gleamynode.net/
>>
>>
>> _______________________________________________
>> netty-users mailing list
>> netty-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>
More information about the netty-users
mailing list