Netty has problem to deal with the "Connection: close" message ... We cannot read the content.
huican ping
pinghuican at gmail.com
Thu Jun 4 11:35:34 EDT 2009
Hello Trustin,
I will let you know for sure ifI saw new problem here. I just synced
to 1327, and seems fine.
Thanks for the help.
Huican
On Thu, Jun 4, 2009 at 1:09 AM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
> That's weird. I've just managed to reproduce the problem. :)
>
> Creating a HttpMessageDecoder with (1024, 1024, 1) always reproduced the
> problem for me. It might be just a similar issue with different cause,
> so please let us know if you still have the problem. The fix has been
> checked in at revision 1322.
>
> Thanks,
> Trustin
>
> On 04-Jun-2009 15:01, huican ping wrote:
>> Hello Trustin,
>>
>> Good news.
>> I rebuilt my whole source code base, and it worked fine now. It is
>> strange that why it is not working by only replacing the the netty lib
>> file.
>>
>> Thanks a lot.
>> Huican
>>
>> 2009/6/4 huican ping <pinghuican at gmail.com>:
>>> no, the 242 is the response size which I got from tomcat server.
>>> The maxChunkSize should be 8192 which is default.
>>>
>>> On Thu, Jun 4, 2009 at 12:19 AM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>> Did you set maxChunkSize to 242?
>>>>
>>>> On 04-Jun-2009 14:17, huican ping wrote:
>>>>> Hello Trustin,
>>>>>
>>>>> Using the HttpClient example, I did see that " } END OF CHUNKED
>>>>> CONTENT " for that "connection: close" response.
>>>>> While in my other code base I am working on, I didn't get the last
>>>>> (3rd) messageReceived() call. It is strange.
>>>>>
>>>>> Do you have any idea about it?
>>>>>
>>>>> # 99th
>>>>> inside messageReceived()
>>>>> First chunk. Setting readingChunks to true
>>>>> inside messageReceived()
>>>>> another chunk chunksize = 242
>>>>> inside messageReceived()
>>>>> another chunk chunksize = 0
>>>>> is last chunk... yes
>>>>>
>>>>> # 100th
>>>>> inside messageReceived()
>>>>> First chunk. Setting readingChunks to true
>>>>> inside messageReceived()
>>>>> inside chunk chunksize = 242
>>>>>
>>>>>
>>>>> On Wed, Jun 3, 2009 at 11:36 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>>>> In revision 1320, I updated the HTTP example to reproduce the problem.
>>>>>> With the HttpServer and HttpClient, I was able to receive the last
>>>>>> HttpChunk successfully for a response without Content-Length. Could you
>>>>>> try to run the HttpClient and let me know if you see any differences.
>>>>>> The following is an example output:
>>>>>>
>>>>>> STATUS: 200 OK
>>>>>> VERSION: HTTP/1.1
>>>>>>
>>>>>> HEADER: Content-Type = text/plain; charset=UTF-8
>>>>>> HEADER: Set-Cookie = another-cookie=bar;my-cookie=foo
>>>>>> HEADER: Transfer-Encoding = chunked
>>>>>>
>>>>>> CHUNKED CONTENT {
>>>>>> WELCOME TO THE WILD WILD WEB SERVER
>>>>>> ===================================
>>>>>> VERSION: HTTP/1.1
>>>>>> HOSTNAME: localhost
>>>>>> REQUEST_URI: http://localhost:8080/hello
>>>>>>
>>>>>> HEADER: Connection = close
>>>>>> HEADER: Cookie = another-cookie=bar;my-cookie=foo
>>>>>> HEADER: Host = localhost
>>>>>>
>>>>>> } END OF CHUNKED CONTENT <-- isLast() returned true
>>>>>>
>>>>>> Thanks in advance,
>>>>>> Trustin
>>>>>>
>>>>>> On 04-Jun-2009 13:16, huican ping wrote:
>>>>>>> FYI, I synced to rev 1320, and "mvn clean install" generated the new jar file.
>>>>>>>
>>>>>>> 2009/6/3 huican ping <pinghuican at gmail.com>:
>>>>>>>> Hello Trustin,
>>>>>>>>
>>>>>>>> Surprisingly, I got the same things as my very last time which is:
>>>>>>>> I saw the first 2 messageReceived() calls, but I didn't see the last
>>>>>>>> messageReceived() call which will tells me chunk.isLast().
>>>>>>>>
>>>>>>>> Is there anything still missing.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Huican
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 3, 2009 at 10:11 PM, "이희승 (Trustin Lee)" <trustin at gmail.com> wrote:
>>>>>>>>> The same fix should have been applied to
>>>>>>>>> ReplayingDecoderBuffer.readable() but it was mistakenly forgotten.
>>>>>>>>> Revision 1317 should now work fine.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Trustin
>>>>>>>>>
>>>>>>>>> On 04-Jun-2009 04:20, huican ping wrote:
>>>>>>>>>> Hello Trustin,
>>>>>>>>>>
>>>>>>>>>> Debugged more, I noticed the 1316 is better than before BETA3, but the
>>>>>>>>>> problem now is:
>>>>>>>>>>
>>>>>>>>>> For a normal request which content is smaller than MAXCHUNKSIZE, there
>>>>>>>>>> are 3 messageReceived() calls.
>>>>>>>>>> The first one is just for Headers, no content; The second one is
>>>>>>>>>> for the content; The last is for chunk.isLast() which satisfies the
>>>>>>>>>> check "if (chunk.isLast())". This "if(chunk.isLast())" is same as the
>>>>>>>>>> http sample code.
>>>>>>>>>>
>>>>>>>>>> Now at rev 1316, I saw the first 2 messageReceived() calls (sorry, I
>>>>>>>>>> said no 2nd time, it was not right), but I didn't see the last
>>>>>>>>>> messageReceived() call which will tells me chunk.isLast(). I need that
>>>>>>>>>> check to do some essential steps at that condition, but now I didn't
>>>>>>>>>> get that condition.
>>>>>>>>>>
>>>>>>>>>> You wont see the problem at http example since the sample code does
>>>>>>>>>> nothing really important at that condition except set readingChunks =
>>>>>>>>>> false.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> huican
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 3, 2009 at 10:36 AM, Huican Ping <pinghuican at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Sorry Trustin,
>>>>>>>>>>>
>>>>>>>>>>> I synced to Rev 1316, and got the same result: the last response's content
>>>>>>>>>>> chunk was not received, e.g, no messageReceived() called second time.
>>>>>>>>>>>
>>>>>>>>>>> For the 100th response like below (FYI, the body is NOT wrapped with chunked
>>>>>>>>>>> byte size, I don't know whether it will cause problem since netty THINKs
>>>>>>>>>>> this response is chunked encoding message but as you see the body is not
>>>>>>>>>>> wrapped the content size as normally chunked message does):
>>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>>>> Connection: close
>>>>>>>>>>>
>>>>>>>>>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
>>>>>>>>>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><ns1:echo
>>>>>>>>>>> xmlns:ns1="http://example1.org/example1"><Text>echo:
>>>>>>>>>>> hello</Text></ns1:echo></soapenv:Body></soapenv:Envelope>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In my httpResponseHandler, the code goes into messageReceived() first time,
>>>>>>>>>>> and the check "respStatus.getCode() == 200 && resp.isChunked()" is true,
>>>>>>>>>>> then I just mark "readingChunk = true". Later on, there is no 2nd
>>>>>>>>>>> messageReceived() call.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The messageReceived method is same as the HttpResponseHandler example
>>>>>>>>>>> logically, such as:
>>>>>>>>>>> if ( !readingChunks )
>>>>>>>>>>> {
>>>>>>>>>>> HttpResponse resp = (HttpResponse) event.getMessage();
>>>>>>>>>>>
>>>>>>>>>>> ...
>>>>>>>>>>>
>>>>>>>>>>> if (respStatus.getCode() == 200 && resp.isChunked() )
>>>>>>>>>>> {
>>>>>>>>>>> // First one is always 0 length, just set readingCHunks to true
>>>>>>>>>>> readingChunks = true;
>>>>>>>>>>> ....
>>>>>>>>>>> }
>>>>>>>>>>> else
>>>>>>>>>>> {
>>>>>>>>>>> // Here is the case of - content-length < 'maxChunkSize'
>>>>>>>>>>> // we need get the content and send resp.
>>>>>>>>>>> final ChannelBuffer buffer = resp.getContent();
>>>>>>>>>>> ....
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>> // for the left chunks
>>>>>>>>>>> else
>>>>>>>>>>> {
>>>>>>>>>>> HttpChunk chunk = (HttpChunk) event.getMessage();
>>>>>>>>>>> ...
>>>>>>>>>>>
>>>>>>>>>>> if (chunk.isLast())
>>>>>>>>>>> {
>>>>>>>>>>> readingChunks = false;
>>>>>>>>>>> // Send the response to the workflow
>>>>>>>>>>> ....
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks Trustin,
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for the quick fix. I will have a try then.
>>>>>>>>>>>> I will let you know in cast if I still see the problem.
>>>>>>>>>>>>
>>>>>>>>>>>> Huican
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Trustin Lee wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Uh, I've just succeeded to reproduce the problem you described by
>>>>>>>>>>>>> myself. There was a regression in ReplayingDecoder when I fixed
>>>>>>>>>>>>> NETTY-142. I've just checked in the fix - revision 1316. Here's the
>>>>>>>>>>>>> filed JIRA issue:
>>>>>>>>>>>>>
>>>>>>>>>>>>> * https://jira.jboss.org/jira/browse/NETTY-164
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>> Trustin
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 03-Jun-2009 17:21, "이희승 (Trustin Lee) wrote:
>>>>>>>>>>>>>> 1) Netty HTTP decoder never closes the connection. If the connection is
>>>>>>>>>>>>>> closed, it's the remote peer (server in your case) or your handler
>>>>>>>>>>>>>> implementation. Please let me know if you still think Netty HTTP decoder
>>>>>>>>>>>>>> closes the connection.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) If there's no way to know the length of the content, Netty transforms
>>>>>>>>>>>>>> a non-chunked HTTP message into a chunked HTTP message.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In case of last response, the length of content is only determined when
>>>>>>>>>>>>>> the connection is closed. Because it is not safe to buffer the whole
>>>>>>>>>>>>>> content until the connection is closed, Netty generates HttpMessage with
>>>>>>>>>>>>>> no content first and then HttpChunks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Your handler should be able to handle this case because any HTTP server
>>>>>>>>>>>>>> can send you a chunked HTTP message. Please make sure that you handle
>>>>>>>>>>>>>> HttpChunk message properly. The HttpClient example already handles such
>>>>>>>>>>>>>> a case, so you could refer to the example.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If you do not like this behavior but just want to limit the content
>>>>>>>>>>>>>> length, you can insert HttpChunkAggregator right after
>>>>>>>>>>>>>> Http(Request|Response)Decoder in the pipeline. Then you will never
>>>>>>>>>>>>>> receive HttpChunk, and an exception will be raised if the content length
>>>>>>>>>>>>>> is too large.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let me know if HttpChunk is not received at all for the last
>>>>>>>>>>>>>> response - it might be a bug then.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> HTH,
>>>>>>>>>>>>>> Trustin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 03-Jun-2009 04:02, Huican Ping wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> More information:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The messageReceived() message is same logic as the ones at http example
>>>>>>>>>>>>>>> where it does
>>>>>>>>>>>>>>> check "respStatus.getCode() == 200&& resp.isChunked() "
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I noticed for the headers at 100th response:
>>>>>>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>>>>>>> Server: Apache-Coyote/1.1
>>>>>>>>>>>>>>> Content-Type: text/xml;charset=UTF-8
>>>>>>>>>>>>>>> Date: Tue, 02 Jun 2009 15:50:12 GMT
>>>>>>>>>>>>>>> Connection: close
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That resp.isChunked() returns true !! While stepping into the code (at
>>>>>>>>>>>>>>> DefaultHttpMessage.java isChunked() method), and I dumpped out
>>>>>>>>>>>>>>> headers, it
>>>>>>>>>>>>>>> has Transfer-Encoding (which I don't know who added it mysteriously).
>>>>>>>>>>>>>>> {Connection=[close], Content-Type=[text/xml;charset=UTF-8], Date=[Tue,
>>>>>>>>>>>>>>> 02
>>>>>>>>>>>>>>> Jun 2009 18:29:59 GMT], Server=[Apache-Coyote/1.1],
>>>>>>>>>>>>>>> Transfer-Encoding=[chunked]}
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Huican Ping wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am using the 3.1.0.Beta3.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That 100th message has no "content-length" header, but has
>>>>>>>>>>>>>>>> "connection:
>>>>>>>>>>>>>>>> close" inside. It is still a valid message.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any suggestion or workaround?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> netty-users mailing list
>>>>>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> netty-users mailing list
>>>>>>>>>>>>> netty-users at lists.jboss.org
>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
> --
> -- Trustin Lee, http://gleamynode.net/
>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>
>
More information about the netty-users
mailing list