HttpMessageDecoder - Skip the footer; does anyone use it?
Trustin Lee (이희승)
trustin at gmail.com
Tue Nov 17 03:00:31 EST 2009
I've just resolved this issue. A new type called HttpChunkTrailer
(extends HttpChunk) has been added. You can down-cast the last chunk
to HttpChunkTrailer to retrieve the trailing headers.
HttpMessageEncoder is now also capable of encoding the trailing
headers.
All changes are in trunk. i did preliminary testing, but please give
it a try and let us know if it works or not. Here's the snapshot:
http://hudson.jboss.org/hudson/view/Netty/job/netty/818/
Wait several minutes as the build is in progress now.
Cheers
— Trustin Lee, http://gleamynode.net/
On Tue, Nov 17, 2009 at 12:47 AM, Trustin Lee (이희승) <trustin at gmail.com> wrote:
> Here's the relevant JIRA issue:
>
> https://jira.jboss.org/jira/browse/NETTY-251
>
> — Trustin Lee, http://gleamynode.net/
>
>
>
> On Sun, Nov 8, 2009 at 12:00 AM, Trustin Lee (이희승) <trustin at gmail.com> wrote:
>> Hi Roger,
>>
>> Oh, there was really someone who uses it! :)
>>
>> I will try to add support for trailing headers in 3.2. Your patch
>> does look very helpful. Thanks a lot!
>>
>> Cheers
>>
>> — Trustin Lee, http://gleamynode.net/
>>
>>
>>
>> On Thu, Nov 5, 2009 at 7:52 AM, Roger F <rfullerton at mimecast.com> wrote:
>>>
>>> Hi Trustin Lee,
>>>
>>> I have been trying to use TrailingHeaders after a Content-Transfer: chunked
>>> message, obviously it does not work, yet.
>>>
>>> This would be useful to allow large files, size unknown, to be streamed in
>>> chunks, followed by the MD5.
>>> This ensures data integrity and prevents out of memory in a server.
>>>
>>> Here is an example :
>>>
>>> PUT / HTTP/1.1
>>> Host: localhost:8081
>>> Transfer-Encoding: chunked
>>> Trailer: Content-MD5
>>>
>>> 64
>>> 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
>>> 0
>>> Content-MD5: 4f30805221b29b36df636869549398fb
>>>
>>>
>>>
>>> The following code changes allow TrailingHeaders and seem to work so far in
>>> testing.
>>>
>>> In HttpMessageDecoder replacing in the decode method the following switch
>>> state READ_CHUNK_FOOTER :
>>>
>>> @Override
>>> protected Object decode(ChannelHandlerContext ctx, Channel channel,
>>> ChannelBuffer buffer, State state) throws Exception {
>>>
>>> ....
>>>
>>> case READ_CHUNK_FOOTER: {
>>> // Do not Skip the footer; some people use it!
>>>
>>> try {
>>>
>>> readFooterHeaders(buffer);
>>>
>>> if (maxChunkSize == 0) {
>>> // Chunked encoding disabled.
>>> return reset();
>>> } else {
>>> reset();
>>> // The last chunk, which is empty
>>> return HttpChunk.LAST_CHUNK;
>>> }
>>> } finally {
>>> checkpoint();
>>> }
>>> }
>>>
>>> ...
>>>
>>>
>>> Then also in HttpMessageDecoder adding the following method :
>>>
>>> private void readFooterHeaders(ChannelBuffer buffer) throws
>>> TooLongFrameException {
>>> HttpFooterHeaders footerHeaders = new HttpFooterHeaders();
>>> headerSize = 0;
>>> String line = readHeader(buffer);
>>> String lastHeader = null;
>>> if (line.length() != 0) {
>>> do {
>>> char firstChar = line.charAt(0);
>>> if (lastHeader != null && (firstChar == ' ' || firstChar ==
>>> '\t')) {
>>> List<String> current = footerHeaders.getHeaders(lastHeader);
>>> int lastPos = current.size() - 1;
>>> String newString = current.get(lastPos) + line.trim();
>>> current.set(lastPos, newString);
>>> } else {
>>> String[] header = splitHeader(line);
>>> footerHeaders.addHeader(header[0], header[1]);
>>> lastHeader = header[0];
>>> }
>>>
>>> line = readHeader(buffer);
>>> } while (line.length() != 0);
>>> }
>>>
>>> Set<String> headerNames = footerHeaders.getHeaderNames();
>>> for (Iterator<String> it = headerNames.iterator(); it.hasNext();) {
>>> String headerName = it.next();
>>> List<String> headerValues = footerHeaders.getHeaders(headerName);
>>> for (String headerValue : headerValues) {
>>> message.addHeader(headerName, headerValue);
>>> }
>>>
>>> }
>>>
>>> }
>>>
>>>
>>> The following class needs to then be added to the package
>>> org.jboss.netty.handler.codec.http :
>>>
>>> package org.jboss.netty.handler.codec.http;
>>>
>>> import java.util.ArrayList;
>>> import java.util.Collections;
>>> import java.util.List;
>>> import java.util.Map;
>>> import java.util.Set;
>>> import java.util.TreeMap;
>>>
>>> import org.jboss.netty.util.internal.CaseIgnoringComparator;
>>>
>>> public class HttpFooterHeaders {
>>>
>>> private final Map<String, List<String>> headers = new TreeMap<String,
>>> List<String>>(CaseIgnoringComparator.INSTANCE);
>>>
>>> public HttpFooterHeaders() {
>>> }
>>>
>>> public void addHeader(final String name, final String value) {
>>> validateHeaderName(name);
>>> validateHeaderValue(value);
>>> if (value == null) {
>>> throw new NullPointerException("value is null");
>>> }
>>> //RFC2616 section 14.40 footer headers MUST NOT contain
>>> Transfer-Encoding, Content-Length or Trailer
>>> if (name.equalsIgnoreCase("Transfer-Encoding") ||
>>> name.equalsIgnoreCase("Content-Length") || name.equalsIgnoreCase("Trailer"))
>>> {
>>> return;
>>> }
>>> if (headers.get(name) == null) {
>>> headers.put(name, new ArrayList<String>(1));
>>> }
>>> headers.get(name).add(value);
>>> }
>>>
>>> private static void validateHeaderName(String name) {
>>> if (name == null) {
>>> throw new NullPointerException("name");
>>> }
>>> for (int i = 0; i < name.length(); i++) {
>>> char c = name.charAt(i);
>>> if (c > 127) {
>>> throw new IllegalArgumentException(
>>> "name contains non-ascii character: " + name);
>>> }
>>>
>>> // Check prohibited characters.
>>> switch (c) {
>>> case '=':
>>> case ',':
>>> case ';':
>>> case ' ':
>>> case ':':
>>> case '\t':
>>> case '\r':
>>> case '\n':
>>> case '\f':
>>> case 0x0b: // Vertical tab
>>> throw new IllegalArgumentException(
>>> "name contains one of the following prohibited
>>> characters: " +
>>> "=,;: \\t\\r\\n\\v\\f: " + name);
>>> }
>>> }
>>> }
>>>
>>> private static void validateHeaderValue(String value) {
>>> if (value == null) {
>>> throw new NullPointerException("value");
>>> }
>>> for (int i = 0; i < value.length(); i++) {
>>> char c = value.charAt(i);
>>> // Check prohibited characters.
>>> switch (c) {
>>> case '\r':
>>> case '\n':
>>> case '\f':
>>> case 0x0b: // Vertical tab
>>> throw new IllegalArgumentException(
>>> "value contains one of the following prohibited
>>> characters: " +
>>> "\\r\\n\\v\\f: " + value);
>>> }
>>> }
>>> }
>>>
>>> public List<String> getHeaders(final String name) {
>>> List<String> values = headers.get(name);
>>> if (values == null) {
>>> return Collections.emptyList();
>>> } else {
>>> return values;
>>> }
>>> }
>>>
>>> public Set<String> getHeaderNames() {
>>> return headers.keySet();
>>> }
>>> }
>>>
>>>
>>> This is probably not the best way as it duplicates some of the code from
>>> HttpMessage as the headers and methods are built into the class.
>>> The problem is then adding the trailing headers to the HttpMessage headers
>>> without clearing them.
>>> This is a quick fix, maybe you understand a better way to implement it.
>>>
>>> I've not found the way to send chunked data from the client side, will
>>> continue looking.
>>>
>>> Thanks for the effort you've done so far on NIO frameworks.
>>> We've been using Mina for a while now and it has been working very well.
>>> We are now migrating onto Netty and so far it looks like you have done an
>>> excellent job.
>>>
>>>
>>> Regards
>>> Roger F
>>> --
>>> View this message in context: http://n2.nabble.com/HttpMessageDecoder-Skip-the-footer-does-anyone-use-it-tp3948901p3948901.html
>>> Sent from the Netty Developer Group mailing list archive at Nabble.com.
>>> _______________________________________________
>>> netty-dev mailing list
>>> netty-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/netty-dev
>>>
>>
>
More information about the netty-dev
mailing list