HttpMessageDecoder - Skip the footer; does anyone use it?
Trustin Lee (이희승)
trustin at gmail.com
Mon Nov 16 10:47:51 EST 2009
Here's the relevant JIRA issue:
https://jira.jboss.org/jira/browse/NETTY-251
— Trustin Lee, http://gleamynode.net/
On Sun, Nov 8, 2009 at 12:00 AM, Trustin Lee (이희승) <trustin at gmail.com> wrote:
> Hi Roger,
>
> Oh, there was really someone who uses it! :)
>
> I will try to add support for trailing headers in 3.2. Your patch
> does look very helpful. Thanks a lot!
>
> Cheers
>
> — Trustin Lee, http://gleamynode.net/
>
>
>
> On Thu, Nov 5, 2009 at 7:52 AM, Roger F <rfullerton at mimecast.com> wrote:
>>
>> Hi Trustin Lee,
>>
>> I have been trying to use TrailingHeaders after a Content-Transfer: chunked
>> message, obviously it does not work, yet.
>>
>> This would be useful to allow large files, size unknown, to be streamed in
>> chunks, followed by the MD5.
>> This ensures data integrity and prevents out of memory in a server.
>>
>> Here is an example :
>>
>> PUT / HTTP/1.1
>> Host: localhost:8081
>> Transfer-Encoding: chunked
>> Trailer: Content-MD5
>>
>> 64
>> 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
>> 0
>> Content-MD5: 4f30805221b29b36df636869549398fb
>>
>>
>>
>> The following code changes allow TrailingHeaders and seem to work so far in
>> testing.
>>
>> In HttpMessageDecoder replacing in the decode method the following switch
>> state READ_CHUNK_FOOTER :
>>
>> @Override
>> protected Object decode(ChannelHandlerContext ctx, Channel channel,
>> ChannelBuffer buffer, State state) throws Exception {
>>
>> ....
>>
>> case READ_CHUNK_FOOTER: {
>> // Do not Skip the footer; some people use it!
>>
>> try {
>>
>> readFooterHeaders(buffer);
>>
>> if (maxChunkSize == 0) {
>> // Chunked encoding disabled.
>> return reset();
>> } else {
>> reset();
>> // The last chunk, which is empty
>> return HttpChunk.LAST_CHUNK;
>> }
>> } finally {
>> checkpoint();
>> }
>> }
>>
>> ...
>>
>>
>> Then also in HttpMessageDecoder adding the following method :
>>
>> private void readFooterHeaders(ChannelBuffer buffer) throws
>> TooLongFrameException {
>> HttpFooterHeaders footerHeaders = new HttpFooterHeaders();
>> headerSize = 0;
>> String line = readHeader(buffer);
>> String lastHeader = null;
>> if (line.length() != 0) {
>> do {
>> char firstChar = line.charAt(0);
>> if (lastHeader != null && (firstChar == ' ' || firstChar ==
>> '\t')) {
>> List<String> current = footerHeaders.getHeaders(lastHeader);
>> int lastPos = current.size() - 1;
>> String newString = current.get(lastPos) + line.trim();
>> current.set(lastPos, newString);
>> } else {
>> String[] header = splitHeader(line);
>> footerHeaders.addHeader(header[0], header[1]);
>> lastHeader = header[0];
>> }
>>
>> line = readHeader(buffer);
>> } while (line.length() != 0);
>> }
>>
>> Set<String> headerNames = footerHeaders.getHeaderNames();
>> for (Iterator<String> it = headerNames.iterator(); it.hasNext();) {
>> String headerName = it.next();
>> List<String> headerValues = footerHeaders.getHeaders(headerName);
>> for (String headerValue : headerValues) {
>> message.addHeader(headerName, headerValue);
>> }
>>
>> }
>>
>> }
>>
>>
>> The following class needs to then be added to the package
>> org.jboss.netty.handler.codec.http :
>>
>> package org.jboss.netty.handler.codec.http;
>>
>> import java.util.ArrayList;
>> import java.util.Collections;
>> import java.util.List;
>> import java.util.Map;
>> import java.util.Set;
>> import java.util.TreeMap;
>>
>> import org.jboss.netty.util.internal.CaseIgnoringComparator;
>>
>> public class HttpFooterHeaders {
>>
>> private final Map<String, List<String>> headers = new TreeMap<String,
>> List<String>>(CaseIgnoringComparator.INSTANCE);
>>
>> public HttpFooterHeaders() {
>> }
>>
>> public void addHeader(final String name, final String value) {
>> validateHeaderName(name);
>> validateHeaderValue(value);
>> if (value == null) {
>> throw new NullPointerException("value is null");
>> }
>> //RFC2616 section 14.40 footer headers MUST NOT contain
>> Transfer-Encoding, Content-Length or Trailer
>> if (name.equalsIgnoreCase("Transfer-Encoding") ||
>> name.equalsIgnoreCase("Content-Length") || name.equalsIgnoreCase("Trailer"))
>> {
>> return;
>> }
>> if (headers.get(name) == null) {
>> headers.put(name, new ArrayList<String>(1));
>> }
>> headers.get(name).add(value);
>> }
>>
>> private static void validateHeaderName(String name) {
>> if (name == null) {
>> throw new NullPointerException("name");
>> }
>> for (int i = 0; i < name.length(); i++) {
>> char c = name.charAt(i);
>> if (c > 127) {
>> throw new IllegalArgumentException(
>> "name contains non-ascii character: " + name);
>> }
>>
>> // Check prohibited characters.
>> switch (c) {
>> case '=':
>> case ',':
>> case ';':
>> case ' ':
>> case ':':
>> case '\t':
>> case '\r':
>> case '\n':
>> case '\f':
>> case 0x0b: // Vertical tab
>> throw new IllegalArgumentException(
>> "name contains one of the following prohibited
>> characters: " +
>> "=,;: \\t\\r\\n\\v\\f: " + name);
>> }
>> }
>> }
>>
>> private static void validateHeaderValue(String value) {
>> if (value == null) {
>> throw new NullPointerException("value");
>> }
>> for (int i = 0; i < value.length(); i++) {
>> char c = value.charAt(i);
>> // Check prohibited characters.
>> switch (c) {
>> case '\r':
>> case '\n':
>> case '\f':
>> case 0x0b: // Vertical tab
>> throw new IllegalArgumentException(
>> "value contains one of the following prohibited
>> characters: " +
>> "\\r\\n\\v\\f: " + value);
>> }
>> }
>> }
>>
>> public List<String> getHeaders(final String name) {
>> List<String> values = headers.get(name);
>> if (values == null) {
>> return Collections.emptyList();
>> } else {
>> return values;
>> }
>> }
>>
>> public Set<String> getHeaderNames() {
>> return headers.keySet();
>> }
>> }
>>
>>
>> This is probably not the best way as it duplicates some of the code from
>> HttpMessage as the headers and methods are built into the class.
>> The problem is then adding the trailing headers to the HttpMessage headers
>> without clearing them.
>> This is a quick fix, maybe you understand a better way to implement it.
>>
>> I've not found the way to send chunked data from the client side, will
>> continue looking.
>>
>> Thanks for the effort you've done so far on NIO frameworks.
>> We've been using Mina for a while now and it has been working very well.
>> We are now migrating onto Netty and so far it looks like you have done an
>> excellent job.
>>
>>
>> Regards
>> Roger F
>> --
>> View this message in context: http://n2.nabble.com/HttpMessageDecoder-Skip-the-footer-does-anyone-use-it-tp3948901p3948901.html
>> Sent from the Netty Developer Group mailing list archive at Nabble.com.
>> _______________________________________________
>> netty-dev mailing list
>> netty-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-dev
>>
>
More information about the netty-dev
mailing list