HttpMessageDecoder - Skip the footer; does anyone use it?
Trustin Lee (이희승)
trustin at gmail.com
Sat Nov 7 10:00:17 EST 2009
Hi Roger,
Oh, there was really someone who uses it! :)
I will try to add support for trailing headers in 3.2. Your patch
does look very helpful. Thanks a lot!
Cheers
— Trustin Lee, http://gleamynode.net/
On Thu, Nov 5, 2009 at 7:52 AM, Roger F <rfullerton at mimecast.com> wrote:
>
> Hi Trustin Lee,
>
> I have been trying to use TrailingHeaders after a Content-Transfer: chunked
> message, obviously it does not work, yet.
>
> This would be useful to allow large files, size unknown, to be streamed in
> chunks, followed by the MD5.
> This ensures data integrity and prevents out of memory in a server.
>
> Here is an example :
>
> PUT / HTTP/1.1
> Host: localhost:8081
> Transfer-Encoding: chunked
> Trailer: Content-MD5
>
> 64
> 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
> 0
> Content-MD5: 4f30805221b29b36df636869549398fb
>
>
>
> The following code changes allow TrailingHeaders and seem to work so far in
> testing.
>
> In HttpMessageDecoder replacing in the decode method the following switch
> state READ_CHUNK_FOOTER :
>
> @Override
> protected Object decode(ChannelHandlerContext ctx, Channel channel,
> ChannelBuffer buffer, State state) throws Exception {
>
> ....
>
> case READ_CHUNK_FOOTER: {
> // Do not Skip the footer; some people use it!
>
> try {
>
> readFooterHeaders(buffer);
>
> if (maxChunkSize == 0) {
> // Chunked encoding disabled.
> return reset();
> } else {
> reset();
> // The last chunk, which is empty
> return HttpChunk.LAST_CHUNK;
> }
> } finally {
> checkpoint();
> }
> }
>
> ...
>
>
> Then also in HttpMessageDecoder adding the following method :
>
> private void readFooterHeaders(ChannelBuffer buffer) throws
> TooLongFrameException {
> HttpFooterHeaders footerHeaders = new HttpFooterHeaders();
> headerSize = 0;
> String line = readHeader(buffer);
> String lastHeader = null;
> if (line.length() != 0) {
> do {
> char firstChar = line.charAt(0);
> if (lastHeader != null && (firstChar == ' ' || firstChar ==
> '\t')) {
> List<String> current = footerHeaders.getHeaders(lastHeader);
> int lastPos = current.size() - 1;
> String newString = current.get(lastPos) + line.trim();
> current.set(lastPos, newString);
> } else {
> String[] header = splitHeader(line);
> footerHeaders.addHeader(header[0], header[1]);
> lastHeader = header[0];
> }
>
> line = readHeader(buffer);
> } while (line.length() != 0);
> }
>
> Set<String> headerNames = footerHeaders.getHeaderNames();
> for (Iterator<String> it = headerNames.iterator(); it.hasNext();) {
> String headerName = it.next();
> List<String> headerValues = footerHeaders.getHeaders(headerName);
> for (String headerValue : headerValues) {
> message.addHeader(headerName, headerValue);
> }
>
> }
>
> }
>
>
> The following class needs to then be added to the package
> org.jboss.netty.handler.codec.http :
>
> package org.jboss.netty.handler.codec.http;
>
> import java.util.ArrayList;
> import java.util.Collections;
> import java.util.List;
> import java.util.Map;
> import java.util.Set;
> import java.util.TreeMap;
>
> import org.jboss.netty.util.internal.CaseIgnoringComparator;
>
> public class HttpFooterHeaders {
>
> private final Map<String, List<String>> headers = new TreeMap<String,
> List<String>>(CaseIgnoringComparator.INSTANCE);
>
> public HttpFooterHeaders() {
> }
>
> public void addHeader(final String name, final String value) {
> validateHeaderName(name);
> validateHeaderValue(value);
> if (value == null) {
> throw new NullPointerException("value is null");
> }
> //RFC2616 section 14.40 footer headers MUST NOT contain
> Transfer-Encoding, Content-Length or Trailer
> if (name.equalsIgnoreCase("Transfer-Encoding") ||
> name.equalsIgnoreCase("Content-Length") || name.equalsIgnoreCase("Trailer"))
> {
> return;
> }
> if (headers.get(name) == null) {
> headers.put(name, new ArrayList<String>(1));
> }
> headers.get(name).add(value);
> }
>
> private static void validateHeaderName(String name) {
> if (name == null) {
> throw new NullPointerException("name");
> }
> for (int i = 0; i < name.length(); i++) {
> char c = name.charAt(i);
> if (c > 127) {
> throw new IllegalArgumentException(
> "name contains non-ascii character: " + name);
> }
>
> // Check prohibited characters.
> switch (c) {
> case '=':
> case ',':
> case ';':
> case ' ':
> case ':':
> case '\t':
> case '\r':
> case '\n':
> case '\f':
> case 0x0b: // Vertical tab
> throw new IllegalArgumentException(
> "name contains one of the following prohibited
> characters: " +
> "=,;: \\t\\r\\n\\v\\f: " + name);
> }
> }
> }
>
> private static void validateHeaderValue(String value) {
> if (value == null) {
> throw new NullPointerException("value");
> }
> for (int i = 0; i < value.length(); i++) {
> char c = value.charAt(i);
> // Check prohibited characters.
> switch (c) {
> case '\r':
> case '\n':
> case '\f':
> case 0x0b: // Vertical tab
> throw new IllegalArgumentException(
> "value contains one of the following prohibited
> characters: " +
> "\\r\\n\\v\\f: " + value);
> }
> }
> }
>
> public List<String> getHeaders(final String name) {
> List<String> values = headers.get(name);
> if (values == null) {
> return Collections.emptyList();
> } else {
> return values;
> }
> }
>
> public Set<String> getHeaderNames() {
> return headers.keySet();
> }
> }
>
>
> This is probably not the best way as it duplicates some of the code from
> HttpMessage as the headers and methods are built into the class.
> The problem is then adding the trailing headers to the HttpMessage headers
> without clearing them.
> This is a quick fix, maybe you understand a better way to implement it.
>
> I've not found the way to send chunked data from the client side, will
> continue looking.
>
> Thanks for the effort you've done so far on NIO frameworks.
> We've been using Mina for a while now and it has been working very well.
> We are now migrating onto Netty and so far it looks like you have done an
> excellent job.
>
>
> Regards
> Roger F
> --
> View this message in context: http://n2.nabble.com/HttpMessageDecoder-Skip-the-footer-does-anyone-use-it-tp3948901p3948901.html
> Sent from the Netty Developer Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-dev mailing list
> netty-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-dev
>
More information about the netty-dev
mailing list