New full duplex HTTP tunnel without need for servlet container

[Trustin Lee (이희승)]

This is a fantastic news, Iain!

I'd love to include your contribution in 3.2 branch.  Actually I was
going to start from your previous work, but it would be an icing on
the cake if you are ready to push better version.

If I remember correctly, you already signed up a contributor license
agreement, so all I can think of is posting the source code here or to
JIRA for review.  Once the contribution is accepted, it would be a
better idea for me to give you the commit access to the official
repository so that you can improve it without any red tapes.  WDYT?

— Trustin Lee, http://gleamynode.net/

On Tue, Sep 29, 2009 at 7:57 PM, Iain McGinniss <iainmcgin at gmail.com> wrote:
> Hello all,
>
> Firstly, a quick introduction: my name is Iain McGinniss, and I am
> presently employed by OneDrum (www.onedrum.com) to contribute to open
> source projects which they use at the core of their product. This
> started as performance and stability fixes for the JXTA peer to peer
> library. JXTA presently uses a hand-crafted NIO TCP stack as it's
> primary means of communication, with a Jetty based HTTP fallback.
> Neither are particularly stable, so once I dealt with some other
> serious issues in their architecture I turned my attention to
> replacing these transports. This brought me to the Netty project,
> after a brief foray with Grizzly. The TCP stack in JXTA has been
> reimplemented to use Netty already, and I have written a new HTTP
> tunnel to replace the HTTP stack.
>
> This brings me to the purpose of this email. I had previously
> contacted Trustin regarding this HTTP tunnel, which I am in the
> process of polishing off just now. The guiding principles of it's
> design were:
>
> 1. Utilise HTTP in a way that will allow it to work through most
> firewalls and proxy servers. This rules out HTTP pipelining as a way
> of getting full duplex behaviour on a single connection. Instead, two
> connections must be used, one to send data from client to server and
> the other to long-poll data back from server to client. This is
> essentially the Comet technique.
> 2. Use only Netty to implement the server and client. Specifically,
> the tunnel should not require a servlet container or any other
> infrastructure, just Netty's TCP stack and HTTP codec.
> 3. Make the tunnel appear like a normal channel to both client and
> server. Emulate the behaviour of a normal TCP connection for the
> tunnel as far as possible, abstracting away the details of how the
> data is actually transmitted.
>
> The above resulted in the creation of:
>
> HttpTunnelClientChannelFactory - takes a ClientSocketChannelFactory
> for the creation of the real TCP connections used to service the tunnel.
> HttpTunnelClientChannel - a virtual channel, wrapping the send and
> poll TCP channels.
> HttpTunnelServerChannelFactory - takes a ServerSocketChannelFactory to
> create the underlying TCP server socket used to accept tunnel
> connections.
> HttpTunnelServerChannel - a virtual server channel
> HttpTunnelAcceptedChannel - a virtual accepted channel, representing
> the server end of a tunnel.
> ... and various other supporting types (Sinks, ChannelHandlers,
> reasonably comprehensive unit tests of each).
>
> No assumptions are made about how long any particular incoming
> connection will last on the server end, so a small component (the
> ServerMessageSwitch) sits in-between the physical channels and the
> virtual ones, routing messages in and out. This provides resilience to
> proxies that reuse HTTP connections or close them periodically in
> response to inactivity.
>
> Design principle 3 above is the one which causes me the most
> difficulty - as I am still relatively new to Netty, I am not sure what
> semantics of channels are regarded as essential and which can be
> treated as optional. For instance, my client channels are not yet
> configurable - they have a SocketChannelConfig based configuration
> object, but it is not used and it's properties are not honoured in any
> way. I also don't do anything with interest ops, and my handling of
> events (exceptions, timeouts, state transitions) is likely to be
> sloppy in places. There are also likely some channel semantics which I
> am completely unaware of.
>
> So, to summarise: I have this new tunnel, which once polished I
> believe will be a good compliment or replacement to the existing HTTP
> tunnel in Netty. It could do with some detailed code review and
> contributions to make it production ready and as high quality as the
> other components in Netty. Also, I am unaware how the contribution
> process works in this community - how should I submit patches, what
> quality gates are there before a contribution is fully accepted and
> committed to trunk? Some pointers around these aspects would be
> greatly appreciated.
>
> Iain
> _______________________________________________
> netty-dev mailing list
> netty-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-dev
>