New full duplex HTTP tunnel without need for servlet container

[Iain McGinniss]

Direct commit access would be good, and I'm happy to work on a branch  
on the side so I don't break anything (a mortal fear of mine,  
especially when I'm still finding my way around Netty's internals).  
I'm presently moving all my code into a checkout of Netty trunk, and  
I'll submit a patch based on those changes to JIRA. There will  
definitely be issues, and I need some guidance on what is important  
(i.e. what will block the contribution being accepted) and what isn't  
(things which are bad or questionable but not necessarily blocking  
issues).

I'll submit the patch ASAP.
Iain

On 29 Sep 2009, at 13:44, Trustin Lee (이희승) wrote:

> This is a fantastic news, Iain!
>
> I'd love to include your contribution in 3.2 branch.  Actually I was
> going to start from your previous work, but it would be an icing on
> the cake if you are ready to push better version.
>
> If I remember correctly, you already signed up a contributor license
> agreement, so all I can think of is posting the source code here or to
> JIRA for review.  Once the contribution is accepted, it would be a
> better idea for me to give you the commit access to the official
> repository so that you can improve it without any red tapes.  WDYT?
>
> — Trustin Lee, http://gleamynode.net/
>
> On Tue, Sep 29, 2009 at 7:57 PM, Iain McGinniss  
> <iainmcgin at gmail.com> wrote:
>> Hello all,
>>
>> Firstly, a quick introduction: my name is Iain McGinniss, and I am
>> presently employed by OneDrum (www.onedrum.com) to contribute to open
>> source projects which they use at the core of their product. This
>> started as performance and stability fixes for the JXTA peer to peer
>> library. JXTA presently uses a hand-crafted NIO TCP stack as it's
>> primary means of communication, with a Jetty based HTTP fallback.
>> Neither are particularly stable, so once I dealt with some other
>> serious issues in their architecture I turned my attention to
>> replacing these transports. This brought me to the Netty project,
>> after a brief foray with Grizzly. The TCP stack in JXTA has been
>> reimplemented to use Netty already, and I have written a new HTTP
>> tunnel to replace the HTTP stack.
>>
>> This brings me to the purpose of this email. I had previously
>> contacted Trustin regarding this HTTP tunnel, which I am in the
>> process of polishing off just now. The guiding principles of it's
>> design were:
>>
>> 1. Utilise HTTP in a way that will allow it to work through most
>> firewalls and proxy servers. This rules out HTTP pipelining as a way
>> of getting full duplex behaviour on a single connection. Instead, two
>> connections must be used, one to send data from client to server and
>> the other to long-poll data back from server to client. This is
>> essentially the Comet technique.
>> 2. Use only Netty to implement the server and client. Specifically,
>> the tunnel should not require a servlet container or any other
>> infrastructure, just Netty's TCP stack and HTTP codec.
>> 3. Make the tunnel appear like a normal channel to both client and
>> server. Emulate the behaviour of a normal TCP connection for the
>> tunnel as far as possible, abstracting away the details of how the
>> data is actually transmitted.
>>
>> The above resulted in the creation of:
>>
>> HttpTunnelClientChannelFactory - takes a ClientSocketChannelFactory
>> for the creation of the real TCP connections used to service the  
>> tunnel.
>> HttpTunnelClientChannel - a virtual channel, wrapping the send and
>> poll TCP channels.
>> HttpTunnelServerChannelFactory - takes a ServerSocketChannelFactory  
>> to
>> create the underlying TCP server socket used to accept tunnel
>> connections.
>> HttpTunnelServerChannel - a virtual server channel
>> HttpTunnelAcceptedChannel - a virtual accepted channel, representing
>> the server end of a tunnel.
>> ... and various other supporting types (Sinks, ChannelHandlers,
>> reasonably comprehensive unit tests of each).
>>
>> No assumptions are made about how long any particular incoming
>> connection will last on the server end, so a small component (the
>> ServerMessageSwitch) sits in-between the physical channels and the
>> virtual ones, routing messages in and out. This provides resilience  
>> to
>> proxies that reuse HTTP connections or close them periodically in
>> response to inactivity.
>>
>> Design principle 3 above is the one which causes me the most
>> difficulty - as I am still relatively new to Netty, I am not sure  
>> what
>> semantics of channels are regarded as essential and which can be
>> treated as optional. For instance, my client channels are not yet
>> configurable - they have a SocketChannelConfig based configuration
>> object, but it is not used and it's properties are not honoured in  
>> any
>> way. I also don't do anything with interest ops, and my handling of
>> events (exceptions, timeouts, state transitions) is likely to be
>> sloppy in places. There are also likely some channel semantics  
>> which I
>> am completely unaware of.
>>
>> So, to summarise: I have this new tunnel, which once polished I
>> believe will be a good compliment or replacement to the existing HTTP
>> tunnel in Netty. It could do with some detailed code review and
>> contributions to make it production ready and as high quality as the
>> other components in Netty. Also, I am unaware how the contribution
>> process works in this community - how should I submit patches, what
>> quality gates are there before a contribution is fully accepted and
>> committed to trunk? Some pointers around these aspects would be
>> greatly appreciated.
>>
>> Iain
>> _______________________________________________
>> netty-dev mailing list
>> netty-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-dev
>>
>
> _______________________________________________
> netty-dev mailing list
> netty-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-dev