Reconnecting a client automatically *without losing state*

[Bruno de Carvalho]

Granted, Netty's ChannelPipeline is meant to be somewhat static but I
beg to disagree that it doesn't help you.

One thing you have to consider with Netty - this is *my* understanding,
so take it with a grain of salt - is that your last handler is the
equivalent of MINA's IoHandlerAdapter; you can call it a "data sink" if
you prefer.

Now, MINA is friendlier here as you clearly understand where the data
hits your application code - the IoHandler - whereas Netty doesn't have
an explicit sink.

What I end up doing most of the times is keeping retry information
(packets that were not sent and stuff like that) outside (read, a layer
above) my last channel handler. This way I can even abstract code as to
not rely solely on Netty or other I/O framework. It also makes
reconnection way simpler as I don't have to reuse the same
ChannelHandler (because the state is kept outside of it).

So, back on the routing part, you would either implement the GET and
POST handling logic directly on your channel handler or, preferably,
you'd have your last channel handler call another module of your code
that'd actually take care of handling the request and producing a
response. Of course, just like it happens with Servlets, you'd have to
pass a reference to the channel through which the request arrived so
that the response would later be sent through that same channel.

You could also have a more complex pipeline and communicate through your
handlers by firing ChannelEvents upwards or downwards, but I'd avoid
this unless it's a very specific case.

Here's a link for a tutorial I've written a couple of weeks ago about
handshaking that includes some more "advanced" stuff with Netty like
modifying a pipeline after handshaking completes and using custom
ChannelEvent's to communicate between my own handlers.

http://bruno.factor45.org/blag/2010/07/15/handshaking-tutorial-with-netty/

Also, I'm about to release a high performance http client based on Netty
(I'm hoping this week, just have to finish up the user guide and the
stylesheets for docbook and javadoc :/ ). In this client I make
extensive usage of this keep-the-important-logic-outside-of-the-handler
philosophy:
- HttpClient has several HttpConnections (the ChannelHandlers);
- HttpClient is a listener on each of the HttpConnections (via
constructor arg);
- HttpClient executes requests through them, but the request queue is
maintained by HttpClient;
- Whenever something happens on the HttpConnection, it calls the
appropriate method on the client to signal something happened.

I've found this approach to be elegant enough to avoid most of the
common confusions with Netty and also useful with reconnections: I can
simply create another bootstrap without the need to reuse the old
handler, since the important state logic was outside.

Hope this sheds some light on the subject.


Cheers,
  Bruno

On Mon, 2010-08-02 at 17:36 -0700, tsuna wrote:
> On Mon, Aug 2, 2010 at 3:19 PM, Bruno de Carvalho <kindernade at gmail.com> wrote:
> > This is one of those cases where I'd ditch the PipelineFactory and
> > create the pipeline manually.
> 
> Oh, yeah, didn't think about that either.  I'll do that.  Netty could
> make it easier to handle disconnections.
> 
> I'm actually considering to stop using Netty's pipeline stuff
> altogether and just use Netty as an async NIO library.
> 
> ChannelPipeline seems only good for simple things but it's not
> suitable for anything non-trivial such as RPC servers (IMO).  Let me
> take a couple of specific examples: a ChannelPipeline is mostly
> static, you can change it, but it's not really designed to change
> dynamically all the time.  For instance if you write an HTTP server,
> you need to handle routing of requests (e.g. GET /foo should trigger
> one piece of code whereas POST /bar should trigger another piece of
> code).  This routing is dynamic by nature: every request needs a
> different "processing pipeline".  But you can't (and actually
> shouldn't) adjust the last handler of your ChannelPipeline to do this
> routing.  Instead you have to do the routing manually, outside of
> Netty (unless I'm seriously missing something).
> 
> Another example is: if your server is serving an RPC, and to serve
> this RPC it may or may not need to send a few other RPCs to other
> servers, handle their responses asynchronously and then respond to the
> original RPC.  The ChannelPipeline doesn't help you at all deal with
> this kind of situation, it's actually the opposite, and once again
> it's something you have to do manually yourself outside of Netty.
> 
> Most of the Netty examples are too trivial to hit those problems.
> They all handle every request in pretty much the same way, which is
> why the static ChannelPipeline approach works well.  A lot of the
> Netty-using code I've read handles the life cycle of the requests
> outside of Netty, manually, everyone has their own home grown
> approach.
> 
> Although I've never used Apache MINA, my understanding of it is that
> with MINA you have to create an FSM (Finite State Machine) to handle
> the lifecycle of a requests.  FSMs are better at doing this kind of
> dynamic processing but they're cumbersome to write and manage, so
> people tend to dislike them (in my experience).
> 
> I implemented Twisted's Deferred API in Java and I've been using it
> with much success inside Netty to handle the dynamic lifecycle of my
> requests.  Deferred allows you to easily build an FSM dynamically and
> *implicitly*, meaning you don't have to explicitly define, implement,
> and manage an FSM.  I'm going to open-source this code in a couple of
> weeks or so.  I think an API like Deferred is good enough to entirely
> replace the notion of ChannelPipeline.  It provides the same basic
> features with many more advantages.
> </offtopic>
>