Newbie questions

[Iain McGinniss]

Hi Trustin,

I fully intend to get the reusable chunks of what I have written  
packaged up and contributed to netty, and perhaps become an active  
contributor to the project. For the last 3 months I have had the  
luxury of being paid to write open source software, by OneDrum (www.onedrum.com 
). I have been fixing a number of issues in the JXTA peer to peer  
framework, mostly performance analysis and optimisation. I have had a  
big impact on the project in my time working on it, and the last major  
piece of work to be delivered (today or tomorrow, eek!) is a complete  
replacement for both the TCP and HTTP transports that are used. I've  
rewritten both using Netty - it's been a challenge, but I've made it  
in the end.

The HTTP tunnel I have written is full duplex and does not require a  
servlet container for the server end, in contrast to the existing HTTP  
tunnel that ships in Netty 3.x. It does this by using two physical  
connections per tunnel much like the "comet" comms strategy that has  
become so popular for rich internet applications. I had originally  
planned to do full duplex using a single connection with request  
pipelining and chunking to keep the overhead as low as possible, but  
this unfortunately does not work with most HTTP proxies, which do not  
relay pipelined requests (they consume them from the client, then  
dispatch them in the normal request-response half duplex fashion to  
the server).

I still have some work to do around connection closing, to make sure  
the virtual tunnel connection on client and server behaves exactly  
like a normal netty channel in terms of the events fired and  
behaviour. The server end was a particular challenge, especially since  
the requests for a particular tunnel could arrive across multiple  
connections over a period of time, with proxies closing connections or  
potentially reusing connections for multiple clients. I also need to  
verify that my usage patterns do not raise red flags with common  
proxies and firewalls.

Tomorrow is my last day working full time on JXTA, so I will need to  
package up everything I've got by tomorrow. I'll send out my HTTP  
tunnel code, regard it as experimental and only partially tested. From  
October onwards I will be working part time on JXTA and most likely  
Netty as well, so if someone else in the community does not pick it up  
I will polish up what I have and get it ready for inclusion in the  
official Netty build.

Overall, my experience with Netty has been quite positive - the API is  
quite clearly documented. I do however feel there needs to be more  
examples of common usage patterns - in my tunnel code, I feel that in  
a number of places I have had to make up an approach based on my own  
instincts, without any real feel for how this will affect the  
performance of behaviour of Netty as a whole. An example would be  
capturing close events and deferring sending them downstream until  
some other key behaviour has occurred, such as sending a close  
handshake to the server. I have had to dip into the implementations of  
the Nio channels a number of times to verify my assumptions on how  
they behave, so perhaps a more comprehensive breakdown of the typical  
event flow and lifecycle of a netty channel would be useful.

Anyway, if I do not have time to submit what I have done directly here  
before tomorrow evening, I will submit it next month.

Iain


On 20 Aug 2009, at 08:42, 이희승 (Trustin Lee) wrote:

> Thanks Iain for sharing your work.  I think it looks fine although I
> didn't run it by myself.  I hope Bozo loves your contribution. :)
>
> You might want to include your work into Netty 3.2.  WDYT?
>
> Cheers,
> Trustin
>
> On 08/18/2009 06:46 PM, Iain McGinniss wrote:
>> Hi Bozo,
>>
>> I'll take a quick stab at this. I've attached something like what  
>> you're
>> looking for, but beware: I've not tested this, I've just written it
>> based on my current understanding of netty. Would be useful to have  
>> it
>> reviewed by others in the community to see if it is a sensible  
>> pattern.
>> The alternative would be to use a WriteTimeoutHandler in your  
>> pipeline,
>> but I do not think this fits the requirements here as it will not
>> trigger an exception if writes are being continuously issued, but  
>> never
>> completing (due to a slow connection).
>>
>> Iain
>>
>>
>>
>> On 18 Aug 2009, at 10:11, Bozo Juretic wrote:
>>
>>> 이희승 (Trustin Lee) wrote:
>>>> You could use org.jboss.netty.util.HashedWheelTimer and schedule a
>>>> timeout for each request, cancel the scheduled timeout when a
>>>> corresponding response is received, and handle the timeout if no
>>>> response was received.
>>>>
>>> Trustin,
>>>
>>> first of all thank you for your kind response. Is it possible that  
>>> you
>>> give me a small (pseudo)code example or a pointer to existing code  
>>> to
>>> see how this can be practically done? I understand the principle,  
>>> but am
>>> all too new to netty to do it.
>>>
>>> Thank you,
>>>
>>> Bozo
>>> _______________________________________________
>>> netty-users mailing list
>>> netty-users at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> netty-users mailing list
>> netty-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/netty-users
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users