How to host a high performing Netty App in a fluctuating network environment?

[John D. Mitchell]

On Jul 27, 2011, at 14:35 , George wrote:

> Thanks for your comments John,
> 
> Part 1 & 3 of your reply, Video stream scaling in terms of network through put is complex as every user needs consistent stream.

Depends on what you mean by "consistent". Client side buffering can deal with an awful lot of jitter.

Back in the days when ATM networking was created (the network kind, not the cash machine kind), video was also used as the bugaboo for things like reserved bandwidth "virtual circuits" and the like. So all sorts of complexity was added to support that stuff. Then folks did a lot of benchmarks and performance testing and still found that "best effort" with non-stupid endpoints still outperformed and had better utilization.

> But i am working on HTTP streaming, so the scaling part is comparatively easier than consistent video streaming. I can redirect user on secs or mins interval, lower per user throughput etc.

So you're just doing HTTP range requests against e.g. big video files? If that's all you need then why not just use e.g. nginx?

> The requests will be 1 or 2KB, but the response could be into MBs, the possibility of hitting OOM should be there from what i understood from other posts. But it depends on the app on how it reacts to channel writeable status. I will read more on this.

Why would it need to hold all of the content in memory instead of reading it in chunks off the disk?

> Real problem and a very well existing one is part 2. Good servers connected to a poor backbone switch. This is the one i am trying to solve. 
> For example, when people report of getting only 300Mbps through put in their 1Gbps NIC cards, basically it means the hosted environment is allowing them that much. This is the real metrics i want to find out.
> From your reply, what i understand  is trying to predict the hosted environment congestion in Netty app using the channels status is not a good solution and it may very well not be a real scenario of the network or the experience of the users.

This is an operations issue, not just an application issue.

You'll need to understand what's going on in your operational environment. From your language, it sounds like you're using some sort of VPS or "cloud" service where you don't know much of anything about the operational environment. Depending on the specifics, you may not be able to reliably know much. :-(  [And yes, this is a huge downside of "clouds".]

If that's the case then there's not much you can do except have your client side be smarter about e.g. gathering a set of candidate servers, test them, and then use them and fast failover to another server. Not sure how you're client-side video is setup in terms of being able to switch sources easily/quickly/seamlessly but that's where I'd start.

Hope this helps,
John