Large response - how to handle asynchronously

[Łukasz Osipiuk]

Hi!

We really need this solved.
If anybody had similar problem and have some thoughts on it we would
really appreciate help.

Best regards, Łukasz Osipiuk

On Thu, Mar 25, 2010 at 16:14, Łukasz Osipiuk <lukasz at osipiuk.net> wrote:
> Hi guys,
>
> We are implementing server application using Netty protocol framework
> and we have problem implementing handling of large responses.
> We will be handling many simultaneously connected clients.
> In protocol we are implementing, responses to client requests can be
> huge (lets assume that larger than server memory).
> While handling request we have to read part of response data from
> external storage - send it to client - wait until he receives the data
> and continue with next part.
>
> We approached problem with following code in handler (simplified
> version, no error handling and so on):
>
>   @Override
>   public void messageReceived(ChannelHandlerContext ctx, MessageEvent
> e) throws Exception {
>       XXXRequest req = (XXXRequest) e.getMessage();
>       writePartResponse(ctx, req);
>   }
>
>    private void writePartResponse(final ChannelHandlerContext ctx,
> final XXXRequest req) {
>        if (moreDataToBeRead(req)) {// anything more to be done?
>            XXXPartResponse resPart = readSomeData(req); // read some
> data from storage
>            ChannelFuture f = ctx.getChannel().write(resPart);
>            f.addListener(new ChannelFutureListener() {
>                public void operationComplete(ChannelFuture f_) throws
> Exception {
>                    writePartResponse(ctx, req);
>                }
>            });
>        }
>    }
>
> Above approach does not work.
>
> Problem 1.
>
> While handling request we have to block other client's requests  so
> they are not handled simultaneously.
> In above code when messageReceived() method terminates any pending
> client requests are delivered to handler and handled even though
> asynchronous handling of previous request might still be in progress.
> We do not know how to block handling of any pending requests.
>
> Problem 2.
>
> I client is quick enough to read data from socket f.addListener
> immediatelly triggers calling of futer.operationComplete in the same
> thread.
> This leads to StackOverflowError.
> (btw. It appears that StackOverflowError it is not handled correctly
> by netty in this place and application is not notified about it being
> thrown - handling of request is just silently terminated).
>
> Any way to avoid above problems?
> I guess the approach is broken - can anybody lead us how to implement
> such (trivial one might think) behaviour?
>
> Best regards, Łukasz Osipiuk
>
>
> --
> --
> Łukasz Osipiuk
> mailto:lukasz at osipiuk.net
>



-- 
-- 
Łukasz Osipiuk
mailto:lukasz at osipiuk.net