On Wed, 17 Jun 2020, 6:07 am Nate, <nate(a)n4te.com> wrote:
I have a request where I need to return a ~100MB file. I want to:
1) do this efficiently, and
2) limit concurrent downloads by IP (ie reject a request if concurrent
downloads by that IP == x).
I dispatch to a worker thread, then read from the file using
FileInputStream and write to exchange.getOutputStream(). However, I noticed
that this writes all the bytes immediately. Digging into why,
exchange.getOutputStream() uses UndertowOutputStream which appears to
buffer all the data given to it. Not only does this seem inefficient to
copy 100MB for every request, it means I don't know when the transfer
completes so I can't track concurrent downloads.
I don't understand exactly what you are doing here, can you share a code
snippet? The stream will always write everything that is given to it, but
it won't allocate 100M of buffers. If you have loaded the whole file into
memory then it is your code that has buffered the full 100M.
Next I tried getResponseChannel, eg:
StreamSinkChannel output = exchange.getResponseChannel();
try (var input = new FileInputStream(file)) {
Channels.transferBlocking(output, input.getChannel(), position,
count);
}
Digging through the code it doesn't seem to be buffering, but it still
completes immediately. For example, I download the file at 10 bytes/sec:
curl --limit-rate 10 --output file
https://example.com/my/file
How can transferBlocking finish right away when this slow transfer will
take ages? What is buffering the 100MB?
It's a non blocking transfer. When The response can take more days then
more is read and transferred.
How can my worker thread write response data one buffer at a time,
blocking as needed for the client to receive the data, until the client has
all the data? This would allow me to know when the transfer is complete so
I can track concurrent downloads by IP.
The stream approach you mention at the start will do it. Just be aware that
your systems socket buffer will also come into play here too, initially
undertow will be able to write a bit without blocking until the buffer
fills up.
Stuart
Thanks for your help!
-Nate
_______________________________________________
undertow-dev mailing list
undertow-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/undertow-dev