3:08 p.m.
I have a request where I need to return a ~100MB file. I want to:
1) do this efficiently, and
2) limit concurrent downloads by IP (ie reject a request if concurrent
downloads by that IP == x).
I dispatch to a worker thread, then read from the file using
FileInputStream and write to exchange.getOutputStream(). However, I noticed
that this writes all the bytes immediately. Digging into why,
exchange.getOutputStream() uses UndertowOutputStream which appears to
buffer all the data given to it. Not only does this seem inefficient to
copy 100MB for every request, it means I don't know when the transfer
completes so I can't track concurrent downloads.
Next I tried getResponseChannel, eg:
StreamSinkChannel output = exchange.getResponseChannel();
try (var input = new FileInputStream(file)) {
Channels.transferBlocking(output, input.getChannel(), position, count);
}
Digging through the code it doesn't seem to be buffering, but it still
completes immediately. For example, I download the file at 10 bytes/sec:
curl --limit-rate 10 --output file https://example.com/my/file
How can transferBlocking finish right away when this slow transfer will
take ages? What is buffering the 100MB?
How can my worker thread write response data one buffer at a time, blocking
as needed for the client to receive the data, until the client has all the
data? This would allow me to know when the transfer is complete so I can
track concurrent downloads by IP.
Thanks for your help!
-Nate
4:17 p.m.
On Wed, 17 Jun 2020, 6:07 am Nate, <nate(a)n4te.com> wrote:
I have a request where I need to return a ~100MB file. I want to:
1) do this efficiently, and
2) limit concurrent downloads by IP (ie reject a request if concurrent
downloads by that IP == x).
I dispatch to a worker thread, then read from the file using
FileInputStream and write to exchange.getOutputStream(). However, I noticed
that this writes all the bytes immediately. Digging into why,
exchange.getOutputStream() uses UndertowOutputStream which appears to
buffer all the data given to it. Not only does this seem inefficient to
copy 100MB for every request, it means I don't know when the transfer
completes so I can't track concurrent downloads.
I don't understand exactly what you are doing here, can you share a code
snippet? The stream will always write everything that is given to it, but
it won't allocate 100M of buffers. If you have loaded the whole file into
memory then it is your code that has buffered the full 100M.
Next I tried getResponseChannel, eg:
StreamSinkChannel output = exchange.getResponseChannel();
try (var input = new FileInputStream(file)) {
Channels.transferBlocking(output, input.getChannel(), position,
count);
}
Digging through the code it doesn't seem to be buffering, but it still
completes immediately. For example, I download the file at 10 bytes/sec:
curl --limit-rate 10 --output file https://example.com/my/file
How can transferBlocking finish right away when this slow transfer will
take ages? What is buffering the 100MB?
It's a non blocking transfer. When The response can take more days then
more is read and transferred.
How can my worker thread write response data one buffer at a time,
blocking as needed for the client to receive the data, until the client has
all the data? This would allow me to know when the transfer is complete so
I can track concurrent downloads by IP.
The stream approach you mention at the start will do it. Just be aware that
your systems socket buffer will also come into play here too, initially
undertow will be able to write a bit without blocking until the buffer
fills up.
Stuart
Thanks for your help!
-Nate
_______________________________________________
undertow-dev mailing list
undertow-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/undertow-dev
5:19 p.m.
On Tue, Jun 16, 2020 at 11:18 PM Stuart Douglas <sdouglas(a)redhat.com> wrote:
On Wed, 17 Jun 2020, 6:07 am Nate, <nate(a)n4te.com> wrote:
> I dispatch to a worker thread, then read from the file using
> FileInputStream and write to exchange.getOutputStream(). However, I noticed
> that this writes all the bytes immediately. Digging into why,
> exchange.getOutputStream() uses UndertowOutputStream which appears to
> buffer all the data given to it. Not only does this seem inefficient to
> copy 100MB for every request, it means I don't know when the transfer
> completes so I can't track concurrent downloads.
>
I don't understand exactly what you are doing here, can you share a code
snippet? The stream will always write everything that is given to it, but
it won't allocate 100M of buffers. If you have loaded the whole file into
memory then it is your code that has buffered the full 100M.
I was reading one byte[] from the InputStream and writing it before reading
the next. I've now stepped through UndertowOutputStream and I see it copies
to a ByteBuffer, then writes that. So I apologize -- I misspoke, it doesn't
buffer the whole 100MB. The behavior I am seeing still has me spooked
though.
I have now switched to transferBlocking, like this:
var rateLimit = RateLimit.obtain(ip); // Throws if the IP has too many
concurrent downloads.
try (var input = new FileInputStream(file)) {
Channels.transferBlocking(exchange.getResponseChannel(),
input.getChannel(), position, length);
exchange.endExchange();
} finally {
rateLimit.release(ip);
}
My concern is that transferBlocking returns very quickly. For example, I
run curl 3 times simultaneously to download the file at 10 bytes/sec. For
each request, all the code above completes right away, even though the 3
curl instances are still downloading and won't finish for ~150 hours.
The stream approach you mention at the start will do it. Just be aware that
your systems socket buffer will also come into play here too,
initially
undertow will be able to write a bit without blocking until the buffer
fills up.
Could the system's socket buffer be responsible for the behavior I see? If
so, maybe it is working correctly and I see the request code complete
quickly because I don't have much load on the server?
It's a little unfortunate to read (and buffer at the system level) the
whole file when the request may be abandoned. In fact, the reason I want to
limit concurrent downloads by IP is that many people in China use
downloader software because of their government firewall. That software
hits my server every few seconds for hundreds or even thousands of times.
It would be nice to avoid doing too much work prematurely, though limiting
concurrent downloads by IP should also solve the problem.
Is there a way I can know when the client has received all the data, even
if it is buffered at the system level?
Thanks!
-Nate
6:02 p.m.
On Wed, 17 Jun 2020 at 08:17, Nate <nate(a)n4te.com> wrote:
On Tue, Jun 16, 2020 at 11:18 PM Stuart Douglas
<sdouglas(a)redhat.com>
wrote:
> On Wed, 17 Jun 2020, 6:07 am Nate, <nate(a)n4te.com> wrote:
>
>> I dispatch to a worker thread, then read from the file using
>> FileInputStream and write to exchange.getOutputStream(). However, I noticed
>> that this writes all the bytes immediately. Digging into why,
>> exchange.getOutputStream() uses UndertowOutputStream which appears to
>> buffer all the data given to it. Not only does this seem inefficient to
>> copy 100MB for every request, it means I don't know when the transfer
>> completes so I can't track concurrent downloads.
>>
>
> I don't understand exactly what you are doing here, can you share a code
> snippet? The stream will always write everything that is given to it,
> but it won't allocate 100M of buffers. If you have loaded the whole file
> into memory then it is your code that has buffered the full 100M.
>
I was reading one byte[] from the InputStream and writing it before
reading the next. I've now stepped through UndertowOutputStream and I see
it copies to a ByteBuffer, then writes that. So I apologize -- I misspoke,
it doesn't buffer the whole 100MB. The behavior I am seeing still has me
spooked though.
NIO needs data to be in direct buffers, if you attempt to write using a
byte[] array backed buffer internally it copies into its own buffer, so
there is no getting around this copy step. For writes of a small amount of
data this buffer step can significantly improve performance as it reduces
the amount of network calls.
I have now switched to transferBlocking, like this:
var rateLimit = RateLimit.obtain(ip); // Throws if the IP has too many
concurrent downloads.
try (var input = new FileInputStream(file)) {
Channels.transferBlocking(exchange.getResponseChannel(),
input.getChannel(), position, length);
exchange.endExchange();
} finally {
rateLimit.release(ip);
}
My concern is that transferBlocking returns very quickly. For example, I
run curl 3 times simultaneously to download the file at 10 bytes/sec. For
each request, all the code above completes right away, even though the 3
curl instances are still downloading and won't finish for ~150 hours.
That is weird. Nothing that ships with Undertow will buffer like that.
Either way though this will always transfer all the data in one go, you
need to manually read the data and then write it in chunks if you want to
manually limit it.
The stream approach you mention at the start will do it. Just be aware
> that your systems socket buffer will also come into play here too,
> initially undertow will be able to write a bit without blocking until the
> buffer fills up.
>
Could the system's socket buffer be responsible for the behavior I see? If
so, maybe it is working correctly and I see the request code complete
quickly because I don't have much load on the server?
I doubt it, unless you have a very weird config.
It's a little unfortunate to read (and buffer at the system level) the
whole file when the request may be abandoned. In fact, the reason I want to
limit concurrent downloads by IP is that many people in China use
downloader software because of their government firewall. That software
hits my server every few seconds for hundreds or even thousands of times.
It would be nice to avoid doing too much work prematurely, though limiting
concurrent downloads by IP should also solve the problem.
Is there a way I can know when the client has received all the data, even
if it is buffered at the system level?
If you call flush() on the OutputStream is won't return until the client
has recieved and acked the data AFAIK.
Stuart
Thanks!
-Nate
6:49 p.m.
On Wed, Jun 17, 2020 at 1:02 AM Stuart Douglas <sdouglas(a)redhat.com> wrote:
Is there a way I can know when the client has received all the data,
even
> if it is buffered at the system level?
>
If you call flush() on the OutputStream is won't return until the client
has recieved and acked the data AFAIK.
Alright, I tried this:
System.out.println("start");
PooledByteBuffer poolBuffer =
exchange.getConnection().getByteBufferPool().getArrayBackedPool().allocate();
byte[] buffer = poolBuffer.getBuffer().array(); // buffer.length is 16KB.
exchange.startBlocking();
OutputStream output = exchange.getOutputStream();
try (var input = new FileInputStream(file)) {
while (true) {
int count = input.read(buffer);
if (count == -1) break;
output.flush(); // Before write() to avoid flushing a closed
OutputStream.
output.write(buffer, 0, count);
}
}
exchange.endExchange();
System.out.println("end");
When I hit the URL, I see "start" and "end" printed immediately in
the
server log even though the curl download takes much, much longer at 10
bytes/sec.
That was on Debian. Next I tried the same on Windows 10 and there I see
only "start" while the download is in progress. That is true even if I
remove the flush.
So it seems it is something specific about Debian or its configuration. Any
idea what is happening, or other ideas on how I can know when the client
has really received the data?
Cheers,
-Nate
3:46 a.m.
It occurs to me on Debian I have Nginx in front of Undertow, so it's almost
certainly Nginx causing the behavior. I'll dig into that now. Thanks for
listening anyway! Undertow has been very nice to use so far, kudos!
On Wed, Jun 17, 2020 at 1:49 AM Nate <nate(a)n4te.com> wrote:
On Wed, Jun 17, 2020 at 1:02 AM Stuart Douglas
<sdouglas(a)redhat.com>
wrote:
> Is there a way I can know when the client has received all the data, even
>> if it is buffered at the system level?
>>
>
> If you call flush() on the OutputStream is won't return until the client
> has recieved and acked the data AFAIK.
>
Alright, I tried this:
System.out.println("start");
PooledByteBuffer poolBuffer =
exchange.getConnection().getByteBufferPool().getArrayBackedPool().allocate();
byte[] buffer = poolBuffer.getBuffer().array(); // buffer.length is 16KB.
exchange.startBlocking();
OutputStream output = exchange.getOutputStream();
try (var input = new FileInputStream(file)) {
while (true) {
int count = input.read(buffer);
if (count == -1) break;
output.flush(); // Before write() to avoid flushing a closed
OutputStream.
output.write(buffer, 0, count);
}
}
exchange.endExchange();
System.out.println("end");
When I hit the URL, I see "start" and "end" printed immediately in
the
server log even though the curl download takes much, much longer at 10
bytes/sec.
That was on Debian. Next I tried the same on Windows 10 and there I see
only "start" while the download is in progress. That is true even if I
remove the flush.
So it seems it is something specific about Debian or its configuration.
Any idea what is happening, or other ideas on how I can know when the
client has really received the data?
Cheers,
-Nate
1727
days inactive
1728
days old
5 comments
2 participants
participants (2)
-
Nate -
Stuart Douglas