[
https://issues.redhat.com/browse/JGRP-2504?page=com.atlassian.jira.plugin...
]
Andrew Skalski commented on JGRP-2504:
--------------------------------------
I tested TCP and TCP_NIO2, and the throughput issue is now resolved. Thank you for
looking into this!
{quote}OK, so I read your comment. I understand socket options are inherited, but what
sense does SO_SNDBUF make here? This option cannot be set on a ServerSocket anyway
(perhaps this can be done in C?), only SO_RCVBUF, so does this mean the send-buffer cannot
be set on a socket returned by accept()?
{quote}
That part of the documentation didn't make sense to me either. Reading through the
Linux sources and verifying empirically, the SO_SNDBUF option is definitely being honored,
even when it is configured on the accepted socket.
{quote}Also:
{quote}If the application explicitly configures SO_RCVBUF, this automatic management is
disabled
{quote}
Does this mean, the value of SO_RCVBUF is the *max* value a receive window can have?
{quote}
Yes. SO_RCVBUF controls the buffer size, which in turn limits the receive window. (The
receive window grows/shrinks over time in response to quantity of data received,
congestion and packet loss, the rate at which the receiving application consumes the data,
etc.) To illustrate, I configured both sites with SO_RCVBUF=200000 and graphed the window
size over time during a send/receive test. !rcvbuf.png!
Poor throughput over high latency TCP connection when recv_buf_size
is configured
---------------------------------------------------------------------------------
Key: JGRP-2504
URL:
https://issues.redhat.com/browse/JGRP-2504
Project: JGroups
Issue Type: Bug
Affects Versions: 5.0.0.Final
Reporter: Andrew Skalski
Assignee: Bela Ban
Priority: Minor
Fix For: 5.1
Attachments: SpeedTest.java, bla5.java, bla6.java, bla7.java, delay-ip.sh,
rcvbuf.png
I recently finished troubleshooting a unidirectional throughput bottleneck involving a
JGroups application (Infinispan) communicating over a high-latency (~45 milliseconds) TCP
connection.
The root cause was JGroups improperly configuring the receive/send buffers on the
listening socket. According to the tcp(7) man page:
{code:java}
On individual connections, the socket buffer size must be set prior to
the listen(2) or connect(2) calls in order to have it take effect.
{code}
However, JGroups does not set the buffer size on the listening side until after
accept().
The result is poor throughput when sending data from client (connecting side) to server
(listening side.) Because the issue is a too-small TCP receive window, throughput is
ultimately latency-bound.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)