[
https://issues.redhat.com/browse/JGRP-2504?page=com.atlassian.jira.plugin...
]
Andrew Skalski commented on JGRP-2504:
--------------------------------------
net.core.rmem_max is the limit on how large of a SO_RCVBUF an application may request. If
the application attempts to request a larger buffer, it is clamped to the value specified
in net.core.rmem_max.
Buffer size and window size are related but distinct. The buffer size is how much kernel
memory is allocated to the socket; the receive window (a number advertised in the TCP
header, indicating how much more data it is willing to receive at a given moment in time)
depends on the buffer size and other factors.
Setting SO_SNDBUF on a listening socket works without issue in C. Here is strace output
of the "iperf3" utility setting up its listening socket with a requested buffer
size of 123456. The setsockopt calls all return 0 (success):
{code:java}
socket(AF_INET6, SOCK_STREAM, IPPROTO_IP) = 3
setsockopt(3, SOL_SOCKET, SO_RCVBUF, [123456], 4) = 0
setsockopt(3, SOL_SOCKET, SO_SNDBUF, [123456], 4) = 0
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
bind(3, {sa_family=AF_INET6, sin6_port=htons(7800), inet_pton(AF_INET6, "::",
&sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
listen(3, 5) = 0
{code}
But based on what I've been able to piece together, it's completely OK – at least
with Linux – to wait until after accepting a connection to configure SO_SNDBUF.
Considering that Java's ServerSocket lacks a setSendBufferSize method, I think
it's safe to assume that it's OK on other platforms as well.
Poor throughput over high latency TCP connection when recv_buf_size
is configured
---------------------------------------------------------------------------------
Key: JGRP-2504
URL:
https://issues.redhat.com/browse/JGRP-2504
Project: JGroups
Issue Type: Bug
Affects Versions: 5.0.0.Final
Reporter: Andrew Skalski
Assignee: Bela Ban
Priority: Minor
Fix For: 5.1
Attachments: SpeedTest.java, bla5.java, bla6.java, bla7.java, delay-ip.sh,
rcvbuf.png
I recently finished troubleshooting a unidirectional throughput bottleneck involving a
JGroups application (Infinispan) communicating over a high-latency (~45 milliseconds) TCP
connection.
The root cause was JGroups improperly configuring the receive/send buffers on the
listening socket. According to the tcp(7) man page:
{code:java}
On individual connections, the socket buffer size must be set prior to
the listen(2) or connect(2) calls in order to have it take effect.
{code}
However, JGroups does not set the buffer size on the listening side until after
accept().
The result is poor throughput when sending data from client (connecting side) to server
(listening side.) Because the issue is a too-small TCP receive window, throughput is
ultimately latency-bound.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)