Poor throughput over high latency TCP connection when recv_buf_size is configured --------------------------------------------------------------------------------- Key: JGRP-2504 URL: https://issues.redhat.com/browse/JGRP-2504 Project: JGroups Issue Type: Bug Affects Versions: 5.0.0.Final Reporter: Andrew Skalski Assignee: Bela Ban Priority: Minor Fix For: 5.1 Attachments: SpeedTest.java, bla5.java, bla6.java, bla7.java, delay-ip.sh, rcvbuf.png I recently finished troubleshooting a unidirectional throughput bottleneck involving a JGroups application (Infinispan) communicating over a high-latency (~45 milliseconds) TCP connection. The root cause was JGroups improperly configuring the receive/send buffers on the listening socket. According to the tcp(7) man page: {code:java} On individual connections, the socket buffer size must be set prior to the listen(2) or connect(2) calls in order to have it take effect. {code} However, JGroups does not set the buffer size on the listening side until after accept(). The result is poor throughput when sending data from client (connecting side) to server (listening side.) Because the issue is a too-small TCP receive window, throughput is ultimately latency-bound.