Change default TOS for UDP -------------------------- Key: JGRP-2453 URL: https://issues.redhat.com/browse/JGRP-2453 Project: JGroups Issue Type: Enhancement Affects Versions: 4.1.9, 4.0.22 Reporter: Dan Berindei Assignee: Bela Ban Priority: Major Fix For: 5.0, 4.2.0 The Linux kernel has some fairly sophisticated queueing disciplines like {{fq_codel}}, but the default one is {{pfifo_fast}}. {quote} pfifo_fast is like three tc-pfifo(8) queues side by side, where packets can be enqueued in any of the three bands based on their Type of Service bits or assigned priority. Not all three bands are dequeued simultaneously - as long as lower bands have traffic, higher bands are never dequeued. This can be used to prioritize interactive traffic or penalize 'lowest cost' traffic. ??[http://man7.org/linux/man-pages/man8/tc-pfifo_fast.8.html#ALGORITHM]?? {quote} These are some examples of band mappings: {quote} {noformat} TOS Bits Means Linux Priority Band ------------------------------------------------------------ 0x0 0 Normal Service 0 Best Effort 1 0x8 4 Maximize Throughput 2 Bulk 2 0x10 8 Minimize Delay 6 Interactive 0 0x18 12 mt+md 4 Int. Bulk 1 {noformat} ??[http://man7.org/linux/man-pages/man8/tc-prio.8.html#QDISC_PARAMETERS]?? {quote} By default {{UDP.tos="8"}}, which makes all UDP traffic go into band 2, lowest priority (bulk). {{FD_ALL}} and {{FD_ALL2}} heartbeats are {{UDP}} traffic, and in some of our benchmarks there is enough client-server traffic in band 1 to delay the FD_ALL heartbeats for more than 10 seconds. We could either set the default TOS to {{0}} (best effort), or {{0x18}} (maximize throughput + minimize delay), the result is the same: band 1.