[infinispan-issues] [JBoss JIRA] (ISPN-11347) Change default TOS for UDP

[Dan Berindei (Jira)]

Dan Berindei created ISPN-11347:
-----------------------------------

             Summary: Change default TOS for UDP
                 Key: ISPN-11347
                 URL: https://issues.redhat.com/browse/ISPN-11347
             Project: Infinispan
          Issue Type: Enhancement
          Components: Configuration, Core
    Affects Versions: 11.0.0.Alpha1, 10.1.2.Final, 9.4.18.Final
            Reporter: Dan Berindei
            Assignee: Dan Berindei
             Fix For: 11.0.0.Beta1


The Linux kernel has some fairly sophisticated queueing disciplines like {{fq_codel}}, but the default one is {{pfifo_fast}}, and that's what our edg-perfXX machines use.

{quote}
pfifo_fast is like three tc-pfifo(8) queues side by side,
       where packets can be enqueued in any of the three bands based on
       their Type of Service bits or assigned priority.

       Not all three bands are dequeued simultaneously - as long as lower
       bands have traffic, higher bands are never dequeued. This can be used
       to prioritize interactive traffic or penalize 'lowest cost' traffic.
??[http://man7.org/linux/man-pages/man8/tc-pfifo_fast.8.html#ALGORITHM]??
{quote}

These are some examples of band mappings:

{quote}
{noformat}
TOS     Bits  Means                    Linux Priority    Band
------------------------------------------------------------
0x0     0     Normal Service           0 Best Effort     1
0x8     4     Maximize Throughput      2 Bulk            2
0x10    8     Minimize Delay           6 Interactive     0
0x18    12    mt+md                    4 Int. Bulk       1
{noformat}
??[http://man7.org/linux/man-pages/man8/tc-prio.8.html#QDISC_PARAMETERS]??
{quote}

By default {{UDP.tos="8"}}, which makes all UDP traffic go into band 2, lowest priority (bulk). HotRod clients and servers don't set a TOS on their sockets, so the client-server packets go into band 1, middle priority (best effort).

{{FD_ALL}} and {{FD_ALL2}} heartbeats are also {{UDP}} traffic, and in some read-only client-server test there is enough client-server traffic in band 1 to delay the FD_ALL heartbeats for more than 10 seconds.

We could either set the default TOS to {{0}} (best effort), or {{0x18}} (maximize throughput + minimize delay), the result is the same: band 1. 

We could also try to set the TOS to {{0x10}} to get them into band 0, but while it would work in Infinispan-only benchmarks, it would probably be unfair to other communications on the same machine.



--
This message was sent by Atlassian Jira
(v7.13.8#713008)