[jboss-jira] [JBoss JIRA] (JGRP-1507) MFC2: multicast flow control

[ https://issues.jboss.org/browse/JGRP-1507?page=com.atlassian.jira.plugin.... ] Bela Ban updated JGRP-1507: --------------------------- Description: The current flow control protocol (MFC) is slowing things down when we have high traffic in multicast networks (UDP based stack), especially when ethernet flow control (802.3) is disabled. One reason is that the number of credits (similar to the window size in TCP) is static, and doesn't change. E.g. it doesn't increase when there's no loss and doesn't shrink when there is loss. (This will be addressed by a different JIRA later) Another reason is that sender credits are currently only used up when *sending* messages, but not when *receiving* messages. Thus, if we have 50MB of credits, a sender A can send 50MB until A blocks. However, when it receives 20MB of traffic from B while it sends the 50MB, it doesn't take the 20MBs into account. Therefore, a receiver S can be overwhelmed by receiving both A's and B's traffic (the switch might even start discarding messages before they reach S). The change from MFC to MFC2 is that a sender only maintains 1 'credits' counter (number of bytes): whenever it sends a message, it decrements credits. Whenever it receives traffic from someone else, it also decrements credits. When credits is 0, the sender blocks (asking for more credits). When credits drops below a certain threshold, it multicasts 1/N of the max credits (N = number of cluster members). (TBD: perhaps it should mcast a fixed number, independent of the cluster size). The advantage of this scheme is that a sender blocks sooner, and not when it's too late (congestion). Example: - credits = 100MB - Concurrently: A sends 50MB, B sends 30MB and C sends 60MB - At time T1, A has sent 20MB, and we've already received 10MB from B and 20 from C, credits is now 50MB - At time T2, A sent another 20MB, and received 20MB from C. credits is now 10MB, and A multicasts 35MB (rounded) - At T3, A receives another 10MB from B, credits=0 - At T4, A blocks trying to send its remaining 10MBs, as credits = 0 - At T5, A receives its 35MB of credits and sends its remaining 10MB, and also receives 20MB from C, and the remaining 10MB from B. credits is now -5 and A would block sending more credits - At T6, A multicasts 35MBs of credits - At T7, A receives 35MB of credits from B and C, credits= ca 65MB - At T8, A receives its own 35MBs of credits, credits=~100MB was: The current flow control protocol (MFC) is slowing things down when we have high traffic in multicast networks (UDP based stack), especially when ethernet flow control (802.3) is disabled. One reason is that the number of credits (similar to the window size in TCP) is static, and doesn't change. E.g. it doesn't increase when there's no loss and doesn't shrink when there is loss. (This will be addressed by a different JIRA later) Another reason is that sender credits are currently only used up when *sending* messages, but not when *receiving* messages. Thus, if we have 50MB of credits, a sender A can send 50MB until A blocks. However, when it receives 20MB of traffic from B while it sends the 50MB, it doesn't take the 20MBs into account. Therefore, a receiver S can be overwhelmed by receiving both A's and B's traffic (the switch might even start discarding messages before they reach S). The change from MFC to MFC2 is that a sender only maintains 1 'credits' counter (number of bytes): whenever it sends a message, it decrements credits. Whenever it receives traffic from someone else, it also decrements credits. When credits is 0, the sender blocks (asking for more credits). When credits drops below a certain threshold, it multicasts 1/N of the max credits (N = number of cluster members). [TBD: perhaps it should mcast a fixed number, independent of the cluster size]. The advantage of this scheme is that a sender blocks sooner, and not when it's too late (congestion). Example: - credits = 100MB - Concurrently: A sends 50MB, B sends 30MB and C sends 60MB - At time T1, A has sent 20MB, and we've already received 10MB from B and 20 from C, credits is now 50MB - At time T2, A sent another 20MB, received 20MB from C. credits is now 10MB, and A multicasts 35MB (rounded) - At T3, A receives another 10MB from C - At T4, A blocks trying to send its remaining 10MBs, as credits = 0 - At T5, A receives its 35MB of credits and sends its remaining 10MB, and also receives 20MB from C, and the remaining 10MB from B. credits is now -5 and A would block sending more credits - At T6, A multicasts 35MBs of credits - At T7, A receives 35MB of credits from B and C, credits= ca 65MB - At T8, A receives its own 35MBs of credits, credits=~100MB -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira