[jboss-jira] [JBoss JIRA] Commented: (JGRP-1154) FC: separate implementations for multicast and unicast messages

[Bela Ban (JIRA)]

    [ https://jira.jboss.org/browse/JGRP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12550502#action_12550502 ] 

Bela Ban commented on JGRP-1154:
--------------------------------

[Email dated Sept 10 2010 to jg-dev]

FYI,

I've added 2 new flow control protocols: MFC (Multicast Flow Control) 
and UFC (Unicast Flow Control). They're an implementation of [1], which 
is about splitting the FC functionality into 2 separate protocols.

The advantages are that this simplifies the code, the code becomes more 
maintainable, that we can enable unicast or multicast flow control 
individually, that I could remove some coarse grained locking, and that 
I created helper classes, e.g. to manage sender or receiver credits, 
which can be unit tested individually, apart from the use in the protocol.

The downside is that we now have MFC and UFC instead of just FC.

My goal for performance was that MFC and UFC should be about the same as 
FC, but not slower.

This morning, I ran a quick perf test [2] on a 4 node cluster, with 4 
nodes (all senders) sending 1 million messages (of varying size) each. 
The numbers are MiBs/sec/node.

The numbers for 4 nodes are:

Size   FC    MFC/UFC

1K    135      135
2K    145      145
4K    145      145
8K    130      131


The numbers for 8 nodes are (2 processes per physical box):

Size   FC    MFC/UFC

1K     72       73
2K     83       85
4K     87       93
8K    107      113


As can be seen, performance is about the same, or even slightly better 
with MFC/UFC when running 2 perf JVMs / physical box. However, the 
second test on 8 nodes is to be taken with a grain of salt, as the 2 
JBMs/box are competing for CPU and the network, and therefore we don't 
saturate the network (bandwidth)...

To use the new folw control protocols, replace

<FC max_credits="2M" min_threshold="0.6" />

with

<UFC max_credits="2M" min_threshold="0.6" />
<MFC max_credits="2M" min_threshold="0.6" />

I will change the configs shipped with JGroups after some more testing 
and updating the documentation.


[1] https://jira.jboss.org/browse/JGRP-1154

[2] http://community.jboss.org/docs/DOC-11594

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss


> FC: separate implementations for multicast and unicast messages
> ---------------------------------------------------------------
>
>                 Key: JGRP-1154
>                 URL: https://jira.jboss.org/browse/JGRP-1154
>             Project: JGroups
>          Issue Type: Feature Request
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>             Fix For: 2.11
>
>
> Currently, FC handles both multicast (=group) and unicast messages. This has a few disadvantages:
> - Complexity
> - Unicast messages are not dependent on multicast messages (and vice versa), so why should this be different for flow control ? E.g. why 
>    should a multicast message block waiting for credits from X because a recent unicast message was sent to X ?
> - If we have separate flow control protocols for multicasts and unicasts, we can selectively remove one or the other. This is currently
>    not possible
> Design (see separate documents in ./doc/design):
> FC_MCAST:
> -----------------
> (see whether some concepts can be borrowed from SFC)
> Sender:
> - Every sender has a credits AtomicLong variable
> - When sending a  message and the message length exceeds credits, the sender blocks (until it gets credits from all current group members)
> - Else credits is decremented by the message length and the message is sent
> - When credits are received from all current group members, credits is reset to the initial value and all blocked senders are unblocked
> - When a new member joins, credits is also reset and all blocked senders are unblocked
> Receiver:
> - Maintains a hashmap of senders and credits (Map<Address,AtomicLong>)
> - When a message is received from S, the credits for S are decremented by the message length
> - If the remaining length is less than the threshold, new credits are sent to S and S's credits are reset to their initial value
> - When a new member joins, all credits (including the newly joined member) are reset to the initial value
> FC_UNICAST:
> --------------------
> Sender:
> - A hashmap of addresses and credits is maintained: every receiver has associated credits
> - When a message is sent to P, if P's credits minus the message length is less than 0, the sender blocks until it gets new credits from the receiver
> - Else P's credits are decremented by the message length and the message is sent
> Receiver:
> -Same as for FC_MCAST

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira