1:28 p.m.
[
https://jira.jboss.org/browse/JGRP-1154?page=com.atlassian.jira.plugin.sy...
]
Bela Ban commented on JGRP-1154:
--------------------------------
[Email dated Sept 10 2010 to jg-dev]
FYI,
I've added 2 new flow control protocols: MFC (Multicast Flow Control)
and UFC (Unicast Flow Control). They're an implementation of [1], which
is about splitting the FC functionality into 2 separate protocols.
The advantages are that this simplifies the code, the code becomes more
maintainable, that we can enable unicast or multicast flow control
individually, that I could remove some coarse grained locking, and that
I created helper classes, e.g. to manage sender or receiver credits,
which can be unit tested individually, apart from the use in the protocol.
The downside is that we now have MFC and UFC instead of just FC.
My goal for performance was that MFC and UFC should be about the same as
FC, but not slower.
This morning, I ran a quick perf test [2] on a 4 node cluster, with 4
nodes (all senders) sending 1 million messages (of varying size) each.
The numbers are MiBs/sec/node.
The numbers for 4 nodes are:
Size FC MFC/UFC
1K 135 135
2K 145 145
4K 145 145
8K 130 131
The numbers for 8 nodes are (2 processes per physical box):
Size FC MFC/UFC
1K 72 73
2K 83 85
4K 87 93
8K 107 113
As can be seen, performance is about the same, or even slightly better
with MFC/UFC when running 2 perf JVMs / physical box. However, the
second test on 8 nodes is to be taken with a grain of salt, as the 2
JBMs/box are competing for CPU and the network, and therefore we don't
saturate the network (bandwidth)...
To use the new folw control protocols, replace
<FC max_credits="2M" min_threshold="0.6" />
with
<UFC max_credits="2M" min_threshold="0.6" />
<MFC max_credits="2M" min_threshold="0.6" />
I will change the configs shipped with JGroups after some more testing
and updating the documentation.
[1] https://jira.jboss.org/browse/JGRP-1154
[2] http://community.jboss.org/docs/DOC-11594
--
Bela Ban
Lead JGroups / Clustering Team
JBoss
FC: separate implementations for multicast and unicast messages
---------------------------------------------------------------
Key: JGRP-1154
URL: https://jira.jboss.org/browse/JGRP-1154
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.11
Currently, FC handles both multicast (=group) and unicast messages. This has a few
disadvantages:
- Complexity
- Unicast messages are not dependent on multicast messages (and vice versa), so why
should this be different for flow control ? E.g. why
should a multicast message block waiting for credits from X because a recent unicast
message was sent to X ?
- If we have separate flow control protocols for multicasts and unicasts, we can
selectively remove one or the other. This is currently
not possible
Design (see separate documents in ./doc/design):
FC_MCAST:
-----------------
(see whether some concepts can be borrowed from SFC)
Sender:
- Every sender has a credits AtomicLong variable
- When sending a message and the message length exceeds credits, the sender blocks
(until it gets credits from all current group members)
- Else credits is decremented by the message length and the message is sent
- When credits are received from all current group members, credits is reset to the
initial value and all blocked senders are unblocked
- When a new member joins, credits is also reset and all blocked senders are unblocked
Receiver:
- Maintains a hashmap of senders and credits (Map<Address,AtomicLong>)
- When a message is received from S, the credits for S are decremented by the message
length
- If the remaining length is less than the threshold, new credits are sent to S and
S's credits are reset to their initial value
- When a new member joins, all credits (including the newly joined member) are reset to
the initial value
FC_UNICAST:
--------------------
Sender:
- A hashmap of addresses and credits is maintained: every receiver has associated
credits
- When a message is sent to P, if P's credits minus the message length is less than
0, the sender blocks until it gets new credits from the receiver
- Else P's credits are decremented by the message length and the message is sent
Receiver:
-Same as for FC_MCAST
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira