[jboss-cvs] JBossAS SVN: r83866 - projects/docs/community/5/Clustering_Guide/en-US.
jboss-cvs-commits at lists.jboss.org
jboss-cvs-commits at lists.jboss.org
Wed Feb 4 15:07:41 EST 2009
Author: bstansberry at jboss.com
Date: 2009-02-04 15:07:41 -0500 (Wed, 04 Feb 2009)
New Revision: 83866
Modified:
projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_JGroups.xml
Log:
[JBAS-6351] Rationalize TOC
Modified: projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_JGroups.xml
===================================================================
--- projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_JGroups.xml 2009-02-04 19:55:26 UTC (rev 83865)
+++ projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_JGroups.xml 2009-02-04 20:07:41 UTC (rev 83866)
@@ -10,10 +10,10 @@
deploying an application that has special network or performance requirements.</para>
<section id="jgroups-configuration">
- <title>JGroups Configuration</title>
+ <title>Configuring a JGroups Channel's Protocol Stack</title>
<para>The JGroups framework provides services to enable peer-to-peer communications between nodes in a
- cluster. It is built on top a stack of network communication protocols that provide transport,
- discovery, reliability and failure detection, and cluster membership management services. <xref linkend="JGroupsStack.fig"/> shows the protocol stack in JGroups.</para>
+ cluster. It is built on top a stack of network communication protocols that provide transport,
+ discovery, reliability and failure detection, and cluster membership management services. <xref linkend="JGroupsStack.fig"/> shows the protocol stack in JGroups.</para>
<figure id="JGroupsStack.fig">
<title>Protocol stack in JGroups</title>
<mediaobject>
@@ -87,7 +87,6 @@
<para>
All the JGroups configuration data is contained in the <Config> element under the JGroups config MBean attribute. This information is used to configure a JGroups Channel; the Channel is conceptually similar to a socket, and manages communication between peers in a cluster. Each element inside the <Config> element defines a particular JGroups Protocol; each Protocol performs one function, and the combination of those functions is what defines the characteristics of the overall Channel. In the next several sections, we will dig into the commonly used protocols and their options and explain exactly what they mean.
</para>
-</section>
<section><title>Common Configuration Properties</title>
<para>The following common properties are exposed by all of the JGroups protocols discussed below:
@@ -796,23 +795,15 @@
</itemizedlist>
</section>
- </section>
-
-
-
- <section id="jgroups-other">
- <title>Other Configuration Options</title>
- <para>In addition to the protocol stacks, you can also configure JGroups network services in the <literal>Config</literal> element.</para>
-
-
- <section id="jgroups-other-gms">
- <title>Group Membership</title>
- <para>The group membership service in the JGroups stack maintains a list of active nodes. It handles
- the requests to join and leave the cluster. It also handles the SUSPECT messages sent by failure
- detection protocols. All nodes in the cluster, as well as the load balancer and client side
- interceptors, are notified if the group membership changes. The group membership service is
- configured in the <literal>pbcast.GMS</literal> sub-element under the JGroups
- <literal>Config</literal> element. Here is an example configuration.</para>
+ </section><section id="jgroups-other-gms">
+ <title>Group Membership (GMS)</title>
+ <para>The group membership service (GMS) protocol in the JGroups stack
+ maintains a list of active nodes. It handles the requests to join and
+ leave the cluster. It also handles the SUSPECT messages sent by failure
+ detection protocols. All nodes in the cluster, as well as the load balancer and client side
+ interceptors, are notified if the group membership changes. The group membership service is
+ configured in the <literal>pbcast.GMS</literal> sub-element under the JGroups
+ <literal>Config</literal> element. Here is an example configuration.</para>
<programlisting>
<pbcast.GMS print_local_addr="true"
join_timeout="3000"
@@ -846,27 +837,28 @@
<para><emphasis role="bold">disable_initial_coord</emphasis> specifies whether to prevent
this node as the cluster coordinator.</para>
</listitem>
- <listitem>
- <para><emphasis role="bold">view_bundling</emphasis> specifies whether multiple JOIN or LEAVE request arriving at the same time are bundled and handled together at the same time, only sending out 1 new view / bundle. This is is more efficient than handling each request separately.
- </para>
+ <listitem>
+ <para><emphasis role="bold">view_bundling</emphasis> specifies whether multiple JOIN or LEAVE request arriving at the same time are bundled and handled together at the same time, only sending out 1 new view / bundle. This is is more efficient than handling each request separately.
+ </para>
</listitem>
-
+
</itemizedlist>
</section>
-
-
+
+
<section id="jgroups-other-fc">
- <title>Flow Control</title>
- <para>The flow control service tries to adapt the sending data rate and the receiving data among
- nodes. If a sender node is too fast, it might overwhelm the receiver node and result in dropped
- packets that have to be retransmitted. In JGroups, the flow control is implemented via a
- credit-based system. The sender and receiver nodes have the same number of credits (bytes) to
- start with. The sender subtracts credits by the number of bytes in messages it sends. The
- receiver accumulates credits for the bytes in the messages it receives. When the sender's credit
- drops to a threshold, the receivers sends some credit to the sender. If the sender's credit is
- used up, the sender blocks until it receives credits from the receiver. The flow control service
- is configured in the <literal>FC</literal> sub-element under the JGroups
- <literal>Config</literal> element. Here is an example configuration.</para>
+ <title>Flow Control (FC)</title>
+ <para>The flow control (FC) protocol tries to adapt the data sending rate
+ to the data receipt rate among nodes. If a sender node is too fast, it
+ might overwhelm the receiver node and result in dropped packets that
+ have to be retransmitted. In JGroups, the flow control is implemented via a
+ credit-based system. The sender and receiver nodes have the same number of credits (bytes) to
+ start with. The sender subtracts credits by the number of bytes in messages it sends. The
+ receiver accumulates credits for the bytes in the messages it receives. When the sender's credit
+ drops to a threshold, the receivers sends some credit to the sender. If the sender's credit is
+ used up, the sender blocks until it receives credits from the receiver. The flow control protocol
+ is configured in the <literal>FC</literal> sub-element under the JGroups
+ <literal>Config</literal> element. Here is an example configuration.</para>
<programlisting>
<FC max_credits="1000000"
@@ -892,58 +884,58 @@
</itemizedlist>
<note><title>Note</title>
- <para>
- Applications that use synchronous group RPC calls primarily do not require FC protocol in their JGroups protocol stack because synchronous communication, where the hread that makes the call blocks waiting for responses from all the members of the group, already slows overall rate of calls. Even though TCP provides flow control by itself, FC is still required in TCP based JGroups stacks because of group communication, where we essentially have to send group messages at the highest speed the slowest receiver can keep up with. TCP flow control only takes into account individual node communications and has not a notion of who's the slowest in the group, which is why FC is required.
- </para>
+ <para>
+ Applications that use synchronous group RPC calls primarily do not require FC protocol in their JGroups protocol stack because synchronous communication, where the hread that makes the call blocks waiting for responses from all the members of the group, already slows overall rate of calls. Even though TCP provides flow control by itself, FC is still required in TCP based JGroups stacks because of group communication, where we essentially have to send group messages at the highest speed the slowest receiver can keep up with. TCP flow control only takes into account individual node communications and has not a notion of who's the slowest in the group, which is why FC is required.
+ </para>
</note>
-
+
<section>
- <title>Why is FC needed on top of TCP ? TCP has its own flow control !</title>
- <para>
-
- The reason is group communication, where we essentially have to send group messages at the highest speed the slowest receiver can keep up with. Let's say we have a cluster {A,B,C,D}. D is slow (maybe overloaded), the rest is fast. When A sends a group message, it establishes TCP connections A-A (conceptually), A-B, A-C and A-D (if they don't yet exist). So let's say A sends 100 million messages to the cluster. Because TCP's flow control only applies to A-B, A-C and A-D, but not to A-{B,C,D}, where {B,C,D} is the group, it is possible that A, B and C receive the 100M, but D only received 1M messages. (BTW: this is also the reason why we need NAKACK, although TCP does its own retransmission).
- </para>
- <para>
- Now JGroups has to buffer all messages in memory for the case when the original sender S dies and a node asks for retransmission of a message of S. Because all members buffer all messages they received, they need to purge stable messages (= messages seen by everyone) every now and then. This is done by the STABLE protocol, which can be configured to run the stability protocol round time based (e.g. every 50s) or size based (whenever 400K data has been received).
- </para>
- <para>
- In the above case, the slow node D will prevent the group from purging messages above 1M, so every member will buffer 99M messages ! This in most cases leads to OOM exceptions. Note that - although the sliding window protocol in TCP will cause writes to block if the window is full - we assume in the above case that this is still much faster for A-B and A-C than for A-D.
- </para>
- <para>
- So, in summary, we need to send messages at a rate the slowest receiver (D) can handle.
- </para>
+ <title>Why is FC needed on top of TCP ? TCP has its own flow control !</title>
+ <para>
+
+ The reason is group communication, where we essentially have to send group messages at the highest speed the slowest receiver can keep up with. Let's say we have a cluster {A,B,C,D}. D is slow (maybe overloaded), the rest is fast. When A sends a group message, it establishes TCP connections A-A (conceptually), A-B, A-C and A-D (if they don't yet exist). So let's say A sends 100 million messages to the cluster. Because TCP's flow control only applies to A-B, A-C and A-D, but not to A-{B,C,D}, where {B,C,D} is the group, it is possible that A, B and C receive the 100M, but D only received 1M messages. (BTW: this is also the reason why we need NAKACK, although TCP does its own retransmission).
+ </para>
+ <para>
+ Now JGroups has to buffer all messages in memory for the case when the original sender S dies and a node asks for retransmission of a message of S. Because all members buffer all messages they received, they need to purge stable messages (= messages seen by everyone) every now and then. This is done by the STABLE protocol, which can be configured to run the stability protocol round time based (e.g. every 50s) or size based (whenever 400K data has been received).
+ </para>
+ <para>
+ In the above case, the slow node D will prevent the group from purging messages above 1M, so every member will buffer 99M messages ! This in most cases leads to OOM exceptions. Note that - although the sliding window protocol in TCP will cause writes to block if the window is full - we assume in the above case that this is still much faster for A-B and A-C than for A-D.
+ </para>
+ <para>
+ So, in summary, we need to send messages at a rate the slowest receiver (D) can handle.
+ </para>
</section>
<section>
- <title>So do I always need FC?</title>
- <para>
- This depends on how the application uses the JGroups channel. Referring to the example above, if there was something about the application that would naturally cause A to slow down its rate of sending because D wasn't keeping up, then FC would not be needed.
- </para>
- <para>
- A good example of such an application is one that makes synchronous group RPC calls (typically using a JGroups RpcDispatcher.) By synchronous, we mean the thread that makes the call blocks waiting for responses from all the members of the group. In that kind of application, the threads on A that are making calls would block waiting for responses from D, thus naturally slowing the overall rate of calls.
- </para>
- <para>
- A JBoss Cache cluster configured for REPL_SYNC is a good example of an application that makes synchronous group RPC calls. If a channel is only used for a cache configured for REPL_SYNC, we recommend you remove FC from its protocol stack.
- </para>
- <para>
- And, of course, if your cluster only consists of two nodes, including FC in a TCP-based protocol stack is unnecessary. There is no group beyond the single peer-to-peer relationship, and TCP's internal flow control will handle that just fine.
- </para>
- <para>
- Another case where FC may not be needed is for a channel used by a JBoss Cache configured for buddy replication and a single buddy. Such a channel will in many respects act like a two node cluster, where messages are only exchanged with one other node, the buddy. (There may be other messages related to data gravitation that go to all members, but in a properly engineered buddy replication use case these should be infrequent. But if you remove FC be sure to load test your application.)
- </para>
+ <title>So do I always need FC?</title>
+ <para>
+ This depends on how the application uses the JGroups channel. Referring to the example above, if there was something about the application that would naturally cause A to slow down its rate of sending because D wasn't keeping up, then FC would not be needed.
+ </para>
+ <para>
+ A good example of such an application is one that makes synchronous group RPC calls (typically using a JGroups RpcDispatcher.) By synchronous, we mean the thread that makes the call blocks waiting for responses from all the members of the group. In that kind of application, the threads on A that are making calls would block waiting for responses from D, thus naturally slowing the overall rate of calls.
+ </para>
+ <para>
+ A JBoss Cache cluster configured for REPL_SYNC is a good example of an application that makes synchronous group RPC calls. If a channel is only used for a cache configured for REPL_SYNC, we recommend you remove FC from its protocol stack.
+ </para>
+ <para>
+ And, of course, if your cluster only consists of two nodes, including FC in a TCP-based protocol stack is unnecessary. There is no group beyond the single peer-to-peer relationship, and TCP's internal flow control will handle that just fine.
+ </para>
+ <para>
+ Another case where FC may not be needed is for a channel used by a JBoss Cache configured for buddy replication and a single buddy. Such a channel will in many respects act like a two node cluster, where messages are only exchanged with one other node, the buddy. (There may be other messages related to data gravitation that go to all members, but in a properly engineered buddy replication use case these should be infrequent. But if you remove FC be sure to load test your application.)
+ </para>
</section>
-
+
-
+
</section>
-
-
-<section><title>Fragmentation</title>
- <para>
- This protocol fragments messages larger than certain size. Unfragments at the receiver's side. It works for both unicast and multicast messages. It is configured in the FRAG2 sub-element under the JGroups Config element. Here is an example configuration.
- </para>
-<programlisting><![CDATA[
- <FRAG2 frag_size="60000" down_thread="false" up_thread="false"/>]]>
+
+
+<section><title>Fragmentation (FRAG2)</title>
+ <para>
+ This protocol fragments messages larger than certain size. Unfragments at the receiver's side. It works for both unicast and multicast messages. It is configured in the FRAG2 sub-element under the JGroups Config element. Here is an example configuration.
+ </para>
+<programlisting><![CDATA[
+ <FRAG2 frag_size="60000" down_thread="false" up_thread="false"/>]]>
</programlisting>
<para>
@@ -951,18 +943,18 @@
</para>
<itemizedlist>
- <listitem><para><emphasis role="bold">frag_size</emphasis> specifies the max frag size in bytes. Messages larger than that are fragmented.</para></listitem>
+ <listitem><para><emphasis role="bold">frag_size</emphasis> specifies the max frag size in bytes. Messages larger than that are fragmented.</para></listitem>
</itemizedlist>
<note><title>Note</title>
- <para>
- TCP protocol already provides fragmentation but a fragmentation JGroups protocol is still needed if FC is used. The reason for this is that if you send a message larger than FC.max_bytes, FC protocol would block. So, frag_size within FRAG2 needs to be set to always be less than FC.max_bytes.
- </para>
+ <para>
+ TCP protocol already provides fragmentation but a fragmentation JGroups protocol is still needed if FC is used. The reason for this is that if you send a message larger than FC.max_bytes, FC protocol would block. So, frag_size within FRAG2 needs to be set to always be less than FC.max_bytes.
+ </para>
</note>
</section>
-
+
<section id="jgroups-other-st">
<title>State Transfer</title>
<para>The state transfer service transfers the state from an existing node (i.e., the cluster
@@ -974,11 +966,11 @@
</programlisting>
</section>
- <section id="jgroups-other-gc">
- <title>Distributed Garbage Collection</title>
+ <section id="jgroups-other-gc">
+ <title>Distributed Garbage Collection (STABLE)</title>
<para>
- In a JGroups cluster, all nodes have to store all messages received for potential retransmission in case of a failure. However, if we store all messages forever, we will run out of memory. So, the distributed garbage collection service in JGroups periodically purges messages that have seen by all nodes from the memory in each node. The distributed garbage collection service is configured in the <literal>pbcast.STABLE</literal> sub-element under the JGroups <literal>Config</literal> element. Here is an example configuration.
- </para>
+ In a JGroups cluster, all nodes have to store all messages received for potential retransmission in case of a failure. However, if we store all messages forever, we will run out of memory. So, the distributed garbage collection service in JGroups periodically purges messages that have seen by all nodes from the memory in each node. The distributed garbage collection service is configured in the <literal>pbcast.STABLE</literal> sub-element under the JGroups <literal>Config</literal> element. Here is an example configuration.
+ </para>
<programlisting>
<pbcast.STABLE stability_delay="1000"
@@ -1000,7 +992,7 @@
<literal>0</literal> disables this service.</para>
</listitem>
<listitem>
- <para><emphasis role="bold">stability_delay</emphasis> specifies delay before we send STABILITY msg (give others a change to send first). If used together with max_bytes, this attribute should be set to a small number.</para>
+ <para><emphasis role="bold">stability_delay</emphasis> specifies delay before we send STABILITY msg (give others a change to send first). If used together with max_bytes, this attribute should be set to a small number.</para>
</listitem>
</itemizedlist>
<note>
@@ -1009,18 +1001,18 @@
</note>
</section>
<section id="jgroups-other-merge">
- <title>Merging</title>
+ <title>Merging (MERGE2)</title>
<para>
- When a network error occurs, the cluster might be partitioned into several different partitions. JGroups has a MERGE service that allows the coordinators in partitions to communicate with each other and form a single cluster back again. The flow control service is configured in the <literal>MERGE2</literal> sub-element under the JGroups <literal>Config</literal> element. Here is an example configuration.
- </para>
+ When a network error occurs, the cluster might be partitioned into several different partitions. JGroups has a MERGE service that allows the coordinators in partitions to communicate with each other and form a single cluster back again. The flow control service is configured in the <literal>MERGE2</literal> sub-element under the JGroups <literal>Config</literal> element. Here is an example configuration.
+ </para>
<programlisting>
<MERGE2 max_interval="10000"
min_interval="2000"
down_thread="false" up_thread="false"/>
</programlisting>
-
-
+
+
<para>The configurable attributes in the <literal>FC</literal> element are as follows.</para>
<itemizedlist>
<listitem>
@@ -1035,11 +1027,17 @@
<para>JGroups chooses a random value between <literal>min_interval</literal> and
<literal>max_interval</literal> to send out the MERGE message.</para>
<note>
- <para>
- The cluster states are not merged in a merger. This has to be done by the application. If <literal>MERGE2</literal> is used in conjunction with TCPPING, the <literal>initial_hosts</literal> attribute must contain all the nodes that could potentially be merged back, in order for the merge process to work properly. Otherwise, the merge process would not merge all the nodes even though shunning is disabled. Alternatively use MPING, which is commonly used with TCP to provide multicast member discovery capabilities, instead of TCPPING to avoid having to specify all the nodes.
- </para>
+ <para>
+ The cluster states are not merged in a merger. This has to be done by the application. If <literal>MERGE2</literal> is used in conjunction with TCPPING, the <literal>initial_hosts</literal> attribute must contain all the nodes that could potentially be merged back, in order for the merge process to work properly. Otherwise, the merge process would not merge all the nodes even though shunning is disabled. Alternatively use MPING, which is commonly used with TCP to provide multicast member discovery capabilities, instead of TCPPING to avoid having to specify all the nodes.
+ </para>
</note>
</section>
+
+
+</section>
+
+ <section id="jgroups-other">
+ <title>Other Configuration Issues</title>
<section><title>Binding JGroups Channels to a particular interface</title>
<para>
@@ -1120,7 +1118,6 @@
<para>
For example, say we have a production cluster of 3 machines, each of which has an HAPartition deployed. On the same network there is also a QA cluster of 3 machines, which also has an HAPartition deployed. The HAPartition group name, multicast address, and multicast port for the production machines must be different from those used on the QA machines.
</para>
- </section>
<section><title>Changing the Group Name</title>
@@ -1165,10 +1162,12 @@
It should be sufficient to just change the address, but there is a problem on several operating systems whereby packets addressed to a particular multicast port are delivered to all listeners on that port, regardless of the multicast address they are listening on. So the recommendation is to change both the address and the port.
</para>
</section>
+ </section>
+ </section>
-
<section><title>JGroups Troubleshooting</title>
- <para><emphasis>Nodes do not form a cluster</emphasis></para>
+ <section>
+ <title>Nodes do not form a cluster</title>
<para>
Make sure your machine is set up correctly for IP multicast. There are 2 test programs that can be used to detect this: McastReceiverTest and McastSenderTest. Go to the <literal>$JBOSS_HOME/server/all/lib</literal> directory and start McastReceiverTest, for example:
More information about the jboss-cvs-commits
mailing list