[jboss-cvs] JBossAS SVN: r92816 - in projects/docs/community/5/Clustering_Guide/en-US: images and 1 other directory.
jboss-cvs-commits at lists.jboss.org
jboss-cvs-commits at lists.jboss.org
Tue Aug 25 19:04:52 EDT 2009
Author: bstansberry at jboss.com
Date: 2009-08-25 19:04:52 -0400 (Tue, 25 Aug 2009)
New Revision: 92816
Added:
projects/docs/community/5/Clustering_Guide/en-US/images/clustering-SharedTransport.png
Modified:
projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_Building_Blocks.xml
Log:
[JBAS-6358] Fill out JGroups section; update HAPartition
Modified: projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_Building_Blocks.xml
===================================================================
--- projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_Building_Blocks.xml 2009-08-25 21:11:49 UTC (rev 92815)
+++ projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_Building_Blocks.xml 2009-08-25 23:04:52 UTC (rev 92816)
@@ -52,17 +52,206 @@
<section id="clustering-blocks-jgroups">
<title>Group Communication with JGroups</title>
- <para></para>
+ <para>JGroups provides the underlying group communication support for
+ JBoss AS clusters. Services deployed on JBoss AS which need group
+ communication with their peers will obtain a JGroups <literal>Channel</literal>
+ and use it to communicate. The <literal>Channel</literal> handles such
+ tasks as managing which nodes are members of the group, detecting node failures,
+ ensuring lossless, first-in-first-out delivery of messages to all group members,
+ and providing flow control to ensure fast message senders cannot overwhelm
+ slow message receivers.
+ </para>
+
+ <para>The characteristics of a JGroups <literal>Channel</literal> are determined
+ by the set of <emphasis>protocols</emphasis> that compose it. Each protocol
+ handles a single aspect of the overall group communication task; for example
+ the <literal>UDP</literal> protocol handles the details of sending and
+ receiving UDP datagrams. A <literal>Channel</literal> that uses the
+ <literal>UDP</literal> protocol is capable of communicating with UDP unicast and
+ multicast; alternatively one that uses the <literal>TCP</literal> protocol
+ uses TCP unicast for all messages. JGroups supports a wide variety of different
+ protocols (see <xref linkend="jgroups-configuration"/> for details), but
+ the AS ships with a default set of channel configurations that should meet
+ most needs.
+ </para>
+
+ <para>By default, UDP multicast is used by all JGroups channels used by
+ the AS (except for one TCP-based channel used by JBoss Messaging).</para>
+ <section id="clustering-blocks-jgroups-channelfactory">
+ <title>The Channel Factory Service</title>
+ <para>A significant difference in JBoss AS 5 versus previous releases
+ is that JGroups Channels needed by clustering services (e.g. a channel
+ used by a distributed HttpSession cache) are no longer configured in
+ detail as part of the consuming service's configuration, and are no longer
+ directly instantiated by the consuming service. Instead, a new
+ <literal>ChannelFactory</literal> service is used as a registry for named
+ channel configurations and as a factory for <literal>Channel</literal> instances.
+ A service that needs a channel requests the channel from the <literal>ChannelFactory</literal>,
+ passing in the name of the desired configuration.</para>
+
+ <para>The ChannelFactory service is deployed in the
+ <literal>server/all/deploy/cluster/jgroups-channelfactory.sar</literal>.
+ On startup the ChannelFactory service parses the
+ <literal>server/all/deploy/cluster/jgroups-channelfactory.sar/META-INF/jgroups-channelfactory-stacks.xml</literal>
+ file, which includes various standard JGroups configurations identified
+ by name (e.g "udp" or "tcp"). Services needing a channel access the channel
+ factory and ask for a channel with a particular named configuration.
+ </para>
+
+ <section id="clustering-blocks-jgroups-stdconfigs">
+ <title>Standard Protocol Stack Configurations</title>
+ <para>The standard protocol stack configurations that ship with AS 5
+ are described below. Note that not all of these are actually used;
+ many are included as a convenience to users who may wish to alter the
+ default server configuration. The configurations actually used in a
+ stock AS 5 <emphasis role="bold">all</emphasis> config are <literal>udp</literal>,
+ <literal>jbm-control</literal> and <literal>jbm-data</literal>, with
+ all clustering services other than JBoss Messaging using <literal>udp</literal>.
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role="bold">udp</emphasis></para>
+ <para>UDP multicast based stack meant to be shared between different channels.
+ Message bundling is disabled, as it can add latency to synchronous
+ group RPCs. Services that only make asynchronous RPCs (e.g. JBoss
+ Cache configured for REPL_ASYNC) and do so in high volume may
+ be able to improve performance by configuring their cache to use
+ the <literal>udp-async</literal> stack below. Services that only
+ make synchronous RPCs (e.g. JBoss Cache configured for REPL_SYNC
+ or INVALIDATION_SYNC) may be able to improve performance by using
+ the <literal>udp-sync</literal> stack below, which does not
+ include flow control.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="bold">udp-async</emphasis></para>
+ <para>Same as the default <literal>udp</literal> stack above,
+ except message bundling is enabled in the transport protocol
+ (<literal>enable_bundling=true</literal>). Useful for services
+ that make high-volume asynchronous RPCs (e.g. high volume JBoss Cache
+ instances configured for REPL_ASYNC) where message bundling may
+ improve performance.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="bold">udp-sync</emphasis></para>
+ <para>UDP multicast based stack, without flow control and without
+ message bundling. This can be used instead of <literal>udp</literal> if
+ (1) synchronous calls are used and (2) the message volume (rate and size)
+ is not that large. Don't use this configuration if you send
+ messages at a high sustained rate, or you might run out of memory.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="bold">tcp</emphasis></para>
+ <para>TCP based stack, with flow control and message bundling.
+ TCP stacks are usually used when IP multicasting cannot be used
+ in a network (e.g. routers discard multicast).</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="bold">tcp-sync</emphasis></para>
+ <para>TCP based stack, without flow control and without message
+ bundling. TCP stacks are usually used when IP multicasting
+ cannot be used in a network (e.g.routers discard multicast).
+ This configuration should be used instead of <literal>tcp</literal>
+ above when (1) synchronous calls are used and (2) the message
+ volume (rate and size) is not that large. Don't use this
+ configuration if you send messages at a high sustained rate,
+ or you might run out of memory.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="bold">jbm-control</emphasis></para>
+ <para>Stack optimized for the JBoss Messaging Control Channel.
+ By default uses the same UDP transport protocol config as is
+ used for the default 'udp' stack defined above. This allows the
+ JBoss Messaging Control Channel to use the same sockets, network
+ buffers and thread pools as are used by the other standard
+ JBoss AS clustered services (see
+ <xref linkend="clustering-blocks-jgroups-sharedtransport"/>.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="bold">jbm-data</emphasis></para>
+ <para>Stack optimized for the JBoss Messaging Data Channel. TCP-based</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>You can add a new stack configuration by adding a new <literal>stack</literal>
+ element to the <literal>server/all/deploy/cluster/jgroups-channelfactory.sar/META-INF/jgroups-channelfactory-stacks.xml</literal>
+ file. You can alter the behavior of an existing configuration by
+ editing this file. Before doing this though, have a look at the
+ other standard configurations the AS ships; perhaps one of those
+ meets your needs. Also, please note that before editing a
+ configuration you should understand what services are using that
+ configuration; make sure the change you are making is appropriate
+ for all affected services. If the change isn't appropriate for a
+ particular service, perhaps its better to create a new configuration
+ and change some services to use that new configuration.</para>
+ </section>
+
+ <para>It's important to note that if several services request a channel
+ with the same configuration name from the ChannelFactory, they are not
+ handed a reference to the same underlying Channel. Each receives its own
+ Channel, but the channels will have an identical configuration. A logical
+ question is how those channels avoid forming a group with each other
+ if each, for example, is using the same multicast address and port.
+ The answer is that when a consuming service connects its Channel, it
+ passes a unique-to-that-service <literal>cluster_name</literal> argument
+ to the <literal>Channel.connect(String cluster_name)</literal> method.
+ The Channel uses that <literal>cluster_name</literal> as one of the factors
+ that determine whether a particular message received over the
+ network is intended for it.</para>
+ </section>
+
<section id="clustering-blocks-jgroups-sharedtransport">
<title>The JGroups Shared Transport</title>
- <para></para>
+ <para>As the number of JGroups-based clustering services running in
+ the AS has risen over the years, the need to share the resources
+ (particularly sockets and threads) used by these channels became
+ a glaring problem. A stock AS 5 <emphasis role="bold">all</emphasis>
+ config will connect 4 JGroups channels during startup, and a total of
+ 7 or 8 will be connected if distributable web apps, clustered EJB3
+ SFSBs and a clustered JPA/Hibernate second level cache are all used.
+ So many channels can consume a lot of resources, and can be a real
+ configuration nightmare if the network environment requires
+ configuration to ensure cluster isolation.</para>
+
+ <para>Beginning with AS 5, JGroups supports sharing of transport
+ protocol instances between channels. A JGroups channel is composed of
+ a stack of individual protocols, each of which is responsible for
+ one aspect of the channel's behavior. A transport protocol is a
+ protocol that is responsible for actually sending messages on the
+ network and receiving them from the network. The resources that are
+ most desirable for sharing (sockets and thread pools) are managed by
+ the transport protocol, so sharing a transport protocol between channels
+ efficiently accomplishes JGroups resource sharing.</para>
+
+ <para>
+ To configure a transport protocol for sharing, simply add a
+ <literal>singleton_name="someName"</literal> attribute to the protocol's
+ configuration. All channels whose transport protocol config uses the
+ same <literal>singleton_name</literal> value will share their transport.
+ All other protocols in the stack will not be shared. The following
+ illustrates 4 services running in a VM, each with its own channel.
+ Three of the services are sharing a transport; the fourth is using
+ its own transport.</para>
+
+
+
+ <figure id="clustering-blocks-jgroups-sharedtp.fig">
+ <title>Services using a Shared Transport</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata align="center" fileref="images/clustering-SharedTransport.png" />
+ </imageobject>
+ </mediaobject>
+ </figure>
+
+ <para>The protocol stack configurations used by the AS 5 ChannelFactory
+ all have a singleton_name configured. In fact, if you add a stack to the
+ ChannelFactory that doesn't include a <literal>singleton_name</literal>,
+ before creating any channels for that stack, the ChannelFactory will
+ synthetically create a <literal>singleton_name</literal> by concatenating
+ the stack name to the string "unnamed_", e.g. unnamed_customStack.</para>
</section>
-
- <section id="clustering-blocks-jgroups-channelfactory">
- <title>The Channel Factory Service</title>
- <para></para>
- </section>
</section>
<section id="clustering-hapartition">
@@ -70,9 +259,12 @@
<para>
HAPartition is a general purpose service used for a variety of tasks
in AS clustering. At its core, it is an abstraction built on top of
- a JGroups Channel that provides support for making/receiving RPC
- invocations on/from one or more cluster members. HAPartition also
- supports a distributed registry of which clustering services are
+ a JGroups <literal>Channel</literal> that provides support for making/receiving RPC
+ invocations on/from one or more cluster members. HAPartition allows
+ services that use it to share a single <literal>Channel</literal> and
+ multiplex RPC invocations over it, eliminating the configuration complexity
+ and runtime overhead of having each service create its own <literal>Channel</literal>.
+ HAPartition also supports a distributed registry of which clustering services are
running on which cluster members. It provides notifications to
interested listeners when the cluster membership changes or the
clustered service registry changes. HAPartition forms the core of many
@@ -84,60 +276,95 @@
- <para>The following example shows the <literal>HAPartition</literal> MBean definition packaged with the standard JBoss AS distribution.
- So, if you simply start JBoss servers with their default clustering settings on a local network, you
- would get a default cluster named <literal>DefaultPartition</literal> that includes all server
- instances as its nodes.</para>
- <programlisting>
-<mbean code="org.jboss.ha.framework.server.ClusterPartition"
- name="jboss:service=DefaultPartition">
+ <para>The following snippet shows the <literal>HAPartition</literal> service definition packaged with the standard JBoss AS distribution.
+ This configuration can be found in the <literal>server/all/deploy/cluster/hapartition-jboss-beans.xml</literal> file.
+ </para>
+ <programlisting><![CDATA[
+ <bean name="HAPartitionCacheHandler"
+ class="org.jboss.ha.framework.server.HAPartitionCacheHandlerImpl">
+ <property name="cacheManager"><inject bean="CacheManager"/></property>
+ <property name="cacheConfigName">ha-partition</property>
+ </bean>
+
+ <bean name="HAPartition"
+ class="org.jboss.ha.framework.server.ClusterPartition">
+
+ <depends>jboss:service=Naming</depends>
+
+ <annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss:service=HAPartition,partition=${jboss.partition.name:DefaultPartition}", exposedInterface=org.jboss.ha.framework.server.ClusterPartitionMBean.class, registerDirectly=true)</annotation>
+
+ <!-- ClusterPartition requires a Cache for state management -->
+ <property name="cacheHandler"><inject bean="HAPartitionCacheHandler"/></property>
+
+ <!-- Name of the partition being built -->
+ <property name="partitionName">${jboss.partition.name:DefaultPartition}</property>
- <! -- Name of the partition being built -->
- <attribute name="PartitionName">
- ${jboss.partition.name:DefaultPartition}
- </attribute>
-
- <! -- The address used to determine the node name -->
- <attribute name="NodeAddress">${jboss.bind.address}</attribute>
-
- <! -- Determine if deadlock detection is enabled -->
- <attribute name="DeadlockDetection">False</attribute>
-
- <! -- Max time (in ms) to wait for state transfer to complete.
- Increase for large states -->
- <attribute name="StateTransferTimeout">30000</attribute>
-
- <! -- The JGroups protocol configuration -->
- <attribute name="PartitionConfig">
- ... ...
- </attribute>
-</mbean>
- </programlisting>
- <para>Here, we omitted the detailed JGroups protocol configuration for this channel. JGroups handles the
- underlying peer-to-peer communication between nodes, and its configuration is discussed in <xref linkend="jgroups.chapt"/>. The following list shows the available configuration attributes
- in the <literal>HAPartition</literal> MBean.</para>
+ <!-- The address used to determine the node name -->
+ <property name="nodeAddress">${jboss.bind.address}</property>
+
+ <!-- Max time (in ms) to wait for state transfer to complete. Increase for large states -->
+ <property name="stateTransferTimeout">30000</property>
+
+ <!-- Max time (in ms) to wait for RPC calls to complete. -->
+ <property name="methodCallTimeout">60000</property>
+
+ <!-- Optionally provide a thread source to allow async connect of our channel -->
+ <property name="threadPool"><inject bean="jboss.system:service=ThreadPool"/></property>
+
+ <property name="distributedStateImpl">
+ <bean name="DistributedState"
+ class="org.jboss.ha.framework.server.DistributedStateImpl">
+ <annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss:service=DistributedState,partitionName=${jboss.partition.name:DefaultPartition}", exposedInterface=org.jboss.ha.framework.server.DistributedStateImplMBean.class, registerDirectly=true)</annotation>
+ <property name="cacheHandler"><inject bean="HAPartitionCacheHandler"/></property>
+ </bean>
+ </property>
+
+ </bean>]]></programlisting>
+ <para>Much of the above is boilerplate; below we'll touch on the key points
+ relevant to end users. There are two beans defined above, the
+ <literal>HAPartitionCacheHandler</literal> and the <literal>HAPartition</literal> itself.</para>
+
+ <para>The <literal>HAPartition</literal> bean itself exposes the following
+ configuration properties:</para>
<itemizedlist>
<listitem>
- <para><emphasis role="bold">PartitionName</emphasis> is an optional attribute to specify the
+ <para><emphasis role="bold">partitionName</emphasis> specifies the
name of the cluster. Its default value is <literal>DefaultPartition</literal>. Use the <literal>-g </literal> (a.k.a. --partition) command line switch to set this value at JBoss startup.</para>
</listitem>
<listitem>
- <para><emphasis role="bold">NodeAddress</emphasis> is an optional attribute used to help generate a unique name for this node.</para>
+ <para><emphasis role="bold">nodeAddress</emphasis> is unused and can be ignored.</para>
</listitem>
<listitem>
- <para><emphasis role="bold">DeadlockDetection</emphasis> is an optional boolean attribute that
- tells JGroups to run message deadlock detection algorithms with every request. Its default
- value is <literal>false</literal>.</para>
+ <para><emphasis role="bold">stateTransferTimeout</emphasis> specifies the timeout (in milliseconds) for initial application state transfer. State transfer refers to the process of obtaining a serialized copy of initial application state from other already-running cluster members at service startup. Its default value is <literal>30000</literal>. </para>
</listitem>
<listitem>
- <para><emphasis role="bold">StateTransferTimeout</emphasis> is an optional attribute to specify the timeout for state replication across the cluster (in milliseconds). State replication refers to the process of obtaining initial application state from other already-running cluster members at service startup. Its default value is <literal>30000</literal>. </para>
+ <para><emphasis role="bold">methodCallTimeout</emphasis> specifies the timeout (in milliseconds) for obtaining responses to group RPCs from the other cluster members. Its default value is <literal>60000</literal>. </para>
</listitem>
+ </itemizedlist>
+
+ <para>The <literal>HAPartitionCacheHandler</literal> is a small utility service that
+ helps the HAPartition integrate with JBoss Cache
+ (see <xref linkend="clustering-blocks-jbc-cachemanager"/>). HAPartition exposes
+ a child service called DistributedState (see <xref linkend="clustering-hapartition-distributedstate"/>)
+ that uses JBoss Cache; the <literal>HAPartitionCacheHandler</literal> helps ensure
+ consistent configuration between the JGroups <literal>Channel</literal> used by
+ Distributed State's cache and the one used directly by HAPartition.</para>
+
+ <itemizedlist>
<listitem>
- <para><emphasis role="bold">PartitionConfig</emphasis> is an element to specify JGroup
- configuration options for this cluster (see <xref linkend="jgroups-configuration"/>).</para>
+ <para><emphasis role="bold">cacheConfigName</emphasis> the name of the
+ JBoss Cache configuration to use for the HAPartition-related cache.
+ Indirectly, this also specifies the name of the JGroups protocol
+ stack configuration HAPartition should use. See
+ <xref linkend="jbosscache-configuration-jgroups"/> for more on
+ how the JGroups protocol stack is configured.</para>
</listitem>
</itemizedlist>
- <para>In order for nodes to form a cluster, they must have the exact same <literal>PartitionName</literal> and the <literal>ParitionConfig</literal> elements. Changes in either element on some but not all nodes would cause the cluster to split.
+
+ <para>In order for nodes to form a cluster, they must have the exact same <literal>partitionName</literal>
+ and the <literal>HAPartitionCacheHandler</literal>'s <literal>cacheConfigName</literal>
+ must specify an identical JBoss Cache configuration. Changes in either
+ element on some but not all nodes would prevent proper clustering behavior.
</para>
<para>You can view the current cluster information by pointing your browser to the JMX console of any
Added: projects/docs/community/5/Clustering_Guide/en-US/images/clustering-SharedTransport.png
===================================================================
(Binary files differ)
Property changes on: projects/docs/community/5/Clustering_Guide/en-US/images/clustering-SharedTransport.png
___________________________________________________________________
Name: svn:mime-type
+ application/octet-stream
More information about the jboss-cvs-commits
mailing list