The UnitTest models our real application. It goes like this: It is an application
processing events for units (areas) where the area is an object stored in the cache and
session affinity is enforced by a message bus. Sometimes during the lifetime of the
application new areas will be added by a coordinator (which is restricted to one of the
nodes, ie. new areas are only ever added at one node).
The message bus acts as a load balancer and will distribute events for areas, using a
sticky algorithm so that session affinity is honored. So:
| * The coordinator adds an area object to the cache.
| * The message bus is notified (by jboss cache listener methods) of the new area. It
then selects a node in the cluster for processing.
| * The working (event sending) threads are notified of the new area and starts sending
events.
|
The problem appears, as I originally wrote, when the events arrive at their deignated node
and a data gravitation occurs. And as far as I have seen only on the backup node. For
example:
| * Node A is coordinator and adds an area (X).
| * Node B becomes buddy backup for X.
| * The message bus assigns X to node C.
| * When messages arrive at node C a data gravitation occurs.
| * Node B receives two consecutive data gravitation cleanup commands, one of which
doesn't seem to let go of its identity lock.
| * After "buddyCommunicationTimeout" (logged as debug) node C get null from
the cache and fails.
|
This is loosely modeled in the UnitTest. In the test worker threads are assigned areas
statically, areas are added in one cache after which the workers are notified of their new
area and attempts to access it. If any of the worker threads gets a null from the cache
the test fails as we know the area should exist (ie. the worker is not notified of its new
area until it has already been added).
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4095516#...
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...