[infinispan-dev] RELAY and Infinispan

[Bela Ban]

RELAY [1] bridges separate local clusters into large virtual global 
clusters, e.g. {A,B,C} and {X,Y,Z} into {A,B,C,X,Y,Z}.

This new global view has local members A, B and C and proxies X, Y, Z on 
the {A,B,C} cluster, and vice versa.

When A sends a message, it is forwarded to the other cluster, but the 
sender is wrapped into a ProxyAddress A/X. This means that the original 
sender is A, but the local address in {X,Y,Z} is X. However, there is a 
problem !

A ProxyAddress A/X's hashCode(), equals() and compareTo() use *X*, which 
means that if we add X, Y, Z and A/X into a *HashSet* (or HashMap), X 
and A/X map to the same address ! So if you have a View (on X) with 
{A/X, B/X, C/X, X, Y, Z}, and add all members to a HashMap, we'll only 
have a size of 3 !!

If we used *A* in A/X for hashCode(), equals() and compareTo(), then 
this problem would not exist, however, this means that we would now 
'know' about the other cluster, and therefore digest handling, flow 
control etc would happen across both clusters, which is something we 
don't want; we want the 2 local cluster to be completely autonomous ! 
E.g. we don't want cluster {X,Y,Z} to block on credits from B in the 
other cluster...


So I was thinking of passing the local view {X,Y,Z} up to Infinispan 
instead of the global view {A/X,B/X,C/X,X,Y,Z}. This would mean 
Infinispan would know only about A, B and C in  the cluster {A,B,C}, and 
about X, Y and Z in {X,Y,Z}.

Now, I want to be able to have backups of keys from {A,B,C} in {X,Y,Z} 
in DIST mode, e.g. with numOwners, key "name" should be stored on A and 
C in the local cluster, and on Z in the remote cluster.

To do this, the consistent hash function would know about the local 
cluster {A,B,C} and the remote cluster {X,Y,Z}. It would get view 
changes by hooking into RELAY.
So when there is a local("name", 3), it would return A, C, A/Z, causing 
Infinispan to fetch the data from or store the data to A,C and Z.

This should work fine I guess, because when Infinispan tries to send 
data to A/Z, JGroups's RELAY will forward the message to the remote Z.
Q: does Infinispan assert that A, C and Z are in the local view, when 
distributing data ? Because then my scheme above wouldn't work...

The other question I have is, can I force Infinispan to do a rehashing ? 
For example, when the consistent hash function in {A,B,C} gets a view 
change for the remote view, going from {X} to {X,Y}, then I'd like 
Infinispan to do a rehashing, checking whether all keys are in the 
correct location and - if not - call into the consistent hash function 
to compute the new locations...


Thoughts ?


[1] http://www.jgroups.org/manual/html/user-advanced.html#RelayAdvanced

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss