On 30 Mar 2010, at 14:09, Bela Ban wrote:
Sanne Grinovero wrote:
>>> You could use the cluster view, which is the same across all nodes,
>>> and pick the first element in the list. Or you could run an
>>> agreement protocol, which deterministically elects a master.
>> Looks simple, deterministic and elegant.
> Bela Ban wrote:
>
> Agreed, that looks like a clean solution, but would scale better if we
> could decide a master for each different index, to spread the load
> instead of choosing always the same node for all the work, ideas on that?
> I could hash the indexname (the identifier) and % cluster view list
> size, WDYT ?
Absolutely, this makes a lot of sense. However, this will require moving
indices around when the cluster view changes.
With a simple modulo based algorithm you'll have a lot of rebalancing
going on, so maybe something like a consistent hash could determine the
server or servers on which a given index is stored. You could even store
an index on more than 1 nodes, similar to what Infinispan does with DIST
and numOwners=2 (for example).
Even better: *if* an index can be recreated from the real data, you
could use a consistent hash and only store an index *once*, and you
could use a Cache instance for it !
Yes, this would be the plan. The indexes themselves are *read* by every node in the
cluster and hence is stored in the cache. It is just the updates that will be processed
by a selected processor singleton.
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org