3:50 p.m.
Hi all,
I just want to share some thoughts on Infinispan. Probably you already
had similar ideas, maybe you can provide some keywords, so I can
search for discussions in the archive.
As I understand, the ConsistentHash works just as a consistent hash
based "locator" for keys in the cluster. Basically one could implement
full replication with a special ConsistentHash that always returns all
addresses in the cluster. A ConsistentHash/"locator" has to be uniform
over the entire cluster to make it work. In principle this is the only
requirement for a "locator".
Currently there is only one parameter for the "locator" - a set with
all nodes in the cluster. If there would be a way to pass (maybe
dynamically) more parameters to a "locator" in a consistent way, many
possibilities would open up: e.g. an asymmetric cache configuration by
feeding the locator with a distributed registry; switching hash
functions at runtime; subscribable keys; keys, that have to stay at a
special node... Also it may be possible to remove the replication mode
and instead provide a special locator. To make this work rebalancing
would have to be more fine grained. E.g. it should be possible for the
locator (or its manager?) to ignore a view change or to trigger some
rebalance work per cache.
What do you think about it?
Cheers,
Christian
10:44 a.m.
Hi Christian, answers in-line.
On 19 Jul 2011, at 21:50, Christian Kulenkampff wrote:
Hi all,
I just want to share some thoughts on Infinispan. Probably you already
had similar ideas, maybe you can provide some keywords, so I can
search for discussions in the archive.
As I understand, the ConsistentHash works just as a consistent hash
based "locator" for keys in the cluster. Basically one could implement
full replication with a special ConsistentHash that always returns all
addresses in the cluster. A ConsistentHash/"locator" has to be uniform
over the entire cluster to make it work. In principle this is the only
requirement for a "locator".
Currently there is only one parameter for the "locator" - a set with
all nodes in the cluster. If there would be a way to pass (maybe
dynamically) more parameters to a "locator" in a consistent way, many
possibilities would open up: e.g. an asymmetric cache configuration by
feeding the locator with a distributed registry; switching hash
functions at runtime; subscribable keys; keys, that have to stay at a
special node…
Possibly, but this breaks the deterministic aspect of a ConsistentHash. I.e., locating a
key by any node in the cluster will not be possible. Unless we share some additional
metadata about each key (such as whether it is pinned, and where it is pinned to, etc).
And the consistency of this metadata will have to be very strong to ensure being able to
find such a key. So it isn't trivial to implement. What specific use case are you
trying to solve here?
Also it may be possible to remove the replication mode
and instead provide a special locator.
We considered this, however there is a lot of additional overhead and we wouldn't be
able to make use of certain JGroups tricks to optimise sending the same state to all
servers in the cluster.
To make this work rebalancing
would have to be more fine grained. E.g. it should be possible for the
locator (or its manager?) to ignore a view change or to trigger some
rebalance work per cache.
We are doing some of this anyway, for other reasons. There is a set of JIRAs targeted for
5.1 to address this.
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan.org
12:35 p.m.
Thank you for the reply!
I currently try to implement a game server system for my diploma
thesis. I would like to use Infinispan for sharing dynamic game
objects between the nodes. Shared parameters for the
locator/ConsistentHash would simplify things for some cases (e.g.
spatial hash and node assignment "by hand"). The implementation of
virtual nodes may profit from a common interface for providing dynamic
parameters to all nodes too (one could add virtual nodes "by
hand"/with special algorithms to the hash wheel).
When these shared parameters (like a key-to-address registry for
providing keys that stay at a special node [usage triggered via group
or special key class...] or a list of nodes that should participate in
providing a cache) are consistently shared with all nodes there should
not be a loss of any deterministic aspects. Changing a parameter would
just trigger something like a node join event. Maybe sharing data in a
broader context (per cache not per locator) makes more sense (e.g.
cache configurations could be send to joining nodes).
But I want to emphasize these are only some thoughts that came up
while thinking about my game server system. Actually I am not sure if
a cache like Infinispan is really a good approach for sharing
write-intensive virtual world data. However using delta aware data
structures I got some promising results. I will implement something
with Infinispan anyway. Of course I will share the results :).
> Also it may be possible to remove the replication mode
> and instead provide a special locator.
We considered this, however there is a lot of additional overhead and we wouldn't be
able to make use of certain JGroups tricks to optimise sending the same state to all
servers in the cluster.
Yeah, I also thought about this. It would require to be able to return
some kind of multicast-adresses with the locator/ConsistentHash and
just by comparing DistributionInterceptor to ReplicationInterceptor
there are indeed many things that one would have to make skippable...
For my use case I am really looking forward to the new applyDelta
method. Since the putMap Command does not support deltas (why,
performance?), maybe an applyDeltas method would even add some more
performance/convenience.
Cheers
Christian
4896
days inactive
4904
days old
infinispan-dev@lists.jboss.org
2 comments
2 participants
participants (2)
-
Christian Kulenkampff -
Manik Surtani