[infinispan-dev] Experiment: Affinity Tagging

[Sanne Grinovero]

Hi all,

I'm playing with an idea for some internal components to be able to
"tag" the key for an entry to be stored into Infinispan in a very
specific segment of the CH.

Conceptually the plan is easy to understand by looking at this patch:

https://github.com/Sanne/infinispan/commit/45a3d9e62318d5f5f950a60b5bb174d23037335f

Hacking the change into ReplicatedConsistentHash is quite barbaric,
please bear with me as I couldn't figure a better way to be able to
experiment with this. I'll probably want to extend this class, but
then I'm not sure how to plug it in?

What would you all think of such a "tagging" mechanism?

# Why I didn't use the KeyAffinityService
- I need to use my own keys, not the meaningless stuff produced by the service
- the extensive usage of Random in there doesn't seem suited for a
performance critical path

# Why I didn't use the Grouping API
- I need to pick the specific storage segment, not just co-locate with
a different key


The general goal is to make it possible to "tag" all entries of an
index, and have an independent index for each segment of the CH. So
the resulting effect would be, that when a primary owner for any key K
is making an update, and this triggers an index update, that update is
 A) going to happen on the same node -> no need to forwarding to a
"master indexing node"
 B) each such writes on the index happen on the same node which is
primary owner for all the written entries of the index.

There are two additional nice consequences:
 - there would be no need to perform a reliable "master election":
ownership singleton is already guaranteed by Infinispan's essential
logic, so it would reuse that
 - the propagation of writes on the index from the primary owner
(which is the local node by definition) to backup owners could use
REPL_ASYNC for most practical use cases.

So net result is that the overhead for indexing is reduced to 0 (ZERO)
blocking RPCs if the async repl is acceptable, or to only one blocking
roundtrip if very strict consistency is required.

Thanks,
Sanne