As consistency models go, Infinispan is primarily strongly consistent (with 2-phase commit
between data owners), with the exception of during a rehash where because of eventual
consistency (inability to get a valid response to a remote GET) forces us to wait for more
responses, a quorum if you like. Not dissimilar to PAXOS [1] in some ways.
I'm wondering whether, for the sake of performance, we should also offer a fully
eventually consistent model? What I am thinking is that changes *always* occur only on
the primary data owner. Single phase, no additional round trips, etc. The primary owner
then asynchronously propagates changes to the other data owners. This would mean things
run much faster in a stable cluster, and durability is maintained. However, during
rehashes when keys are moved, the notion of the primary owner may change. So to deal with
this, we could use vector clocks [2] to version each entry. Vector clocks allow us to
"merge" state nicely in most cases, and in the case of reads, we'd flip back
to a PAXOS style quorum during a rehash to get the most "correct" version.
In terms of implementation, almost all of this would only affect the
DistributionInterceptor and the DistributionManager, so we could easily have eventually
consistent flavours of these two components.
Thoughts?
Cheers
Manik
[1]
http://en.wikipedia.org/wiki/Paxos_algorithm
[2]
http://en.wikipedia.org/wiki/Vector_clock
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan.org