When thinking about this I didn't even realize that we utilize a Replicated consistent hash for invalidation mode.  This all strikes me as just wrong.  To me an invalidation cache is just a local cache that sends broadcasted invalidation messages to other members.  It shouldn't have any need for a consistent hash.

Currently though we have an invalidation cache that will try to acquire remote locks [2], reports ownership of keys by the consistent hash (instead of each node owning every key as it would be in a local cache [3]), which will participate in initial state transfer (REPL based) [4] among other things.  We even have invalidation commands that are tied to given topologies [5].

I was originally planning on cutting the cord between invalidation caches and state transfer, but I am finding so many places that assume we have a topology that it requires more code that I would have thought.  So I wanted to check if anyone had any objections.  If there are objections I can end up taking the quick and easy route of just forcing disabling of initial state transfer and leaving everything as it is otherwise.  I personally feel we should completely separate these concepts to guarantee what I would think as proper behavior.

Thanks,

 - Will

[2] https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/lock/InvalidationModePessimisticLockReleaseTest.java

[3] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/interceptors/locking/ClusteringDependentLogic.java#L300

[4] https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/statetransfer/NonTxStateTransferInvalidationTest.java#L56

[5] https://issues.jboss.org/browse/ISPN-4016