[infinispan-dev] State Transfer & Invalidation Caches

[Sanne Grinovero]

Hi Will,
thanks for asking! It's a big decision indeed, and I might have some
objections :) although I can be easily persuaded otherwise as I'm not
really sure how it's currently implemented.

My doubts would be around cache state consistency across nodes. I
agree with you that state transfer is unexpected in the scope of
invalidation, but the consistent hash distribution is also used to
identify the "primary owner" for each entry - which I think still is
an essential role for example to implement atomic operations and
locking.

One might say that such things are not needed for some use cases, but
I think the contract of Invalidation with a client is around the lines
of "it's ok to return null if you don't have a local copy" but it's
not relaxing the consistency and transactional guarantees which we
normally provide with other cache modes.
We could of course argue if it could be useful to provide a relaxed
consistency mode, but I'd really not drop the stronger model.

-- Sanne


On 17 June 2015 at 19:32, William Burns <mudokonman at gmail.com> wrote:
> Recently [1] was found.  The underlying cause is that now initial state
> transfer is enabled by default including invalidation caches.
>
> When thinking about this I didn't even realize that we utilize a Replicated
> consistent hash for invalidation mode.  This all strikes me as just wrong.
> To me an invalidation cache is just a local cache that sends broadcasted
> invalidation messages to other members.  It shouldn't have any need for a
> consistent hash.
>
> Currently though we have an invalidation cache that will try to acquire
> remote locks [2], reports ownership of keys by the consistent hash (instead
> of each node owning every key as it would be in a local cache [3]), which
> will participate in initial state transfer (REPL based) [4] among other
> things.  We even have invalidation commands that are tied to given
> topologies [5].
>
> I was originally planning on cutting the cord between invalidation caches
> and state transfer, but I am finding so many places that assume we have a
> topology that it requires more code that I would have thought.  So I wanted
> to check if anyone had any objections.  If there are objections I can end up
> taking the quick and easy route of just forcing disabling of initial state
> transfer and leaving everything as it is otherwise.  I personally feel we
> should completely separate these concepts to guarantee what I would think as
> proper behavior.
>
> Thanks,
>
>  - Will
>
> [1] https://issues.jboss.org/browse/ISPN-5560
> [2]
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/lock/InvalidationModePessimisticLockReleaseTest.java
> [3]
> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/interceptors/locking/ClusteringDependentLogic.java#L300
> [4]
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/statetransfer/NonTxStateTransferInvalidationTest.java#L56
> [5] https://issues.jboss.org/browse/ISPN-4016
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev