<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Wed, Jun 17, 2015 at 3:08 PM Sanne Grinovero <<a href="mailto:sanne@infinispan.org">sanne@infinispan.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Will,<br>
thanks for asking! It's a big decision indeed, and I might have some<br>
objections :) although I can be easily persuaded otherwise as I'm not<br>
really sure how it's currently implemented.<br></blockquote><div><br></div><div>Yeah that is part of it is I am not sure the original intent for invalidation mode. We all know it gets left ignored in regards to our attention, unless something goes wrong :P</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
My doubts would be around cache state consistency across nodes. I<br>
agree with you that state transfer is unexpected in the scope of<br>
invalidation, but the consistent hash distribution is also used to<br>
identify the "primary owner" for each entry - which I think still is<br>
an essential role for example to implement atomic operations and<br>
locking.<br></blockquote><div><br></div><div>That is the part that is odd, sorry I didn't go into more details.</div><div><br></div><div>Atomic operations are only guaranteed for the local node data in nontx or tx mode. For example if you have 2 nodes and only node1 has a value for key1. If a replace(key1, newvalue) is ran on node2 it will fail since only node1 has a value for key1.</div><div><br></div><div>Locks are the thing that are really weird: </div><div>- No remote locks are acquired for a nontx cache.</div><div><br></div><div>- Remote locks are only acquired for tx caches. Since data modification is already protected by just a local lock, this only serves to prevent 2 invalidation messages to be sent from different nodes at the same time. </div><div><br></div><div>The usefulness of remote locks in tx mode seems dubious as it will slow down uncontended writes since it will require replicating the lock information to all owners (includes backups) and having to include this owner in the tx commit handling to clean up said lock. We also have to handle a topology change to move the lock information between nodes.</div><div><br></div><div>Maybe there is some use case here I am not aware of with invalidation and tx mode locking on remote nodes?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
One might say that such things are not needed for some use cases, but<br>
I think the contract of Invalidation with a client is around the lines<br>
of "it's ok to return null if you don't have a local copy" but it's<br>
not relaxing the consistency and transactional guarantees which we<br>
normally provide with other cache modes.<br>
We could of course argue if it could be useful to provide a relaxed<br>
consistency mode, but I'd really not drop the stronger model.<br>
<br>
-- Sanne<br>
<br>
<br>
On 17 June 2015 at 19:32, William Burns <<a href="mailto:mudokonman@gmail.com" target="_blank">mudokonman@gmail.com</a>> wrote:<br>
> Recently [1] was found. The underlying cause is that now initial state<br>
> transfer is enabled by default including invalidation caches.<br>
><br>
> When thinking about this I didn't even realize that we utilize a Replicated<br>
> consistent hash for invalidation mode. This all strikes me as just wrong.<br>
> To me an invalidation cache is just a local cache that sends broadcasted<br>
> invalidation messages to other members. It shouldn't have any need for a<br>
> consistent hash.<br>
><br>
> Currently though we have an invalidation cache that will try to acquire<br>
> remote locks [2], reports ownership of keys by the consistent hash (instead<br>
> of each node owning every key as it would be in a local cache [3]), which<br>
> will participate in initial state transfer (REPL based) [4] among other<br>
> things. We even have invalidation commands that are tied to given<br>
> topologies [5].<br>
><br>
> I was originally planning on cutting the cord between invalidation caches<br>
> and state transfer, but I am finding so many places that assume we have a<br>
> topology that it requires more code that I would have thought. So I wanted<br>
> to check if anyone had any objections. If there are objections I can end up<br>
> taking the quick and easy route of just forcing disabling of initial state<br>
> transfer and leaving everything as it is otherwise. I personally feel we<br>
> should completely separate these concepts to guarantee what I would think as<br>
> proper behavior.<br>
><br>
> Thanks,<br>
><br>
> - Will<br>
><br>
> [1] <a href="https://issues.jboss.org/browse/ISPN-5560" rel="noreferrer" target="_blank">https://issues.jboss.org/browse/ISPN-5560</a><br>
> [2]<br>
> <a href="https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/lock/InvalidationModePessimisticLockReleaseTest.java" rel="noreferrer" target="_blank">https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/lock/InvalidationModePessimisticLockReleaseTest.java</a><br>
> [3]<br>
> <a href="https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/interceptors/locking/ClusteringDependentLogic.java#L300" rel="noreferrer" target="_blank">https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/interceptors/locking/ClusteringDependentLogic.java#L300</a><br>
> [4]<br>
> <a href="https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/statetransfer/NonTxStateTransferInvalidationTest.java#L56" rel="noreferrer" target="_blank">https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/statetransfer/NonTxStateTransferInvalidationTest.java#L56</a><br>
> [5] <a href="https://issues.jboss.org/browse/ISPN-4016" rel="noreferrer" target="_blank">https://issues.jboss.org/browse/ISPN-4016</a><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</blockquote></div></div>