[infinispan-dev] Read Committed Distributed Cache Concerns

[William Burns]

I was recently refactoring code dealing with isolation levels and
found how ReadCommitted is implemented and I have a few concerns I
wanted to bring up.

ReadCommitted read operations work by storing a reference to the value
from the data store in its caller's context.  Thus whenever another
transaction is committed that updates the data store value any context
that has that reference now sees the latest committed value.  This
works well for Local and Replicated caches since all data stores are
updated with the latest value upon completion of the transaction.
However Distributed caches only the owners see the update in their
data store and thus any non owner will still have the old value they
previously read before the commit occurred.

This seems quite inconsistent that Distributed caches run in a mix of
Repeatable Read/Read Committed depending on what node and what key you
are using.

To operate properly we could track requests similar to how it works
for L1 so we can tell non owners to clear out their context values for
values they read remotely that they haven't updated (since Read
Committed writes should return the same written value).  That seems
like quite a bit of additional overhead though.

I am wondering is it worth it to try to keep Read Committed isolation
level though?  It seems that Repeatable Read would be simpler and most
likely more performant as you wouldn't need all the additional remote
calls to get it to work properly.  Or is it okay that we have
different isolation levels for some keys on some nodes?  This could be
quite confusing if a user was using a local and remote transaction and
a transaction may not see the other's committed changes when they
expect to.

What do you guys think?

 - Will

P.S.

I also found a bug with Read Committed for all caches where if you do
a write that changes the underlying InternalCacheEntry to a new type,
that reads won't see subsequent committed values.  This is caused
because the underlying data is changed to a new reference and a read
would still be holding onto a reference of the old InternalCacheEntry.
 This can happen when using the various overridden put methods for
example.  We should have a good solution for it, but may not be
required if we find that Read Committed itself is flawed beyond
saving.