[infinispan-dev] Meaning of locking in Infinispan: ISPN-1546 and better general throughput

[Manik Surtani]

On 7 Dec 2011, at 13:27, Sanne Grinovero wrote:

> So in Infinispan we have a single type of lock associated with each
> entry; I assume it's designed like it is to save on memory, as are
> features like lock striping, and as the reason to not support both
> read and write locks.
> 
> A problem I'm having more and more often, is that the "lock" is being
> used both as something that leaks out as a way to control data
> consistency from a "user" point of view, and as an internal artifact
> to ensure consistent internal mutations of our data structure.
> 
> So the same Lock instance can be acquired for long term by a user
> process, and prevent for example the data to be passivated on a
> CacheLoader as this will attempt to grab the same lock.
> These should really be two different things!

Yes agreed.  One is a data lock - a logical, user-centric lock on some data that should not be concurrently mutated - versus an internal, data-structure lock which prevents concurrent modification of parts of the internal data structure.  Is this what you are getting at?

> It might bring up some confusion as well, like with the
> FineGrainedAtomicMap: the feature I need is to be able to lock values
> in an atomic map individually, in terms of data consistency. This does
> NOT imply that Infinispan should not be allowed to acquire locks on
> the AtomicMap itself for brief moments, to proceed with other internal
> processes.. be it to create a threadsafe iterator, passivate the entry
> to a CacheLoader or even transfer the element to a new owned in the
> grid.
> 
> Finally, to save memory I think that we don't need to guarantee these
> lock instances exists all the time; granted it might be more efficient
> to keep them as part of the Entry to avoid re-creating them too often,
> but in some cases it might not be the case, so we might even make it
> possible to create a different kind of Entry to optimise for specific
> usage patterns.
> 
> In pratical terms, it's currently quite hard to design a consistent
> data access using Infinispan's locks from a user point of view if you
> have to consider that Infinispan might lock the keys for it's own
> internal needs.
> 
> To solve ISPN-1546, I think it's totally fine to acquire a lock on the
> FGAM for the time needed to create an iterator. But this lock needs to
> be a different instance than the entry itself, and will be very short
> lived, and not clustered in any way. it's just a means to guarantee we
> can make a safe copy of the needed Array, and acquiring this lock
> should have nothing to do with the "data experience" of preventing
> some entries of the FGAM to be updated.
> 
> thoughts?

This is all quite a big change though.  Do we have a  JIRA for this?  Perhaps we should start my creating one, and outlining some designs on the wiki.

Cheers
Manik
--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org