[infinispan-dev] L1 Data Container

Wed Jun 19 11:28:04 EDT 2013

On Wed, Jun 19, 2013 at 10:19 AM, Sanne Grinovero <sanne at infinispan.org>wrote:

> On 19 June 2013 13:44, William Burns <mudokonman at gmail.com> wrote:
> > All the L1 data for a DIST cache is stored in the same data container as
> the
> > actual distributed data itself.  I wanted to propose breaking this out so
> > there is a separate data container for the L1 cache as compared to the
> > distributed data.
> >
> > I thought of a few quick benefits/drawbacks:
> >
> > Benefits:
> > 1. L1 cache can be separately tuned - L1 maxEntries for example
>
> -1!
>   I don't think thats a benefit actually, from the point of view of a user:
> as a user I only know I have a certain amount of memory available on
> each node, and the application is going to use certain data way more
> often than others.
> The eviction strategy should be put in condition to be able to make an
> optimal choice about which entries - among all - are better kept in
> memory vs. passivated.
> I don't see a specific reason to "favour" keeping in memory owned
> entries over an L1 entry: the L1 entry might be very hot, and the
> owned entry might be almost-never read.
> Considering that even serving a Get operation to another node (as
> owners of the entry) makes the entry less likely to be passivated (it
> counts as a "hit"), the current design naturally provides an optimal
> balance for memory usage.
>
> At the opposite site, I don't see how - as a user - I could optimally
> tune a separate container.
>
I agree that is more difficult to configure, this was one of my points as
both a drawback and benefit.   It sounds like in general you don't believe
the benefits outweigh the drawbacks then.

>
> > 2. L1 values will not cause eviction of real data
>
> -1
> That's not a benefit, as I explained above. "Real Data" is not less
> important, especially if it's never used.
> Granted I'm making some assumptions about the application having some
> hot-data and some less hot data, and not being able to take advantage
> of node pinning or affinity strategies.. but that is another way to
> say that I'm assuming the user needs L1: if it was possible to apply
> these more advanced strategies I'd disable L1 altogether.
>
In regards to "real data" versus L1.  I don't see how having the containers
separated would be an issue for very hot data either way.  In either case
the hottest data would be promoted in the LIRS in either cache.  The only
way having the containers together would be different was if you had more
hot L1 entries than your currently sized L1 maxEntries and the data
container could have removed some of the "real data" to make room, but that
seems unlikely unless you sized the L1 cache very small.  But either way L1
always has a lifespan, so even if it is the hottest data ever it will be
removed at the end of that lifespan no matter what.

To me a L1 cache value is of lower priority than real data - especially the
fact that we can't even guarantee how long the L1 value will be around for.
 I guess that assumption may be incorrect though.

>
> > 3. Would make https://issues.jboss.org/browse/ISPN-3229 an easy fix
> > 4. Could add a new DataContainer implementation specific to L1 with
> > additional optimizations
>
> You have some example of what you have in mind?
> Considering you would need to consider the optimal balance usage of
> the available heap space, I suspect that would be quite hard.
>
 Nothing that substantial right now.  I was thinking you could remove a lot
of additional checks since you know that they would always be mortal
entries and you wouldn't want to passivate such entries - since it will
usually be faster to just ask the owning node for the value.

>
> > 5. Help with some concurrency issues with L1 without requiring wider
> locking
> > (such as locking a key for an entire ClusteredGet rpc call) -
> > https://issues.jboss.org/browse/ISPN-3197.
>
> I don't understand this. L1 entries require the same level of
> consistency than any other entry so I suspect you would need the same
> locking patterns replicated. The downside would be that you get
> duplication of the same logic.
> Remember also that L1 is having some similarities with entries still
> "hanging around" when they where previously stored in this node after
> a state transfer.. today these are considered L1-active entries, if
> you change the storage you would need to design a migration of state
> from one container to the other; migration of state might not be too
> hard, doing it while guaranteeing consistent locking is going to be I
> guess as hard as considering the L1 problem today.
>
There are 2 sides to this, transactional and non transactional.

In the non transactional case we only ever write to the L1 cache on a Get
and an Invalidation, which allows for quite a bit of optimization.  In this
case we can get away with only locking quickly before and after the get rpc
call and still guarantee consistency with the L1.  And if the additional
optimization I mentioned on Pedro's comments we can eliminate the locking
as we know only a single get is done at a time.

Transactional needs to be changed to support locking I talk about in the
non tx paragraph to fix https://issues.jboss.org/browse/ISPN-2965.  Writes
in the tx model acquire the lock for the duration of the commit rpc so we
are already covered from that perspective.

The one thing I wasn't quite clear on was rehash events, which I was
talking to Dan about a bit, but as you mention should be doable.

>
> >
> > Drawbacks:
> > 1. Would require, depending on configuration, an additional thread for
> > eviction
> > 2. Users upgrading could have double memory used up due to 2 data
> containers
>
> This drawback specifically is to be considered very seriously. I don't
> think people would be happy to buy and maintain a twice as large
> datacenter than what they actually need.
>
I was only saying if they left the configuration as is and the default
value was the same as the other data container size.  As mentioned by Manik
and Pedro even with those settings this would unlikely scale to double.

>
> Sanne
>
> >
> > Both?:
> > 1. Additional configuration available
> >    a. Add maxEntires just like the normal data container (use data
> container
> > size if not configured?)
> >    b. Eviction wakeup timer?  We could just reuse the task cleanup
> > frequency?
> >    c. Eviction strategy?  I would think the default data container's
> would
> > be sufficient.
> >
> > I was wondering what you guys thought.
> >
> > Thanks,
> >
> >  - Will
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20130619/1e3b1263/attachment-0001.html 