All the L1 data for a DIST cache is stored in the same data container as the actual distributed data itself. I wanted to propose breaking this out so there is a separate data container for the L1 cache as compared to the distributed data. I thought of a few quick benefits/drawbacks: Benefits: 1. L1 cache can be separately tuned - L1 maxEntries for example 2. L1 values will not cause eviction of real data 3. Would make https://issues.jboss.org/browse/ISPN-3229 an easy fix 4. Could add a new DataContainer implementation specific to L1 with additional optimizations 5. Help with some concurrency issues with L1 without requiring wider locking (such as locking a key for an entire ClusteredGet rpc call) - https://issues.jboss.org/browse/ISPN-3197. Drawbacks: 1. Would require, depending on configuration, an additional thread for eviction

2. Users upgrading could have double memory used up due to 2 data containers

Both?: 1. Additional configuration available a. Add maxEntires just like the normal data container (use data container size if not configured?) b. Eviction wakeup timer? We could just reuse the task cleanup frequency? c. Eviction strategy? I would think the default data container's would be sufficient. I was wondering what you guys thought. Thanks, - Will _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I don't think so. We should only query the L1DC just before performing the remote get. This way, only the DistributionInterceptor (and sub-classes) will keep a reference to L1DC to query, store, etc.

+1 always having it be bounded would be really good. The configuration could have a default and there could be a check to make sure they have > 1 configured if provided. I don't know what would be the best default here though? 64? 128? 1024?

All the L1 data for a DIST cache is stored in the same data container as the actual distributed data itself. I wanted to propose breaking this out so there is a separate data container for the L1 cache as compared to the distributed data. I thought of a few quick benefits/drawbacks: Benefits: 1. L1 cache can be separately tuned - L1 maxEntries for example

2. L1 values will not cause eviction of real data

3. Would make https://issues.jboss.org/browse/ISPN-3229 an easy fix 4. Could add a new DataContainer implementation specific to L1 with additional optimizations

5. Help with some concurrency issues with L1 without requiring wider locking (such as locking a key for an entire ClusteredGet rpc call) - https://issues.jboss.org/browse/ISPN-3197.

Drawbacks: 1. Would require, depending on configuration, an additional thread for eviction 2. Users upgrading could have double memory used up due to 2 data containers

Both?: 1. Additional configuration available a. Add maxEntires just like the normal data container (use data container size if not configured?) b. Eviction wakeup timer? We could just reuse the task cleanup frequency? c. Eviction strategy? I would think the default data container's would be sufficient. I was wondering what you guys thought. Thanks, - Will _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

/ >> Benefits: >> 1. L1 cache can be separately tuned - L1 maxEntries for example > -1! > I don't think thats a benefit actually, from the point of view of a user: > [...] > At the opposite site, I don't see how - as a user - I could optimally tune a separate container. / There is 1 place where this is important from the User's view. Users want (and actually need) to be able to tune their expectations for their Cache/Grid provider's L1 capability join-point. Specifically, a user can be greatly empowered if their provider provides an API that assist the user's preference for "where he wants to be" on the get(K) probability of "hit@L1" vs. get(K) probability of "readThrough@remoteContainer" distribution curve.

On 19 June 2013 15:35, cotton-ben <ben.cotton(a)alumni.rutgers.edu&gt; wrote: > / >>> Benefits: >>> 1. L1 cache can be separately tuned - L1 maxEntries for example > >> -1! >> I don't think thats a benefit actually, from the point of view of a user: >> [...] >> At the opposite site, I don't see how - as a user - I could optimally > tune a separate container. / > > There is 1 place where this is important from the User's view. Users want > (and actually need) to be able to tune their expectations for their > Cache/Grid provider's L1 capability join-point. Specifically, a user can be > greatly empowered if their provider provides an API that assist the user's > preference for "where he wants to be" on the get(K) probability of "hit@L1" > vs. get(K) probability of "readThrough@remoteContainer" distribution curve. Indeed, IFF you separate the two storage areas you get in that kind of need, but I think it's an unnecessary complexity. The world is actually quite simple: you have N gigs of heap available, you can estimate you want your LIRS strategy to target some M number of entries. Which entries exactly, you will never be able to make a good choice, but LIRS (or other eviction strategies) are able to make a decision which is easy to model. This whole discussion is starting from the assumption that it is important to keep the owned entries in memory; I don't think we should blindly assume that. If someone could explain an important reason to always favour owned entries over *hotter* entries I'd be less skeptical, but even so that would be a step backwards in ease of configuration as today the grid basically self-tunes ergonomically.

> tune a separate container.

I agree that is more difficult to configure, this was one of my points as both a drawback and benefit. It > sounds like in general you don't believe the benefits outweigh the drawbacks then./

On 19 June 2013 16:44, cotton-ben <ben.cotton(a)alumni.rutgers.edu&gt; wrote: > > />> At the opposite site, I don't see how - as a user - I could optimally >>> tune a separate container. > >> I agree that is more difficult to configure, this was one of my points as >> both a drawback and benefit. It > sounds like in general you don't >> believe the benefits outweigh the drawbacks then./ > > Hi William. The benefits of your ambition to provide L1 capability > enhancements -- for /certain/ user's completeness requirements-- definitely > outweigh the drawbacks . This is a FACT. I have to disagree ;-) It certainly is a fact that he's very well intentioned to make enhancements, but I don't this strategy is proven the be superior; I'm actually convinced of the opposite. We simply cannot assume that the "real data" and the L1 stored entries will have the same level of hotness, it's actually very likely (since you like stats) that the entries stored in L1 are frequently accessed, to the opposite of other entries which - for as far as we know - could be large and dormant for years.

Storing useless data in memory is going to force to evict entries from L1 which are hot by definition (as they wouldn't be in L1 otherwise - as you pointed out there likely is an expiry) is a strategy which actively strives towards a less efficient storage. Also L1 timeout can be disabled, making for a very nice self-tuning adaptive system.

Sanne > > However, for the /typical/ user's completeness requirements, this FACT > probably won't be relevant. > > > > -- > View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-L1-D... > Sent from the Infinispan Developer List mailing list archive at Nabble.com. > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 19 June 2013 17:17, William Burns <mudokonman(a)gmail.com&gt; wrote: > > > > On Wed, Jun 19, 2013 at 11:56 AM, Sanne Grinovero <sanne(a)infinispan.org&gt; > wrote: >> >> On 19 June 2013 16:44, cotton-ben <ben.cotton(a)alumni.rutgers.edu&gt; wrote: >> > >> > />> At the opposite site, I don't see how - as a user - I could >> > optimally >> >>> tune a separate container. >> > >> >> I agree that is more difficult to configure, this was one of my points >> >> as >> >> both a drawback and benefit. It > sounds like in general you don't >> >> believe the benefits outweigh the drawbacks then./ >> > >> > Hi William. The benefits of your ambition to provide L1 capability >> > enhancements -- for /certain/ user's completeness requirements-- >> > definitely >> > outweigh the drawbacks . This is a FACT. >> >> I have to disagree ;-) It certainly is a fact that he's very well >> intentioned to make enhancements, but I don't this strategy is proven >> the be superior; I'm actually convinced of the opposite. >> >> We simply cannot assume that the "real data" and the L1 stored entries >> will have the same level of hotness, it's actually very likely (since >> you like stats) that the entries stored in L1 are frequently accessed, >> to the opposite of other entries which - for as far as we know - could >> be large and dormant for years. > > Actually this is only half true, we know that the values are hot on this > node specifically. Other nodes could be requesting the "cold" data quite > frequently as well. I see where you're coming from, but my point is the opposite: if other nodes would be requesting this data quite frequently, it woudln't be considered "cold": by using a single data container the eviction strategy automatically takes this into account as well. A hit is a hit in all senses.

> This could lead to L1 values pushing out distributed > data leaving it where nodes have L1 cached values for which the owner > doesn't even own. That's just another excellent reason to keep a unified datacontainer: if a different node uses the value frequently, allow it to be cached for read operations, even if the primary owner is passivating it. Write operations are inherently safe as they have to go through the owner and trigger entry activation as needed.

> And when the L1 cache value expires there will be no more > backup (not including passivation). This is a very odd situation though, > since you can't do conditional operations then as well. Conditional operations would hit the owner, and by doing so trigger loading. Not too odd, as it's the design today.

> And even with separate containers this is possible to have the L1 > discrepency, but actually having a lower lifespan would help remedy this > since every once in a while the "hot" value would have to retrieved again > from the owning node. Should be unnecessary, consistency is guaranteed no matter what timeouts of lifespans are set.

>> >> >> Storing useless data in memory is going to force to evict entries from >> L1 which are hot by definition (as they wouldn't be in L1 otherwise - >> as you pointed out there likely is an expiry) is a strategy which >> actively strives towards a less efficient storage. >> >> Also L1 timeout can be disabled, making for a very nice self-tuning >> adaptive system. > > The value cannot be disabled for the timeout. Unless by disable you mean to > increase the timeout so high that it will never be hit? Is that a common > use case? Sure why not, any read-mostly system would benefit from it: think of video streaming services, online stores (amazon like or app stores), news papers. Basically, most systems on which I'd suggest to enable L1. BTW I think L1's lifespan accepts a -1 as well, which should result in immortal cache entries.

Sanne _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I have to disagree ;-) It certainly is a fact that he's very well intentioned to make enhancements, but I don't this strategy is proven the be superior; I'm actually convinced of the opposite.

We simply cannot assume that the "real data" and the L1 stored entries will have the same level of hotness,

On 19 June 2013 16:44, cotton-ben <ben.cotton(a)alumni.rutgers.edu&gt; wrote: > > />> At the opposite site, I don't see how - as a user - I could optimally >>> tune a separate container. > >> I agree that is more difficult to configure, this was one of my points as >> both a drawback and benefit. It > sounds like in general you don't >> believe the benefits outweigh the drawbacks then./ > > Hi William. The benefits of your ambition to provide L1 capability > enhancements -- for /certain/ user's completeness requirements-- definitely > outweigh the drawbacks . This is a FACT. I have to disagree ;-) It certainly is a fact that he's very well intentioned to make enhancements, but I don't this strategy is proven the be superior; I'm actually convinced of the opposite. We simply cannot assume that the "real data" and the L1 stored entries will have the same level of hotness, it's actually very likely (since you like stats) that the entries stored in L1 are frequently accessed, to the opposite of other entries which - for as far as we know - could be large and dormant for years. Storing useless data in memory is going to force to evict entries from L1 which are hot by definition (as they wouldn't be in L1 otherwise - as you pointed out there likely is an expiry) is a strategy which actively strives towards a less efficient storage. Also L1 timeout can be disabled, making for a very nice self-tuning adaptive system.

On 19 June 2013 13:44, William Burns <mudokonman(a)gmail.com&gt; wrote: > All the L1 data for a DIST cache is stored in the same data container as the > actual distributed data itself. I wanted to propose breaking this out so > there is a separate data container for the L1 cache as compared to the > distributed data. > > I thought of a few quick benefits/drawbacks: > > Benefits: > 1. L1 cache can be separately tuned - L1 maxEntries for example -1! I don't think thats a benefit actually, from the point of view of a user: as a user I only know I have a certain amount of memory available on each node, and the application is going to use certain data way more often than others. The eviction strategy should be put in condition to be able to make an optimal choice about which entries - among all - are better kept in memory vs. passivated. I don't see a specific reason to "favour" keeping in memory owned entries over an L1 entry: the L1 entry might be very hot, and the owned entry might be almost-never read.

owners of the entry) makes the entry less likely to be passivated (it counts as a "hit"), the current design naturally provides an optimal balance for memory usage. At the opposite site, I don't see how - as a user - I could optimally tune a separate container. > 2. L1 values will not cause eviction of real data -1 That's not a benefit, as I explained above. "Real Data" is not less important, especially if it's never used. Granted I'm making some assumptions about the application having some hot-data and some less hot data, and not being able to take advantage of node pinning or affinity strategies.. but that is another way to say that I'm assuming the user needs L1: if it was possible to apply these more advanced strategies I'd disable L1 altogether.

> 3. Would make https://issues.jboss.org/browse/ISPN-3229 an easy fix > 4. Could add a new DataContainer implementation specific to L1 with > additional optimizations You have some example of what you have in mind? Considering you would need to consider the optimal balance usage of the available heap space, I suspect that would be quite hard. > 5. Help with some concurrency issues with L1 without requiring wider locking > (such as locking a key for an entire ClusteredGet rpc call) - > https://issues.jboss.org/browse/ISPN-3197. I don't understand this. L1 entries require the same level of consistency than any other entry so I suspect you would need the same locking patterns replicated. The downside would be that you get duplication of the same logic. Remember also that L1 is having some similarities with entries still "hanging around" when they where previously stored in this node after a state transfer.. today these are considered L1-active entries, if you change the storage you would need to design a migration of state from one container to the other; migration of state might not be too hard, doing it while guaranteeing consistent locking is going to be I guess as hard as considering the L1 problem today. > > Drawbacks: > 1. Would require, depending on configuration, an additional thread for > eviction > 2. Users upgrading could have double memory used up due to 2 data containers This drawback specifically is to be considered very seriously. I don't think people would be happy to buy and maintain a twice as large datacenter than what they actually need. Sanne > > Both?: > 1. Additional configuration available > a. Add maxEntires just like the normal data container (use data container > size if not configured?) > b. Eviction wakeup timer? We could just reuse the task cleanup > frequency? > c. Eviction strategy? I would think the default data container's would > be sufficient. > > I was wondering what you guys thought. > > Thanks, > > - Will > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 19 June 2013 19:00, Dan Berindei <dan.berindei(a)gmail.com&gt; wrote: > > > > On Wed, Jun 19, 2013 at 5:19 PM, Sanne Grinovero <sanne(a)infinispan.org&gt; > wrote: >> >> On 19 June 2013 13:44, William Burns <mudokonman(a)gmail.com&gt; wrote: >>> All the L1 data for a DIST cache is stored in the same data container as >>> the >>> actual distributed data itself. I wanted to propose breaking this out >>> so >>> there is a separate data container for the L1 cache as compared to the >>> distributed data. >>> >>> I thought of a few quick benefits/drawbacks: >>> >>> Benefits: >>> 1. L1 cache can be separately tuned - L1 maxEntries for example >> >> -1! >> I don't think thats a benefit actually, from the point of view of a >> user: >> as a user I only know I have a certain amount of memory available on >> each node, and the application is going to use certain data way more >> often than others. >> The eviction strategy should be put in condition to be able to make an >> optimal choice about which entries - among all - are better kept in >> memory vs. passivated. >> I don't see a specific reason to "favour" keeping in memory owned >> entries over an L1 entry: the L1 entry might be very hot, and the >> owned entry might be almost-never read. >> >> Considering that even serving a Get operation to another node (as >> owners of the entry) makes the entry less likely to be passivated (it >> counts as a "hit"), the current design naturally provides an optimal >> balance for memory usage. >> >> At the opposite site, I don't see how - as a user - I could optimally >> tune a separate container. >> >>> 2. L1 values will not cause eviction of real data >> >> -1 >> That's not a benefit, as I explained above. "Real Data" is not less >> important, especially if it's never used. >> Granted I'm making some assumptions about the application having some >> hot-data and some less hot data, and not being able to take advantage >> of node pinning or affinity strategies.. but that is another way to >> say that I'm assuming the user needs L1: if it was possible to apply >> these more advanced strategies I'd disable L1 altogether. >> > > You're also assuming that data can always be recreated from another source, > but I don't think that's always the case. If Infinispan is the canonical > data store and there is no cache store, you can't enable eviction for the > "real data" and with a single container you can't enable eviction for the L1 > entries either. Well spotted. True we are making many assumptions, I guess each of us is having some use case in mind and designing for it. While our configuration options are flexible, we can design to provide an optimal structure for all combinations, especially as some configuration options can be mixed in ways which would never be used in a real-world solution. In the specific situation, if I where to use Infinispan as a "canonical data store", and without having a cache store, I think I would at the very least isolate it from its clients by using Hot Rod; in such case L1 would be disabled for obvious reasons.