[infinispan-dev] L1 Data Container

[Pedro Ruivo]

Hi

+1 :)

some comments inline.

Pedro

On 06/19/2013 02:16 PM, Manik Surtani wrote:
> I have often thought of this.  Comments inline:
>
> On 19 Jun 2013, at 13:44, William Burns <mudokonman at gmail.com
> <mailto:mudokonman at gmail.com>> wrote:
>
>> All the L1 data for a DIST cache is stored in the same data container
>> as the actual distributed data itself.  I wanted to propose breaking
>> this out so there is a separate data container for the L1 cache as
>> compared to the distributed data.
>>
>> I thought of a few quick benefits/drawbacks:
>>
>> Benefits:
>> 1. L1 cache can be separately tuned - L1 maxEntries for example

+1 for the maxEntries. I think the L1 should be always bounded to avoid 
any memory issues

>> 2. L1 values will not cause eviction of real data
>> 3. Would make https://issues.jboss.org/browse/ISPN-3229 an easy fix
>> 4. Could add a new DataContainer implementation specific to L1 with
>> additional optimizations
>> 5. Help with some concurrency issues with L1 without requiring wider
>> locking (such as locking a key for an entire ClusteredGet rpc call) -
>> https://issues.jboss.org/browse/ISPN-3197.
>>
>> Drawbacks:
>> 1. Would require, depending on configuration, an additional thread for
>> eviction
>
> Not really - the same eviction thread used for the main data container
> could also clean things up in the L1 DC.
>
>> 2. Users upgrading could have double memory used up due to 2 data
>> containers
>
> This is just a fixed (and small) overhead.  Not actually double the
> memory used by the entire grid.

agree.

>
> Additional drawbacks:
> - More complex lookups in the DC.  Code that would otherwise just call
> DC.get() would now need to also check L1DC.  This would be scattered all
> over the code-base - unless perhaps the main DC would implicitly also
> check the contents of the L1DC (if one is available).

I don't think so. We should only query the L1DC just before performing 
the remote get. This way, only the DistributionInterceptor (and 
sub-classes) will keep a reference to L1DC to query, store, etc.

possible advantages:

* use it own lock manager
* try reduce some network traffic by keep tracking the remote gets and 
if the key requested does not exist in the L1DC but another thread 
already made the remote get, it could skip a second remote get... this 
could be discussed later...

>
>>
>> Both?:
>> 1. Additional configuration available
>>    a. Add maxEntires just like the normal data container (use data
>> container size if not configured?)
>>    b. Eviction wakeup timer?  We could just reuse the task cleanup
>> frequency?
>>    c. Eviction strategy?  I would think the default data container's
>> would be sufficient.

for the beginning the current data container should work fine. But in 
the future, if we found that some parts can be optimized for the L1 uses 
cases, I think an optimized DC would be better.
>>
>> I was wondering what you guys thought.
>>
>> Thanks,
>>
>>  - Will
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik at jboss.org <mailto:manik at jboss.org>
> twitter.com/maniksurtani <http://twitter.com/maniksurtani>
>
> Platform Architect, JBoss Data Grid
> http://red.ht/data-grid
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>