[infinispan-dev] Eviction thread and purging expired entries

[Mircea Markus]

On 22 Jul 2010, at 17:21, Galder Zamarreño wrote:

> 
> On Jul 22, 2010, at 5:25 PM, Mircea Markus wrote:
> 
>> 
>> On 22 Jul 2010, at 09:11, Galder Zamarreño wrote:
>> 
>>> 
>>> On Jul 22, 2010, at 3:49 AM, Mircea Markus wrote:
>>> 
>>>> 
>>>> On 21 Jul 2010, at 19:29, Galder Zamarreño wrote:
>>>> 
>>>>> 
>>>>> On Jul 21, 2010, at 3:24 PM, Mircea Markus wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> Eviction thread does two things right now:
>>>>>> - evict stuff from DataContainer
>>>>>> - purge entries from a CacheStore
>>>>>> 
>>>>>> CacheStore.purge might slow down eviction as it is generally an expensive operation. It might not even be needed, if the users don't use expiration. 
>>>>> 
>>>>> It's not considered here, but I think it'd be hard to figure out that expiration is not in use at runtime. You'd have to track calls with expiration and make sure there's at least one expiration that has not expired. Probably not practical.
>>>> well spotted! Not hard to implement, and definitely simple for the users, as doesn't need an additional config.
>>> 
>>> Not hard to implement? You'd need a separate collection to track those nodes to which expiration has been set, including those nodes that might have evicted from memory but are still present in the cache store (imagine a node with expire=1 day, but for eviction +passivation reasons, it has been moved to cache store after 1 min). On top of that, you'd need to update the collection everytime a node is expired to remove it from there. Then, in every call to purge entries from a cache store, you'd have to check this collection to see whether it's empty or not to decide whether to call purge entries on cache store. The latter is not difficult, but the tracking can get complicated v easily. IMO, not worth it. 
>> That would be hard to implement indeed :)
>> What I have in mind is: cache store has a a boolean field set to true if it has entries that can expire, false otherwise. It is in full control of the modifications applied to it and it can update this flag accordingly. 
>> if the flag is false, then when asked to purge the store won't do anything. So if one doesn't use expiry at all, it won't even have to configure anything to disable it.
>> I've implemented this for BucketBasedCacheStore and is really simple (attached). 
>> <purging_optimization.patch>
>> 
>> Further on this can be refined: use a counter that gets incremented whenever an expirable entry is added, and decremented when it is purged (not ALL CS will support this).
> 
> That looks indeed simpler although I'm not sure it'll work as expected. When are you planning to set it to needsPurging to false?
never. this would only work if the user doesn't use expiry/purging, as an alternative to using a configuration param that would explicitly disable purging (second solution you suggested).
> 
> If after super.purgeExpired() has been called you knew that no other node is left that can expire, then yes. Otherwise, even after purging expired, there might be some left in the cache store to expire. I think you need more than just a boolean to track this.
For this scenario I would need more, indeed. Something something like an counter, to count number of expirable entries (see my prev email). 
> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev