Guys, A problem with eviction, passivation and the new data container has been reported in ISPN-582.  A quick summary, eviction now takes place within the data container as containers are naturally bounded.  This means once a size threshold is reached entries are automatically removed.  More accurately, they are just stored in a separate collection and the eviction thread picks them off this collection and calls evict() on each one.  Now we have some issues here: 1) This means notifications are often dispatched later than when the actual eviction takes place.  No biggie though, eviction is best-effort. 2)  Passivation too is delayed.  This is huge.  This means there is a window where a thread may ask for an entry but the entry is in limbo - not in the data container, not yet passivated to a cache store. I am currently testing a fix for this, where the data container handles notification + passivation (if enabled) directly, when an entry is evicted from the primary data collection.  So far this seems to work, however this all happens on the caller's thread now.  So in a sense, I wonder if the EvictionThreadPolicy is relevant. However, my fix is only a temporary one - since it still doesn't work when using LIRS, as LIRS maintains 2 separate stacks and only the LIR stack is queried on a get and not the HIR stack.  So for now we should consider LIRS + Passivation as an invalid configuration.        https://jira.jboss.org/browse/ISPN-598 Anyway the key thing is, it all works fine now, so lets open this discussion again when Vladimir is back from vacation.

Guys, A problem with eviction, passivation and the new data container has been reported in ISPN-582. A quick summary, eviction now takes place within the data container as containers are naturally bounded. This means once a size threshold is reached entries are automatically removed. More accurately, they are just stored in a separate collection and the eviction thread picks them off this collection and calls evict() on each one. Now we have some issues here: 1) This means notifications are often dispatched later than when the actual eviction takes place. No biggie though, eviction is best-effort. 2) Passivation too is delayed. This is huge. This means there is a window where a thread may ask for an entry but the entry is in limbo - not in the data container, not yet passivated to a cache store. I am currently testing a fix for this, where the data container handles notification + passivation (if enabled) directly, when an entry is evicted from the primary data collection. So far this seems to work, however this all happens on the caller's thread now. So in a sense, I wonder if the EvictionThreadPolicy is relevant.

However, my fix is only a temporary one - since it still doesn't work when using LIRS, as LIRS maintains 2 separate stacks and only the LIR stack is queried on a get and not the HIR stack. So for now we should consider LIRS + Passivation as an invalid configuration.

On 16 Aug 2010, at 09:22, Vladimir Blagojevic wrote: > Hey, > > Very interesting. So what we essentially have to do is to consider evicted entries to be in container until evict is called? > > On 2010-08-12, at 10:22 AM, Manik Surtani wrote: > >> Guys, >> >> A problem with eviction, passivation and the new data container has been reported in ISPN-582. A quick summary, eviction now takes place within the data container as containers are naturally bounded. This means once a size threshold is reached entries are automatically removed. More accurately, they are just stored in a separate collection and the eviction thread picks them off this collection and calls evict() on each one. Now we have some issues here: >> >> 1) This means notifications are often dispatched later than when the actual eviction takes place. No biggie though, eviction is best-effort. >> 2) Passivation too is delayed. This is huge. This means there is a window where a thread may ask for an entry but the entry is in limbo - not in the data container, not yet passivated to a cache store. >> >> I am currently testing a fix for this, where the data container handles notification + passivation (if enabled) directly, when an entry is evicted from the primary data collection. So far this seems to work, however this all happens on the caller's thread now. So in a sense, I wonder if the EvictionThreadPolicy is relevant. > > > Ok, so if I understand you correctly you are suggesting a removal of default dedicated eviction thread and use only piggyback eviction. That way we easily achieve objective of "consider evicted entries to be in container until evict is called" since we are calling evict from caller's thread rather then allowing a window of vulnerability to open until dedicated eviction thread calls evict. Correct? Yes. However I am not suggesting we *only* go with the piggyback option. It was just that the piggyback option was the easiest/quickest for me to get working again. We should still look at deferring the eviction process/taking it off the critical path while still keeping things coherent. Would need some thought. >> However, my fix is only a temporary one - since it still doesn't work when using LIRS, as LIRS maintains 2 separate stacks and only the LIR stack is queried on a get and not the HIR stack. So for now we should consider LIRS + Passivation as an invalid configuration. > > What do you mean only LIR stack is queried on get? get hashes into segment and if the entry is there in the segment it is returned. Both LIRS structures are only helpers to figure out which entries to remove from that particular segment once execute is called on a segment (execute is called when threshold is reached and can be triggered by either put or get) Hmm, could be something else that is wrong then. Have a look at EvictionWithPassivationTest in trunk and branch/4.1.x and enable all tests (including the LIRS ones that I have disabled). You'll see what I mean. Cheers Manik -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev