[infinispan-dev] Fwd: CloudCacheStore Bug

[Manik Surtani]

On 4 Feb 2010, at 17:36, Philippe Van Dyck wrote:

> Anyway, bad news, still have the problem without async and with purgeSynchronously="true".
> 
> It is easy to test, create a transaction with 100000 updates (file store) and use a maxEntries of "2" for the eviction.
> 
> While the transaction is being committed, the eviction thread wakes up and deletes entries.
> 
> I don't think this behavior is intended (?)

Right, similar to using an async queue (which in effect is a batch write to the store), a transaction too is a batch write to the store when the transaction completes.  So when the tx does finish, writes are flushed.  prior to that, concurrent threads not seeing the entry is expected since we don't support READ_UNCOMMITTED semantics.

> 
> phil
> 
> On Thu, Feb 4, 2010 at 6:19 PM, Philippe Van Dyck <pvdyck at gmail.com> wrote:
> Totally agree... as long as any failing async transaction is logged...
> 
> BTW, since none of my cache entries expire, they are all instances of ImmortalCacheEntry.
> But since ImmortalCacheEntry does not update the "lastUsed" field, LRU or FIFO are useless eviction strategies...
> 
> My own eviction strategy, getting rid of a % of the size of the cache in memory -> LRU first, does not work...
> 
> Any idea ? Should I use my own timer (even if there is one in InternalCacheEntry) ?
> 
> cheers,
> 
> phil  
> 
> On Thu, Feb 4, 2010 at 5:49 PM, Manik Surtani <manik at jboss.org> wrote:
> 
> On 4 Feb 2010, at 16:27, Philippe Van Dyck wrote:
> 
>> 
>>> 
>>> Am I missing something ? Loosing data is something I cannot afford ! I Plan to use this store as a *permanent* one... I have no backup ! (Actually S3 is the backup) - So, no, I don't want this ... at any price ;-)
>> 
>> Then set <async enabled="false" /> in your cache store config.  :-)
>> 
>> That is exactly what I planned to do... for the FileCacheStore since the latency is quite low and the failure rate almost zero.
>> But the S3 store is very slow, and asynchronism is not a luxury...
>> 
>> Right now, I am trying to make my own custom solution based on the size of the cache in memory (as trigger) and then I will evict specific oldest entries... hoping that async transactions are fully committed.
>>  
>> 
>>>  
>>> reduced by looking through the async queue as well, before checking the underlying store.  But as I said, this just reduces the size of this window and not eliminate it altogether, since this is async and there is no guarantee that the cache store has finished writing internally (e.g., an fsync() operation or in the case of S3, Amazon's eventual consistency model).
>>> 
>>> 
>>> Why should eviction be transactional?  I don't need eviction to be an all-or-nothing, reversible event. :)  If an entry gets evicted, cool.  If not (for whatever reason), too bad, move on to the next evictable entry.  
>>> 
>>> You are right, we don't want to rollback evictions... but maybe we should use a priority queue to be sure that evictions are done after any other command ? Doesn't it solve it all ? 
>>> 
>>> 1) The eviction thread runs (we could lower the priority of this thread too)
>>> 2) It fills a queue of keys to evict
>>> 3) The async queue is prioritized and evicts entries ... when there is nothing else to do (suddenly it looks like garbage collecting)
>> 
>> That is a possibility.  But I don't expect to be making any drastic changes to the existing eviction code anymore.  Don't know if you have been following discussions re: LIRS, lock amortization, etc., but Vladimir is working on some very interesting self-evicting, bounded data containers which would mean that the eviction threads, etc all get ripped out.
>> 
>> 
>> Sounds terrific... 
>> 
>> Just to close the subject, shouldn't the documentation explicitly say that async and eviction are not "compatible" ?
> 
> I don't think this really has anything to do with "incompatibilities".  It's just the effects of queued/batched processing in the cache store async threads.  You will see the same problem if you:
> 
> 1.  put (K, V)
> 2.  The put is enqueued in the cache store
> 3.  Restart the cache 
> 4.  get (K) // Data loss!?  Just an async write that didn't have time to complete.
> 
> And the above has nothing to do with eviction.
> 
> Cheers
> Manik
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100204/3070a764/attachment-0002.html