[infinispan-dev] Fwd: CloudCacheStore Bug

[Philippe Van Dyck]

Anyway, bad news, still have the problem without async and
with purgeSynchronously="true".

It is easy to test, create a transaction with 100000 updates (file store)
and use a maxEntries of "2" for the eviction.

While the transaction is being committed, the eviction thread wakes up and
deletes entries.

I don't think this behavior is intended (?)

phil

On Thu, Feb 4, 2010 at 6:19 PM, Philippe Van Dyck <pvdyck at gmail.com> wrote:

> Totally agree... as long as any failing async transaction is logged...
>
> BTW, since none of my cache entries expire, they are all instances of
> ImmortalCacheEntry.
> But since ImmortalCacheEntry does not update the "lastUsed" field, LRU or
> FIFO are useless eviction strategies...
>
> My own eviction strategy, getting rid of a % of the size of the cache in
> memory -> LRU first, does not work...
>
> Any idea ? Should I use my own timer (even if there is one in
> InternalCacheEntry) ?
>
> cheers,
>
> phil
>
> On Thu, Feb 4, 2010 at 5:49 PM, Manik Surtani <manik at jboss.org> wrote:
>
>>
>> On 4 Feb 2010, at 16:27, Philippe Van Dyck wrote:
>>
>>
>>
>>> Am I missing something ? Loosing data is something I cannot afford ! I
>>> Plan to use this store as a *permanent* one... I have no backup ! (Actually
>>> S3 is the backup) - So, no, I don't want this ... at any price ;-)
>>>
>>>
>>> Then set <async enabled="false" /> in your cache store config.  :-)
>>>
>>
>> That is exactly what I planned to do... for the FileCacheStore since the
>> latency is quite low and the failure rate almost zero.
>> But the S3 store is very slow, and asynchronism is not a luxury...
>>
>> Right now, I am trying to make my own custom solution based on the size of
>> the cache in memory (as trigger) and then I will evict specific oldest
>> entries... hoping that async transactions are fully committed.
>>
>>
>>>
>>>
>>>
>>>> reduced by looking through the async queue as well, before checking the
>>>> underlying store.  But as I said, this just reduces the size of this window
>>>> and not eliminate it altogether, since this is async and there is no
>>>> guarantee that the cache store has finished writing internally (e.g., an
>>>> fsync() operation or in the case of S3, Amazon's eventual consistency
>>>> model).
>>>>
>>>>
>>>> Why should eviction be transactional?  I don't need eviction to be an
>>>> all-or-nothing, reversible event. :)  If an entry gets evicted, cool.  If
>>>> not (for whatever reason), too bad, move on to the next evictable entry.
>>>>
>>>
>>> You are right, we don't want to rollback evictions... but maybe we should
>>> use a priority queue to be sure that evictions are done after any other
>>> command ? Doesn't it solve it all ?
>>>
>>> 1) The eviction thread runs (we could lower the priority of this thread
>>> too)
>>> 2) It fills a queue of keys to evict
>>> 3) The async queue is prioritized and evicts entries ... when there is
>>> nothing else to do (suddenly it looks like garbage collecting)
>>>
>>>
>>> That is a possibility.  But I don't expect to be making any drastic
>>> changes to the existing eviction code anymore.  Don't know if you have been
>>> following discussions re: LIRS, lock amortization, etc., but Vladimir is
>>> working on some very interesting self-evicting, bounded data containers
>>> which would mean that the eviction threads, etc all get ripped out.
>>>
>>>
>> Sounds terrific...
>>
>> Just to close the subject, shouldn't the documentation explicitly say that
>> async and eviction are not "compatible" ?
>>
>>
>> I don't think this really has anything to do with "incompatibilities".
>>  It's just the effects of queued/batched processing in the cache store async
>> threads.  You will see the same problem if you:
>>
>> 1.  put (K, V)
>> 2.  The put is enqueued in the cache store
>> 3.  Restart the cache
>> 4.  get (K) // Data loss!?  Just an async write that didn't have time to
>> complete.
>>
>> And the above has nothing to do with eviction.
>>
>> Cheers
>> Manik
>>  --
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100204/4d39b5e1/attachment-0002.html