[infinispan-dev] [ISPN-116] Async cache store: aggregation of multiple changes on a single key

[Galder Zamarreno]

On 07/04/2009 08:12 AM, Amin Abbaspour wrote:
> Hi,
>
> It would be great if both of these are present and developer decides
> which one to use. In many cases, the custom logic of the application
> can perform better then a general purpose data structure to aggregate
> changes to a single value.

Please try to keep the infinispan developer list cc'd so that we can 
discuss the different options among every one.

Hmmm, on one side, I'm not sure of the need of implementing the two 
methods. I think we need simply one that performs best.

Also, I'm not sure I understand the need for the application to apply 
any logic here. The application has likely made several put operations 
on a key and so the most likely outcome is the latter one should be 
applied, unless there has been some kind of cluster partition but this 
is a separate concern.

> If every change and its coalition process is running inside a
> transactional block, then this custom logic is quite feasible and
> safe.

Again, this seems to be separate concern that addresses how to deal with 
merging of cache state when there has been some kind of network 
partition. This won't be resolved at the cache loader level though.

>
> IMHO both 1st and 2nd options require similar changes to code in
> persistence mechanism, I think we'd better implement simple 2nd option
> first and then switch to enhanced version with 1st option.
>
> Regards
> Amin
>
> On 6/30/09, Galder Zamarreno<galder.zamarreno at redhat.com>  wrote:
>> Hi,
>>
>> Re: https://jira.jboss.org/jira/browse/ISPN-116
>>
>> I can see two ways of providing such feature:
>>
>> 1. Rather than using a queue, using a data structure similar to the one
>> used in the data container so that when a newer value for a key already
>> present in the queue is present, the value can be swapped (map like
>> lookup required on the key so that O(1) is maintained) while maintaining
>> queue-like FIFO operations required to empty it.
>>
>> 2. Based on suggestion coming amin59 in
>> http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4240980#4240980,
>> make
>> the current queue only store keys and make the draining process query
>> the cache for the latest value. Because from the time the data was put
>> to the cache to the time it's stored, the data could have expired or had
>> been removed all together, such query would need to make sure it doesn't
>> go to the cache loader again. Also, if the data had been removed, how
>> would the Async store know that from cache.get()? The queue would need
>> to record somehow that a key was actually removed.
>>
>> The benefit of the 1st option is that we can take advantage of fast
>> drainTo() like operations when draining it, and also the fact that no
>> contention is added to the cache itself. The downside would be the need
>> for a more complex data structure.
>>
>> The benefit of the 2nd option is the keeping a simple queue, less
>> complex data structure, but requires that each time a key is to be
>> drained, we have to query the cache. This could slow down the draining
>> process.
>>
>> My preference is with 1. but for to use it, I'd need the data container
>> collection to be made more generic.
>>
>> Thoughts?
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>

-- 
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache