[infinispan-dev] [ISPN-116] Async cache store: aggregation of multiple changes on a single key

[Mircea Markus]

Galder Zamarreno wrote:
> Hi,
>
> Re: https://jira.jboss.org/jira/browse/ISPN-116
>
> I can see two ways of providing such feature:
>
> 1. Rather than using a queue, using a data structure similar to the 
> one used in the data container so that when a newer value for a key 
> already present in the queue is present, the value can be swapped (map 
> like lookup required on the key so that O(1) is maintained) while 
> maintaining queue-like FIFO operations required to empty it.
Is the FIFO order required? We apply the 'final state' when we do the 
flush of the structure, I don't think the order of the operation is 
relevant.
If this is the case, our structure might be a Map, with null values for 
removed keys, as a convention.
>
> 2. Based on suggestion coming amin59 in 
> http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4240980#4240980, 
> make the current queue only store keys and make the draining process 
> query the cache for the latest value. Because from the time the data 
> was put to the cache to the time it's stored, the data could have 
> expired or had been removed all together, such query would need to 
> make sure it doesn't go to the cache loader again. Also, if the data 
> had been removed, how would the Async store know that from 
> cache.get()? The queue would need to record somehow that a key was 
> actually removed.
>
> The benefit of the 1st option is that we can take advantage of fast 
> drainTo() like operations when draining it, and also the fact that no 
> contention is added to the cache itself. The downside would be the 
> need for a more complex data structure.
>
> The benefit of the 2nd option is the keeping a simple queue, less 
> complex data structure, but requires that each time a key is to be 
> drained, we have to query the cache. This could slow down the draining 
> process.
draining process being slowed down is not critical, as it is performed 
async.
>
> My preference is with 1. but for to use it, I'd need the data 
> container collection to be made more generic.
>
> Thoughts?