[infinispan-dev] New API to iterate over current entries in cache

[William Burns]

On Mon, Mar 17, 2014 at 10:45 AM, Radim Vansa <rvansa at redhat.com> wrote:
> Why listeners are not invoked? JCache iterator() notifies the listeners.

Like I mentioned this can be changed.  However, I have not seen a
cache entry visitor listener in JCache.  The only listeners I am aware
of are for created, removed, expired and updated which wouldn't be
affected by this.

>
> Can the iterator remove entries?

Sorry I forgot to mention this, but no it would not be able to remove
with what I was planning.  This could be added easily, however this
would be problematic for when using repeatable read. Would the remove
take part of the transaction and if so it causes an inconsistency
since we aren't storing all the values that were read.  I figured it
would be easier to just not support it and if the user wants they can
simply call cache.remove(key) which this would do anyways.

>
> I assume there's no ordering guarantee, but behaviour under concurrent
> changes would be rather tricky. I don't like the idea of iterating over
> changing structure.

There are no ordering of elements.  This would have guarantees more
similar to the ConcurrentMap entrySet iteration in that you are
guaranteed to see a value that was valid for the given key that may or
may not be the most up to date.  This would work similarly to how our
Cache works now currently though.

>
> If you need to iterate through all entries, I'd rather introduce the
> snapshot ability and then iterate over the snapshot. Then, you wouldn't
> have to mess with tx and introduce non-tx operation on tx cache.

That would definitely be helpful, but we don't have support for
snapshot atm.  The way this is implemented currently though it would
be pretty simple to add an option for this though, as all you would
need to do is pass along the version of the snapshot in the request
command.

The reason I was saying not to support this for tx right now, is
because of repeatable read, there is no way we can hold all the values
of the cache in the current context.

>
> My 2c
>
> Radim
>
> On 03/17/2014 02:30 PM, William Burns wrote:
>> While working on ISPN-4068 to add the current state to listeners that
>> were added I found that what I essentially needed was a way to iterate
>> over the entries of the cache.  I am thinking of adding this to the
>> public API available on the AdvancedCache interface.
>>
>> I wanted to get your guy's opinions if you don't think we should add
>> it or any changes you might suggest.
>>
>> My thought was to add 2 overloaded methods:
>>
>> <C> Iterator<Map.Entry<K, C>> entryIterator(KeyValueFilter<? super K,
>> ? super V> filter, Converter<? super K, ? super V, C> converter);
>>
>> and
>>
>> Iterator<Map.Entry<K, V>> entryIterator(KeyValueFilter<? super K, ?
>> super V> filter);
>>
>> The method would return almost immediately after invocation and the
>> iterator would queue entries and block as entries are required to be
>> returned.  The filter and converter are applied on each of the remote
>> nodes and are required to be serializable or have an externalizer
>> registered.
>>
>> Internally the iterator would use chunking to help prevent memory
>> saturation.  The max memory usage would be (chunkSize * N) + local
>> entries where N is the number of nodes.
>>
>> These methods would be different than other methods on the
>> Cache/AdvancedCache in the following things:
>>
>> 1. This operation is treated as nontx and thus won't store them into
>> the context and thus repeatable read semantics would not be
>> guaranteed.  This doesn't preclude manually adding values to the
>> context.  Also prior writes in the current context would be ignored
>> (current data returned), although this could be changed if desired.
>> 2. Values are not activated from loaders and visited listeners would
>> not be notified of access.  The latter could be sensibly changed if
>> desired.
>>
>>   - Will
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss DataGrid QA
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev