[infinispan-dev] BulkGet in hotrod (ISPN-516)

[Manik Surtani]

On 14 Jul 2010, at 15:29, Galder Zamarreño wrote:

> 
> On Jul 14, 2010, at 3:08 PM, Manik Surtani wrote:
> 
>> 
>> On 14 Jul 2010, at 13:23, Mircea Markus wrote:
>> 
>>> thanks, see bellow.
>>> On 14 Jul 2010, at 11:49, Galder Zamarreño wrote:
>>> 
>>>> Hi Mircea,
>>>> 
>>>> You should make it explicit in the title that this is a design document to avoid confusion.
>>> done
>>>> I'm not sure I understand what the filter and exclusion parameters are about.
>>> Filtering is about excluding from the set of entries returned by the server to the client of those filtering keys specified in the request (filter size] [filter key size 1] [filter key 1] [filter key size 2] [filter key 2]...).
>>>> A multi get, as I see is about giving a list of keys and returning them. So, why use filtering?
>>> We need filtering because of RemoteCacheStore: it needs to implement  CacheLoader.loadAllKeys(Set<Object> keysToExclude)
>> 
>> Like I said (on the wiki) - this may not be needed.  See my reasons there.
> 
> In which case, I only see an immediate need for a getAll operation where all keys are retrieved. 

You still need to be able to provide a limit of how many entries to retrieve.  Instant OOM otherwise.  :)

> I'm still not convinced for the need of a multi get op. Multi gets on a subset could be done by the client parallelizing/pipelining individual get requests. You could also do multi-gets doing async or non-blocking gets. IOW, if you want to get 10 keys, do 10 non-blocking gets and then wait for the results of all of them and it's here where netty based client could be more performant. This way you don't have to wait for each response before moving on and this is simpler than paralelizing. 

I think you've misunderstood the use case, Galder.  This is BULK_GET operation is not the same as memcached's multi_get.  The client has no knowledge of the keys in question.  The client is just interested in retrieving N entries, presumably for a warm cache backed by a RemoteCacheStore.  See the JIRA for more details/examples.  :)

> Very interesting stuff to read in http://code.google.com/p/memcached/wiki/FAQ#Batch_your_requests_with_get_multi and http://lists.danga.com/pipermail/memcached/2007-July/004528.html

This is generally interesting as well, and probably should be discussed on the HotRod client wiki page as a pattern, but is generally irrelevant to ISPN-516.

Cheers
Manik

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org