[infinispan-dev] BulkGet in hotrod (ISPN-516)

[Galder Zamarreño]

On Jul 14, 2010, at 4:36 PM, Manik Surtani wrote:

> 
> On 14 Jul 2010, at 15:29, Galder Zamarreño wrote:
> 
>> 
>> On Jul 14, 2010, at 3:08 PM, Manik Surtani wrote:
>> 
>>> 
>>> On 14 Jul 2010, at 13:23, Mircea Markus wrote:
>>> 
>>>> thanks, see bellow.
>>>> On 14 Jul 2010, at 11:49, Galder Zamarreño wrote:
>>>> 
>>>>> Hi Mircea,
>>>>> 
>>>>> You should make it explicit in the title that this is a design document to avoid confusion.
>>>> done
>>>>> I'm not sure I understand what the filter and exclusion parameters are about.
>>>> Filtering is about excluding from the set of entries returned by the server to the client of those filtering keys specified in the request (filter size] [filter key size 1] [filter key 1] [filter key size 2] [filter key 2]...).
>>>>> A multi get, as I see is about giving a list of keys and returning them. So, why use filtering?
>>>> We need filtering because of RemoteCacheStore: it needs to implement  CacheLoader.loadAllKeys(Set<Object> keysToExclude)
>>> 
>>> Like I said (on the wiki) - this may not be needed.  See my reasons there.
>> 
>> In which case, I only see an immediate need for a getAll operation where all keys are retrieved. 
> 
> You still need to be able to provide a limit of how many entries to retrieve.  Instant OOM otherwise.  :)
> 
>> I'm still not convinced for the need of a multi get op. Multi gets on a subset could be done by the client parallelizing/pipelining individual get requests. You could also do multi-gets doing async or non-blocking gets. IOW, if you want to get 10 keys, do 10 non-blocking gets and then wait for the results of all of them and it's here where netty based client could be more performant. This way you don't have to wait for each response before moving on and this is simpler than paralelizing. 
> 
> I think you've misunderstood the use case, Galder.  This is BULK_GET operation is not the same as memcached's multi_get.  The client has no knowledge of the keys in question.  The client is just interested in retrieving N entries, presumably for a warm cache backed by a RemoteCacheStore.  See the JIRA for more details/examples.  :)

Right, the JIRA is much clearer on the requirements. Reading the wiki without the JIRA link, it appeared to me that we're trying to do a multi get.

Mircea, maybe you can add a note in the introduction to avoid confusion with multi-get and a link to the JIRA?

I'm not sure this is related to the target use case but I was wondering the following: If a client asks for N keys, what N keys will it get? The first N keys as per a particular order or just random? IOW, are there any guarantees to what keys it'll get if the client does two get N keys one after the other? According to the CacheLoader interface, no guarantees are provided. Just double checking.

> 
>> Very interesting stuff to read in http://code.google.com/p/memcached/wiki/FAQ#Batch_your_requests_with_get_multi and http://lists.danga.com/pipermail/memcached/2007-July/004528.html
> 
> This is generally interesting as well, and probably should be discussed on the HotRod client wiki page as a pattern, but is generally irrelevant to ISPN-516.

+1 on adding these kind of patterns to Hot Rod client wiki. In fact, I can probably do that as part of https://jira.jboss.org/browse/ISPN-366 cos I was planning to add clearer info on the versioned API, probably using the info in the Hot Rod preso for JUDCon as a way to explain the underlying motivations for this kind of API.

> 
> Cheers
> Manik
> 
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache