[infinispan-dev] cache loader/preload

[Galder Zamarreño]

On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism at gmail.com> wrote:

> Btw how do we deal with preloading in a cluster?

Each node preloads locally only.

> Eg how do we prevent every node from preloading the same data?

Preloaded data is not replicated/distributed to other nodes, in order to speed up process. For replicated caches, each node loads its own stuff (from local or shared). For distributed caches, I don't think we're intelligent enough to only load data that belongs to a particular node (I think there's some jiras on this topic…)

Cheers,

> 
> On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus at redhat.com> wrote:
> 
>> 
>> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne at infinispan.org> wrote:
>> 
>>> On 14 August 2013 17:59, Mircea Markus <mmarkus at redhat.com> wrote:
>>>> 
>>>> On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne at infinispan.org> wrote:
>>>> 
>>>>> On 14 August 2013 11:15, Mircea Markus <mmarkus at redhat.com> wrote:
>>>>>> Currently the "preload" attribute is configured at *loaders* level and the first cache loader in the chain is used for preloading.
>>>>>> A better place for preload might be the individual loader element: would indicate clearly which loader is used for preloading and also consistent with the fetchPeristentState which is again defined on a per loader basis.
>>>>>> 
>>>>>> How does it sound?
>>>>> 
>>>>> Very confusing actually :-)
>>>>> Are you going to prevent me to enable "preload" on multiple Cache
>>>>> Loaders elements?
>>>> 
>>>> I'm not going to take anything from you, this is something you're not allowed to do ATM :-)
>>> 
>>> That's exactly my point. I'm not allowed to do it ATM, but if you
>>> change the position of this attribute to *each* loader it will look
>>> like I could enable/disable it differently on a per-instance.
>> 
>> Gotcha, I'll either add a check to allow specifying preload on a single store, or (if time allows) I'll implement the multiple preload feature.
>> 
>>> 
>>>> 
>>>>> And what if I'm actually storing entries in
>>>>> different CacheStore instanced based on some discriminator, like the
>>>>> value (key) type?
>>>>> 
>>>>> To make a practical example, the Lucene Directory CacheLoader only
>>>>> loads some specific types, and only from named indexes which match a
>>>>> configured name. You could want multiple instances using different
>>>>> configurations, so that for each index name stored in the same cache
>>>>> you actually store them in different cachestores.
>>>>> 
>>>>> Similar situation, with the JPA CacheStore you might want to use
>>>>> multiple independent instances, one per mapped type. Not necessarily
>>>>> needed (one could improve the CacheStore implementation as well) but
>>>>> it gives lot of flexibility, like connecting to different RDBMs
>>>>> instances or with different credentials/schemas, etc...
>>>> 
>>>> Indeed user might want this in the future. Moving "preload" at "loader" level would only facilitate that.
>>> 
>>> Right. I'm making those examples to highlight the benefit of moving
>>> the attribute to each cachestore, as you proposed, but this only makes
>>> sense if you actually allow the option to be applied differently for
>>> each instance.
>> 
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org