Re: [infinispan-dev] cache loader/preload

On Fri, Aug 16, 2013 at 5:05 AM, Galder Zamarreño <galder(a)redhat.com <mailto:galder@redhat.com>> wrote: On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism(a)gmail.com <mailto:saturnism@gmail.com>> wrote: > Btw how do we deal with preloading in a cluster? Each node preloads locally only. How is this done? Is it through loadAllKeys first, and then determining which keys are local? Or do you mean it preloads locally from a non-shared cache store? What if the cache store is shared?

> Eg how do we prevent every node from preloading the same data? Preloaded data is not replicated/distributed to other nodes, in order to speed up process. For replicated caches, each node loads its own stuff (from local or shared). For distributed caches, I don't think we're intelligent enough to only load data that belongs to a particular node (I think there's some jiras on this topic…) For replicated cache, when it loads the data, does it put to the local cache container only? (i.e., avoid unnecessarily replicating to other nodes of course).

Cheers, > > On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus(a)redhat.com <mailto:mmarkus@redhat.com>> wrote: > >> >> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org <mailto:sanne@infinispan.org>> wrote: >> >>> On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com <mailto:mmarkus@redhat.com>> wrote: >>>> >>>> On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org <mailto:sanne@infinispan.org>> wrote: >>>> >>>>> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com <mailto:mmarkus@redhat.com>> wrote: >>>>>> Currently the "preload" attribute is configured at *loaders* level and the first cache loader in the chain is used for preloading. >>>>>> A better place for preload might be the individual loader element: would indicate clearly which loader is used for preloading and also consistent with the fetchPeristentState which is again defined on a per loader basis. >>>>>> >>>>>> How does it sound? >>>>> >>>>> Very confusing actually :-) >>>>> Are you going to prevent me to enable "preload" on multiple Cache >>>>> Loaders elements? >>>> >>>> I'm not going to take anything from you, this is something you're not allowed to do ATM :-) >>> >>> That's exactly my point. I'm not allowed to do it ATM, but if you >>> change the position of this attribute to *each* loader it will look >>> like I could enable/disable it differently on a per-instance. >> >> Gotcha, I'll either add a check to allow specifying preload on a single store, or (if time allows) I'll implement the multiple preload feature. >> >>> >>>> >>>>> And what if I'm actually storing entries in >>>>> different CacheStore instanced based on some discriminator, like the >>>>> value (key) type? >>>>> >>>>> To make a practical example, the Lucene Directory CacheLoader only >>>>> loads some specific types, and only from named indexes which match a >>>>> configured name. You could want multiple instances using different >>>>> configurations, so that for each index name stored in the same cache >>>>> you actually store them in different cachestores. >>>>> >>>>> Similar situation, with the JPA CacheStore you might want to use >>>>> multiple independent instances, one per mapped type. Not necessarily >>>>> needed (one could improve the CacheStore implementation as well) but >>>>> it gives lot of flexibility, like connecting to different RDBMs >>>>> instances or with different credentials/schemas, etc... >>>> >>>> Indeed user might want this in the future. Moving "preload" at "loader" level would only facilitate that. >>> >>> Right. I'm making those examples to highlight the benefit of moving >>> the attribute to each cachestore, as you proposed, but this only makes >>> sense if you actually allow the option to be applied differently for >>> each instance. >> >> Cheers, >> -- >> Mircea Markus >> Infinispan lead (www.infinispan.org <http://www.infinispan.org>) >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño galder(a)redhat.com <mailto:galder@redhat.com> twitter.com/galderz <http://twitter.com/galderz> Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev