[infinispan-dev] cache loader/preload

[Pedro Ruivo]

On 08/16/2013 07:51 PM, Ray Tsang wrote:
>
>
> On Fri, Aug 16, 2013 at 5:05 AM, Galder Zamarreño <galder at redhat.com
> <mailto:galder at redhat.com>> wrote:
>
>
>     On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism at gmail.com
>     <mailto:saturnism at gmail.com>> wrote:
>
>      > Btw how do we deal with preloading in a cluster?
>
>     Each node preloads locally only.
>
>
> How is this done?  Is it through loadAllKeys first, and then determining
> which keys are local?
> Or do you mean it preloads locally from a non-shared cache store?  What
> if the cache store is shared?

the behaviour is the same for non-shared/shared caches. Each node loads 
all the data from the cache store and puts it in their local container 
(without any replication/distribution neither any communication with 
other nodes).

>
>
>      > Eg how do we prevent every node from preloading the same data?
>
>     Preloaded data is not replicated/distributed to other nodes, in
>     order to speed up process. For replicated caches, each node loads
>     its own stuff (from local or shared). For distributed caches, I
>     don't think we're intelligent enough to only load data that belongs
>     to a particular node (I think there's some jiras on this topic…)
>
>
> For replicated cache, when it loads the data, does it put to the local
> cache container only? (i.e., avoid unnecessarily replicating to other
> nodes of course).

yes

Cheers,
Pedro

>
>
>     Cheers,
>
>      >
>      > On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus at redhat.com
>     <mailto:mmarkus at redhat.com>> wrote:
>      >
>      >>
>      >> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne at infinispan.org
>     <mailto:sanne at infinispan.org>> wrote:
>      >>
>      >>> On 14 August 2013 17:59, Mircea Markus <mmarkus at redhat.com
>     <mailto:mmarkus at redhat.com>> wrote:
>      >>>>
>      >>>> On 14 Aug 2013, at 15:26, Sanne Grinovero
>     <sanne at infinispan.org <mailto:sanne at infinispan.org>> wrote:
>      >>>>
>      >>>>> On 14 August 2013 11:15, Mircea Markus <mmarkus at redhat.com
>     <mailto:mmarkus at redhat.com>> wrote:
>      >>>>>> Currently the "preload" attribute is configured at *loaders*
>     level and the first cache loader in the chain is used for preloading.
>      >>>>>> A better place for preload might be the individual loader
>     element: would indicate clearly which loader is used for preloading
>     and also consistent with the fetchPeristentState which is again
>     defined on a per loader basis.
>      >>>>>>
>      >>>>>> How does it sound?
>      >>>>>
>      >>>>> Very confusing actually :-)
>      >>>>> Are you going to prevent me to enable "preload" on multiple Cache
>      >>>>> Loaders elements?
>      >>>>
>      >>>> I'm not going to take anything from you, this is something
>     you're not allowed to do ATM :-)
>      >>>
>      >>> That's exactly my point. I'm not allowed to do it ATM, but if you
>      >>> change the position of this attribute to *each* loader it will look
>      >>> like I could enable/disable it differently on a per-instance.
>      >>
>      >> Gotcha, I'll either add a check to allow specifying preload on a
>     single store, or (if time allows) I'll implement the multiple
>     preload feature.
>      >>
>      >>>
>      >>>>
>      >>>>> And what if I'm actually storing entries in
>      >>>>> different CacheStore instanced based on some discriminator,
>     like the
>      >>>>> value (key) type?
>      >>>>>
>      >>>>> To make a practical example, the Lucene Directory CacheLoader
>     only
>      >>>>> loads some specific types, and only from named indexes which
>     match a
>      >>>>> configured name. You could want multiple instances using
>     different
>      >>>>> configurations, so that for each index name stored in the
>     same cache
>      >>>>> you actually store them in different cachestores.
>      >>>>>
>      >>>>> Similar situation, with the JPA CacheStore you might want to use
>      >>>>> multiple independent instances, one per mapped type. Not
>     necessarily
>      >>>>> needed (one could improve the CacheStore implementation as
>     well) but
>      >>>>> it gives lot of flexibility, like connecting to different RDBMs
>      >>>>> instances or with different credentials/schemas, etc...
>      >>>>
>      >>>> Indeed user might want this in the future. Moving "preload" at
>     "loader" level would only facilitate that.
>      >>>
>      >>> Right. I'm making those examples to highlight the benefit of moving
>      >>> the attribute to each cachestore, as you proposed, but this
>     only makes
>      >>> sense if you actually allow the option to be applied
>     differently for
>      >>> each instance.
>      >>
>      >> Cheers,
>      >> --
>      >> Mircea Markus
>      >> Infinispan lead (www.infinispan.org <http://www.infinispan.org>)
>      >>
>      >>
>      >>
>      >>
>      >>
>      >> _______________________________________________
>      >> infinispan-dev mailing list
>      >> infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>      >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>      > _______________________________________________
>      > infinispan-dev mailing list
>      > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>      > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>     --
>     Galder Zamarreño
>     galder at redhat.com <mailto:galder at redhat.com>
>     twitter.com/galderz <http://twitter.com/galderz>
>
>     Project Lead, Escalante
>     http://escalante.io
>
>     Engineer, Infinispan
>     http://infinispan.org
>
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>