12:15 p.m.
Currently the "preload" attribute is configured at *loaders* level and the first
cache loader in the chain is used for preloading.
A better place for preload might be the individual loader element: would indicate clearly
which loader is used for preloading and also consistent with the fetchPeristentState which
is again defined on a per loader basis.
How does it sound?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
4:26 p.m.
On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com> wrote:
Currently the "preload" attribute is configured at
*loaders* level and the first cache loader in the chain is used for preloading.
A better place for preload might be the individual loader element: would indicate clearly
which loader is used for preloading and also consistent with the fetchPeristentState which
is again defined on a per loader basis.
How does it sound?
Very confusing actually :-)
Are you going to prevent me to enable "preload" on multiple Cache
Loaders elements? And what if I'm actually storing entries in
different CacheStore instanced based on some discriminator, like the
value (key) type?
To make a practical example, the Lucene Directory CacheLoader only
loads some specific types, and only from named indexes which match a
configured name. You could want multiple instances using different
configurations, so that for each index name stored in the same cache
you actually store them in different cachestores.
Similar situation, with the JPA CacheStore you might want to use
multiple independent instances, one per mapped type. Not necessarily
needed (one could improve the CacheStore implementation as well) but
it gives lot of flexibility, like connecting to different RDBMs
instances or with different credentials/schemas, etc...
Cheers,
Sanne
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
6:59 p.m.
On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org> wrote:
On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com>
wrote:
> Currently the "preload" attribute is configured at *loaders* level and the
first cache loader in the chain is used for preloading.
> A better place for preload might be the individual loader element: would indicate
clearly which loader is used for preloading and also consistent with the
fetchPeristentState which is again defined on a per loader basis.
>
> How does it sound?
Very confusing actually :-)
Are you going to prevent me to enable "preload" on multiple Cache
Loaders elements?
I'm not going to take anything from you, this is something you're not allowed to
do ATM :-)
And what if I'm actually storing entries in
different CacheStore instanced based on some discriminator, like the
value (key) type?
To make a practical example, the Lucene Directory CacheLoader only
loads some specific types, and only from named indexes which match a
configured name. You could want multiple instances using different
configurations, so that for each index name stored in the same cache
you actually store them in different cachestores.
Similar situation, with the JPA CacheStore you might want to use
multiple independent instances, one per mapped type. Not necessarily
needed (one could improve the CacheStore implementation as well) but
it gives lot of flexibility, like connecting to different RDBMs
instances or with different credentials/schemas, etc...
Indeed user might want this in the future. Moving "preload" at
"loader" level would only facilitate that.
Cheers,
Sanne
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
9:53 p.m.
On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com> wrote:
On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org> wrote:
> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com> wrote:
>> Currently the "preload" attribute is configured at *loaders* level and
the first cache loader in the chain is used for preloading.
>> A better place for preload might be the individual loader element: would indicate
clearly which loader is used for preloading and also consistent with the
fetchPeristentState which is again defined on a per loader basis.
>>
>> How does it sound?
>
> Very confusing actually :-)
> Are you going to prevent me to enable "preload" on multiple Cache
> Loaders elements?
I'm not going to take anything from you, this is something you're not allowed to
do ATM :-)
That's exactly my point. I'm not allowed to do it ATM, but if you
change the position of this attribute to *each* loader it will look
like I could enable/disable it differently on a per-instance.
> And what if I'm actually storing entries in
> different CacheStore instanced based on some discriminator, like the
> value (key) type?
>
> To make a practical example, the Lucene Directory CacheLoader only
> loads some specific types, and only from named indexes which match a
> configured name. You could want multiple instances using different
> configurations, so that for each index name stored in the same cache
> you actually store them in different cachestores.
>
> Similar situation, with the JPA CacheStore you might want to use
> multiple independent instances, one per mapped type. Not necessarily
> needed (one could improve the CacheStore implementation as well) but
> it gives lot of flexibility, like connecting to different RDBMs
> instances or with different credentials/schemas, etc...
Indeed user might want this in the future. Moving "preload" at
"loader" level would only facilitate that.
Right. I'm making those examples to highlight the benefit of moving
the attribute to each cachestore, as you proposed, but this only makes
sense if you actually allow the option to be applied differently for
each instance.
>
> Cheers,
> Sanne
>
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
10:09 p.m.
On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org> wrote:
On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com>
wrote:
>
> On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org> wrote:
>
>> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>> Currently the "preload" attribute is configured at *loaders* level
and the first cache loader in the chain is used for preloading.
>>> A better place for preload might be the individual loader element: would
indicate clearly which loader is used for preloading and also consistent with the
fetchPeristentState which is again defined on a per loader basis.
>>>
>>> How does it sound?
>>
>> Very confusing actually :-)
>> Are you going to prevent me to enable "preload" on multiple Cache
>> Loaders elements?
>
> I'm not going to take anything from you, this is something you're not allowed
to do ATM :-)
That's exactly my point. I'm not allowed to do it ATM, but if you
change the position of this attribute to *each* loader it will look
like I could enable/disable it differently on a per-instance.
Gotcha, I'll either add a check to allow specifying preload on a single store, or (if
time allows) I'll implement the multiple preload feature.
>
>> And what if I'm actually storing entries in
>> different CacheStore instanced based on some discriminator, like the
>> value (key) type?
>>
>> To make a practical example, the Lucene Directory CacheLoader only
>> loads some specific types, and only from named indexes which match a
>> configured name. You could want multiple instances using different
>> configurations, so that for each index name stored in the same cache
>> you actually store them in different cachestores.
>>
>> Similar situation, with the JPA CacheStore you might want to use
>> multiple independent instances, one per mapped type. Not necessarily
>> needed (one could improve the CacheStore implementation as well) but
>> it gives lot of flexibility, like connecting to different RDBMs
>> instances or with different credentials/schemas, etc...
>
> Indeed user might want this in the future. Moving "preload" at
"loader" level would only facilitate that.
Right. I'm making those examples to highlight the benefit of moving
the attribute to each cachestore, as you proposed, but this only makes
sense if you actually allow the option to be applied differently for
each instance.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
4:25 a.m.
Btw how do we deal with preloading in a cluster?
Eg how do we prevent every node from preloading the same data?
On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus(a)redhat.com> wrote:
On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org> wrote:
> On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>
>> On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org> wrote:
>>
>>> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>>> Currently the "preload" attribute is configured at *loaders*
level and the first cache loader in the chain is used for preloading.
>>>> A better place for preload might be the individual loader element: would
indicate clearly which loader is used for preloading and also consistent with the
fetchPeristentState which is again defined on a per loader basis.
>>>>
>>>> How does it sound?
>>>
>>> Very confusing actually :-)
>>> Are you going to prevent me to enable "preload" on multiple Cache
>>> Loaders elements?
>>
>> I'm not going to take anything from you, this is something you're not
allowed to do ATM :-)
>
> That's exactly my point. I'm not allowed to do it ATM, but if you
> change the position of this attribute to *each* loader it will look
> like I could enable/disable it differently on a per-instance.
Gotcha, I'll either add a check to allow specifying preload on a single store, or (if
time allows) I'll implement the multiple preload feature.
>
>>
>>> And what if I'm actually storing entries in
>>> different CacheStore instanced based on some discriminator, like the
>>> value (key) type?
>>>
>>> To make a practical example, the Lucene Directory CacheLoader only
>>> loads some specific types, and only from named indexes which match a
>>> configured name. You could want multiple instances using different
>>> configurations, so that for each index name stored in the same cache
>>> you actually store them in different cachestores.
>>>
>>> Similar situation, with the JPA CacheStore you might want to use
>>> multiple independent instances, one per mapped type. Not necessarily
>>> needed (one could improve the CacheStore implementation as well) but
>>> it gives lot of flexibility, like connecting to different RDBMs
>>> instances or with different credentials/schemas, etc...
>>
>> Indeed user might want this in the future. Moving "preload" at
"loader" level would only facilitate that.
>
> Right. I'm making those examples to highlight the benefit of moving
> the attribute to each cachestore, as you proposed, but this only makes
> sense if you actually allow the option to be applied differently for
> each instance.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
11:05 a.m.
On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism(a)gmail.com> wrote:
Btw how do we deal with preloading in a cluster?
Each node preloads locally only.
Eg how do we prevent every node from preloading the same data?
Preloaded data is not replicated/distributed to other nodes, in order to speed up process.
For replicated caches, each node loads its own stuff (from local or shared). For
distributed caches, I don't think we're intelligent enough to only load data that
belongs to a particular node (I think there's some jiras on this topic…)
Cheers,
On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus(a)redhat.com> wrote:
>
> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org> wrote:
>
>> On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>>
>>> On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org>
wrote:
>>>
>>>> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>>>> Currently the "preload" attribute is configured at
*loaders* level and the first cache loader in the chain is used for preloading.
>>>>> A better place for preload might be the individual loader element:
would indicate clearly which loader is used for preloading and also consistent with the
fetchPeristentState which is again defined on a per loader basis.
>>>>>
>>>>> How does it sound?
>>>>
>>>> Very confusing actually :-)
>>>> Are you going to prevent me to enable "preload" on multiple
Cache
>>>> Loaders elements?
>>>
>>> I'm not going to take anything from you, this is something you're not
allowed to do ATM :-)
>>
>> That's exactly my point. I'm not allowed to do it ATM, but if you
>> change the position of this attribute to *each* loader it will look
>> like I could enable/disable it differently on a per-instance.
>
> Gotcha, I'll either add a check to allow specifying preload on a single store, or
(if time allows) I'll implement the multiple preload feature.
>
>>
>>>
>>>> And what if I'm actually storing entries in
>>>> different CacheStore instanced based on some discriminator, like the
>>>> value (key) type?
>>>>
>>>> To make a practical example, the Lucene Directory CacheLoader only
>>>> loads some specific types, and only from named indexes which match a
>>>> configured name. You could want multiple instances using different
>>>> configurations, so that for each index name stored in the same cache
>>>> you actually store them in different cachestores.
>>>>
>>>> Similar situation, with the JPA CacheStore you might want to use
>>>> multiple independent instances, one per mapped type. Not necessarily
>>>> needed (one could improve the CacheStore implementation as well) but
>>>> it gives lot of flexibility, like connecting to different RDBMs
>>>> instances or with different credentials/schemas, etc...
>>>
>>> Indeed user might want this in the future. Moving "preload" at
"loader" level would only facilitate that.
>>
>> Right. I'm making those examples to highlight the benefit of moving
>> the attribute to each cachestore, as you proposed, but this only makes
>> sense if you actually allow the option to be applied differently for
>> each instance.
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
8:51 p.m.
On Fri, Aug 16, 2013 at 5:05 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism(a)gmail.com> wrote:
> Btw how do we deal with preloading in a cluster?
Each node preloads locally only.
How is this done? Is it through loadAllKeys first, and then determining
which keys are local?
Or do you mean it preloads locally from a non-shared cache store? What if
the cache store is shared?
> Eg how do we prevent every node from preloading the same data?
Preloaded data is not replicated/distributed to other nodes, in order to
speed up process. For replicated caches, each node loads its own stuff
(from local or shared). For distributed caches, I don't think we're
intelligent enough to only load data that belongs to a particular node (I
think there's some jiras on this topic…)
For replicated cache, when it loads the data, does it put to the local
cache container only? (i.e., avoid unnecessarily replicating to other nodes
of course).
Cheers,
>
> On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus(a)redhat.com> wrote:
>
>>
>> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org> wrote:
>>
>>> On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com> wrote:
>>>>
>>>> On 14 Aug 2013, at 15:26, Sanne Grinovero <sanne(a)infinispan.org>
wrote:
>>>>
>>>>> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com>
wrote:
>>>>>> Currently the "preload" attribute is configured at
*loaders* level
and the first cache loader in the chain is used for preloading.
>>>>>> A better place for preload might be the individual loader
element:
would indicate clearly which loader is used for preloading and also
consistent with the fetchPeristentState which is again defined on a per
loader basis.
>>>>>>
>>>>>> How does it sound?
>>>>>
>>>>> Very confusing actually :-)
>>>>> Are you going to prevent me to enable "preload" on
multiple Cache
>>>>> Loaders elements?
>>>>
>>>> I'm not going to take anything from you, this is something
you're not
allowed to do ATM :-)
>>>
>>> That's exactly my point. I'm not allowed to do it ATM, but if you
>>> change the position of this attribute to *each* loader it will look
>>> like I could enable/disable it differently on a per-instance.
>>
>> Gotcha, I'll either add a check to allow specifying preload on a single
store, or (if time allows) I'll implement the multiple preload feature.
>>
>>>
>>>>
>>>>> And what if I'm actually storing entries in
>>>>> different CacheStore instanced based on some discriminator, like
the
>>>>> value (key) type?
>>>>>
>>>>> To make a practical example, the Lucene Directory CacheLoader only
>>>>> loads some specific types, and only from named indexes which match
a
>>>>> configured name. You could want multiple instances using different
>>>>> configurations, so that for each index name stored in the same
cache
>>>>> you actually store them in different cachestores.
>>>>>
>>>>> Similar situation, with the JPA CacheStore you might want to use
>>>>> multiple independent instances, one per mapped type. Not
necessarily
>>>>> needed (one could improve the CacheStore implementation as well)
but
>>>>> it gives lot of flexibility, like connecting to different RDBMs
>>>>> instances or with different credentials/schemas, etc...
>>>>
>>>> Indeed user might want this in the future. Moving "preload"
at
"loader" level would only facilitate that.
>>>
>>> Right. I'm making those examples to highlight the benefit of moving
>>> the attribute to each cachestore, as you proposed, but this only makes
>>> sense if you actually allow the option to be applied differently for
>>> each instance.
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
12:02 p.m.
On 08/16/2013 07:51 PM, Ray Tsang wrote:
On Fri, Aug 16, 2013 at 5:05 AM, Galder Zamarreño <galder(a)redhat.com
<mailto:galder@redhat.com>> wrote:
On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism(a)gmail.com
<mailto:saturnism@gmail.com>> wrote:
> Btw how do we deal with preloading in a cluster?
Each node preloads locally only.
How is this done? Is it through loadAllKeys first, and then determining
which keys are local?
Or do you mean it preloads locally from a non-shared cache store? What
if the cache store is shared?
the behaviour is the same for non-shared/shared caches. Each node loads
all the data from the cache store and puts it in their local container
(without any replication/distribution neither any communication with
other nodes).
> Eg how do we prevent every node from preloading the same data?
Preloaded data is not replicated/distributed to other nodes, in
order to speed up process. For replicated caches, each node loads
its own stuff (from local or shared). For distributed caches, I
don't think we're intelligent enough to only load data that belongs
to a particular node (I think there's some jiras on this topic…)
For replicated cache, when it loads the data, does it put to the local
cache container only? (i.e., avoid unnecessarily replicating to other
nodes of course).
yes
Cheers,
Pedro
Cheers,
>
> On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus(a)redhat.com
<mailto:mmarkus@redhat.com>> wrote:
>
>>
>> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org
<mailto:sanne@infinispan.org>> wrote:
>>
>>> On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com
<mailto:mmarkus@redhat.com>> wrote:
>>>>
>>>> On 14 Aug 2013, at 15:26, Sanne Grinovero
<sanne(a)infinispan.org <mailto:sanne@infinispan.org>> wrote:
>>>>
>>>>> On 14 August 2013 11:15, Mircea Markus <mmarkus(a)redhat.com
<mailto:mmarkus@redhat.com>> wrote:
>>>>>> Currently the "preload" attribute is configured
at *loaders*
level and the first cache loader in the chain is used for preloading.
>>>>>> A better place for preload might be the individual loader
element: would indicate clearly which loader is used for preloading
and also consistent with the fetchPeristentState which is again
defined on a per loader basis.
>>>>>>
>>>>>> How does it sound?
>>>>>
>>>>> Very confusing actually :-)
>>>>> Are you going to prevent me to enable "preload" on
multiple Cache
>>>>> Loaders elements?
>>>>
>>>> I'm not going to take anything from you, this is something
you're not allowed to do ATM :-)
>>>
>>> That's exactly my point. I'm not allowed to do it ATM, but if
you
>>> change the position of this attribute to *each* loader it will look
>>> like I could enable/disable it differently on a per-instance.
>>
>> Gotcha, I'll either add a check to allow specifying preload on a
single store, or (if time allows) I'll implement the multiple
preload feature.
>>
>>>
>>>>
>>>>> And what if I'm actually storing entries in
>>>>> different CacheStore instanced based on some discriminator,
like the
>>>>> value (key) type?
>>>>>
>>>>> To make a practical example, the Lucene Directory CacheLoader
only
>>>>> loads some specific types, and only from named indexes which
match a
>>>>> configured name. You could want multiple instances using
different
>>>>> configurations, so that for each index name stored in the
same cache
>>>>> you actually store them in different cachestores.
>>>>>
>>>>> Similar situation, with the JPA CacheStore you might want to
use
>>>>> multiple independent instances, one per mapped type. Not
necessarily
>>>>> needed (one could improve the CacheStore implementation as
well) but
>>>>> it gives lot of flexibility, like connecting to different
RDBMs
>>>>> instances or with different credentials/schemas, etc...
>>>>
>>>> Indeed user might want this in the future. Moving
"preload" at
"loader" level would only facilitate that.
>>>
>>> Right. I'm making those examples to highlight the benefit of
moving
>>> the attribute to each cachestore, as you proposed, but this
only makes
>>> sense if you actually allow the option to be applied
differently for
>>> each instance.
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org <http://www.infinispan.org>)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
<mailto:infinispan-dev@lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
<mailto:infinispan-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com <mailto:galder@redhat.com>
twitter.com/galderz <http://twitter.com/galderz>
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3:24 p.m.
On 19 August 2013 11:02, Pedro Ruivo <pedro(a)infinispan.org> wrote:
On 08/16/2013 07:51 PM, Ray Tsang wrote:
>
>
> On Fri, Aug 16, 2013 at 5:05 AM, Galder Zamarreño <galder(a)redhat.com
> <mailto:galder@redhat.com>> wrote:
>
>
> On Aug 15, 2013, at 4:25 AM, Ray Tsang <saturnism(a)gmail.com
> <mailto:saturnism@gmail.com>> wrote:
>
> > Btw how do we deal with preloading in a cluster?
>
> Each node preloads locally only.
>
>
> How is this done? Is it through loadAllKeys first, and then determining
> which keys are local?
> Or do you mean it preloads locally from a non-shared cache store? What
> if the cache store is shared?
the behaviour is the same for non-shared/shared caches. Each node loads
all the data from the cache store and puts it in their local container
(without any replication/distribution neither any communication with
other nodes).
Let's note that Pedro and Galder are correctly replying to "how is
this done", but this is not functionally complete and we're aware of
it.
The problem is that if you intend to stop the whole grid to restart it
and reload the existing state from the CacheStore(s), the nodes get
likely assigned a different position in the view, and therefore in the
hashwheel, and the content of each local CacheStore won't match the
expected keys.
Today the only viable configuration is to use a single shared CacheStore..
further design discussion is on
-https://community.jboss.org/wiki/ControlledClusterShutdownWithDataRestoreFromPersistentStorage
-- Sanne
>
>
> > Eg how do we prevent every node from preloading the same data?
>
> Preloaded data is not replicated/distributed to other nodes, in
> order to speed up process. For replicated caches, each node loads
> its own stuff (from local or shared). For distributed caches, I
> don't think we're intelligent enough to only load data that belongs
> to a particular node (I think there's some jiras on this topic…)
>
>
> For replicated cache, when it loads the data, does it put to the local
> cache container only? (i.e., avoid unnecessarily replicating to other
> nodes of course).
yes
Cheers,
Pedro
>
>
> Cheers,
>
> >
> > On Aug 14, 2013, at 16:09, Mircea Markus <mmarkus(a)redhat.com
> <mailto:mmarkus@redhat.com>> wrote:
> >
> >>
> >> On 14 Aug 2013, at 20:53, Sanne Grinovero <sanne(a)infinispan.org
> <mailto:sanne@infinispan.org>> wrote:
> >>
> >>> On 14 August 2013 17:59, Mircea Markus <mmarkus(a)redhat.com
> <mailto:mmarkus@redhat.com>> wrote:
> >>>>
> >>>> On 14 Aug 2013, at 15:26, Sanne Grinovero
> <sanne(a)infinispan.org <mailto:sanne@infinispan.org>> wrote:
> >>>>
> >>>>> On 14 August 2013 11:15, Mircea Markus
<mmarkus(a)redhat.com
> <mailto:mmarkus@redhat.com>> wrote:
> >>>>>> Currently the "preload" attribute is
configured at *loaders*
> level and the first cache loader in the chain is used for preloading.
> >>>>>> A better place for preload might be the individual
loader
> element: would indicate clearly which loader is used for preloading
> and also consistent with the fetchPeristentState which is again
> defined on a per loader basis.
> >>>>>>
> >>>>>> How does it sound?
> >>>>>
> >>>>> Very confusing actually :-)
> >>>>> Are you going to prevent me to enable "preload"
on multiple Cache
> >>>>> Loaders elements?
> >>>>
> >>>> I'm not going to take anything from you, this is something
> you're not allowed to do ATM :-)
> >>>
> >>> That's exactly my point. I'm not allowed to do it ATM, but
if you
> >>> change the position of this attribute to *each* loader it will
look
> >>> like I could enable/disable it differently on a per-instance.
> >>
> >> Gotcha, I'll either add a check to allow specifying preload on a
> single store, or (if time allows) I'll implement the multiple
> preload feature.
> >>
> >>>
> >>>>
> >>>>> And what if I'm actually storing entries in
> >>>>> different CacheStore instanced based on some
discriminator,
> like the
> >>>>> value (key) type?
> >>>>>
> >>>>> To make a practical example, the Lucene Directory
CacheLoader
> only
> >>>>> loads some specific types, and only from named indexes
which
> match a
> >>>>> configured name. You could want multiple instances using
> different
> >>>>> configurations, so that for each index name stored in the
> same cache
> >>>>> you actually store them in different cachestores.
> >>>>>
> >>>>> Similar situation, with the JPA CacheStore you might want
to use
> >>>>> multiple independent instances, one per mapped type. Not
> necessarily
> >>>>> needed (one could improve the CacheStore implementation as
> well) but
> >>>>> it gives lot of flexibility, like connecting to different
RDBMs
> >>>>> instances or with different credentials/schemas, etc...
> >>>>
> >>>> Indeed user might want this in the future. Moving
"preload" at
> "loader" level would only facilitate that.
> >>>
> >>> Right. I'm making those examples to highlight the benefit of
moving
> >>> the attribute to each cachestore, as you proposed, but this
> only makes
> >>> sense if you actually allow the option to be applied
> differently for
> >>> each instance.
> >>
> >> Cheers,
> >> --
> >> Mircea Markus
> >> Infinispan lead (www.infinispan.org <http://www.infinispan.org>)
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev(a)lists.jboss.org
> <mailto:infinispan-dev@lists.jboss.org>
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev(a)lists.jboss.org
> <mailto:infinispan-dev@lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarreño
> galder(a)redhat.com <mailto:galder@redhat.com>
> twitter.com/galderz <http://twitter.com/galderz>
>
> Project Lead, Escalante
> http://escalante.io
>
> Engineer, Infinispan
> http://infinispan.org
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
10:57 a.m.
JSR-107 comes with an interesting twist on preloading. It's not really a cache
writer/loader configuration, but instead there's a high level loadAll API in Cache to
load data from a cache store [1]. I wonder if it'd make sure for us to go that way
too, although the main problem we have is that once a cache is retrieved from the cache
manager, it's already started, and our current preloading strategy works as part of
the start() procedure.
Also, JSR-107 seems to be enable loading of a subset of keys and I'm not sure how you
can load all entries in the store.
In spite of this, it's something worth bearing in mind for preloading discussions...
[1]
https://github.com/jsr107/jsr107spec/blob/master/src/main/java/javax/cach...
On Aug 14, 2013, at 12:15 PM, Mircea Markus <mmarkus(a)redhat.com> wrote:
Currently the "preload" attribute is configured at
*loaders* level and the first cache loader in the chain is used for preloading.
A better place for preload might be the individual loader element: would indicate clearly
which loader is used for preloading and also consistent with the fetchPeristentState which
is again defined on a per loader basis.
How does it sound?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
4170
days inactive
4175
days old
infinispan-dev@lists.jboss.org
10 comments
5 participants
participants (5)
-
Galder Zamarreño -
Mircea Markus -
Pedro Ruivo -
Ray Tsang -
Sanne Grinovero