[infinispan-dev] Preloading from disk versus state transfer Re: ISPN-1384 - InboundInvocationHandlerImpl should wait for cache to be started? (not just defined)

[Galder Zamarreño]

On Oct 24, 2011, at 12:58 PM, Dan Berindei wrote:

> Hi Galder
> 
> On Mon, Oct 24, 2011 at 1:46 PM, Galder Zamarreño <galder at redhat.com> wrote:
>> 
>> On Oct 24, 2011, at 12:04 PM, Dan Berindei wrote:
>> 
>>> ISPN-1470 (https://issues.jboss.org/browse/ISPN-1470) raises an
>>> interesting question: if the preloading happens before joining, the
>>> preloading code won't know anything about the consistent hash. It will
>>> load everything from the cache store, including the keys that are
>>> owned by other nodes.
>> 
>> It's been defined to work that way:
>> https://docs.jboss.org/author/display/ISPN/CacheLoaders
>> 
>> Tbh, that will only happen in shared cache stores. In non-shared ones, you'll only have data that belongs to that node.
>> 
> 
> Not really... in distributed mode, every time the cache starts it will
> have another position on the hash wheel.
> That means even with a non-shared cache store, it's likely most of the
> stored keys will no longer be local.
> 
> Actually I just noticed that you've fixed ISPN-1404, which looks like
> it would solves my problem when the cache is created by a HotRod
> server. I would like to extend it to work like this by default, e.g.
> by using the transport's nodeName as the seed.
> 
>>> I think there is a check in place already so that the joiner won't
>>> push stale data from its cache store to the other nodes, but we should
>>> also discard the keys that don't map locally or we'll have stale data
>>> (since we don't have a way to check if those keys are stale and
>>> register to receive invalidations for those keys).
>> 
>> +1, only for shared cache stores.
>> 
>>> 
>>> What do you think, should I discard the non-local keys with the fix
>>> for ISPN-1470 or should I let them be and warn the user about
>>> potentially stale data?
>> 
>> Discard only for shared cache stores.
>> 
>> Cache configurations should be symmetrical, so if other nodes preload, they'll preload only data local to them with your change.
>> 
> 
> Discarding works fine from the correctness POV, but for performance
> it's not that great: we may do a lot of work to preload keys and have
> nothing to show for it at the end.

I agree, I thought of that when replying to this. It'd be great if you could only bring that data that will belong to you, but for that we'd need to store the hash of the key as well.

> 
> Enabling the fixed hash seed by default should make the performance
> issue go away. I think it would also require virtual nodes enabled by
> default and a way to ensure that the nodeNames are unique across the
> cluster.
> 
> Cheers
> Dan
> 
> 
>>> 
>>> Cheers
>>> Dan
>>> 
>>> 
>>> On Mon, Oct 3, 2011 at 3:09 AM, Manik Surtani <manik at jboss.org> wrote:
>>>> 
>>>> On 28 Sep 2011, at 10:56, Dan Berindei wrote:
>>>> 
>>>> I'm not sure if the comment is valid though, since the old
>>>> StateTransferManager had priority 55 and it also cleared the data
>>>> container before applying the state from the coordinator. I'm not sure
>>>> how preloading and state transfer are supposed to interact, maybe
>>>> Manik can help clear this up?
>>>> 
>>>> Hmm - this is interesting.  I think preloading should happen first, since
>>>> the cache store may contain old data.
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> twitter.com/maniksurtani
>>>> Lead, Infinispan
>>>> http://www.infinispan.org
>>>> 
>>>> 
>>>> 
>> 
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>> 
>> 

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache