[infinispan-dev] Preloading from disk versus state transfer Re: ISPN-1384 - InboundInvocationHandlerImpl should wait for cache to be started? (not just defined)

[Dan Berindei]

Hi Galder

On Mon, Oct 24, 2011 at 1:46 PM, Galder Zamarreño <galder at redhat.com> wrote:
>
> On Oct 24, 2011, at 12:04 PM, Dan Berindei wrote:
>
>> ISPN-1470 (https://issues.jboss.org/browse/ISPN-1470) raises an
>> interesting question: if the preloading happens before joining, the
>> preloading code won't know anything about the consistent hash. It will
>> load everything from the cache store, including the keys that are
>> owned by other nodes.
>
> It's been defined to work that way:
> https://docs.jboss.org/author/display/ISPN/CacheLoaders
>
> Tbh, that will only happen in shared cache stores. In non-shared ones, you'll only have data that belongs to that node.
>

Not really... in distributed mode, every time the cache starts it will
have another position on the hash wheel.
That means even with a non-shared cache store, it's likely most of the
stored keys will no longer be local.

Actually I just noticed that you've fixed ISPN-1404, which looks like
it would solves my problem when the cache is created by a HotRod
server. I would like to extend it to work like this by default, e.g.
by using the transport's nodeName as the seed.

>> I think there is a check in place already so that the joiner won't
>> push stale data from its cache store to the other nodes, but we should
>> also discard the keys that don't map locally or we'll have stale data
>> (since we don't have a way to check if those keys are stale and
>> register to receive invalidations for those keys).
>
> +1, only for shared cache stores.
>
>>
>> What do you think, should I discard the non-local keys with the fix
>> for ISPN-1470 or should I let them be and warn the user about
>> potentially stale data?
>
> Discard only for shared cache stores.
>
> Cache configurations should be symmetrical, so if other nodes preload, they'll preload only data local to them with your change.
>

Discarding works fine from the correctness POV, but for performance
it's not that great: we may do a lot of work to preload keys and have
nothing to show for it at the end.

Enabling the fixed hash seed by default should make the performance
issue go away. I think it would also require virtual nodes enabled by
default and a way to ensure that the nodeNames are unique across the
cluster.

Cheers
Dan


>>
>> Cheers
>> Dan
>>
>>
>> On Mon, Oct 3, 2011 at 3:09 AM, Manik Surtani <manik at jboss.org> wrote:
>>>
>>> On 28 Sep 2011, at 10:56, Dan Berindei wrote:
>>>
>>> I'm not sure if the comment is valid though, since the old
>>> StateTransferManager had priority 55 and it also cleared the data
>>> container before applying the state from the coordinator. I'm not sure
>>> how preloading and state transfer are supposed to interact, maybe
>>> Manik can help clear this up?
>>>
>>> Hmm - this is interesting.  I think preloading should happen first, since
>>> the cache store may contain old data.
>>> --
>>> Manik Surtani
>>> manik at jboss.org
>>> twitter.com/maniksurtani
>>> Lead, Infinispan
>>> http://www.infinispan.org
>>>
>>>
>>>
>
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
>
>