Hi Galder
On Mon, Oct 24, 2011 at 1:46 PM, Galder Zamarreño <galder(a)redhat.com> wrote:
On Oct 24, 2011, at 12:04 PM, Dan Berindei wrote:
> ISPN-1470 (
https://issues.jboss.org/browse/ISPN-1470) raises an
> interesting question: if the preloading happens before joining, the
> preloading code won't know anything about the consistent hash. It will
> load everything from the cache store, including the keys that are
> owned by other nodes.
It's been defined to work that way:
https://docs.jboss.org/author/display/ISPN/CacheLoaders
Tbh, that will only happen in shared cache stores. In non-shared ones, you'll only
have data that belongs to that node.
Not really... in distributed mode, every time the cache starts it will
have another position on the hash wheel.
That means even with a non-shared cache store, it's likely most of the
stored keys will no longer be local.
Actually I just noticed that you've fixed ISPN-1404, which looks like
it would solves my problem when the cache is created by a HotRod
server. I would like to extend it to work like this by default, e.g.
by using the transport's nodeName as the seed.
> I think there is a check in place already so that the joiner
won't
> push stale data from its cache store to the other nodes, but we should
> also discard the keys that don't map locally or we'll have stale data
> (since we don't have a way to check if those keys are stale and
> register to receive invalidations for those keys).
+1, only for shared cache stores.
>
> What do you think, should I discard the non-local keys with the fix
> for ISPN-1470 or should I let them be and warn the user about
> potentially stale data?
Discard only for shared cache stores.
Cache configurations should be symmetrical, so if other nodes preload, they'll
preload only data local to them with your change.
Discarding works fine from the correctness POV, but for performance
it's not that great: we may do a lot of work to preload keys and have
nothing to show for it at the end.
Enabling the fixed hash seed by default should make the performance
issue go away. I think it would also require virtual nodes enabled by
default and a way to ensure that the nodeNames are unique across the
cluster.
Cheers
Dan
>
> Cheers
> Dan
>
>
> On Mon, Oct 3, 2011 at 3:09 AM, Manik Surtani <manik(a)jboss.org> wrote:
>>
>> On 28 Sep 2011, at 10:56, Dan Berindei wrote:
>>
>> I'm not sure if the comment is valid though, since the old
>> StateTransferManager had priority 55 and it also cleared the data
>> container before applying the state from the coordinator. I'm not sure
>> how preloading and state transfer are supposed to interact, maybe
>> Manik can help clear this up?
>>
>> Hmm - this is interesting. I think preloading should happen first, since
>> the cache store may contain old data.
>> --
>> Manik Surtani
>> manik(a)jboss.org
>>
twitter.com/maniksurtani
>> Lead, Infinispan
>>
http://www.infinispan.org
>>
>>
>>
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache