5:53 a.m.
Very nice.
wouldn't it make sense to be able to run Infinispan in a "migration
mode", to cover all possible future cases in which a hash function is
upgraded, or for those cases people want to replace their function?
We could provide a feature to use a secondary function to try fetching
what was not found using the primary function, and when this triggers
it should also take care of moving data around the stores.
You could see this operation as an extension to the rehashing process,
so even if it has a performance hit it could reuse some code and
benefit of all optimizations you do during rehashing.
Thinking it as a rehashing operation, means we could potentially offer
means to change the hash algorithm on the fly; quite nice to
experiment on big datasets with different implementations.
Sanne
2011/1/14 Manik Surtani <manik(a)jboss.org>:
We do support rolling upgrade. And further, we also support the
following scenario:
1) Run a cluster, using a cache store that is *not* shared. Entries get hashed to
specific nodes.
2) Shut down the cluster, upgrade Infinispan.
3) Should still be able to locate your entries, as they would be loaded from the cache
stores.
If the hash function changes between upgrades, then requests for keys would now be mapped
to different nodes, and the keys won't be found in those nodes' cache stores.
On 14 Jan 2011, at 09:45, Sanne Grinovero wrote:
> I'm wondering why you're considering backwards compatibility of the
> hashing distribution. Is Infinispan supposed to support multiple
> different versions to be connected in a cluster, providing some form
> of rolling upgrade?
>
> Cheers,
> Sanne
>
> 2011/1/14 Emmanuel Bernard <emmanuel(a)hibernate.org>:
>> Or release a 4.3.
>>
>> On 14 janv. 2011, at 09:32, Bela Ban wrote:
>>
>>> I would fix the existing MurmurHash; IMO backward compat doesn't extend
>>> to consistent hashing...
>>>
>>> Does leaving the 1 bit out create extreme anomalies wrt hashing
>>> distribution ? I wouldn't think so...
>>>
>>> I'm fine with the other solution, too.
>>>
>>>
>>>
>>> On 1/13/11 8:35 PM, Manik Surtani wrote:
>>>> Guys,
>>>>
>>>> The MurmurHash2 impl we have in 4.2.0 is buggy in that my translation
from the original C source was faulty and it effectively hashes over just 31 bits instead
of 32. It means the distribution result isn't as good as it could be.
>>>>
>>>> Now it isn't that easy for me to just *fix* this in 4.2.1, since it
means keys mapped to nodes using 4.2.0 may not map to the same node in 4.2.1.
>>>>
>>>> So here is what I propose:
>>>>
>>>> 1) Fix it in 4.2.x as MurmurHash2A
>>>> 2) Use MurmurHash2A by default, *unless* a config flag is provided that
forces the use of MurmurHash2. (e.g.,<hash function="MurmurHash2">)
>>>>
>>>> This will even give us the ability to use MurmurHash3 in 5.0 when we have
it.
>>>>
>>>> WDYT?
>>>
>>>
>>> --
>>> Bela Ban
>>> Lead JGroups / Clustering Team
>>> JBoss
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev