Manik, Being new here, I'm not totally sure of JBoss' stance on including other libraries. Hadoop has a Murmur2 hash implementation that I have been using at work for a while with no problems. Just a thought.

Guys, The MurmurHash2 impl we have in 4.2.0 is buggy in that my translation from the original C source was faulty and it effectively hashes over just 31 bits instead of 32. It means the distribution result isn't as good as it could be. Now it isn't that easy for me to just *fix* this in 4.2.1, since it means keys mapped to nodes using 4.2.0 may not map to the same node in 4.2.1. So here is what I propose: 1) Fix it in 4.2.x as MurmurHash2A 2) Use MurmurHash2A by default, *unless* a config flag is provided that forces the use of MurmurHash2. (e.g.,<hash function="MurmurHash2">) This will even give us the ability to use MurmurHash3 in 5.0 when we have it. WDYT?

Or release a 4.3. On 14 janv. 2011, at 09:32, Bela Ban wrote: > I would fix the existing MurmurHash; IMO backward compat doesn't extend > to consistent hashing... > > Does leaving the 1 bit out create extreme anomalies wrt hashing > distribution ? I wouldn't think so... > > I'm fine with the other solution, too. > > > > On 1/13/11 8:35 PM, Manik Surtani wrote: >> Guys, >> >> The MurmurHash2 impl we have in 4.2.0 is buggy in that my translation from the original C source was faulty and it effectively hashes over just 31 bits instead of 32.  It means the distribution result isn't as good as it could be. >> >> Now it isn't that easy for me to just *fix* this in 4.2.1, since it means keys mapped to nodes using 4.2.0 may not map to the same node in 4.2.1. >> >> So here is what I propose: >> >> 1) Fix it in 4.2.x as MurmurHash2A >> 2) Use MurmurHash2A by default, *unless* a config flag is provided that forces the use of MurmurHash2.  (e.g.,<hash function="MurmurHash2">) >> >> This will even give us the ability to use MurmurHash3 in 5.0 when we have it. >> >> WDYT? > > > -- > Bela Ban > Lead JGroups / Clustering Team > JBoss > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

We do support rolling upgrade.  And further, we also support the following scenario: 1) Run a cluster, using a cache store that is *not* shared.  Entries get hashed to specific nodes. 2) Shut down the cluster, upgrade Infinispan. 3) Should still be able to locate your entries, as they would be loaded from the cache stores. If the hash function changes between upgrades, then requests for keys would now be mapped to different nodes, and the keys won't be found in those nodes' cache stores. On 14 Jan 2011, at 09:45, Sanne Grinovero wrote: > I'm wondering why you're considering backwards compatibility of the > hashing distribution. Is Infinispan supposed to support multiple > different versions to be connected in a cluster, providing some form > of rolling upgrade? > > Cheers, > Sanne > > 2011/1/14 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: >> Or release a 4.3. >> >> On 14 janv. 2011, at 09:32, Bela Ban wrote: >> >>> I would fix the existing MurmurHash; IMO backward compat doesn't extend >>> to consistent hashing... >>> >>> Does leaving the 1 bit out create extreme anomalies wrt hashing >>> distribution ? I wouldn't think so... >>> >>> I'm fine with the other solution, too. >>> >>> >>> >>> On 1/13/11 8:35 PM, Manik Surtani wrote: >>>> Guys, >>>> >>>> The MurmurHash2 impl we have in 4.2.0 is buggy in that my translation from the original C source was faulty and it effectively hashes over just 31 bits instead of 32.  It means the distribution result isn't as good as it could be. >>>> >>>> Now it isn't that easy for me to just *fix* this in 4.2.1, since it means keys mapped to nodes using 4.2.0 may not map to the same node in 4.2.1. >>>> >>>> So here is what I propose: >>>> >>>> 1) Fix it in 4.2.x as MurmurHash2A >>>> 2) Use MurmurHash2A by default, *unless* a config flag is provided that forces the use of MurmurHash2.  (e.g.,<hash function="MurmurHash2">) >>>> >>>> This will even give us the ability to use MurmurHash3 in 5.0 when we have it. >>>> >>>> WDYT? >>> >>> >>> -- >>> Bela Ban >>> Lead JGroups / Clustering Team >>> JBoss >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Or release a 4.3. On 14 janv. 2011, at 09:32, Bela Ban wrote: > I would fix the existing MurmurHash; IMO backward compat doesn't extend > to consistent hashing... > > Does leaving the 1 bit out create extreme anomalies wrt hashing > distribution ? I wouldn't think so... > > I'm fine with the other solution, too. > > > > On 1/13/11 8:35 PM, Manik Surtani wrote: >> Guys, >> >> The MurmurHash2 impl we have in 4.2.0 is buggy in that my translation from the original C source was faulty and it effectively hashes over just 31 bits instead of 32. It means the distribution result isn't as good as it could be. >> >> Now it isn't that easy for me to just *fix* this in 4.2.1, since it means keys mapped to nodes using 4.2.0 may not map to the same node in 4.2.1. >> >> So here is what I propose: >> >> 1) Fix it in 4.2.x as MurmurHash2A >> 2) Use MurmurHash2A by default, *unless* a config flag is provided that forces the use of MurmurHash2. (e.g.,<hash function="MurmurHash2">) >> >> This will even give us the ability to use MurmurHash3 in 5.0 when we have it. >> >> WDYT? > > > -- > Bela Ban > Lead JGroups / Clustering Team > JBoss > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Guys, The MurmurHash2 impl we have in 4.2.0 is buggy in that my translation from the original C source was faulty and it effectively hashes over just 31 bits instead of 32. It means the distribution result isn't as good as it could be. Now it isn't that easy for me to just *fix* this in 4.2.1, since it means keys mapped to nodes using 4.2.0 may not map to the same node in 4.2.1. So here is what I propose: 1) Fix it in 4.2.x as MurmurHash2A 2) Use MurmurHash2A by default, *unless* a config flag is provided that forces the use of MurmurHash2. (e.g., <hash function="MurmurHash2">) This will even give us the ability to use MurmurHash3 in 5.0 when we have it. WDYT?

Cheers Manik -- Manik Surtani manik(a)jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev