On 18 Mar 2010, at 21:26, Alex Kluge wrote:
Hi,
I wrestled with the same question when going down this path.
> but the only way this can be meaningfully used is if the client has the ability
> to calculate the hash code of the key in the same manner the servers do.
Definitely the case. However, even if the client gets it wrong, the
message should still be routed to the correct server.
Yes, but if this is the norm rather than the exception then there is little point in
attempting client-side hashing.
> Firstly, this is hard when consumed by non-Java clients as you'd need to
> implement the way the JDK calculates the hash code of a byte array.
This is a much easier problem if you don't use the built in hashing.
This is true. The current impl simply calls key.hashcode() and passes the result through
a Wang/Jenkins based bit spreader. I suppose we could be smarter and for keys which are
byte arrays (and this will *always* be the case if interacting via HotRod) something like
FNV-1 or SuperFastHash could be used. I was even looking at MurmurHash.
http://sites.google.com/site/murmurhash/
We probably wouldn't even need a bit spreader in this case, and a MurmurHash impls
exist for several platforms.
http://en.wikipedia.org/wiki/MurmurHash
Cheers
Manik
There are
a number of hash algorithms that can be used, including
FNV 1 (
http://www.isthe.com/chongo/tech/comp/fnv/)
and others
http://www.azillionmonkeys.com/qed/hash.html.
These are implementable in other languages, are fast, and provide good distributions of
results. We can use, and similarly document, an integer hash used to further spread the
hash values if needed. If these are chosen to be implementable in multiple languages,
clearly documented, and don't change too often, it should be reasonable to put them
into
the client.
Using this approach removes the dependency on the hash that the VM happens to be
using. Indeed, the hash for a byte array may simply be the address of the array, which
makes it very poor for our use.
Alex
--- On Thu, 3/18/10, Manik Surtani <manik(a)jboss.org> wrote:
> From: Manik Surtani <manik(a)jboss.org>
> Subject: [infinispan-dev] HotRod, ClientIntelligence and client-side key location
> To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org>
> Date: Thursday, March 18, 2010, 11:59 AM
> I've been thinking about how we
> handle this, and I think we have a problem with smart
> clients where clients have the ability to locate the key on
> the server cluster in order to direct the request to the
> specific node.
>
> The problem is in hash code calculation. The HotRod
> protocol caters for this with regards to calculating node
> address hash code by passing this in the topology map (see
> "Hasher Client Topology Change Header" in [1]), but the only
> way this can be meaningfully used is if the client has the
> ability to calculate the hash code of the key in the same
> manner the servers do. Firstly, this is hard when
> consumed by non-Java clients as you'd need to implement the
> way the JDK calculates the hash code of a byte array.
> Second, you'd need detailed and specific knowledge of any
> bit spreading that takes place within Infinispan - and this
> is internal implementation detail which may change from
> release to release.
>
> So the way I see it I can't see how non-Java clients will
> be able to locate keys and then direct requests to the
> necessary nodes. In fact, even with Java clients the
> only way this could be done would be to send back marshalled
> Addresses in the topology map, *and* have the same version
> of the Infinispan server libs installed on the client, *and*
> ensure that the same JDK/JVM version is used on the
> client.
>
> Can we think of a better way to do this? If not, is
> it worth still supporting client-side consistent hash based
> key location for the weird but vaguely workable scenario for
> Java-based clients?
>
> Thoughts?
>
> Cheers
> Manik
>
>
> [1]
http://community.jboss.org/wiki/HotRodProtocol
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
>
http://www.infinispan.org
>
http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org