[infinispan-dev] HotRod, ClientIntelligence and client-side key location

[Alex Kluge]

Hi,

 I wrestled with the same question when going down this path.

 > but the only way this can be meaningfully used is if the client has the ability
 > to calculate the hash code of the key in the same manner the servers do.

 Definitely the case. However, even if the client gets it wrong, the
 message should still be routed to the correct server.

 > Firstly, this is hard when consumed by non-Java clients as you'd need to
 >  implement the way the JDK calculates the hash code of a byte array.

 This is a much easier problem if you don't use the built in hashing. There are
 a number of hash algorithms that can be used, including 
FNV 1 (http://www.isthe.com/chongo/tech/comp/fnv/)
 and others http://www.azillionmonkeys.com/qed/hash.html.

 These are implementable in other languages, are fast, and provide good distributions of
 results. We can use, and similarly document, an integer hash used to further spread the
 hash values if needed. If these are chosen to be implementable in multiple languages,
 clearly documented, and don't change too often, it should be reasonable to put them into
 the client.

 Using this approach removes the dependency on the hash that the VM happens to be
 using. Indeed, the hash for a byte array may simply be the address of the array, which
 makes it very poor for our use.
                                                                    Alex

--- On Thu, 3/18/10, Manik Surtani <manik at jboss.org> wrote:

> From: Manik Surtani <manik at jboss.org>
> Subject: [infinispan-dev] HotRod, ClientIntelligence and client-side key location
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Date: Thursday, March 18, 2010, 11:59 AM
> I've been thinking about how we
> handle this, and I think we have a problem with smart
> clients where clients have the ability to locate the key on
> the server cluster in order to direct the request to the
> specific node.
> 
> The problem is in hash code calculation.  The HotRod
> protocol caters for this with regards to calculating node
> address hash code by passing this in the topology map (see
> "Hasher Client Topology Change Header" in [1]), but the only
> way this can be meaningfully used is if the client has the
> ability to calculate the hash code of the key in the same
> manner the servers do.  Firstly, this is hard when
> consumed by non-Java clients as you'd need to implement the
> way the JDK calculates the hash code of a byte array. 
> Second, you'd need detailed and specific knowledge of any
> bit spreading that takes place within Infinispan - and this
> is internal implementation detail which may change from
> release to release.
> 
> So the way I see it I can't see how non-Java clients will
> be able to locate keys and then direct requests to the
> necessary nodes.  In fact, even with Java clients the
> only way this could be done would be to send back marshalled
> Addresses in the topology map, *and* have the same version
> of the Infinispan server libs installed on the client, *and*
> ensure that the same JDK/JVM version is used on the
> client.  
> 
> Can we think of a better way to do this?  If not, is
> it worth still supporting client-side consistent hash based
> key location for the weird but vaguely workable scenario for
> Java-based clients?
> 
> Thoughts?
> 
> Cheers
> Manik
> 
> 
> [1] http://community.jboss.org/wiki/HotRodProtocol
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>