[infinispan-dev] Ideas for Hot Rod protocol improvements to support new hashing routing table

[Galder Zamarreño]

On Sep 7, 2012, at 5:03 PM, Dan Berindei <dan.berindei at gmail.com> wrote:

> On Fri, Sep 7, 2012 at 1:09 PM, Galder Zamarreño <galder at redhat.com> wrote:
> Hey Dan,
> 
> Re: https://issues.jboss.org/browse/ISPN-2205
> 
> Sorry we couldn't have a proper chat but we can carry on via email. Extending to Infinispan Dev.
> 
> | >  [11:03]  <dberindei> this is what I had in mind for the hotrod update: https://gist.github.com/3664504
> 
> What is "1st owner's index"? Is that the position in the array of servers in topology above? I guess os, if so, I think https://gist.github.com/3664504 looks fine.
> 
> 
> Yes, it's the position in the array of members.
> 
>  
> That's assuming the clients provide the normalized hash for the key (which they already do today), they should have all the info to locate in.
> 
> Do we still need hash space? I think we can assume this is fixed now to be all positive integers.
> 
> 
> I figured I'd keep the new protocol a superset of 1.1, since the new CH format is quite generic it could theoretically used for any CH.

Yeah, let's keep that as is.

> 
>  
> | >  [11:04]  <dberindei> but thinking about it some more, I think it might be a good idea to talk about having a CH per cache on the client as well
> 
> Supporting a CH per cache would not require any changes to the protocol, since all operations are per-cache, it would just need the client to cache the info accordingly.
> 
> 
> I think we might need some logic on the server as well, but you're probably right that the protocol doesn't need changes.
> 
>  
> | > [11:07]  <dberindei> something else that came to mind: we allow the server to specify a hash function version, but the serialization of the CH is fixed based only on the client's version
> | > [11:08]  <dberindei> maybe we could specify in the protocol a different serialization for each function version?
> | > [11:09]  <dberindei> that way we could have a single client that can support any server CH via a very generic client CH like the new DCH
> | > [11:10]  <dberindei> and at the same time, if the server knows that the client supports a fancier CH, it can send the other CH instead
> 
> The Hash Function is there to define what 'hash function' the client uses. Whether that's MurmurHash3, MurmurHash2…etc. The serialization, or how we transmit the rest of hashing information should really be part of the protocol itself. I don't think we should change this.
> 
> 
> I think there's a slight confusion in the protocol/client here, the field is called 'hash function' but the client actually uses it to choose a ConsistentHash implementation (either ConsistentHashV1 or ConsistentHashV2). You're right that the only difference between the two is the actual hash function though.
> 
> I was thinking of using this as a sort of negotiation protocol: the client says it supports a protocol version, and the server picks one of the consistent hashes that it knows and is compatible with the client. But perhaps this would be better implemented as a proper negotiation, with the client specifying a range of supported protocol versions and the server replying with a single common version.

Remember that this is not the only change I want to add to the protocol, I also want to add other stuff (see https://issues.jboss.org/browse/ISPN-2205)

On top of that, I don't wanna be subversioning parts of the protocol, and hash function already has a defined objective, which is switching between the actual hash function (murmurhash2, murmurhash3, …etc), and any future hash functions we come accross.

Cheers,

> 
> Cheers
> Dan
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org