but the only way this can be meaningfully used is if the client has the ability to calculate the hash code of the key in the same manner the servers do.

Firstly, this is hard when consumed by non-Java clients as you'd need to implement the way the JDK calculates the hash code of a byte array.

From: Manik Surtani <manik(a)jboss.org&gt; Subject: [infinispan-dev] HotRod, ClientIntelligence and client-side key location To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; Date: Thursday, March 18, 2010, 11:59 AM I've been thinking about how we handle this, and I think we have a problem with smart clients where clients have the ability to locate the key on the server cluster in order to direct the request to the specific node. The problem is in hash code calculation.  The HotRod protocol caters for this with regards to calculating node address hash code by passing this in the topology map (see "Hasher Client Topology Change Header" in [1]), but the only way this can be meaningfully used is if the client has the ability to calculate the hash code of the key in the same manner the servers do.  Firstly, this is hard when consumed by non-Java clients as you'd need to implement the way the JDK calculates the hash code of a byte array.  Second, you'd need detailed and specific knowledge of any bit spreading that takes place within Infinispan - and this is internal implementation detail which may change from release to release. So the way I see it I can't see how non-Java clients will be able to locate keys and then direct requests to the necessary nodes.  In fact, even with Java clients the only way this could be done would be to send back marshalled Addresses in the topology map, *and* have the same version of the Infinispan server libs installed on the client, *and* ensure that the same JDK/JVM version is used on the client.  Can we think of a better way to do this?  If not, is it worth still supporting client-side consistent hash based key location for the weird but vaguely workable scenario for Java-based clients? Thoughts? Cheers Manik [1] http://community.jboss.org/wiki/HotRodProtocol -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 18 Mar 2010, at 21:26, Alex Kluge wrote: > Hi, > > I wrestled with the same question when going down this path. > >> but the only way this can be meaningfully used is if the client has the ability >> to calculate the hash code of the key in the same manner the servers do. > > Definitely the case. However, even if the client gets it wrong, the > message should still be routed to the correct server. Yes, but if this is the norm rather than the exception then there is little point in attempting client-side hashing. > >> Firstly, this is hard when consumed by non-Java clients as you'd need to >> implement the way the JDK calculates the hash code of a byte array. > > This is a much easier problem if you don't use the built in hashing. This is true. The current impl simply calls key.hashcode() and passes the result through a Wang/Jenkins based bit spreader. I suppose we could be smarter and for keys which are byte arrays (and this will *always* be the case if interacting via HotRod) something like FNV-1 or SuperFastHash could be used. I was even looking at MurmurHash. http://sites.google.com/site/murmurhash/ We probably wouldn't even need a bit spreader in this case, and a MurmurHash impls exist for several platforms. http://en.wikipedia.org/wiki/MurmurHash

> There are > a number of hash algorithms that can be used, including > FNV 1 (http://www.isthe.com/chongo/tech/comp/fnv/) > and others http://www.azillionmonkeys.com/qed/hash.html. > > These are implementable in other languages, are fast, and provide good distributions of > results. We can use, and similarly document, an integer hash used to further spread the > hash values if needed. If these are chosen to be implementable in multiple languages, > clearly documented, and don't change too often, it should be reasonable to put them into > the client. > > Using this approach removes the dependency on the hash that the VM happens to be > using. Indeed, the hash for a byte array may simply be the address of the array, which > makes it very poor for our use. > Alex > > --- On Thu, 3/18/10, Manik Surtani <manik(a)jboss.org&gt; wrote: > >> From: Manik Surtani <manik(a)jboss.org&gt; >> Subject: [infinispan-dev] HotRod, ClientIntelligence and client-side key location >> To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; >> Date: Thursday, March 18, 2010, 11:59 AM >> I've been thinking about how we >> handle this, and I think we have a problem with smart >> clients where clients have the ability to locate the key on >> the server cluster in order to direct the request to the >> specific node. >> >> The problem is in hash code calculation. The HotRod >> protocol caters for this with regards to calculating node >> address hash code by passing this in the topology map (see >> "Hasher Client Topology Change Header" in [1]), but the only >> way this can be meaningfully used is if the client has the >> ability to calculate the hash code of the key in the same >> manner the servers do. Firstly, this is hard when >> consumed by non-Java clients as you'd need to implement the >> way the JDK calculates the hash code of a byte array. >> Second, you'd need detailed and specific knowledge of any >> bit spreading that takes place within Infinispan - and this >> is internal implementation detail which may change from >> release to release. >> >> So the way I see it I can't see how non-Java clients will >> be able to locate keys and then direct requests to the >> necessary nodes. In fact, even with Java clients the >> only way this could be done would be to send back marshalled >> Addresses in the topology map, *and* have the same version >> of the Infinispan server libs installed on the client, *and* >> ensure that the same JDK/JVM version is used on the >> client. >> >> Can we think of a better way to do this? If not, is >> it worth still supporting client-side consistent hash based >> key location for the weird but vaguely workable scenario for >> Java-based clients? >> >> Thoughts? >> >> Cheers >> Manik >> >> >> [1] http://community.jboss.org/wiki/HotRodProtocol >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org

He also hosts both a big endian and an endian neutral, but slower, implementation. Some research and testing would be required.

--- On Fri, 3/19/10, Manik Surtani <manik(a)jboss.org&gt; wrote: > From: Manik Surtani <manik(a)jboss.org&gt; > Subject: Re: [infinispan-dev] HotRod, ClientIntelligence and client-side key location > To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; > Date: Friday, March 19, 2010, 9:31 AM > > On 19 Mar 2010, at 14:20, Manik Surtani wrote: > >> >> On 18 Mar 2010, at 21:26, Alex Kluge wrote: >> >>> Hi, >>> >>> I wrestled with the same question when going down > this path. >>> >>>> but the only way this can be meaningfully used > is if the client has the ability >>>> to calculate the hash code of the key in the > same manner the servers do. >>> >>> Definitely the case. However, even if the client > gets it wrong, the >>> message should still be routed to the correct > server. >> >> Yes, but if this is the norm rather than the exception > then there is little point in attempting client-side > hashing. >> >>> >>>> Firstly, this is hard when consumed by > non-Java clients as you'd need to >>>> implement the way the JDK calculates the hash > code of a byte array. >>> >>> This is a much easier problem if you don't use the > built in hashing. >> >> This is true. The current impl simply calls > key.hashcode() and passes the result through a Wang/Jenkins > based bit spreader. I suppose we could be smarter and > for keys which are byte arrays (and this will *always* be > the case if interacting via HotRod) something like FNV-1 or > SuperFastHash could be used. I was even looking at > MurmurHash. >> >> http://sites.google.com/site/murmurhash/ >> >> We probably wouldn't even need a bit spreader in this > case, and a MurmurHash impls exist for several platforms. >> >> http://en.wikipedia.org/wiki/MurmurHash > > Maybe MurmurHash isn't such a good idea. A comment in > Austin Appleby's C++ impl: > > // 2. It will not produce the same results on little-endian > and big-endian > // machines. > > So while JVMs shield you from CPU endian-ness, clients > would need to be run on machines of the same CPU-endian-ness > to make it work. > > >>> There are >>> a number of hash algorithms that can be used, > including >>> FNV 1 (http://www.isthe.com/chongo/tech/comp/fnv/) >>> and others http://www.azillionmonkeys.com/qed/hash.html. >>> >>> These are implementable in other languages, are > fast, and provide good distributions of >>> results. We can use, and similarly document, an > integer hash used to further spread the >>> hash values if needed. If these are chosen to be > implementable in multiple languages, >>> clearly documented, and don't change too often, it > should be reasonable to put them into >>> the client. >>> >>> Using this approach removes the dependency on the > hash that the VM happens to be >>> using. Indeed, the hash for a byte array may > simply be the address of the array, which >>> makes it very poor for our use. >>> > > > > Alex >>> >>> --- On Thu, 3/18/10, Manik Surtani <manik(a)jboss.org&gt; > wrote: >>> >>>> From: Manik Surtani <manik(a)jboss.org&gt; >>>> Subject: [infinispan-dev] HotRod, > ClientIntelligence and client-side key location >>>> To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; >>>> Date: Thursday, March 18, 2010, 11:59 AM >>>> I've been thinking about how we >>>> handle this, and I think we have a problem > with smart >>>> clients where clients have the ability to > locate the key on >>>> the server cluster in order to direct the > request to the >>>> specific node. >>>> >>>> The problem is in hash code calculation. > The HotRod >>>> protocol caters for this with regards to > calculating node >>>> address hash code by passing this in the > topology map (see >>>> "Hasher Client Topology Change Header" in > [1]), but the only >>>> way this can be meaningfully used is if the > client has the >>>> ability to calculate the hash code of the key > in the same >>>> manner the servers do. Firstly, this is > hard when >>>> consumed by non-Java clients as you'd need to > implement the >>>> way the JDK calculates the hash code of a byte > array. >>>> Second, you'd need detailed and specific > knowledge of any >>>> bit spreading that takes place within > Infinispan - and this >>>> is internal implementation detail which may > change from >>>> release to release. >>>> >>>> So the way I see it I can't see how non-Java > clients will >>>> be able to locate keys and then direct > requests to the >>>> necessary nodes. In fact, even with Java > clients the >>>> only way this could be done would be to send > back marshalled >>>> Addresses in the topology map, *and* have the > same version >>>> of the Infinispan server libs installed on the > client, *and* >>>> ensure that the same JDK/JVM version is used > on the >>>> client. >>>> >>>> Can we think of a better way to do this? > If not, is >>>> it worth still supporting client-side > consistent hash based >>>> key location for the weird but vaguely > workable scenario for >>>> Java-based clients? >>>> >>>> Thoughts? >>>> >>>> Cheers >>>> Manik >>>> >>>> >>>> [1] http://community.jboss.org/wiki/HotRodProtocol >>>> -- >>>> Manik Surtani >>>> manik(a)jboss.org >>>> Lead, Infinispan >>>> Lead, JBoss Cache >>>> http://www.infinispan.org >>>> http://www.jbosscache.org >>>> >>>> >>>> >>>> >>>> >>>> > _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

See below: On Thu, 18 Mar 2010 22:26:44 +0100, Alex Kluge <java_kluge(a)yahoo.com&gt; wrote: > </snip> > >> Firstly, this is hard when consumed by non-Java clients as you'd need > to >> implement the way the JDK calculates the hash code of a byte array. > > This is a much easier problem if you don't use the built in hashing. > There are > a number of hash algorithms that can be used, including > FNV 1 (http://www.isthe.com/chongo/tech/comp/fnv/) > and others http://www.azillionmonkeys.com/qed/hash.html. > > These are implementable in other languages, are fast, and provide good > distributions of > results. We can use, and similarly document, an integer hash used to > further spread the > hash values if needed. If these are chosen to be implementable in > multiple languages, > clearly documented, and don't change too often, it should be reasonable > to put them into > the client. > > Using this approach removes the dependency on the hash that the VM > happens to be > using. Indeed, the hash for a byte array may simply be the address of > the array, which > makes it very poor for our use. This is a very good point. This could even be done internally. If your starting Infinispan normally, use default CHA. If you're starting a Hot Rod server, the implementation could inject one of these CHA which are clearly documented and are easy to implement in other languages. That way you'd get the best of both worlds. You don't need to expose your internal details for normal Infinispan and we have a robust, stable and easy to implement algorithm in other languages.

Cheers, > Alex > > --- On Thu, 3/18/10, Manik Surtani <manik(a)jboss.org&gt; wrote: > >> From: Manik Surtani <manik(a)jboss.org&gt; >> Subject: [infinispan-dev] HotRod, ClientIntelligence and client-side >> key location >> To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; >> Date: Thursday, March 18, 2010, 11:59 AM >> I've been thinking about how we >> handle this, and I think we have a problem with smart >> clients where clients have the ability to locate the key on >> the server cluster in order to direct the request to the >> specific node. >> >> The problem is in hash code calculation. The HotRod >> protocol caters for this with regards to calculating node >> address hash code by passing this in the topology map (see >> "Hasher Client Topology Change Header" in [1]), but the only >> way this can be meaningfully used is if the client has the >> ability to calculate the hash code of the key in the same >> manner the servers do. Firstly, this is hard when >> consumed by non-Java clients as you'd need to implement the >> way the JDK calculates the hash code of a byte array.Second, you'd need >> detailed and specific knowledge of any >> bit spreading that takes place within Infinispan - and this >> is internal implementation detail which may change from >> release to release. >> >> So the way I see it I can't see how non-Java clients will >> be able to locate keys and then direct requests to the >> necessary nodes. In fact, even with Java clients the >> only way this could be done would be to send back marshalled >> Addresses in the topology map, *and* have the same version >> of the Infinispan server libs installed on the client, *and* >> ensure that the same JDK/JVM version is used on the >> client. >> Can we think of a better way to do this? If not, is >> it worth still supporting client-side consistent hash based >> key location for the weird but vaguely workable scenario for >> Java-based clients? >> >> Thoughts? >> >> Cheers >> Manik >> >> >> [1] http://community.jboss.org/wiki/HotRodProtocol >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

See below: On Mon, 22 Mar 2010 13:02:15 +0100, Manik Surtani <manik(a)jboss.org&gt; wrote: > > On 22 Mar 2010, at 11:16, Galder Zamarreno wrote: > >> See below: >> >> On Thu, 18 Mar 2010 22:26:44 +0100, Alex Kluge <java_kluge(a)yahoo.com&gt; >> wrote: >> >>> </snip> >>> >>>> Firstly, this is hard when consumed by non-Java clients as you'd need >>> to >>>> implement the way the JDK calculates the hash code of a byte array. >>> >>> This is a much easier problem if you don't use the built in hashing. >>> There are >>> a number of hash algorithms that can be used, including >>> FNV 1 (http://www.isthe.com/chongo/tech/comp/fnv/) >>> and others http://www.azillionmonkeys.com/qed/hash.html. >>> >>> These are implementable in other languages, are fast, and provide good >>> distributions of >>> results. We can use, and similarly document, an integer hash used to >>> further spread the >>> hash values if needed. If these are chosen to be implementable in >>> multiple languages, >>> clearly documented, and don't change too often, it should be reasonable >>> to put them into >>> the client. >>> >>> Using this approach removes the dependency on the hash that the VM >>> happens to be >>> using. Indeed, the hash for a byte array may simply be the address of >>> the array, which >>> makes it very poor for our use. >> >> This is a very good point. This could even be done internally. If your >> starting Infinispan normally, use default CHA. If you're starting a Hot >> Rod server, the implementation could inject one of these CHA which are >> clearly documented and are easy to implement in other languages. That >> way >> you'd get the best of both worlds. You don't need to expose your >> internal >> details for normal Infinispan and we have a robust, stable and easy to >> implement algorithm in other languages. > > On no, it shouldn't even be as complex as that. What I suggest is this: > > DefaultConsistentHash currently uses a Wang/Jenkins hash as a > bit-spreader on key.hashcode(), and address.hashcode(). > > This is poor, since this is JVM dependent. Secondly (and for a > different reason) the W/J hash isn't providing us with adequate spread, > as we've found out. > > So step 1 is to identify a better-spread hash, some have been suggested > on this thread, preferably one that can operate directly on byte[]'s and > eliminate the need for a bit spreader. > > The next step would be to change DefaultConsistentHash to: > > * For addresses, use ${HASH_FUNCTION} on address.hashcode()

> * For keys which are byte[]s, use ${HASH_FUNCTION} directly on the > byte[] (this would directly benefit use via HotRod) > * For keys which are Strings, use ${HASH_FUNCTION} directly on the > String (this is an optimisation) > * For keys which are Objects, use ${HASH_FUNCTION} on object.hashcode() > (for in-VM use) Yeah, this would much simpler to implement and maintain, hence +1 > > We would need to document ${HASH_FUNCTION} as a part of the HotRod > protocol, and to successfully locate entries, clients would need the > following info: > > * Server endpoints on the backend and their address.hashcode() values We have that in the protocol. > * Hash space size (the modulus for all modular arithmetic, hard-coded > for now, may change in future). An int. That would need adding to the response header, wouldn't it? http://community.jboss.org/wiki/HotRodProtocol#HashDistributionAware_Clie... > * Hash function version. This could point to details on the spec. This > could be a short. You've included that, good. > * Num owners the servers have been configured to use. This again could > be a short as far as HotRod is concerned. > > WDYT? > > Cheers > Manik > >> Cheers, >> >>> Alex >>> >>> --- On Thu, 3/18/10, Manik Surtani <manik(a)jboss.org&gt; wrote: >>> >>>> From: Manik Surtani <manik(a)jboss.org&gt; >>>> Subject: [infinispan-dev] HotRod, ClientIntelligence and client-side >>>> key location >>>> To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; >>>> Date: Thursday, March 18, 2010, 11:59 AM >>>> I've been thinking about how we >>>> handle this, and I think we have a problem with smart >>>> clients where clients have the ability to locate the key on >>>> the server cluster in order to direct the request to the >>>> specific node. >>>> >>>> The problem is in hash code calculation. The HotRod >>>> protocol caters for this with regards to calculating node >>>> address hash code by passing this in the topology map (see >>>> "Hasher Client Topology Change Header" in [1]), but the only >>>> way this can be meaningfully used is if the client has the >>>> ability to calculate the hash code of the key in the same >>>> manner the servers do. Firstly, this is hard when >>>> consumed by non-Java clients as you'd need to implement the >>>> way the JDK calculates the hash code of a byte array.Second, you'd >>>> need >>>> detailed and specific knowledge of any >>>> bit spreading that takes place within Infinispan - and this >>>> is internal implementation detail which may change from >>>> release to release. >>>> >>>> So the way I see it I can't see how non-Java clients will >>>> be able to locate keys and then direct requests to the >>>> necessary nodes. In fact, even with Java clients the >>>> only way this could be done would be to send back marshalled >>>> Addresses in the topology map, *and* have the same version >>>> of the Infinispan server libs installed on the client, *and* >>>> ensure that the same JDK/JVM version is used on the >>>> client. >>>> Can we think of a better way to do this? If not, is >>>> it worth still supporting client-side consistent hash based >>>> key location for the weird but vaguely workable scenario for >>>> Java-based clients? >>>> >>>> Thoughts? >>>> >>>> Cheers >>>> Manik >>>> >>>> >>>> [1] http://community.jboss.org/wiki/HotRodProtocol >>>> -- >>>> Manik Surtani >>>> manik(a)jboss.org >>>> Lead, Infinispan >>>> Lead, JBoss Cache >>>> http://www.infinispan.org >>>> http://www.jbosscache.org >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Galder Zamarreño >> Sr. Software Engineer >> Infinispan, JBoss Cache >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

> > * Hash space size (the modulus for all modular arithmetic, hard-coded > for now, may change in future). An int. That would need adding to the response header, wouldn't it? http://community.jboss.org/wiki/HotRodProtocol#HashDistributionAware_Clie...

On Tue, 23 Mar 2010 14:12:19 +0100, Manik Surtani <manik(a)jboss.org&gt; wrote: > > On 23 Mar 2010, at 07:48, Galder Zamarreno wrote: > >>> >>> * Hash space size (the modulus for all modular arithmetic, hard-coded >>> for now, may change in future). An int. >> >> That would need adding to the response header, wouldn't it? >> http://community.jboss.org/wiki/HotRodProtocol#HashDistributionAware_Clie... > > > Umm, not sure what you mean. This is has nothing to do with server (or > even endpoint) count... I would have thought this is sent along with > hash function version. Oh right, so what you're implying is that the version itself defines the corresponding hash space size? When I first read it, I thought clients could pass different space size for each version, hence why I was wondering if it should be part of the protocol.

> > Cheers > Manik > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

So, back to my original point, if you're sending this together with the hash function version, you need to include "hash space size" in the response header, don't u? To refresh memory, when a server detects a topology change, it will send back the cluster formation plus: hash function version and number of owners. So, if you want the client to apply the hash function correctly, you'd need to let it know about the hash space size as well, correct?

> I foresee making the hash space size a tuning parameter in the future. This is independent of hash function used. Are all the hash functions going to have an hash space parameter? Other functions might need some additional config params. What about not "hardcoding" params in the protocol itself, but use a generic way to send them over: ... bytes[ h f version] [param number] [param name 1] [param value 1] etc. Each function should be able to read its own params.

On 6 Apr 2010, at 19:37, Manik Surtani wrote: > > On 2 Apr 2010, at 00:59, Mircea Markus wrote: > >>> I foresee making the hash space size a tuning parameter in the future. This is independent of hash function used. >> Are all the hash functions going to have an hash space parameter? Other functions might need some additional config params. What about not "hardcoding" params in the protocol itself, but use a generic way to send them over: >> ... bytes[ h f version] [param number] [param name 1] [param value 1] etc. >> Each function should be able to read its own params. > > > I don't see what other params could exist? Any examples? nr of virtual nodes for a different implementation. In future we might also want to send the number of owners. Another impl might come with its own params. > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 14 Apr 2010, at 10:22, Galder Zamarreno wrote: > Apologies for the delay getting back to this. I think Mircea does have a point here. > > I think his suggestion might make the the hash function parameters more future proof at the cost of a vInt for the parameter number and a String with the parameter name in the response. +1. This is more generic and the cost is only one additional param (int?), that's only sent once per HR client. > > I can't however give individual examples on parameters that'd be needed in the future. What about virtual node count, for a function that uses virtual nodes?

> > Mircea, what did you refer to by sending number of owners? We already do that in the current protocol spec. I missed that. > > Cheers, > > ----- "Mircea Markus" <mircea.markus(a)jboss.com&gt; wrote: > >> On 6 Apr 2010, at 19:37, Manik Surtani wrote: >> >>> >>> On 2 Apr 2010, at 00:59, Mircea Markus wrote: >>> >>>>> I foresee making the hash space size a tuning parameter in the >> future. This is independent of hash function used. >>>> Are all the hash functions going to have an hash space parameter? >> Other functions might need some additional config params. What about >> not "hardcoding" params in the protocol itself, but use a generic way >> to send them over: >>>> ... bytes[ h f version] [param number] [param name 1] [param value >> 1] etc. >>>> Each function should be able to read its own params. >>> >>> >>> I don't see what other params could exist? Any examples? >> nr of virtual nodes for a different implementation. In future we might >> also want to send the number of owners. Another impl might come with >> its own params. >>> >>> -- >>> Manik Surtani >>> manik(a)jboss.org >>> Lead, Infinispan >>> Lead, JBoss Cache >>> http://www.infinispan.org >>> http://www.jbosscache.org >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 20 Apr 2010, at 16:58, Manik Surtani wrote: > > On 16 Apr 2010, at 23:45, Mircea Markus wrote: > >> >> On 14 Apr 2010, at 10:22, Galder Zamarreno wrote: >> >>> Apologies for the delay getting back to this. I think Mircea does have a point here. >>> >>> I think his suggestion might make the the hash function parameters more future proof at the cost of a vInt for the parameter number and a String with the parameter name in the response. >> +1. This is more generic and the cost is only one additional param (int?), that's only sent once per HR client. >>> >>> I can't however give individual examples on parameters that'd be needed in the future. >> What about virtual node count, for a function that uses virtual nodes? > > There is no guarantee that we'd ever implement a virt node mechanism. sure, but we might implement it, and somebody else might implement other functions that need other params