On 16 May 2013, at 19:12, cotton-ben <ben.cotton(a)ALUMNI.RUTGERS.EDU&gt; wrote: Added to the FAQ: https://docs.jboss.org/author/display/ISPN/Grouping+API+vs+Key+Affinity+S... Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org)

Done. see https://issues.jboss.org/browse/ISPN-3112 -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

Want to point out that without this signature added to the API *K getKeyForAddress(Address address, K otherKey); * /*compute address specific version of otherKey*/ we are finding the KeyAffinityService unusable. It is impossible for a node, that is not the same as the node that produced the AffinityKey, to operate on the object. i.e. it cannot reproduce the AffinityKey! -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

We need a way for a "foreign" node to be able to come up with the same affinityKey produced earlier by a "local" node. Thanks for the response, Mircea. Let me elaborate. E.g. consider the infinispan quick start that is made up of AbstractNode.java, Node0.java, Node1.java, Node2.java Let's say that @t=0 we execute the following code within Node0.java: //assume keySvc reference constructed and .start() invoked Object pinnedDimensionKeyAffinityAtNode0 = keySvc.getCollocatedKey(myObjectNaturalKey); this.dimensionsPinnedCache.put(pinnedDimensionKeyAffinityAtNode0, myObject); Now, let's say it is @t=1 and we want to get() 'myObject' from code within Node2.java ... how? I don't think it is possible with the existing API. However if you add the signature requested, I could do something like this: Object pinnedKey = keySvc.getKeyForAddress(getAddress(Node0), myObjectNaturalKey);//need this API! myObject = this.dimensionsPinnedCache.get(pinnedKey); I need some way for Node2.java to know that it can re-compute the affinityKey rendered by Node0.java ... that way Node2 can operate on <K,V> sets pinned by affinity to Node0 Maybe I am missing something, is there some with the existing 5.3 API for Node2 to do this? Or do we need to add the signature suggested at https://issues.jboss.org/browse/ISPN-3112? As far as I can tell, I think we need to add the requested signature. Without it, I don't think Node2.java can do the operation described above. -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

Thanks for this reply, Mircea. Very interesting approach. By dispatching a distributed executor back to the node (node 0) that produced the pinned key affinity for the object's natural key, we could indeed do an iterative search (from node 2) to find the original 'myObject' in the pinned cache (on node 0). As you point out, however, this approach is prohibitively costly -- we are effectively doing an iterative search across nodes. Again, we see the best approach as being to use the enhancement to the Key Affinity Service API suggested in https://issues.jboss.org/browse/ISPN-3112. Once *K getKeyForAddress(Address pinnedNodeAddress, K objectNaturalKey);* is provided, we will be able to proceed as simply as follows: //given that all nodes "know" that all birthday Entry(s) are pinned to Node 0 @t=0, from node=0, pin Entry to cache at Node 0 Object pinnedKey = getKeyForAddress(getAddress("Node0"), "1962-08-10"); cache.put(pinnedKey,"Ben D. Cotton III birthday"); Later @t=1, from node =2, we can use the improved API (suggested in the JIRA) from a 'foreign' node without any need to do a distributed, iterative search, i.e. Object pinnedKey = getKeyForAddress(getAddress("Node0"), "1962-08-10"); Object birthdayCelebrant = cache.get(pinnedKey); //return "Ben D. Cotton III birthday" Of course, for us to be able to do this, the ISPN team would have to deliver to us the API recommended in the JIRA https://issues.jboss.org/browse/ISPN-3112. Don't you agree such an API enhancement would be an awesome capability improvement for ISPN users wishing to operate in a distributed fashion on pinned Caches? :-) As always, thanks to you and this forum for its fine support, Ben -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

/It's probably possible to define something like this instead: PinnedKey<K> getKeyForAddress(address, key) / This is fine. Do it this way. /Even so, keeping the result key pinned to the same address after a rebalance would be tricky - probably requiring some changes to the ConsistentHash implementation. Of course, the node to which the key is pinned may also die, and I'm not sure where the keys should be located then. / But, shouldn't some combination of my usage of the ISPN Grouper<T> and the ISPN KeyAffintyService capabilities cover me? In my understanding, the Grouper capability can allow me to "group" my keys to a specific node (however node identity is anonymous). Grouper capability's ambition is to transparently survive topology changes of nodes participating in my grid. In my understanding, the KAS capability can allow me to "pin" my keys to a specific node (by node identity). A potetnailly great tutorial document for ISPN users might be "How to harmoniously use the ISPN Grouper and ISP KeyAffinityService Capabilities". How to use these 2 meritorious capabilies together (and in harmony) is still unclear to me. /Speaking of which, how do you know which keys should map to which node in your application?/ We just know. :-) Seriously, we do have a mechanism for resolving pinnedKey identity to Node identity custody. /If the rules are simple enough, you may be able to create multiple caches, each of them with a custom ConsistentHashFactory that locates *all* the keys to the same primary node (with random backup owners)./ Very true. But we don't think that is as nice as extending the API in the way recommend. To be blunt, we could just do something as simple as this: @t=0, from node=0, pin Entry to cache at Node 0 Object pinnedKey = getKeyForAddress(getAddress("Node0"),"1962-08-10"); globalReplicatedPinnedKeyCache.put("1962-08-10", pinnedKey); //cache the pinnedKeys globally pinnedCache.put(pinnedKey,"Ben D. Cotton III birthday"); Later @t=1, from node =2 Object pinnedKey = globalReplicatedPinnedKeyCache.get("1962-08-10"); Object birthdayCelebrant = pinnedCache.get(pinnedKey); //return "Ben D. Cotton III birthday" But that is effectively forcing us to do 2 explicit Cache<K,V>#get() operations in application code. First to get the pinnedKey (given the naturalKey). Second to get the pinnedValue (given the pinnedKey). Not preferred -- but DOABLE, FOR SURE. What do you think? Just for the "niceness" of it, do you think we might be able to have the API suggested at https://issues.jboss.org/browse/ISPN-3112 ? -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

To be honest, Dan, your last post leaves me slightly concerned. / No. that would be no good. /> But it has been my impression that you want something that would keep the key pinned to the same node as long as that node was alive. If that's the case, then implementing it would require significant additional work./ Yes. that is what I want. /> I'm just wondering, wouldn't it be simpler if you created a custom ConsistentHashFactory to map keys to nodes using the same mechanism you already have? Not as easy as the API you proposed, but it wouldn't require a separate cache either.../ I have no basis for commenting on this competently. I am just not familiar with ConsistentHashFactory, etc. I do know the API I propose would be easy for me. But that is irrelevant if it is not doable. /> With this solution, "pinned" keys could also move from the initial target node to another node as the cache topology changes (and not all keys will move to the same node). / Well, I guess I don't care if they move from pinnedNodeX to pinnedNodeY as long as you gave me a reliable API that empowers me to answer for myself "Where dey at?". i.e. as long as this requested API works the way I want it to work *K getKeyForAddress(Address pinnedNodeAddress, K objectNaturalKey); * /> My main concern is that you actually want something more than the KeyAffinityService is able to give you ATM - even if we did add the "niceness" in the API, the keys still won't be pinned to a certain node for as long as that node is alive./ Ok. Greatly appreciate the candor of this comment. Just sharing with you my hopes for what I would like to see in an ideal API (if delivery of that API is in fact possible). BTW, how cool is it that a little ole ISPN user like me can come here and get this kind of interactive dialog and support? SUPERB! And appreciated. :-) Thanks. -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

so, in the event that no topology change takes place, may I otherwise consider this key2node association /reliable/? Mircea Markus-2 wrote -- View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/KeyAffinityService-... Sent from the Infinispan Developer List mailing list archive at Nabble.com.

Sent from my iPhone On 14 Jun 2013, at 13:43, cotton-ben <ben.cotton(a)ALUMNI.RUTGERS.EDU&gt; wrote: My solution only makes sense if the vast majority of requests are processed by the node that is aware of the key. Another thing you can do is have a replicated Cache holding the mapping between the actual keys and the affinity keys. Whenever you need to lookup an obj based on the actual key, you'd do a lookup in this cache(local) and then use this key for getting the object you're after. This cache shoul be pretty small as well. I understand your requirements and also the usefulness of such a service :-) I don't think it is possible to implement this without a severe limitation: if the topology changes between the moment the key was created and the moment the method is invoked, it would return an incorrect value(i'm not even sure that this woul work but i have an idea at least). Whilst this might be enough for your application, it's not sure it is a general purpose functionality for a fwk.

Sent from my iPhone On 14 Jun 2013, at 19:14, cotton-ben <ben.cotton(a)ALUMNI.RUTGERS.EDU&gt; wrote: As I've mentioned in the previous email, for your requirement, the only solution i can think of, *if* possible, has severe limitations when it comes to implementing it. Can you leave with this limitations?

Sent from my iPhone On 14 Jun 2013, at 19:39, cotton-ben <ben.cotton(a)ALUMNI.RUTGERS.EDU&gt; wrote: Just to reiterate, your *never* expects a node to crash?

Sent from my iPhone On 14 Jun 2013, at 20:02, cotton-ben <ben.cotton(a)ALUMNI.RUTGERS.EDU&gt; wrote: I'm just trying to help Ben.