[infinispan-issues] [JBoss JIRA] Issue Comment Edited: (ISPN-1090) Hot Rod protocol improvements for version 2

[Galder Zamarreño (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12612485#comment-12612485 ] 

Galder Zamarreño edited comment on ISPN-1090 at 7/7/11 8:31 AM:
----------------------------------------------------------------

To support virtual nodes more efficiently, the hash topology header must change:
- -When virtual nodes are enabled, we're sending host:port information repeated for each of the hash ids belonging to a HotRod server. So, the protocol needs to be more efficient in v2 sending the host:port only once and then N hash ids, where N is the number of configured virtual nodes. If virtual nodes is disabled, the default is 1 virtual node, so N=1.-

Actually, forget what I just said cos as Manik rightly pointed out in a chat we had earlier, it won't be atypical to configure with 500-1000 vnodes. So, even if we applied the suggested change, that'd still require, for a cluster of 10 nodes, shipping 10*1000*4bytes=~40kb which is not very optimal.

During the chat it was suggested that the clients themselves generate the hash ids, but in order to do that, they need to have the same hash seed that the Infinispan server does, and currently that is org.infinispan.remoting.transport.Address which represents the cluster address. Clearly, this won't work for clients but what about if the server hashed on String UTF-8 version of org.infinispan.remoting.transport.Address? If that worked, then a hash topology update need only shipping: existing header params plus num virtual nodes, and tuples for host:port:clusterAddress(UTF-8). The key thing here is that the number of tuples to send back would be equal to the number of nodes in the cluster. The clients could then generate the hash codes for all virtual nodes themselves.

At the moment, Infinispan cluster addresses are backed by JGroups address which are UUID driven, so its toString() representation seems deterministic. However, if there was ever a move back to IP based addresses that would break because toString() representations rely on DNS lookups which means the toString() representation could vary over time.

@Pete, could you confirm whether this change on the key on which cluster addresses are hashed would work as expected?

The hash topology header size could be further reduced if clients were assumed to be able to keep state in which case the server could just send information about nodes that have joined or left. Since nodes pick fixed positions in the hash wheel, this would in theory work cos the addition of a node would not require other nodes to recalculate hashes.

      was (Author: galder.zamarreno):
    To support virtual nodes more efficiently, the hash topology header must change:
- When virtual nodes are enabled, we're sending host:port information repeated for each of the hash ids belonging to a HotRod server. So, the protocol needs to be more efficient in v2 sending the host:port only once and then N hash ids, where N is the number of configured virtual nodes. If virtual nodes is disabled, the default is 1 virtual node, so N=1.
  
> Hot Rod protocol improvements for version 2
> -------------------------------------------
>
>                 Key: ISPN-1090
>                 URL: https://issues.jboss.org/browse/ISPN-1090
>             Project: Infinispan
>          Issue Type: Enhancement
>          Components: Cache Server
>            Reporter: Galder Zamarreño
>            Assignee: Galder Zamarreño
>             Fix For: 5.2.0.Final
>
>
> This JIRA will act as a place to keep track of enhancements to the Hot Rod protocol that should be included in version 2 of the protocol:
> - We need a new error status code so that clients can detect when a node has been suspected and so react accordingly (i.e. not bubble it up but instead failover). Currently, this can be solved by checking whether the error message contains SuspectException but this is not nice.
> - Since we're not gonna be supporting asymmetric clusters in the mean time, we need a new error status code to deal with situations where the cache is not defined in the server. Currently we check for CacheNotFoundException in the message but that's not nice.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira