[infinispan-issues] [JBoss JIRA] Updated: (ISPN-1408) Reduce Hot Rod topology information memory consumption

[ https://issues.jboss.org/browse/ISPN-1408?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-1408: ----------------------------------- Description: Hot Rod server storage of topology views should be trimmed. Currently it stores: ||key||value|| |view|view_id,[sever_endpoint,cluster_addr [cache_name:hash_id]]
| This results in storing far too much information, and particularly when virtual nodes are enabled, this explodes in size. This affects both Hot Rod memory consumption and network activity since these ids and server endpoint information needs to be send back to client. For example, with 1000 virtual nodes, 8 nodes and 10 caches and taking in account each hash id is a 4 byte integer, that results in ~300kb of data. On top of that, this is stored in a single key which means that concurrent startups can be problematic. There's a couple of things that can be done to reduce the size of this: # Get clients to calculate hash ids. This will require configured number of virtual nodes to be sent back to clients and also ISPN-1404 so that both the client and the server can hash node positions on the same thing. # Piggy back on JGroups' view id to avoid Hot Rod having to maintain its own view id. The end result is: ||key||value|| |cluster|sever_endpoint| Which reduces the memory consumption and avoids concurrent startup issues. This would require a new version of the Hot Rod protocol, potentially a 1.1 version rather than 2.0 was: Hot Rod server storage of topology views should be trimmed. Currently it stores: ||key||value|| |view|view_id, [sever_endpoint, cluster_addr, [cache_name:hash_id]]
| This results in storing far too much information, and particularly when virtual nodes are enabled, this explodes in size. This affects both Hot Rod memory consumption and network activity since these ids and server endpoint information needs to be send back to client. For example, with 1000 virtual nodes, 8 nodes and 10 caches and taking in account each hash id is a 4 byte integer, that results in ~300kb of data. On top of that, this is stored in a single key which means that concurrent startups can be problematic. There's a couple of things that can be done to reduce the size of this: # Get clients to calculate hash ids. This will require configured number of virtual nodes to be sent back to clients and also ISPN-1404 so that both the client and the server can hash node positions on the same thing. # Piggy back on JGroups' view id to avoid Hot Rod having to maintain its own view id. The end result is: ||key||value|| |cluster, sever_endpoint| Which reduces the memory consumption and avoids concurrent startup issues. This would require a new version of the Hot Rod protocol, potentially a 1.1 version rather than 2.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira