[infinispan-dev] Dynamic data placement

Hi all, In the Cloud-TM project we are working on a technique to change dynamically the distribution of data across the nodes of an ISPN data grid. The idea is basically to determine at run-time which are the keys most accessed by each member/node and move these keys to that nodes. The ultimate goal is to maximize data locality and reduce the number of remote get operation. One of the ingredients of the solution is to define a mechanism that allows the entire data grid, which is running in a configuration A, having a given data distribution strategy (determined by the current hash function implementation), to switch to a configuration B, in which all nodes have agreed on a new data distribution strategy, and have completed to relocate accordingly the keys. Our idea therefore is to add a functionality in ISPN, which allows to support the dynamic change of the data distribution criterion (namely the consistent hash function). To us this mechanism resembles closely the state transfer problem, except that in the latter case the keys are relocated when a node joins or leaves the system. We were thinking to implement this by introducing a command (say, RedistributedDataCommand) that would take as input a new hash function, and that would internally trigger a state transfer based on the hash function passed as input parameter. Another alternative would be to change the CacheViewControlCommand, allowing it to distributed across all the nodes the new hash function, as well as to trigger the state transfer. The point is that we would like to do this in a clean way, in order to simplify a possible future integration. We will therefore be grateful for any feedback that you will be able to provide. Cheers, Pedro Ruivo