[infinispan-dev] Dynamic data placement

Mircea Markus mmarkus at redhat.com
Mon Jul 2 07:27:05 EDT 2012


Hi Pedro,

Agreed that this looks a lot like the state transfer, and +1 to implement it on top of it.
Do you plan to implement this on top of 5.1.x? Or 5.2.x with the new NBST? If later, I think you should coordinate with Dan/Adrian who are working on it right now.

Cheers,
Mircea   

----- Original Message -----
> From: "Pedro Ruivo" <pruivo at gsd.inesc-id.pt>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>, "cloudtm-discussion"
> <cloudtm-discussion at lists.sourceforge.net>
> Cc: "Paolo Romano" <romano at inesc-id.pt>
> Sent: Thursday, June 28, 2012 5:33:49 PM
> Subject: [infinispan-dev] Dynamic data placement
> 
> Hi all,
> 
> In the Cloud-TM project we are working on a technique to change
> dynamically the distribution of data across the nodes of an ISPN data
> grid. The idea is basically to determine at run-time which are the
> keys
> most accessed by each member/node and move these keys to that nodes.
> The
> ultimate goal is to maximize data locality and reduce the number of
> remote get operation.
> 
> One of the ingredients of the solution is to define a mechanism that
> allows the entire data grid, which is running in a configuration A,
> having a given data distribution strategy (determined by the current
> hash function implementation), to switch to a configuration B, in
> which
> all nodes have agreed on a new data distribution strategy, and have
> completed to relocate accordingly the keys.
> 
> Our idea therefore is to add a functionality in ISPN, which allows to
> support the dynamic change of the data distribution criterion (namely
> the consistent hash function).
> 
> To us this mechanism resembles closely the state transfer problem,
> except that in the latter case the keys are relocated when a node
> joins
> or leaves the system.
> 
> We were thinking to implement this by introducing a command (say,
> RedistributedDataCommand) that would take as input a new hash
> function,
> and that would internally trigger a state transfer based on the hash
> function passed as input parameter. Another alternative would be to
> change the CacheViewControlCommand, allowing it to distributed across
> all the nodes the new hash function, as well as to trigger the state
> transfer.
> 
> The point is that we would like to do this in a clean way, in order
> to
> simplify a possible future integration. We will therefore be grateful
> for any feedback that you will be able to provide.
> 
> Cheers,
> Pedro Ruivo
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 



More information about the infinispan-dev mailing list