Hi Pedro,
Agreed that this looks a lot like the state transfer, and +1 to implement it on top of
it.
Do you plan to implement this on top of 5.1.x? Or 5.2.x with the new NBST? If later, I
think you should coordinate with Dan/Adrian who are working on it right now.
Cheers,
Mircea
----- Original Message -----
From: "Pedro Ruivo" <pruivo(a)gsd.inesc-id.pt>
To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org>,
"cloudtm-discussion"
<cloudtm-discussion(a)lists.sourceforge.net>
Cc: "Paolo Romano" <romano(a)inesc-id.pt>
Sent: Thursday, June 28, 2012 5:33:49 PM
Subject: [infinispan-dev] Dynamic data placement
Hi all,
In the Cloud-TM project we are working on a technique to change
dynamically the distribution of data across the nodes of an ISPN data
grid. The idea is basically to determine at run-time which are the
keys
most accessed by each member/node and move these keys to that nodes.
The
ultimate goal is to maximize data locality and reduce the number of
remote get operation.
One of the ingredients of the solution is to define a mechanism that
allows the entire data grid, which is running in a configuration A,
having a given data distribution strategy (determined by the current
hash function implementation), to switch to a configuration B, in
which
all nodes have agreed on a new data distribution strategy, and have
completed to relocate accordingly the keys.
Our idea therefore is to add a functionality in ISPN, which allows to
support the dynamic change of the data distribution criterion (namely
the consistent hash function).
To us this mechanism resembles closely the state transfer problem,
except that in the latter case the keys are relocated when a node
joins
or leaves the system.
We were thinking to implement this by introducing a command (say,
RedistributedDataCommand) that would take as input a new hash
function,
and that would internally trigger a state transfer based on the hash
function passed as input parameter. Another alternative would be to
change the CacheViewControlCommand, allowing it to distributed across
all the nodes the new hash function, as well as to trigger the state
transfer.
The point is that we would like to do this in a clean way, in order
to
simplify a possible future integration. We will therefore be grateful
for any feedback that you will be able to provide.
Cheers,
Pedro Ruivo
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev