Actually I'd like regular users to be able to define their own RebalancePolicies and ConsistentHashFactories, without modifying ClusterTopologyManagerImpl/ClusterCacheStatus, so I see this as a good opportunity to modify our implementation to allow it.


On Tue, Feb 12, 2013 at 6:04 PM, Pedro Ruivo <pruivo@gsd.inesc-id.pt> wrote:
Can I modify the existing one?

I'm thinking in the following:

ClusterTopologyManagerImpl.handleNewMappings(...) { //new method
  ClusterCacheStatus status = //get status for cache name
  status.setNewMappings(...) //synchronized of course
  rebalancePolicy.updateCacheStatus(...);
}


Like I said, I'd like to keep ClusterTopologyManager as generic as possible wrt rebalance strategies, so I think your DataPlacementManager should call your custom RebalancePolicy directly.

The rebalance policy could keep the new mappings in a map on its own, although maybe it would be a nice touch to allow storing custom state in ClusterCacheStatus.

 
DefaultRebalancePolicy.updateCacheStatus(...) { //modified
  ...
  if (!status.hasJoiners() && isBalanced(...) && !status.hasNewMappings()) { //added last condition
    return;
  }
  ...
}

I guess you'd also need to clean up the old "new mappings" here after the rebalance is done.

 

ClusterTopologyManagerImpl.startRebalance(...) { //modifed
  ...
  chFactory.rebalance(ch);
  chFactory.applyMappings(ch, status.getNewMappings()); //added.
  ... //if it is the same ch, no state transfer is triggered
}



This would require ClusterTopologyManagerImpl to know about your custom ConsistentHashFactory, and it wouldn't work with another ConsistentHashFactory that requires different custom data. So I'd rather we add a generic parameter to ConsistentHashFactory.rebalance.


What do you think?

Thanks,
Pedro


On 2/12/13 3:39 PM, Dan Berindei wrote:
Sorry, I didn't read your code so I just assumed you're writing your own RebalancePolicy.

I think you need to implement your own RebalancePolicy, because ClusterTopologyManagerImpl by itself doesn't remember that a rebalance was triggerred. So if you call startRebalance, but there is already a rebalance in progress, it is just ignored. When the in-progress rebalance finishes, it calls RebalancePolicy.updateCacheStatus, and it's the RebalancePolicy implementation's job to start a new rebalance if needed.


On Tue, Feb 12, 2013 at 5:28 PM, Pedro Ruivo <pruivo@gsd.inesc-id.pt> wrote:
Hi Dan,


On 2/12/13 3:12 PM, Dan Berindei wrote:
Hi Pedro

When I split off the RebalancePolicy I was thinking that when a RebalancePolicy needs to collaborate with a ConsistentHashFactory, they should do so via another cache manager-scoped component. But that doesn't really work (yet?), because ConsistentHashFactory can't access any components.
I didn't understand the previous sentence... Do I need to invoke anything in the RebalancePolicy?

So far, I'm invoking directly in the ClusterTopologyManager: https://github.com/pruivo/infinispan/blob/cloudtm_v2/core/src/main/java/org/infinispan/dataplacement/DataPlacementManager.java#L246

Thanks!

Cheers,
Pedro


I think it would be better to extend ClusterTopologyManager.triggerRebalance (and ConsistentHashFactory.rebalance) to accept an arbitrary Object parameter. Then RebalancePolicy could use this parameter to pass extra information to the CHF, like your Mappings object, and then when ClusterTopologyManagerImpl asks for a balanced CH, the CHF will include the Mappings in the result CH. What do you think?

In order to trigger the rebalance you have to call startRebalance, and the new ("balanced") consistent hash must not be equal to the existing consistent hash. See https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/topology/ClusterTopologyManagerImpl.java#L389

Cheers
Dan




On Thu, Feb 7, 2013 at 10:05 PM, Pedro Ruivo <pruivo@gsd.inesc-id.pt> wrote:
Hi,

I'm working in a way to rebase auto-placer on top of NBST and I have one
question...
If you have already forgot, auto-placer analyzes the workload and tries
to move the most remote accessed keys to the corresponding requester.

After calculating the new mappings, I want to trigger the NBST with this
mapping. I'm thinking to add a new method in the ClusterTopologyManager,
something like:

triggerAutoPlacer(String cacheName, Mappings newMappings);

and this method it will be a duplicate of triggerRebalance but instead
of doing chFactory.rebalance(CH) (in the startRebalance() method) I'm
thinking to do chFactory.autoPlacer(CH, Mappings). The last method will
override the defautl CH location.

Question: will this solution trigger the NBST or do I have to create the
triggerAutoPlacer() method in another class?

ps. forget the methods names... I will think in better names later

Thanks!!

Cheers,
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev