[infinispan-dev] Trigger NBST for Auto-Placer

[Dan Berindei]

Actually I'd like regular users to be able to define their own
RebalancePolicies and ConsistentHashFactories, without modifying
ClusterTopologyManagerImpl/ClusterCacheStatus, so I see this as a good
opportunity to modify our implementation to allow it.


On Tue, Feb 12, 2013 at 6:04 PM, Pedro Ruivo <pruivo at gsd.inesc-id.pt> wrote:

> **
> Can I modify the existing one?
>
> I'm thinking in the following:
>
> ClusterTopologyManagerImpl.handleNewMappings(...) { //new method
>   ClusterCacheStatus status = //get status for cache name
>   status.setNewMappings(...) //synchronized of course
>   rebalancePolicy.updateCacheStatus(...);
> }
>
>
Like I said, I'd like to keep ClusterTopologyManager as generic as possible
wrt rebalance strategies, so I think your DataPlacementManager should call
your custom RebalancePolicy directly.

The rebalance policy could keep the new mappings in a map on its own,
although maybe it would be a nice touch to allow storing custom state in
ClusterCacheStatus.



>  DefaultRebalancePolicy.updateCacheStatus(...) { //modified
>   ...
>   if (!status.hasJoiners() && isBalanced(...) && !status.hasNewMappings())
> { //added last condition
>     return;
>   }
>   ...
> }
>

I guess you'd also need to clean up the old "new mappings" here after the
rebalance is done.



>
> ClusterTopologyManagerImpl.startRebalance(...) { //modifed
>   ...
>   chFactory.rebalance(ch);
>   chFactory.applyMappings(ch, status.getNewMappings()); //added.
>   ... //if it is the same ch, no state transfer is triggered
> }
>
>
This would require ClusterTopologyManagerImpl to know about your custom
ConsistentHashFactory, and it wouldn't work with another
ConsistentHashFactory that requires different custom data. So I'd rather we
add a generic parameter to ConsistentHashFactory.rebalance.


What do you think?
>
> Thanks,
> Pedro
>
>
> On 2/12/13 3:39 PM, Dan Berindei wrote:
>
>  Sorry, I didn't read your code so I just assumed you're writing your own
> RebalancePolicy.
>
>  I think you need to implement your own RebalancePolicy, because
> ClusterTopologyManagerImpl by itself doesn't remember that a rebalance was
> triggerred. So if you call startRebalance, but there is already a rebalance
> in progress, it is just ignored. When the in-progress rebalance finishes,
> it calls RebalancePolicy.updateCacheStatus, and it's the RebalancePolicy
> implementation's job to start a new rebalance if needed.
>
>
> On Tue, Feb 12, 2013 at 5:28 PM, Pedro Ruivo <pruivo at gsd.inesc-id.pt>wrote:
>
>>  Hi Dan,
>>
>>
>> On 2/12/13 3:12 PM, Dan Berindei wrote:
>>
>>  Hi Pedro
>>
>>  When I split off the RebalancePolicy I was thinking that when a
>> RebalancePolicy needs to collaborate with a ConsistentHashFactory, they
>> should do so via another cache manager-scoped component. But that doesn't
>> really work (yet?), because ConsistentHashFactory can't access any
>> components.
>>
>>  I didn't understand the previous sentence... Do I need to invoke
>> anything in the RebalancePolicy?
>>
>> So far, I'm invoking directly in the ClusterTopologyManager:
>> https://github.com/pruivo/infinispan/blob/cloudtm_v2/core/src/main/java/org/infinispan/dataplacement/DataPlacementManager.java#L246
>>
>> Thanks!
>>
>> Cheers,
>> Pedro
>>
>>
>>  I think it would be better to extend
>> ClusterTopologyManager.triggerRebalance (and
>> ConsistentHashFactory.rebalance) to accept an arbitrary Object parameter.
>> Then RebalancePolicy could use this parameter to pass extra information to
>> the CHF, like your Mappings object, and then when
>> ClusterTopologyManagerImpl asks for a balanced CH, the CHF will include the
>> Mappings in the result CH. What do you think?
>>
>>  In order to trigger the rebalance you have to call startRebalance, and
>> the new ("balanced") consistent hash must not be equal to the existing
>> consistent hash. See
>> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/topology/ClusterTopologyManagerImpl.java#L389
>>
>>  Cheers
>>  Dan
>>
>>
>>
>>
>> On Thu, Feb 7, 2013 at 10:05 PM, Pedro Ruivo <pruivo at gsd.inesc-id.pt>wrote:
>>
>>> Hi,
>>>
>>> I'm working in a way to rebase auto-placer on top of NBST and I have one
>>> question...
>>> If you have already forgot, auto-placer analyzes the workload and tries
>>> to move the most remote accessed keys to the corresponding requester.
>>>
>>> After calculating the new mappings, I want to trigger the NBST with this
>>> mapping. I'm thinking to add a new method in the ClusterTopologyManager,
>>> something like:
>>>
>>> triggerAutoPlacer(String cacheName, Mappings newMappings);
>>>
>>> and this method it will be a duplicate of triggerRebalance but instead
>>> of doing chFactory.rebalance(CH) (in the startRebalance() method) I'm
>>> thinking to do chFactory.autoPlacer(CH, Mappings). The last method will
>>> override the defautl CH location.
>>>
>>> Question: will this solution trigger the NBST or do I have to create the
>>> triggerAutoPlacer() method in another class?
>>>
>>> ps. forget the methods names... I will think in better names later
>>>
>>> Thanks!!
>>>
>>> Cheers,
>>> Pedro
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing listinfinispan-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
> _______________________________________________
> infinispan-dev mailing listinfinispan-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20130212/4485e159/attachment-0001.html