[infinispan-dev] State transfer testing

Mircea Markus mircea.markus at jboss.com
Tue Jul 19 13:16:32 EDT 2011


On 19 Jul 2011, at 18:00, Bela Ban wrote:

> 
> 
> On 7/19/11 6:49 PM, Mircea Markus wrote:
> 
>>> When there's a merge into {A,B}  and A turns out to be the (existing and
>>> new) primary owner of x, then it'll update B.x=1 (*if* B is the backup
>>> owner of x, or else B will remove x).
>> This is precisely the thing that worries me: with previous code B.x=2 didn't get lost. User's @Merge listener were called instead and the user could do some stuff to merge the state, but updates were not lost.
>> I agree that the previous approach placed a lot of burden on user's shoulder: merge the state "manually", but that was for a reason - not to loose consistency, which the current approach does. Hence my surprise to see it.
> 
> 
> Wait ! The @Merged annotation just invoked a callback which gave you a 
> MergeEvent (similar to @View giving you a ViewEvent).
> 
> I don't think it was ever intended to be used to merge state, at least 
> the documentation of @Merged never mentioned this (IMO)...
> 
> I also don't think that anybody (us or users) has ever written any code 
> to merge data on a MergeEvent...
> 
> Note that this is probably what should be done, but we should tackle 
> this in more sophisticated way than throwing a callback at the user and 
> telling them to handle the merge.
> 
> *If* we used version numbers that were incremented on changes, we could 
> do some of the merging ourselves, and throw the conflicting changes to 
> the users to handle. Perhaps we could even provide some merge strategies 
> that a user could pick.
> 
> However, this is not exactly trivial, e.g. how do conflicting removes 
> and changes get handled, tombstones ?
Agreed on the versioning for auto-merge. I'm just wandering weather it wouldn't make more sense, for now, not to do any (incorrect) merge as we do now, but stick with the old approach of migrating this responsibility to the user.
When versioning is in place we can start merging things indeed, but ATM I think it's better not to do anything than do something that creates inconsistencies. 
Or make this auto-merge behaviour configurable and let the user decide?
> 
> This is something we should discuss separately from the new rebalancing 
> code. Note that this also applies to replication, not only distribution.
Just a note on state transfer with replication: 5.1's state transfer (REPL) uses the rehashing code from distribution to move state around: the basic idea is that if numOwners>clusterSize, when a node joins you practically have state transfer.
So post 5.1, state-transfer and rebalancing are the same thing. Almost :-)  


More information about the infinispan-dev mailing list