[infinispan-dev] Partial state transfer

[Vladimir Blagojevic]

In another words the essential problem is that digest and channel state 
are per channel abstractions and they do not fit nicely with higher 
level abstractions like substates? We use partial state transfer in 
Infinispan and we need to address this.

What is the time frame here? 5.0 final release?

Vladimir

On 11-06-15 11:39 AM, Bela Ban wrote:
> I looked into adding partial state transfer back into JGroups, but found
> out that partial state transfer is fundamentally flawed, something I've
> always suspected ! (Regular state transfer is correct, and has always
> been correct).
>
> - Say we have node A and B. B requests the state from A
> - There are partial states X and Y
> - Message M1 modifies X, M2 modifies Y
>
> Here's what happens:
>
> T1: A multicasts M1
> T2: A delivers M1, and changes X
> T3: B sends a GET_STATE("Y") request to A    // partial state request
> for state Y
> T4: A multicasts M2
> T5: A delivers M2, changing Y
> T6: A receives the GET_STATE request, sends a SET_STATE response back
> including Y and the digest (including M1 and M2's seqnos)
> T7: B receives the SET_STATE response, sets its digest (includes now M1
> and M2) and state Y *BUT NOT* state X !
> T8: *** B receives M1, discards it because it is already in its digest ***
> T9: B receives M2, and also discards it
>
>
> At time T8, M1 (which would have changed state X) is discarded, because
> it is already in the digest sent with the SET_STATE response. Therefore
> state X is now incorrect, as M1 was never applied !
>
> As a summary, if we get a number of updates to partial states, and don't
> receive all of them before requesting the partial state, the last update
> includes in the digest wins...
>
> I'm a real idiot, as I've written this down before, in 2006: see [1] for
> details.
>
> In a nutshell, [1] shows that partial state transfer doesn't work,
> unless virtual synchrony (FLUSH) is used.
>
> So I propose Infinispan and JBoss AS look into how they can replace
> their use of partial state transfer. I suggest Infinispan uses the same
> approach already used for state transfer with mode=distribution.
>
> Opinions ?
>
> [1]
> https://github.com/belaban/JGroups/blob/master/doc/design/PartialStateTransfer.txt
>