Re: [infinispan-dev] Partial state transfer

I looked into adding partial state transfer back into JGroups, but found out that partial state transfer is fundamentally flawed, something I've always suspected ! (Regular state transfer is correct, and has always been correct). - Say we have node A and B. B requests the state from A - There are partial states X and Y - Message M1 modifies X, M2 modifies Y Here's what happens: T1: A multicasts M1 T2: A delivers M1, and changes X T3: B sends a GET_STATE("Y") request to A // partial state request for state Y T4: A multicasts M2 T5: A delivers M2, changing Y T6: A receives the GET_STATE request, sends a SET_STATE response back including Y and the digest (including M1 and M2's seqnos) T7: B receives the SET_STATE response, sets its digest (includes now M1 and M2) and state Y *BUT NOT* state X ! T8: *** B receives M1, discards it because it is already in its digest *** T9: B receives M2, and also discards it At time T8, M1 (which would have changed state X) is discarded, because it is already in the digest sent with the SET_STATE response. Therefore state X is now incorrect, as M1 was never applied ! As a summary, if we get a number of updates to partial states, and don't receive all of them before requesting the partial state, the last update includes in the digest wins... I'm a real idiot, as I've written this down before, in 2006: see [1] for details. In a nutshell, [1] shows that partial state transfer doesn't work, unless virtual synchrony (FLUSH) is used. So I propose Infinispan and JBoss AS look into how they can replace their use of partial state transfer. I suggest Infinispan uses the same approach already used for state transfer with mode=distribution. Opinions ? [1] https://github.com/belaban/JGroups/blob/master/doc/design/PartialStateTra...