[infinispan-dev] Non-blocking state transfer (ISPN-1424)

[Bela Ban]

Thanks for the feedback and clarifications Dan !

Comments inline...



>> Over the last couple of days, I've exchanged a couple of emails with the
>> Cloud-TM guys, and I'm more and more convinced that their total order
>> solution is the simpler approach to (1) transactional updates and (2)
>> state transfer. They don't have a solution for (2) yet, but I believe
>> this can be done as another totally ordered transaction, applied in the
>> correct location within the update stream. Or, we could possibly use a
>> flush: as we don't need to wait for pending TXs to complete and release
>> their locks, this should be quite fast.
>>
>
> Sounds intriguing, but I'm not sure how making the entire state
> transfer a single transaction would allow us to handle transactions
> while that state is being transferred.


We're discussing 2 scenarios, one being the use of flush (but 
short-lived as we don't have any locks in play) and the other being 
Pedro's proposal of a start-state TO-cast (shipped with a keys set), 
followed by a state transfer. Transactions with any keys in the key set 
are queued and applied in order after the state transfer is done 
(signalled with abother stop-state TO-cast).

If you have anything to add, let's discuss it on the other thread.



>> So my 5 cents:
>>
>> #1 We should focus on the total order approach, and get rid of the 2PC
>> and locking business for transactional updates
>>
>
> The biggest problem I remember total order having is TM transactions
> that have other participants (as opposed to cache-only transactions).
> I haven't followed the TO discussion on the mailing list very closely,
> does that work now?


No, I don't think that's addressed by TOM, good point in favor of having 
2 (or more) approaches to partial replication and state transfer !

I think a lot of systems would still benefit from TOM though as there 
are no external participants, for instance session replication: it uses 
an Infinispan batch to ship session updates to the session owners, and 
there are no other participants (as a matter of fact, this is not even a 
JTA transaction anyway).


> Regarding state transfer in particular, remember, non-blocking state
> transfer with 2PC sounded very easy as well, before we got into the
> details.

True :-)



>> #2 Really focus on the eventual consistency approach
>>
>
> Again, it's very tempting, but I fear much of what makes eventual
> consistency tempting is that we don't know enough of its
> implementation details yet.


True again...


> Manik has been investigating eventual consistency for a while, I
> wonder what he thinks...


Yep - Manik, do you have any status update on eventual consistency ?


-- 
Bela Ban, JGroups lead (http://www.jgroups.org)