[infinispan-dev] Prepare and Rollback messages order

[Sebastiano Peluso]

Hi Mircea,

thanks for the answer. But can you confirm that the rollback command is 
sent after the reception of all the participants' acknowledgements even 
if we run Infinispan in *distribution* mode?

In particular, does the prepare phase break the cycle at line 335 of the 
CommandAwareRpcDispatcher [1] if one of the remote prepares throws an 
exception? In this case I think that the get() method of the associated 
Future object throws an exception as well.

Thank you very much.


Cheers,

    Sebastiano


[1] 
https://github.com/infinispan/infinispan/blob/0f8df66dad98d172b5f4314ff4e85e56874e27a6/core/src/main/java/org/infinispan/remoting/transport/jgroups/CommandAwareRpcDispatcher.java

On 5/31/12 3:33 PM, Mircea Markus wrote:
> On 31 May 2012, at 10:21, Sebastiano Peluso wrote:
>
>> Hi all,
>>
>> I have a question about the messages order guaranteed in the execution
>> of the 2PC for committing a transaction in Infinispan (all versions).
>>
>> In particular I have noticed that both in replication and distribution
>> mode the RollbackCommand command is multicast by using a priority queue.
>> This means that:
>>      1) In replication mode, the visitRollbackCommand method of the
>> ReplicationInterceptor class invokes the broadcastRpcCommand method on
>> the RpcManager by using a true value for the usePriorityQueue parameter;
>>      2) In distribution mode, the visitRollbackCommand method of the
>> DistributionInterceptor class invokes the invokeRemotely method on the
>> RpcManager by using a true value for the usePriorityQueue parameter.
>>
>> Therefore the JGroups' packets containing the RollbackCommand commands
>> will be marked with the org.jgroups.Message.OOB and will be sent and
>> received by means of the Jgroups' out-of-band (OOB) thread pool.
>>
>> If I'm not wrong until now, I deduce that since OBB threads don't
>> guarantee ordering, then it can happen that a message M2 multicast by
>> node N1 after it has multicast message M1, can be delivered to a node N2
>> before message M1. So the sending order is M1 ->  M2 and the delivery
>> order is M2 ->  M1 [1].
> yes, you're right.
>> If this is true as well, I can have runs in which the PreparedCommand
>> command of a transaction T involved in the execution of a 2PC, can be
>> received by a 2PC's participant node after a RollbackCommand command for
>> T, even if the JGroups' messages containing the PrepareCommand commands
>> are actually managed by means of the JGroups's regular thread pool.
> The rollbackCommand is only sent once the prepare execution is acknowledged. So rollback should always arrive *after* prepare: and this ordering is not enforced at jgroups level, but by the infinispan logic.
>> For instance, we can consider a scenario in which a transaction T
>> requests a commit on node N0 and N0 multicasts the T's prepare message
>> on a large set of nodes (e.g. 100 nodes) S={N1,...,N100}. If the lock
>> acquisition fails on at least one of the 100 nodes, N0 can multicasts a
>> rollback message without waiting for the reply of the prepare from all
>> the nodes in S.
> That's only possible if lockAcquisitionTimeout(default to 10secs) is higher than the replicationTimeout(15 secs by default).
>
>> This means that the last rollback message can be
>> delivered to a node in S, say N100, before the T's prepare message and
>> that the locks successfully acquired by N100 for T will not be released
>> anymore.
>>
>> Am I missing anything?
>>
>> Thanks.
>>
>> Best Regards,
>>
>>      Sebastiano Peluso
>>
>>
>> [1] http://www.jgroups.org/manual/html/user-advanced.html
>>
>>
>> -- 
>> Sebastiano Peluso, PhD student
>>
>> Department of Computer, Control and Management Engineering
>> Sapienza University of Rome
>> Via Ariosto 25, 00185, Rome, Italy
>> Tel. +39 06 77274108
>> Webpage http://www.dis.uniroma1.it/~peluso
>>
>> Distributed Systems Group, INESC-ID
>> Instituto Superior Técnico
>> Rua Alves Redol 9, 1000-029, Lisbon, Portugal
>> Webpage http://www.gsd.inesc-id.pt/~peluso
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Sebastiano Peluso, PhD student

Department of Computer, Control and Management Engineering
Sapienza University of Rome
Via Ariosto 25, 00185, Rome, Italy
Tel. +39 06 77274108
Webpage http://www.dis.uniroma1.it/~peluso

Distributed Systems Group, INESC-ID
Instituto Superior Técnico
Rua Alves Redol 9, 1000-029, Lisbon, Portugal
Webpage http://www.gsd.inesc-id.pt/~peluso