[infinispan-dev] Prepare and Rollback messages order

[Sebastiano Peluso]

Hi all,

I have a question about the messages order guaranteed in the execution 
of the 2PC for committing a transaction in Infinispan (all versions).

In particular I have noticed that both in replication and distribution 
mode the RollbackCommand command is multicast by using a priority queue. 
This means that:
     1) In replication mode, the visitRollbackCommand method of the 
ReplicationInterceptor class invokes the broadcastRpcCommand method on 
the RpcManager by using a true value for the usePriorityQueue parameter;
     2) In distribution mode, the visitRollbackCommand method of the 
DistributionInterceptor class invokes the invokeRemotely method on the 
RpcManager by using a true value for the usePriorityQueue parameter.

Therefore the JGroups' packets containing the RollbackCommand commands 
will be marked with the org.jgroups.Message.OOB and will be sent and 
received by means of the Jgroups' out-of-band (OOB) thread pool.

If I'm not wrong until now, I deduce that since OBB threads don't 
guarantee ordering, then it can happen that a message M2 multicast by 
node N1 after it has multicast message M1, can be delivered to a node N2 
before message M1. So the sending order is M1 -> M2 and the delivery 
order is M2 -> M1 [1].

If this is true as well, I can have runs in which the PreparedCommand 
command of a transaction T involved in the execution of a 2PC, can be 
received by a 2PC's participant node after a RollbackCommand command for 
T, even if the JGroups' messages containing the PrepareCommand commands 
are actually managed by means of the JGroups's regular thread pool.

For instance, we can consider a scenario in which a transaction T 
requests a commit on node N0 and N0 multicasts the T's prepare message 
on a large set of nodes (e.g. 100 nodes) S={N1,...,N100}. If the lock 
acquisition fails on at least one of the 100 nodes, N0 can multicasts a 
rollback message without waiting for the reply of the prepare from all 
the nodes in S. This means that the last rollback message can be 
delivered to a node in S, say N100, before the T's prepare message and 
that the locks successfully acquired by N100 for T will not be released 
anymore.

Am I missing anything?

Thanks.

Best Regards,

     Sebastiano Peluso


[1] http://www.jgroups.org/manual/html/user-advanced.html


-- 
Sebastiano Peluso, PhD student

Department of Computer, Control and Management Engineering
Sapienza University of Rome
Via Ariosto 25, 00185, Rome, Italy
Tel. +39 06 77274108
Webpage http://www.dis.uniroma1.it/~peluso

Distributed Systems Group, INESC-ID
Instituto Superior Técnico
Rua Alves Redol 9, 1000-029, Lisbon, Portugal
Webpage http://www.gsd.inesc-id.pt/~peluso