[infinispan-dev] Prepare and Rollback messages order

Hi all, I have a question about the messages order guaranteed in the execution of the 2PC for committing a transaction in Infinispan (all versions). In particular I have noticed that both in replication and distribution mode the RollbackCommand command is multicast by using a priority queue. This means that: 1) In replication mode, the visitRollbackCommand method of the ReplicationInterceptor class invokes the broadcastRpcCommand method on the RpcManager by using a true value for the usePriorityQueue parameter; 2) In distribution mode, the visitRollbackCommand method of the DistributionInterceptor class invokes the invokeRemotely method on the RpcManager by using a true value for the usePriorityQueue parameter. Therefore the JGroups' packets containing the RollbackCommand commands will be marked with the org.jgroups.Message.OOB and will be sent and received by means of the Jgroups' out-of-band (OOB) thread pool. If I'm not wrong until now, I deduce that since OBB threads don't guarantee ordering, then it can happen that a message M2 multicast by node N1 after it has multicast message M1, can be delivered to a node N2 before message M1. So the sending order is M1 -> M2 and the delivery order is M2 -> M1 [1]. If this is true as well, I can have runs in which the PreparedCommand command of a transaction T involved in the execution of a 2PC, can be received by a 2PC's participant node after a RollbackCommand command for T, even if the JGroups' messages containing the PrepareCommand commands are actually managed by means of the JGroups's regular thread pool. For instance, we can consider a scenario in which a transaction T requests a commit on node N0 and N0 multicasts the T's prepare message on a large set of nodes (e.g. 100 nodes) S={N1,...,N100}. If the lock acquisition fails on at least one of the 100 nodes, N0 can multicasts a rollback message without waiting for the reply of the prepare from all the nodes in S. This means that the last rollback message can be delivered to a node in S, say N100, before the T's prepare message and that the locks successfully acquired by N100 for T will not be released anymore. Am I missing anything? Thanks. Best Regards, Sebastiano Peluso [1] http://www.jgroups.org/manual/html/user-advanced.html -- Sebastiano Peluso, PhD student Department of Computer, Control and Management Engineering Sapienza University of Rome Via Ariosto 25, 00185, Rome, Italy Tel. +39 06 77274108 Webpage http://www.dis.uniroma1.it/~peluso Distributed Systems Group, INESC-ID Instituto Superior Técnico Rua Alves Redol 9, 1000-029, Lisbon, Portugal Webpage http://www.gsd.inesc-id.pt/~peluso