Here is a transcript of conversation that I had with Brian regarding the algorithm details of JBCACHE-315: Replying privately, but think we should take this to jbcache-dev or the forum. This is complex and Vladimir Blagojevic wrote: To be even more specific: On the remote node (i.e. the one we're working on) the JG up-handler thread will be blocked while the prepare() call waits to acquire a lock. That thread will block until there is a lock timeout. This will occur whether we are using REPL_ASYNC or REPL_SYNC. One effect of this is that no other JG messages will be received until there is a timeout. Note also that *I think* that having another thread roll back the tx associated with the prepare call will not cause the JG up-handler thread to unblock!! If REPL_SYNC, on the node that originated the GTX, the client thread that committed the tx will be blocking waiting for a response to the prepare() call. Can this be done in the TxInterceptor? I.e. isn't it always before any LockInterceptor? I would think it would be. I expect Manik would put up a fuss about doing tx-related stuff outside TxInterceptor; the whole reason it was added in 1.3.0 was to encapsulate stuff that was previously spread around other interceptors. +1. No reason to let a tx that hasn't acquired any locks go through and cause trouble. The concern I have with this is we give up one of the key goals -- not rolling back a tx if its not hurting anything. Here we assume that an ACTIVE locally originated tx is going to cause a problem by blocking a remote prepare() call. So we roll back the tx. Actually the odds of a remote prepare() call being blocked are pretty low. How about this: 1) receive block call 2) flip a latch in TxInterceptor (I'm assuming it will work putting it here instead of InvocationContextInterceptor). This latch is used at 2 or 3 different control points to block any threads that: a) are not associated with a GTX (i.e. non-transactional reads/writes from the local server) b) are associated with a GTX, but not yet in TransactionTable (your idea above) c) are associated with a locally originated GTX and are about to enter the beforeCompletion phase (i.e. the original idea of preventing the tx from proceeding to making a prepare() call.) 3) Loop through the GTXs in the TransactionTable. Create and throw in a map little object for each GTX. Object is a state machine that uses the JTA Transaction.STATUS, the elapsed time and whether the tx is locally or remotely originated to govern its state transitions. Keep looping through the TransactionTable, create more of these objects if more GTXs appear and for each GTX update the object with the current Transaction.STATUS, then read the object state. The object state tells you whether you need to rollback the tx, etc. 4) If a the state machine is for a *remotely initiated* GTX that's in ACTIVE status, after some elapsed time its state will tell you that its likely held up by a lock conflict with a locally originated tx. At that point we have a choice. a) roll back all locally originated tx's that are ACTIVE or PREPARED. Con: indiscriminately breaks transactions. Con: if tx has already entered beforeCompletion() we don't know whether its in prepare() call or later. We can only roll it back during beforeCompletion(); otherwise we introduce a heuristic. b) roll back the remotely originated tx. Pro: doesn't indiscriminately break transactions. Con: I *think* this rollback won't unblock the JG up-handler thread. 5) We'd need to work out all the state transitions; i.e. what conditions lead to tx rollback. 6) flip latch back and allow transactions to proceed 7) return from block ok Here's a question for you about FLUSH: When a service returns from block() or sends blockOK, does the channel immediately block? Is there coordination across the cluster? My concern: Node A doesn't have much going on, quickly returns from block(), so his channel is blocked. Node B takes a little longer; has some txs the completion of which requires sending messages to A. Those messages don't get through due to A being blocked. View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3991386#... Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...