[jboss-dev-forums] [Design of JBossCache] - Re: Locking tree before setting initial state

[bstansberry@jboss.com]

Following the previous post last March, a rather extensive discussion took place on private e-mail.  6 months later I'm going to recreate the highlights of that discussion here so the whole thread will be available in one place as we discuss these issues again next week:
 
"bela at jboss.com" wrote : I thought about this, and I think the best thing is to *not* break the locks ! Let's look at the use cases.
  | 
  | #1 PREPARE was received and now state transfer request is received (state provider side)
  | 
  | We cannot break the locks acquired by PREPARE because we returned OK from PREPARE, so the coordinator assumes the TX will be successful (assuming for now everyone returned OK). Returning OK from PREPARE means we will be able to apply the changes, so we cannot break the locks acquired by PREPARE ! Otherwise, the result might be that everyone applied a TX, except the state provider, so we end up with inconsistent state...
  | 
  | Looks like we have to wait for the PREPARE locks to be released. This shouldn't take long though
  | 
  |  
  | #2 PREPARE is received after state transfer request has been received (again, on state provider side)
  | 
  | We cannot queue the PREPAREs, for similar reasons as discussed in #1: returning OK from a PREPARE means we successfully acquired the locks, but if we queue, we don't know whether we will be able to acquire the locks later ! 
  | 
  | We can throw an exception so we return FAIL from PREPARE, which means that during state transfer, nobody will be able to commit transactions across the cluster !
  | 
  | #3 PREPARE and FLUSH
  | 
  | Once we have FLUSH, we do the following:
  |  
  | a) On state transfer, we run the FLUSH protocol 
  | b) It flushes pending multicasts out of the system so that all members have received the same messages after a FLUSH 
  | c) It blocks members from sending new messages, this is done via the block() callback in MembershipListener 
  | d) The block() needs to do the following: 
  | --- Stop sending new messages (in ReplicationInterceptor) 
  | --- Complete existing PREPARE rounds, e.g. return from block() only when a 2PC has been committed or rolled back 
  | e) This means that when the state provider receives a state transfer request, the entire tree will be *lock-free*, because all 2PCs have completed. The only time this isn't true is when the 2PC leader crashed so we have dangling locks. However, Brian fixed this recently. 
  | 
  | So what do we need to do once we have FLUSH and the block() callback is implemented correctly ? 
  | We simply revert back to the original state transfer code, which acquires all locks on the state provider side (with small timeouts), and we don't need to break any locks ! 
  | 
  | 
  | So the strategy is to throw exceptions to PREPARE in JBossCache 1.3.0 (#2b), and in 1.4.0 we use solution #3 (depends on JGroups 2.4 with FLUSH).
  | 
  | 

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3970498#3970498

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=3970498