[jbosscache-dev] Non Blocking State Transfer Status (& Integration with JGroups)

Hi All, I wanted to summarize my initial research into NBST. The planned design (outlined in the wiki: http://www.jboss.org/community/docs/DOC-10275) only needs to block transactional activity once, at the end of the process when sending the tx log. Unfortunately it appears that flush and partial flush can not be used for this, since the application needs the ability to send state (tx log) during the flush. I.e. transactions need to be paused by only 2 nodes, while the transfer state. This however is not a big deal because we can just do this in JBoss Cache using a normal RPC message that flips a gate. In addition, the state transfer and streaming state transfer facilities in jgroups can not be used (since they are designed around blocking the entire group). This means JBoss Cache needs to stream state itself. Ideally this would be a separate point-to-point connection, since we don't want to pollute multicast traffic with potentially huge volumes of noise. Currently jgroups does not yet support a streaming API like this: https://jira.jboss.org/jira/browse/JGRP-653 IMO This leaves us with 3 options: 1. Wait on JGRP-653 (upping its priority), also add requirements for a p2p connection. 2. Implement our own p2p connection using tcp (probably using xnio). 3. Somehow enhance state transfer / partial flush to meet our needs Option 1 seems to be a useful feature for other applications. Although we need feedback from Bela and Vladimir about that. Option 2 would give us more flexibility in the implementation, however care has to be taken to ensure that communication can only happen between group members (for security reasons), and that the network address configurations are somehow reused. Option 3 I am less found of, since we would likely end up adding a bunch of JBoss Cache specific code to JGroups that no one else would use. -- Jason T. Greene JBoss, a division of Red Hat