[jbosscache-dev] Non Blocking State Transfer Status (& Integration with JGroups)

[Jason T. Greene]

Hi All,

I wanted to summarize my initial research into NBST. The planned design 
(outlined in the wiki: http://www.jboss.org/community/docs/DOC-10275) 
only needs to block transactional activity once, at the end of the 
process when sending the tx log. Unfortunately it appears that flush and 
partial flush can not be used for this, since the application needs the 
ability to send state (tx log) during the flush. I.e. transactions need 
to be paused by only 2 nodes, while the transfer state. This however is 
not a big deal because we can just do this in JBoss Cache using a normal 
RPC message that flips a gate.

In addition, the state transfer and streaming state transfer facilities 
in jgroups can not be used (since they are designed around blocking the 
entire group). This means JBoss Cache needs to stream state itself. 
Ideally this would be a separate point-to-point connection, since we 
don't want to pollute multicast traffic with potentially huge volumes of 
noise. Currently jgroups does not yet support a streaming API like this:
https://jira.jboss.org/jira/browse/JGRP-653

IMO This leaves us with 3 options:

1. Wait on JGRP-653 (upping its priority), also add requirements for a 
p2p connection.
2. Implement our own p2p connection using tcp (probably using xnio).
3. Somehow enhance state transfer / partial flush to meet our needs

Option 1 seems to be a useful feature for other applications. Although 
we need feedback from Bela and Vladimir about that.

Option 2 would give us more flexibility in the implementation, however 
care has to be taken to ensure that communication can only happen 
between group members (for security reasons), and that the network 
address configurations are somehow reused.

Option 3 I am less found of, since we would likely end up adding a bunch 
of JBoss Cache specific code to JGroups that no one else would use.

-- 
Jason T. Greene
JBoss, a division of Red Hat