[jbosscache-dev] Re: Non Blocking State Transfer Status (& Integration with JGroups)

Brian Stansberry wrote: > Jason T. Greene wrote: >> Hi All, >> >> I wanted to summarize my initial research into NBST. The planned >> design (outlined in the wiki: >> http://www.jboss.org/community/docs/DOC-10275) only needs to block >> transactional activity once, at the end of the process when sending >> the tx log. Unfortunately it appears that flush and partial flush can >> not be used for this, since the application needs the ability to send >> state (tx log) during the flush. I.e. transactions need to be paused >> by only 2 nodes, while the transfer state. This however is not a big >> deal because we can just do this in JBoss Cache using a normal RPC >> message that flips a gate. >> >> In addition, the state transfer and streaming state transfer >> facilities in jgroups can not be used (since they are designed around >> blocking the entire group). This means JBoss Cache needs to stream >> state itself. Ideally this would be a separate point-to-point >> connection, since we don't want to pollute multicast traffic with >> potentially huge volumes of noise. Currently jgroups does not yet >> support a streaming API like this: >> https://jira.jboss.org/jira/browse/JGRP-653 >> >> IMO This leaves us with 3 options: >> >> 1. Wait on JGRP-653 (upping its priority), also add requirements for >> a p2p connection. >> 2. Implement our own p2p connection using tcp (probably using xnio). >> 3. Somehow enhance state transfer / partial flush to meet our needs >> >> Option 1 seems to be a useful feature for other applications. >> Although we need feedback from Bela and Vladimir about that. >> >> Option 2 would give us more flexibility in the implementation, >> however care has to be taken to ensure that communication can only >> happen between group members (for security reasons), and that the >> network address configurations are somehow reused. >> >> Option 3 I am less found of, since we would likely end up adding a >> bunch of JBoss Cache specific code to JGroups that no one else would >> use. >> > > Option 2 makes me nervous. Two separate communication frameworks, > added dependencies, opening new sockets etc. Sounds like integration > hassles for sure. > Yes there are definitely integration hassles that make this option less desirable than the first. From a dependency perspective, we are already using non-jgroups p2p with TCPCacheServer (currently Java sockets based), although I believe Manik was evaluating xnio for it since it would simplify development. While it is an added dep for JBC, it will eventually be part of AS, since Remoting 3 depends on it.