Re: [infinispan-dev] Distribution across multiple sites

Hi Manik, On 1/10/12 7:02 PM, Manik Surtani wrote: > Sorry for the late response on this thread. > > I think solution #2 is overkill - this could be a huge bottleneck. Something like solution #1 will work, but that's still expensive (in terms of memory consumption, and consequently GC). Yes > Are the communications between relay coordinators (RC) asynchronous? Yes, everything is asynchronous. Synchronous constructs, such as RPCs, are then run over this async 'transport'. This is the same model JGroups has in a purely local cluster. Bridging is done at the transport; this can be compared to a switch, bridging subnets: whenever the switch is down, any IP packets sent during this time are not forwarded to the other subnet and a higher-up layer has to do repair. TCP for instance does this, whereas UDP doesn't. > I realise the communication is not on the critical path of a transaction in either data centre, but it could still be sync. If this were the case, we could have something like this: That would slow things down though ! Plus, it would require an ack, so that's additional traffic (not much though). The big issue here is that latency is usually high between sites, so the round trip time would kill us !

> Assume {A, B, C} and {X, Y, Z}. A --> X synchronously, but offline (in a separate thread) so calling transactions aren't blocked. Assume key K is on {A, B, Z}. A change to K would synchronously update A and B, and put the update on the RC (and backup RC)'s processing queue. A then flushes to X, and on receiving the ack from X, informs B that the message was delivered. If X crashes/doesn't ack, A keeps retrying, potentially batching queued updates. While this could certainly be done, what happens if X itself crashes ?