[infinispan-dev] Distribution across multiple sites

[Manik Surtani]

On 11 Jan 2012, at 09:33, Bela Ban wrote:

> Hi Manik,
> 
> On 1/10/12 7:02 PM, Manik Surtani wrote:
>> Sorry for the late response on this thread.
>> 
>> I think solution #2 is overkill - this could be a huge bottleneck.  Something like solution #1 will work, but that's still expensive (in terms of memory consumption, and consequently GC).
> 
> 
> Yes
> 
> 
>> Are the communications between relay coordinators (RC) asynchronous?
> 
> 
> Yes, everything is asynchronous. Synchronous constructs, such as RPCs, 
> are then run over this async 'transport'. This is the same model JGroups 
> has in a purely local cluster.
> 
> Bridging is done at the transport; this can be compared to a switch, 
> bridging subnets: whenever the switch is down, any IP packets sent 
> during this time are not forwarded to the other subnet and a higher-up 
> layer has to do repair. TCP for instance does this, whereas UDP doesn't.
> 
> 
>>  I realise the communication is not on the critical path of a 
> transaction in either data centre, but it could still be sync.  If this 
> were the case, we could  have something like this:
> 
> That would slow things down though ! Plus, it would require an ack, so 
> that's additional traffic (not much though). The big issue here is that 
> latency is usually high between sites, so the round trip time would kill 
> us !

Only if the relay call is on the critical path.  If it is offloaded to a separate executor, then it's fine; it won't hold up any application threads.

> 
> 
>> Assume {A, B, C} and {X, Y, Z}.  A -->  X synchronously, but offline (in a separate thread) so calling transactions aren't blocked.  Assume key K is on {A, B, Z}.  A change to K would synchronously update A and B, and put the update on the RC (and backup RC)'s processing queue.  A then flushes to X, and on receiving the ack from X, informs B that the message was delivered.  If X crashes/doesn't ack, A keeps retrying, potentially batching queued updates.
> 
> While this could certainly be done, what happens if X itself crashes ?

Then A doesn't get it's ack and A will not GC the message, but retry to the newly nominated proxy on site-2.

Cheers
Manik

--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org