[infinispan-dev] XSite synchronous replication

[Mircea Markus]

Thanks Bela.
On 23 Apr 2013, at 16:27, Bela Ban wrote:

> Erik and I had a call and concluded that
> - the regular thread pool should have a queue enabled
is that something you plan to do in the JGroups sample TCP configuration Erik mentioned? Or just something we should recommend for x-site bridge in particular?
> - for sync replication between sites, RPCs are *not* tagged as OOB, but they should ! Mircea, any idea why this deviates from the default (local) replication where sync RPCs are OOB ?
It shouldn't: https://issues.jboss.org/browse/ISPN-3043

> 
> On 4/22/13 11:29 PM, Erik Salter (esalter) wrote:
>> Hi guys,
>> 
>> While we wait for the threading model to change in ISPN 5.3, I was doing
>> a deep-dive into the existing xsite implementation, and I noticed that
>> all messages originating from the bridge use the regular/default/in-band
>> thread pool, even those that are marked as synchronous in ISPN.
>> 
>> Ex:
>> 
>> 2013-05-23 13:03:14,153 TRACE [org.jgroups.protocols.TCP]
>> (Incoming-2,erm-cluster,adht1-12627(DC1)) sending msg to
>> _bdht5-37320:DC2, src=_adht1-12627:DC1, headers are RequestCorrelator:
>> id=200, type=REQ, id=146, rsp_expected=true, RELAY2: DATA
>> [dest=SiteMaster(DC2), sender=adht5-23034:DC1], UNICAST2: DATA, seqno=4,
>> conn_id=5, TCP: [channel_name=erm-bridge]
>> 
>> 2013-05-23 13:03:14,153 TRACE [org.jgroups.protocols.TCP]
>> (Incoming-2,erm-cluster,adht1-12627(DC1)) dest=10.30.16.134:44572 (1269
>> bytes)
>> 
>> 2013-05-23 13:03:14,164 TRACE [org.jgroups.protocols.TCP]
>> (OOB-9,erm-bridge,_adht1-12627:DC1) received [dst: _adht1-12627:DC1,
>> src: _bdht5-37320:DC2 (4 headers), size=4 bytes, flags=OOB|DONT_BUNDLE],
>> headers are RequestCorrelator: id=200, type=RSP, id=146,
>> rsp_expected=false, RELAY2: DATA [dest=adht5-23034:DC1,
>> sender=SiteMaster(DC2)], UNICAST2: DATA, seqno=2, conn_id=6, TCP:
>> [channel_name=erm-bridge]
>> 
>> Shouldn’t this message from the bridge end to the remote site use the
>> OOB thread pool, since a response is expected?
>> 
>> I ask because in JGroups 3.2.x, the sample TCP configuration shows that
>> the default thread pool has queuing disabled:
>> thread_pool.queue_enabled="false".  If I enable queuing for my async
>> replication use case, my performance for sync very much degrades.  But I
>> can easily flood/abuse the incoming thread pool if I disable queuing –
>> i.e. messages get dropped (XSite replication, internal JGroups
>> communication).
>> 
>> Thanks,
>> 
>> Erik Salter
>> 
>> Technical Leader I
>> 
>> Cisco Systems, SPVTG
>> 
>> (404) 317-0693
>> 
> 
> -- 
> Bela Ban, JGroups lead (http://www.jgroups.org)

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)