Re: [infinispan-dev] XSite synchronous replication

Erik and I had a call and concluded that - the regular thread pool should have a queue enabled

- for sync replication between sites, RPCs are *not* tagged as OOB, but they should ! Mircea, any idea why this deviates from the default (local) replication where sync RPCs are OOB ?

On 4/22/13 11:29 PM, Erik Salter (esalter) wrote: > Hi guys, > > While we wait for the threading model to change in ISPN 5.3, I was doing > a deep-dive into the existing xsite implementation, and I noticed that > all messages originating from the bridge use the regular/default/in-band > thread pool, even those that are marked as synchronous in ISPN. > > Ex: > > 2013-05-23 13:03:14,153 TRACE [org.jgroups.protocols.TCP] > (Incoming-2,erm-cluster,adht1-12627(DC1)) sending msg to > _bdht5-37320:DC2, src=_adht1-12627:DC1, headers are RequestCorrelator: > id=200, type=REQ, id=146, rsp_expected=true, RELAY2: DATA > [dest=SiteMaster(DC2), sender=adht5-23034:DC1], UNICAST2: DATA, seqno=4, > conn_id=5, TCP: [channel_name=erm-bridge] > > 2013-05-23 13:03:14,153 TRACE [org.jgroups.protocols.TCP] > (Incoming-2,erm-cluster,adht1-12627(DC1)) dest=10.30.16.134:44572 (1269 > bytes) > > 2013-05-23 13:03:14,164 TRACE [org.jgroups.protocols.TCP] > (OOB-9,erm-bridge,_adht1-12627:DC1) received [dst: _adht1-12627:DC1, > src: _bdht5-37320:DC2 (4 headers), size=4 bytes, flags=OOB|DONT_BUNDLE], > headers are RequestCorrelator: id=200, type=RSP, id=146, > rsp_expected=false, RELAY2: DATA [dest=adht5-23034:DC1, > sender=SiteMaster(DC2)], UNICAST2: DATA, seqno=2, conn_id=6, TCP: > [channel_name=erm-bridge] > > Shouldn’t this message from the bridge end to the remote site use the > OOB thread pool, since a response is expected? > > I ask because in JGroups 3.2.x, the sample TCP configuration shows that > the default thread pool has queuing disabled: > thread_pool.queue_enabled="false". If I enable queuing for my async > replication use case, my performance for sync very much degrades. But I > can easily flood/abuse the incoming thread pool if I disable queuing – > i.e. messages get dropped (XSite replication, internal JGroups > communication). > > Thanks, > > Erik Salter > > Technical Leader I > > Cisco Systems, SPVTG > > (404) 317-0693 > -- Bela Ban, JGroups lead (http://www.jgroups.org)