[infinispan-dev] JGRP-1317 and large clusters

[Bela Ban]

On 9/18/13 1:38 PM, Galder Zamarreño wrote:
> Great stuff Bela! How many nodes did you test this in?

The number of nodes to test is not relevant in this case (although this 
work came out of testing 500+ nodes). It will probably not make view 
installation faster either, but if we have 1000 nodes, then we'll use 
less bandwidth to install views. On an almost saturated network, this 
can make a difference.


> On Sep 12, 2013, at 4:15 PM, Bela Ban <bban at redhat.com> wrote:
>
>> FYI,
>>
>>
>> if you're running large clusters, you might be interested to know that I
>> implemented another optimization: JGRP-1317 [1].
>>
>> The effect of it is that (in decreasing order of importance):
>>
>> - Regular view messages to existing members don't carry a digest any
>> longer. This is a significant size reduction, as a digest has 2 longs
>> per member (although compressed for network transmission)
>>
>> - Where we have a view and a digest (e.g. JoinRsp, merge response,
>> MergeView), the digest's 'members' field and the view's 'members' field
>> point to the same memory location
>>
>> - When we send such an object across the network, the 'members' field is
>> only marshalled once, leading to smaller messages
>>
>>
>> Next up, I'm going to tackle [2] which will reduce the size of view
>> messages even further: instead of sending the *full view* every time, we
>> only send delta views. I believe JGRP-1317 and JGRP-1354 will have a
>> huge impact on systems with a large number of nodes.
>>
>>
>> [1] https://issues.jboss.org/browse/JGRP-1317
>> [2] https://issues.jboss.org/browse/JGRP-1354

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)