[infinispan-dev] distribution's impact on performance

[Bela Ban]

Hmm, interesting !

For GETs, I think the only reason the line degrades a only slightly 
after 5 nodes is that the chance of hitting youself (therefore no 
communication needed) decreases with increasing cluster size. I wonder 
how this would line would look like on cluster sizes of 30+ ...

For PUTs, the degradation is more pronounced. I assume this is with 
numOwners=2 ? In that case, the chance of hitting yourself is twice of 
that for GETs.

Can you re-run this test with numOwners=1 ? I guess we should then see a 
similar line for PUTs than for GETs.

I assume you're sending instructions on how to run this ideal distribution ?


On 2/1/11 2:11 PM, Mircea Markus wrote:
> Hi,
>
> I've enhanced Radargun with an ideal distribution function: this is  a perfectly correct behaving function that is aware of the keys passed to it and  makes sure that all the nodes have the same share of keys(I'll send an email soon detailing this behavior),
> I've just run a 3-9 test on the cluster{01..10}.mw.lab.eng.bos.redhat.com, results attached.
> The ideal distribution causes the performance to be more linear. Also for puts we have an constant 10-35% performance increase.
> In the case of default CH 3 nodes, the get has a peek, this can be explained by the fact that two of the nodes moved really fast, almost as in replicated mode, as they hold most the data (90%) (see bellow num of keys on each node);.
>
> Below is the distribution of keys/nodes[1] :
>
> Configuration : dist-sync.xml
> Cluster size: 3 ->  ( 2930 2612 458)
> Cluster size: 5 ->  ( 1922 1930 660 2969 2519)
> Cluster size: 7 ->  ( 4617 1271 720 1028 4394 328 1642)
> Cluster size: 9 ->  ( 2586 4507 1153 556 1292 274 582 2560 4490)
>
> Configuration : idealdistribution/dist-sync-ideal-distribution.xml
> Cluster size: 3 ->  ( 2000 2000 2000)
> Cluster size: 5 ->  ( 2000 2000 2000 2000 2000)
> Cluster size: 7 ->  ( 2000 2000 2000 2000 2000 2000 2000)
> Cluster size: 9 ->  ( 2000 2000 2000 2000 2000 2000 2000 2000 2000)
>
> [1] this can now be obtained by running (./bin/dist.sh - just added).

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss