[infinispan-issues] [JBoss JIRA] (ISPN-1801) Virtual nodes should be enabled by default

[Manik Surtani (JIRA)]

     [ https://issues.jboss.org/browse/ISPN-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Manik Surtani updated ISPN-1801:
--------------------------------

       Summary: Virtual nodes should be enabled by default  (was: Enable virtual nodes in the default configuration  )
    Complexity: Low

    
> Virtual nodes should be enabled by default
> ------------------------------------------
>
>                 Key: ISPN-1801
>                 URL: https://issues.jboss.org/browse/ISPN-1801
>             Project: Infinispan
>          Issue Type: Feature Request
>          Components: Distributed Cache
>    Affects Versions: 5.1.0.FINAL
>            Reporter: Mircea Markus
>            Assignee: Manik Surtani
>             Fix For: 5.1.1.CR1, 5.1.1.FINAL
>
>
> ATM the default value for virtualNodes is 1. This means that the wheel-share each node has can be very uneven for small(up to 15 nodes) clusters.
> Increasing this value even to a small number(10-30) would significantly improve each node's share of wheel and the chance for a well balanced data distribution over the cluster. 
> Here are some suggestions from an email from Dan:
> <snip>
> I've been working on a test to search for an optimal default value here:
> https://github.com/danberindei/infinispan/commit/983c0328dc40be9609fcabb767dd46f9b98af464
> I'm measuring both the number of keys for which a node is primary
> owner and the number of keys for which it is one of the owners
> compared to the ideal distribution (K/N keys on each node). The former
> tells us how much more work the node could be expected to do, the
> latter how much memory the node is likely to need.
> I'm only running 10000 loops, so the max figure is not the absolute
> maximum. But it's certainly bigger than the 0.9999 percentile.
> The full results are here:
> http://fpaste.org/cI1r/
> The uniformity of the distribution goes up with the number of virtual
> nodes but down with the number of physical nodes. I think we should go
> with a default of 48 nodes (or 50 if you prefer decimal). With 32
> nodes, there's only a 0.1% chance that a node will hold more than 1.35
> * K/N keys, and a 0.1% chance that the node will be primary owner for
> more than 1.5 * K/N keys.
> We could go higher, but we run against the risk of node addresses
> colliding on the hash wheel. According to the formula on the Birthday
> Paradox page (http://en.wikipedia.org/wiki/Birthday_problem), we only
> need 2072 addresses on our 2^31 hash wheel to get a 0.1% chance of
> collision. That means 21 nodes * 96 virtual nodes, 32 nodes * 64
> virtual nodes or 43 nodes * 48 virtual nodes.
> </snip>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira