wrote:

wrote:

NonBlockingHashMap is great *except* that it relies on sun.misc.Unsafe, which makes portability a bit iffy. I *think* that it could be ported to use Atomic* instead, but it looked like quite a bit of work to do so (not to mention testing). The problem with CHM is that they are *big*, so you don't want to have a lot of them - *especially* if you're using a high concurrency level, because they get even bigger in that case. - DML On 09/15/2009 11:59 AM, Christian Vest Hansen wrote: > Using NonBlockingHashMap from highscalelib could be another option. > > On Tue, Sep 15, 2009 at 5:59 PM, Bela Ban <bban(a)redhat.com&gt; wrote: >> FYI, >> >> This might be important for you guys as well (if you haven't done >> so): >> I'll add code to configure ConcurrentHashMaps based on system >> properties, e.g. capacity, load factory etc. >> >> This will be important for large clusters. E.g. when we have 1000 >> nodes >> in the cluster, most hashmaps in JGroups will contain 1000 >> entries, and >> if we only have 16 buckets (the default IIRC), there would be a >> lot of >> contention. >> >> >> https://jira.jboss.org/jira/browse/JGRP-1049 >> >> ConcurrentHashMap >> ----------------- >> >> CCHMs have a default initial capacity (16), load factor (0.75) and >> concurrency level (16). These are OK for most >> scenarios, but we have to investigate whether these values are >> sufficient for 1000 node clusters. >> >> When for example 1000 threads from different senders access the same >> CCHM, we need to make sure we don't have high >> contention, ie. by spreading a 1000 senders over 16 buckets. >> >> Investigate whether we should add CCHM initial sizes, load factors >> and >> concurrency levels as properties. >> >> With ergonomics [1], we could for example set bucket sizes for CCHMs >> dynamically, e.g. based on cluster size. >> >> >> >> [1] https://jira.jboss.org/jira/browse/JGRP-1037 >> >> -- >> Bela Ban >> Lead JGroups / Clustering Team >> JBoss >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

David M. Lloyd wrote: > NonBlockingHashMap is great *except* that it relies on sun.misc.Unsafe, > which makes portability a bit iffy. I *think* that it could be ported to > use Atomic* instead, but it looked like quite a bit of work to do so (not > to mention testing). > > The problem with CHM is that they are *big*, so you don't want to have a > lot of them - *especially* if you're using a high concurrency level, > because they get even bigger in that case. Understood that CHMs are big, but back to my question: how doesn Infinispan intend to address hashmaps with 1000 different keys ? Or are you guys already parameterizing creation of CHMs ?

On Sep 16, 2009, at 10:22 AM, Jason T. Greene wrote: > Bela Ban wrote: >> >> David M. Lloyd wrote: >>> NonBlockingHashMap is great *except* that it relies on >>> sun.misc.Unsafe, >>> which makes portability a bit iffy. I *think* that it could be >>> ported to >>> use Atomic* instead, but it looked like quite a bit of work to do >>> so (not >>> to mention testing). >>> >>> The problem with CHM is that they are *big*, so you don't want to >>> have a >>> lot of them - *especially* if you're using a high concurrency >>> level, >>> because they get even bigger in that case. >> >> Understood that CHMs are big, but back to my question: how doesn >> Infinispan intend to address hashmaps with 1000 different keys ? Or >> are >> you guys already parameterizing creation of CHMs ? >> > > There is a concurrency level setting that the user can set based off > of > expected size (it also affects striping). I think this only has to do with the lock striping, and is rather related to the number of concurrent users expected to access the cache, not with the size of the data. AFAIK, there's no correlation between the number of cache entries, and the way the CHMs are being instantiated. Manik can put some more light on this on his return, but I do see a lot of sense for this correlation.

> -- > Jason T. Greene > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

NonBlockingHashMap is great *except* that it relies on sun.misc.Unsafe, which makes portability a bit iffy. I *think* that it could be ported to use Atomic* instead, but it looked like quite a bit of work to do so (not to mention testing).

The problem with CHM is that they are *big*, so you don't want to have a lot of them - *especially* if you're using a high concurrency level, because they get even bigger in that case. - DML On 09/15/2009 11:59 AM, Christian Vest Hansen wrote: > Using NonBlockingHashMap from highscalelib could be another option. > > On Tue, Sep 15, 2009 at 5:59 PM, Bela Ban <bban(a)redhat.com&gt; wrote: >> FYI, >> >> This might be important for you guys as well (if you haven't done >> so): >> I'll add code to configure ConcurrentHashMaps based on system >> properties, e.g. capacity, load factory etc. >> >> This will be important for large clusters. E.g. when we have 1000 >> nodes >> in the cluster, most hashmaps in JGroups will contain 1000 >> entries, and >> if we only have 16 buckets (the default IIRC), there would be a >> lot of >> contention. >> >> >> https://jira.jboss.org/jira/browse/JGRP-1049 >> >> ConcurrentHashMap >> ----------------- >> >> CCHMs have a default initial capacity (16), load factor (0.75) and >> concurrency level (16). These are OK for most >> scenarios, but we have to investigate whether these values are >> sufficient for 1000 node clusters. >> >> When for example 1000 threads from different senders access the same >> CCHM, we need to make sure we don't have high >> contention, ie. by spreading a 1000 senders over 16 buckets. >> >> Investigate whether we should add CCHM initial sizes, load factors >> and >> concurrency levels as properties. >> >> With ergonomics [1], we could for example set bucket sizes for CCHMs >> dynamically, e.g. based on cluster size. >> >> >> >> [1] https://jira.jboss.org/jira/browse/JGRP-1037 >> >> -- >> Bela Ban >> Lead JGroups / Clustering Team >> JBoss >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev