[
https://issues.jboss.org/browse/ISPN-4842?page=com.atlassian.jira.plugin....
]
Dan Berindei commented on ISPN-4842:
------------------------------------
Takayoshi, the JGroups channel is already shared. There are probably some per-cache
threads that could be merged, like the TXCleanupService thread that you already issued a
PR for, however most threads should be shared.
Sharing the consistent hash would require all those caches to be symmetric - i.e. if cache
{{c}} is running on one node, it must also run on all the other nodes in the cluster. We
are considering allowing only symmetric caches for 8.0 (adding support for multiple cache
managers on FORKed JChannels instead), which should allow sharing the consistent hash.
However, 8.0 is quite far away, so I would focus on reducing cost of computing the
consistent hash (e.g. [TopologyAware]SyncConsistentHash results may be cached) and the
merging per-cache threads into global threads. Please create separate issues for other
per-cache threads that you've found.
Shared consistent hash
----------------------
Key: ISPN-4842
URL:
https://issues.jboss.org/browse/ISPN-4842
Project: Infinispan
Issue Type: Feature Request
Components: Core
Affects Versions: 7.0.0.CR1
Reporter: Takayoshi Kimura
A user is testing 500 nodes cluster with 500 dist caches defined, and plans to expand it
to 3000 caches.
Infinispan manages consistent hash per cache, uses a JGroups channel per cache and uses
several threads per cache. It gives significant overhead with this large size cluster.
When tested with this size, Infinispan easily exhausted all threads in the thread pools
and deadlocks, and requires several thousands threads to handle massive JOIN requests -
the coord receives 499 * 3000 JOIN requests.
It would be great if we can share the consistent hash and resources between caches. For
example, define a "master" dist cache and allow other caches to refer to the
master cache for resource sharing.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)