[infinispan-issues] [JBoss JIRA] (ISPN-4842) Reduce the overhead of clustered caches

[Sanne Grinovero (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040859#comment-13040859 ] 

Sanne Grinovero commented on ISPN-4842:
---------------------------------------

As discussed face to face, I think it's time to reconsider the asymmetric Caches idea.

Just make them all initialized eagerly, don't allow starting new caches after the CacheManager has started, but make it easier for people to start multiple (independent) CacheManager(s) on a FORKed JGroups channel.

This would also require for the same application (classloader and classes) to be available on each node participating in the CacheManager group; that's not a drawback but a benefit: would allow us to load many more extensions points to the CacheManager from the user context: from Infinispan configuration files but up to custom interceptors et al.

> Reduce the overhead of clustered caches
> ---------------------------------------
>
>                 Key: ISPN-4842
>                 URL: https://issues.jboss.org/browse/ISPN-4842
>             Project: Infinispan
>          Issue Type: Feature Request
>          Components: Core
>    Affects Versions: 7.0.0.CR1
>            Reporter: Takayoshi Kimura
>
> A user is testing 500 nodes cluster with 500 dist caches defined, and plans to expand it to 3000 caches.
> Infinispan manages consistent hash per cache, uses a JGroups channel per cache and uses several threads per cache. It gives significant overhead with this large size cluster. When tested with this size, Infinispan easily exhausted all threads in the thread pools and deadlocks, and requires several thousands threads to handle massive JOIN requests - the coord receives 499 * 3000 JOIN requests.
> It would be great if we can share the consistent hash and resources between caches. For example, define a "master" dist cache and allow other caches to refer to the master cache for resource sharing.



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)