[
https://issues.jboss.org/browse/ISPN-4979?page=com.atlassian.jira.plugin....
]
Dan Berindei commented on ISPN-4979:
------------------------------------
[~sannegrinovero], replacing all address collections with masks over Views was always
going to be a challenge because not all members have all the views. E.g. a joiner might
receive an old message referencing a view that it doesn't know about. It has become
even trickier with partition handling, because we now have to track both nodes that are in
the current consistent hash but not in the current JGroups view, and at the same time
nodes that are in the current JGroups view but not in the JGroups view in which the CH was
created.
[~william.burns], I didn't expect to reuse addresses (or anything else) across
different CacheStatusResponse instances. But since we're talking about 300 nodes x
3000 caches (potentially), I think the savings from intra-CSR reuse would be big enough.
In fact, my first thought was to reuse the address just inside a
{{DefaultConsistentHash}}, writing the members list first and referencing the members via
their indices in the other collections.
I have one small concern about your approach, as the additional {{ThreadLocal}} and
{{IdentityHashMap}} accesses might slow down the more more common case of having just one
address in the command. E.g. in a PrepareCommand, it's just the address of the
originator in the {{GlobalTransaction}} object.
Since the number of addresses is quite limited and doesn't change from cache to cache
(or even from manager to manager, with FORK), I guess we could also add a static map in
{{JGroupsAddress}}, limited to {{$\{jgroups.uuid_cache.max_elements\}}} entries (similar
to JGroups' UUID cache), and keeping the wire format as it is.
If necessary, we could try reusing the entire CH across CSRs. But that might be easier to
do with a {{ResponseFilter}} in the {{GET_STATUS}} invocation, again keeping the wire
format as it is.
CacheStatusResponse map uses too much memory
--------------------------------------------
Key: ISPN-4979
URL:
https://issues.jboss.org/browse/ISPN-4979
Project: Infinispan
Issue Type: Bug
Components: Core, State Transfer
Affects Versions: 7.0.0.Final
Reporter: Dan Berindei
Assignee: William Burns
Priority: Critical
Fix For: 7.1.0.Final
When the cluster is large and there are a log of caches, the {{CacheStatusResponse}} map
on the new coordinator can get quite large. One of the problems that seems to be that the
addresses in {{DefaultConsistentHash}} are duplicated on serialization, so the
deserialized version occupies more memory.
We need to investigate why the objects are not "shared" by the River
marshaller, and maybe work around the problem by de-duplicating the addresses in the
externalizer.
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)