[mod_cluster-issues] [JBoss JIRA] (MODCLUSTER-288) SystemMemoryUsageLoadMetric is not correct on Linux/Unix
Michal Babacek (JIRA)
jira-events at lists.jboss.org
Thu Mar 22 15:31:47 EDT 2012
[ https://issues.jboss.org/browse/MODCLUSTER-288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michal Babacek reopened MODCLUSTER-288:
---------------------------------------
@[~pferraro], I am sorry that my point is not clear, I will try to rephrase and compress the description of this JIRA:
I understand the possible use cases, as you have mentioned, e.g. starting external processes or any other "extra-jvm work". The problem is, that any Linux/UNIX system that tries to utilize as much RAM as possible as e.g. a cache will always (no matter how many external non JVM processes you start) report full load with this SystemMemoryUsageLoadMetric implementation. Therefore, your statement is not right: If I have an application that does significant extra-jvm work and I want to monitor system memory, the current SystemMemoryUsageLoadMetric implementation is not gonna help me.
In my humble opinion, we have to modify SystemMemoryUsageLoadMetric in such a a way, that *buffers/cache* value is actually *subtracted* from the *free* number.
Thank you for spending some time commenting on this JIRA.
> SystemMemoryUsageLoadMetric is not correct on Linux/Unix
> --------------------------------------------------------
>
> Key: MODCLUSTER-288
> URL: https://issues.jboss.org/browse/MODCLUSTER-288
> Project: mod_cluster
> Issue Type: Bug
> Affects Versions: 1.0.10, 1.1.0.Final, 1.1.1.Final, 1.2.0.Final
> Environment: Linux/Unix operating systems with standard memory management
> Reporter: Michal Babacek
> Assignee: Paul Ferraro
>
> I am convinced that the [SystemMemoryUsageLoadMetric.java|https://github.com/modcluster/mod_cluster/blob/master/core/src/main/java/org/jboss/modcluster/load/metric/impl/SystemMemoryUsageLoadMetric.java] is of no use to anyone.
> I have tested it by allocating some memory outside the JVM control with [NativeMemoryStress.c|https://github.com/Karm/clusterbench/blob/system-memory-load/clusterbench-native/src/main/native/NativeMemoryStress.c].
> The results from Mod_cluster are such that the system is almost fully loaded all the time despite the fact that I work with 8GB RAM box with almost nothing running on it.
> This [picture|http://hudson.qa.jboss.com/hudson/view/mod_cluster-QE/view/mod_cluster-QE-eap-6x/job/eap-6x-mod_cluster-rhel6-x86_64-load-mem/4/artifact/report/load.png] displays time on X axis and (100-(Load from Mod_cluster)) on Y axis. As you may observe, all 4 nodes appear to be under full load all the time and there is a spike on each of them. This spike represents a period of time when the aforementioned [NativeMemoryStress.c|https://github.com/Karm/clusterbench/blob/system-memory-load/clusterbench-native/src/main/native/NativeMemoryStress.c] was called with the following setting:
> {noformat}
> megabytes 500;
> milliseconds 60000;
> {noformat}
> What had happened is that kernel has cleared some cache and garbage data from the RAM in order to accommodate new demands, so by the time I freed the memory, there was all of sudden actually more free RAM then before.
> The actual problem with the current [SystemMemoryUsageLoadMetric.java|https://github.com/modcluster/mod_cluster/blob/master/core/src/main/java/org/jboss/modcluster/load/metric/impl/SystemMemoryUsageLoadMetric.java] implementation using [OperatingSystemMXBean|http://docs.oracle.com/javase/6/docs/jre/api/management/extension/com/sun/management/OperatingSystemMXBean.html] is that this code:
> {code:lang=java}
> +++
> public static final String FREE_MEMORY = "FreePhysicalMemorySize";
> public static final String TOTAL_MEMORY = "TotalPhysicalMemorySize";
> +++
> double free = ((Number) this.server.getAttribute(name, FREE_MEMORY)).doubleValue();
> double total = ((Number) this.server.getAttribute(name, TOTAL_MEMORY)).doubleValue();
> return (total - free) / total;
> +++
> {code}
> Is somewhat similar to this (on 8GB RAM machine with almost nothing running on at the moment):
> {noformat}
> [karm at killerbox karm]$ free -m
> total used free shared buffers cached
> Mem: 7933 6755 1177 0 44 141
> -/+ buffers/cache: 6569 1363
> Swap: 16383 58 16325
> {noformat}
> Obviously, this does not mean that I could get only 1177MB of RAM if I want to. The "free" actually means "wasted". The majority of RAM is being used as cache. If I ask for e.g. 2GB, I am gonna get them, because kernel will simply sacrifice some cache, clears some garbage from RAM and free up space so as to accommodate my demands.
> So, according to my opinion, on a sane, well-behaving caching operating system, the [SystemMemoryUsageLoadMetric.java|https://github.com/modcluster/mod_cluster/blob/master/core/src/main/java/org/jboss/modcluster/load/metric/impl/SystemMemoryUsageLoadMetric.java] will always show ~ 98% RAM usage.
> Due to this fact, this metric is unlikely to be of some use to anyone on Linux/UNIX. What do you think?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
More information about the mod_cluster-issues
mailing list