[Performance Tuning] - JBoss AS 4.2.3.GA + Seam 2.1.0.GA - 1 hit = 100% CPU
by rhills
We're trying to work out why a single page hit on our application pushes the CPU to 100%.
Our setup is JBoss AS 4.2.3.GA + Seam 2.1.0.GA. We're using the APR and serving over HTTPS.
The 100% CPU hit happens with any page that does any "work" (DB lookup + rendering results). It occurs after multiple hits on the same page. It occurs on two dev machines and a staging server.
The web application feels very slow for a single user, even running off the staging server. I've got to do performance testing at some stage, but there seems little point while it's so sluggish for a single user.
WRT memory, my dev machine has 2GB of RAM. The JVM config is: -Xms300m -Xmx300m -XX:NewSize=100m -XX:MaxNewSize=100m and profiling IAW http://www.cecmg.de/doc/tagung_2007/agenda07/24-mai/2b3-peter-johnson/ind... suggests that the GC process is running smoothly. On this machine, there is about 800MB RAM free before JBoss startup and our app doesn't seem to consume the 300MB heap (not in short runs anyway).
I've been through the server optimisation/slimming process detailed at http://www.jboss.org/community/docs/DOC-10217, with minimal impact on this issue. Not that I was surprised as most of the recommendations in that doc appear to be targeted at scaling performance and our problem is with a single user.
When I profile the classes to try and establish what is causing the CPU hit, 95% of the CPU time vanishes in two APR methods: org.apache.tomcat.jni.Poll.poll and org.apache.tomcat.jni.Socket.accept.
Almost no CPU time is consumed by our own classes.
The profiling is done with JProfiler and AFAICT, the CPU time recording is only for the JBoss server, not other apps on this machine (browser etc.). For the record, the DB accessed by this app is on a separate server and all tables are quite small (the largest is 12K rows, most are < a few hundred records).
I'm not sure what this is telling me. Is this normal behaviour for JBoss serving HTTPS or have we got our server configuration all wrong? Does anyone have any thoughts on where I should look next?
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4193645#4193645
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4193645
15 years, 5 months
[Performance Tuning] - Re: JBoss Community DOC-10217 - advice on tomcat connector t
by rhills
Hi Peter,
"PeterJ" wrote : The problem with using recommendations is that often they are specific to the environment of the user. And sometimes the recommendations are not necessarily the best especially if the recommendation is based on a performance test. For example, one of the recommendations is for setting sun.rmi.dgc.client.gcInterval to 60 minutes. Why 60 minutes? Because the specjappserver performance test runs in under 60 minutes so the "recommended" setting prevents stray full GCs from being called during the test run. But in a real life production environment, I would set it to once a day or once a week.
That's an excellent example. It's just a shame that the docs don't include that kind of qualification as well. I agree with you that it's almost impossible to provide recommendations that suit everybody. However I think that a set of recommendations for a specific set of circumstances with some explanation of why each value has been chosen would be a very useful document that most of us could then extrapolate to our own circumstances.
"PeterJ" wrote : Having said that, the Tomcat documents, since they describe the settings in general terms, are better references on how the settings work.
|
| (Examples below use the default values)
|
| MinSpareThreads is the minimum number of unused threads that will be kept available to handle new incoming requests. Example: if 20 threads are actively in use, there will be at least 24 threads (20 in use + 4 spare).
|
| A side effect is that this setting also dictates the initial number of threads created. Example: 0 threads in use, thus 4 threads. (I believe this is where the 'peak load' recommendation comes from - in a test such as specjappserver you can easily calculate the peak load and thus set minsparethreads to create that number of threads as part of system startup.)
|
| At some point the system might get busy and you end up with, say. 200 threads. Then everyone goes to lunch and only 20 threads are busy. When the idle thread timeout is reached, the thread count will be reduced to 70 (20 active plus 50 spare).
I agree, the Tomcat docs are better in this area, but I find your descriptions above even better still. It's a shame they're not in the Tomcat docs.
"PeterJ" wrote : Having said all of this, my testing has shown that Tomcat 6, and thus the versions of JBoss Web Server based on it, while it still accepts and stores these values no longer uses they to manage the threads. The min and max idle thread values are no longer checked in the source code (at least, it is no longer where it used to be checked and I haven't found any alternate locations where they are checked), and the observed behavior appears to validate the claim that they are no longer used.
Hmmm, for some silly reason, I'd thought JBoss was using 5.5 - I probably saw the number "5.5" somewhere and made the assumption without checking. Looking then at the http://tomcat.apache.org/tomcat-6.0-doc/config/http.html, no mention is made of the maxSpareThreads and minSpareThreads attributes, so that adds further evidence to your observations. Thanks for pointing that out.
Presumably, Tomcat now tunes itself for these parameters?
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4193636#4193636
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4193636
15 years, 5 months