Thanks Jay, Just to be clear, did you mean that each individual request is now taking 100+ seconds as compared to 3 seconds previously? Jay Balunas wrote: > Performance Review of current Seam trunk fo 2.1.1 > -------------------------------------------------------- > > Working with the latest trunk from today (r9557) and testing the > performance changes that were made. Previous tests were done with > (r9017). Just as a refresher my baseline test uses the wiki example > with all of the sfwk.org <http://sfwk.org> data up to July 31st 2008. > For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. > Jmeter is used to load test and calculate the results and graphs. I > then use JProfiler to to identify either blocking threads, call > graphs, and CPU usage. > Unfortunately the results were not good. I was using a mixture of 25 > and 50 users - hitting the server 25 times each. As before they were > accessing the first page of the user forum. > > With r9017 the 25 user x 25 requests averaged 3 seconds a request. > With trunk they were 100+ seconds. Thinking something was wrong > with the system I replaced the 9557 wiki.war with the 9017 and reran > with all other variables the same. Again the 9017 saw about 3 > seconds for the average over the 625 requests. > > I then profiled the server under load as I did before. The methods > below appear to be the primary offenders although as with most > blocking threads there are some others waiting on the same monitor. > I'll follow up this email with the stack traces, and more > information from my investigation. > > ----------------------------------- > 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) > - This appears to be the biggest issue. Every requests are > generating many of these calls. > - > http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... > > - It looks like every interaction with any transaction causing > synchronization issues with this call. > - We'll need to find a way to limit these calls. > - I'm guessing some of the changes made for JBSEAM-3519 may be the > cause although I have not had time to look deeper. > - See : > http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... > <http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... > > > http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... > <http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... > > ----------------------------------- > 2) org.jboss.naming.ENCFactory.getObjectInstance() > javax.naming.Context.lookup(java.lang.String) > > This appears to be the second biggest offender and it looks like we > are no longer blocking on retrieving the InitialContext, but now > blocking on performing the lookups using the context. > ----------------------------------- > 3) > org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() > > > org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() > > > This is the third biggest issue, but much less than the others. 1 or > 2 dozen blocks on 60 requests. These are all related to hibernate > calls and database access from what I've seen so far. > ----------------------------------- > I'll follow up this email tomorrow with the typical stack traces seen > for each of these. > > Shane could you review, and I'll get more information on these tomorrow. > Thanks, > -Jay > > -- > blog: http://in.relation.to/Bloggers/Jay > ------------------------------------------------------------------------ > > _______________________________________________ > seam-dev mailing list > seam-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/seam-dev > _______________________________________________ seam-dev mailing list seam-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/seam-dev

Yes when run under load (25 users) the system seems to becomes unresponsive. The requests do eventually return - jmeter was reporting an average request time of 100+ seconds. When accessed during the warm up requests (my manual requests to prime the application) the application functioned just fine. -Jay On Wed, Nov 12, 2008 at 8:20 PM, Shane Bryzak <shane.bryzak(a)jboss.com&gt;wrote: > Thanks Jay, > > Just to be clear, did you mean that each individual request is now taking > 100+ seconds as compared to 3 seconds previously? > > Jay Balunas wrote: > >> Performance Review of current Seam trunk fo 2.1.1 >> -------------------------------------------------------- >> >> Working with the latest trunk from today (r9557) and testing the >> performance changes that were made. Previous tests were done with (r9017). >> Just as a refresher my baseline test uses the wiki example with all of the >> sfwk.org <http://sfwk.org> data up to July 31st 2008. >> For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. >> Jmeter is used to load test and calculate the results and graphs. I then >> use JProfiler to to identify either blocking threads, call graphs, and CPU >> usage. >> Unfortunately the results were not good. I was using a mixture of 25 and >> 50 users - hitting the server 25 times each. As before they were accessing >> the first page of the user forum. >> >> With r9017 the 25 user x 25 requests averaged 3 seconds a request. With >> trunk they were 100+ seconds. Thinking something was wrong with the system >> I replaced the 9557 wiki.war with the 9017 and reran with all other >> variables the same. Again the 9017 saw about 3 seconds for the average over >> the 625 requests. >> >> I then profiled the server under load as I did before. The methods below >> appear to be the primary offenders although as with most blocking threads >> there are some others waiting on the same monitor. I'll follow up this >> email with the stack traces, and more information from my investigation. >> >> ----------------------------------- >> 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) >> - This appears to be the biggest issue. Every requests are generating >> many of these calls. >> - >> http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... >> - It looks like every interaction with any transaction causing >> synchronization issues with this call. >> - We'll need to find a way to limit these calls. >> - I'm guessing some of the changes made for JBSEAM-3519 may be the cause >> although I have not had time to look deeper. >> - See : >> http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... >> http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... >> > >> >> http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... >> http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... >> > >> ----------------------------------- >> 2) org.jboss.naming.ENCFactory.getObjectInstance() >> javax.naming.Context.lookup(java.lang.String) >> >> This appears to be the second biggest offender and it looks like we are >> no longer blocking on retrieving the InitialContext, but now blocking on >> performing the lookups using the context. >> ----------------------------------- >> 3) >> org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() >> >> org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() >> >> This is the third biggest issue, but much less than the others. 1 or 2 >> dozen blocks on 60 requests. These are all related to hibernate calls and >> database access from what I've seen so far. >> ----------------------------------- >> I'll follow up this email tomorrow with the typical stack traces seen for >> each of these. >> >> Shane could you review, and I'll get more information on these tomorrow. >> Thanks, >> -Jay >> >> -- >> blog: http://in.relation.to/Bloggers/Jay >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> seam-dev mailing list >> seam-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/seam-dev >> >> > > -- blog: http://in.relation.to/Bloggers/Jay

By the way the goal is to eventually get these scripts and instructions into svn so that anyone can run them. On Wed, Nov 12, 2008 at 10:37 PM, Jay Balunas <tech4j(a)gmail.com&gt; wrote: Shane, Here is my basic 25 user script for the wiki example. It does assume a page "wiki/Community/SeamUsers" exists for the testing and I am not certain if such a page exists by default in the wiki example. I have imported the live seamframework.org data from July 31st in my system. However the script is easy enough to modify with JMeter 2.3.2 and you should be able to change the page that will be accessed without a problem. If needed I can forward the instructions and data for that import, but it is not quick. Depending on what you find - tomorrow I will run the same test on the booking example and the dvd example to get some more data points. Let me know if you find anything or would like me to try something specific. Talk to you later, Jay On Wed, Nov 12, 2008 at 10:22 PM, Jay Balunas <tech4j(a)gmail.com&gt; wrote: Yes when run under load (25 users) the system seems to becomes unresponsive. The requests do eventually return - jmeter was reporting an average request time of 100+ seconds. When accessed during the warm up requests (my manual requests to prime the application) the application functioned just fine. -Jay On Wed, Nov 12, 2008 at 8:20 PM, Shane Bryzak <shane.bryzak(a)jboss.com&gt; wrote: Thanks Jay, Just to be clear, did you mean that each individual request is now taking 100+ seconds as compared to 3 seconds previously? Jay Balunas wrote: Performance Review of current Seam trunk fo 2.1.1 -------------------------------------------------------- Working with the latest trunk from today (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org <http://sfwk.org> data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. Unfortunately the results were not good. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. I'll follow up this email with the stack traces, and more information from my investigation. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... <http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... > http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... <http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... > ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org .jboss .resource .connectionmanager.CachedConnectionManager.unregisterConnection() org .jboss .resource .connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. ----------------------------------- I'll follow up this email tomorrow with the typical stack traces seen for each of these. Shane could you review, and I'll get more information on these tomorrow. Thanks, -Jay -- blog: http://in.relation.to/Bloggers/Jay ------------------------------------------------------------------------ _______________________________________________ seam-dev mailing list seam-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/seam-dev -- blog: http://in.relation.to/Bloggers/Jay -- blog: http://in.relation.to/Bloggers/Jay -- blog: http://in.relation.to/Bloggers/Jay

wrote:

On Nov 13, 2008, at 04:37 , Jay Balunas wrote: Here is my basic 25 user script for the wiki example. It does assume a > page "wiki/Community/SeamUsers" exists for the testing and I am not certain > if such a page exists by default in the wiki example. > Testing this page is not a good idea. If you do not have an authenticated user, it will run a particularly heavy SQL query once and then cache the result in the page fragment cache. Most of the page is then served from the cache. If you have an authenticated user the cache is not used at all, then the time for the SQL query will dominate the result. I recommend using the 'dev' deploy of the wiki and the default startpage. This is a good mix of database access and lots of code loops. The live sfwk.org data should probably be used to find other performance bottlenecks that are inherent in the wiki. We know that the SQL query on the forum overview page is an issue. But, a forum posting with 50 replies is quite slow to render also. Why? _______________________________________________ seam-dev mailing list seam-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/seam-dev