[jbossseam-issues] [JBoss JIRA] Created: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[jbossseam-issues] [JBoss JIRA]...

Jay Balunas (JIRA)

Monday, 17 November 2008 Mon, 17 Nov '08

2:12 p.m.

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Priority: Blocker Fix For: 2.1.1.CR2 Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. ----------------------------------- -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

Show replies by date

Jay Balunas (JIRA)

Monday, 17 November Mon, 17 Nov

2:14 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- There was discussion that the problem could be using seam's "debug" option. I verified that both version are setting debug=false, and that the log output shows that seam is setting this correctly to "false".

...

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

Jay Balunas (JIRA)

2:16 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Updated: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas updated JBSEAM-3712: -------------------------------- Attachment: Test Plan.jmx Attached my basic 25 user script for the wiki example. It does assume a page "wiki/Community/SeamUsers" exists for the testing and I am not certain if such a page exists by default in the wiki example. I have imported the live seamframework.org data from July 31st in my system. However the script is easy enough to modify with JMeter 2.3.2 and you should be able to change the page that will be accessed without a problem. If needed I can forward the instructions and data for that import, but it is not a quick process.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

Wednesday, 26 November Wed, 26 Nov

1:33 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- Dan and I have been looking at this further. Things that we tested: 1) Removing the debug jar from the WAR incase certain setting are dependent on that - No change. 2) Reverting r9556 with Naming.java - so we are not caching the initial context as a static variable for all threads - No change 3) Slowed the ramp up speed of the tested threads so that the threads do not stack up so quickly. - this actually helped the throughput but it is still in 60 second range. Whats next: We need to look at the changes made with JBSEAM-3519 - this r9552, r9550. Because the primary issue seems to be transaction related reverting these may help us to understand what is causing this.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

2:18 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- I just reverted the changes made for https://jira.jboss.org/jira/browse/JBSEAM-3519 and nothing changed in the load tests. Profiling the load test has not really uncovered anything new - beyond what I originally found and posted in the jira.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Dan Allen (JIRA)

2:27 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Dan Allen commented on JBSEAM-3712: ----------------------------------- I think what we need in order for other devs to help out is a test script that we can run against a demo that we can get running in a reasonable amount of time. Then, we need to run that test with Seam 2.0.2.GA and then trunk so we can observe the change. Otherwise, we are shooting in the dark as we work on fixes. I generated a quick seam-gen application and found it hard to notice a different in performance. Perhaps the right answer is the booking example. This also allows us to prove to the community that we actually did improve performance. The wiki example is important to test, but has a high barrier to entry.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Pete Muir (JIRA)

8:31 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Pete Muir commented on JBSEAM-3712: ----------------------------------- Yes, and this script needs to run in hudson :-)

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Shane Bryzak (JIRA)

Monday, 1 December Mon, 1 Dec

4:29 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Assigned: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Shane Bryzak reassigned JBSEAM-3712: ------------------------------------ Assignee: Shane Bryzak

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Shane Bryzak Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

Tuesday, 2 December Tue, 2 Dec

3:02 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- I have narrowed down the issue to r9196 < http://fisheye.jboss.org/changelog/Seam/?cs=9196 > this is a wiki specific change to the WikiPluginMacro.java class The good news is that this is most likely a wiki specific issue and not an issue with seam core. The bad new is that is may be effecting the wiki application now, plus all the time wasted on this issue. The wiki code needs to be profiled specifically for this to determine the root cause - my guess is excessive CPU cycles in the equals() and hashcode() but I am not sure (and they don't look bad). I'll ping Christian on this and see what he thinks. However, what I really think needs to happen first is that we need to setup the dvdstore example for perf/load testing as discussed this morning. This will provide an easier way for dev and users to run the same tests, and will provide a second performance benchmark to avoid this type of issue in the future. I'll wait until Shane or Christian gets a chance to look at this issue before resolving.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Shane Bryzak Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

3:33 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- Working with Christian we have it narrowed down to the override on the hashcode method. I'll profile and see what is happening.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Shane Bryzak Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

4:56 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- The profiler was of very limited use (can't unfilter java.lang.Object to see when hashCode is called), and to be honest it is not going to tell us anything we do not already know. The hashCode() method and the super.hashCode() method needs to be optimized - probably through caching the inner hashcode calls since some of the values called within do not appear to change. Below are the two hashcode methods. I'll try out some different caching approaches tomorrow morning and see how this affects the perf. However for this release I do believe that this is blocker, and I'll move it down. If the fix ( I'll try tomorrow ) is safe and easy we can get it in, otherwise we'll have to wait. WikiPluginMacro hashCode: @Override public int hashCode() { int result = super.hashCode(); result = 31 * result + uniqueId.hashCode(); return result; } WikiTextMacro hashCode (the super of ^^^) public int hashCode() { int result; result = (position != null ? position.hashCode() : 0); result = 31 * result + name.hashCode(); return result; }

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Shane Bryzak Priority: Blocker Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

4:56 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Updated: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas updated JBSEAM-3712: -------------------------------- Priority: Critical (was: Blocker)

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Shane Bryzak Priority: Critical Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

Wednesday, 3 December Wed, 3 Dec

4:01 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- I was able to narrow the issue down to the "result = 31 * result + uniqueId.hashCode(); " line. Comment that out and everything work fine. The hashCode() method of a String is fairly complex, but it should not be slowing things down this much. I even tried caching the hashCode value along with the String, but that is not it either. From the code this does not look like it needs to be a String - perhaps using a long value will be sufficient.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Shane Bryzak Priority: Critical Fix For: 2.1.1.CR2 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

Thursday, 4 December Thu, 4 Dec

10:18 a.m.

New subject: [jbossseam-issues] [JBoss JIRA] Updated: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas updated JBSEAM-3712: -------------------------------- Fix Version/s: 2.1.2.CR1 (was: 2.1.1.CR2) Assignee: Christian Bauer (was: Shane Bryzak) Priority: Major (was: Critical)

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Christian Bauer Fix For: 2.1.2.CR1 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Jay Balunas (JIRA)

4:06 p.m.

New subject: [jbossseam-issues] [JBoss JIRA] Commented: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Jay Balunas commented on JBSEAM-3712: ------------------------------------- My new theory is that this has nothing to do with the processing time of the hashcode() method call. I have timed it, tried different JVMs, etc... The hashcode call itself never reports more than 0 ms. Granted the granularity of the of the time is not great, but if this was the true cause of the 100+ second response time it should show something. I believe that the real cause is that now all of these objects have truly unique hashcodes. This in turn is causing more of these objects to be treated uniquely and because of that there are more of these object performing their various tasks. That is why the blocks that I saw had more to do with transactions, and mysql calls than anything else. All of these unique objects are performing these activities and that is causing the blockage. I can't spend anymore time on this atm.

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Assignee: Christian Bauer Fix For: 2.1.2.CR1 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Christian Bauer (JIRA)

Tuesday, 8 December Tue, 8 Dec

3:54 a.m.

New subject: [JBoss JIRA] Assigned: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Christian Bauer reassigned JBSEAM-3712: --------------------------------------- Assignee: (was: Christian Bauer)

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Fix For: 2.2.1.CR1 Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

Pete Muir (JIRA)

Wednesday, 20 January Wed, 20 Jan

9:06 a.m.

New subject: [JBoss JIRA] Updated: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1

[ https://jira.jboss.org/jira/browse/JBSEAM-3712?page=com.atlassian.jira.pl... ] Pete Muir updated JBSEAM-3712: ------------------------------ Fix Version/s: The future (was: 2.2.1.CR1)

...

Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1 ------------------------------------------------------- Key: JBSEAM-3712 URL: https://jira.jboss.org/jira/browse/JBSEAM-3712 Project: Seam Issue Type: Bug Components: Core, Platform interoperability Affects Versions: 2.1.0.SP1 Environment: Fedora 8 JDK 5 Reporter: Jay Balunas Fix For: The future Attachments: Test Plan.jmx Working with the latest trunk (r9557) and testing the performance changes that were made. Previous tests were done with (r9017). Just as a refresher my baseline test uses the wiki example with all of the sfwk.org data up to July 31st 2008. For these tests I use JBoss AS 4.2.3 with JDK5 on my linux machine. Jmeter is used to load test and calculate the results and graphs. I then use JProfiler to to identify either blocking threads, call graphs, and CPU usage. I was using a mixture of 25 and 50 users - hitting the server 25 times each. As before they were accessing the first page of the user forum. With r9017 the 25 user x 25 requests averaged 3 seconds a request. With trunk they were 100+ seconds. Thinking something was wrong with the system I replaced the 9557 wiki.war with the 9017 and reran with all other variables the same. Again the 9017 saw about 3 seconds for the average over the 625 requests. I then profiled the server under load as I did before. The methods below appear to be the primary offenders although as with most blocking threads there are some others waiting on the same monitor. ----------------------------------- 1) com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate.findLock(..) - This appears to be the biggest issue. Every requests are generating many of these calls. - http://viewvc.jboss.org/cgi-bin/viewvc.cgi/labs/labs/jbosstm/branches/JBO... - It looks like every interaction with any transaction causing synchronization issues with this call. - We'll need to find a way to limit these calls. - I'm guessing some of the changes made for JBSEAM-3519 may be the cause although I have not had time to look deeper. - See : http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/transa... http://fisheye.jboss.org/browse/Seam/trunk/src/main/org/jboss/seam/util/W... ----------------------------------- 2) org.jboss.naming.ENCFactory.getObjectInstance() javax.naming.Context.lookup(java.lang.String) This appears to be the second biggest offender and it looks like we are no longer blocking on retrieving the InitialContext, but now blocking on performing the lookups using the context. ----------------------------------- 3) org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection() org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection() This is the third biggest issue, but much less than the others. 1 or 2 dozen blocks on 60 requests. These are all related to hibernate calls and database access from what I've seen so far. -----------------------------------

5373

days inactive

5802

days old

seam-issues@lists.jboss.org

Manage subscription

16 comments

5 participants

tags (0)

participants (5)

Christian Bauer (JIRA)
Dan Allen (JIRA)
Jay Balunas (JIRA)
Pete Muir (JIRA)
Shane Bryzak (JIRA)

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[jbossseam-issues] [JBoss JIRA] Created: (JBSEAM-3712) Seam performance regression from 2.1.0.SP1 to 2.1.1.CR1