[seam-dev] Initial performance findings and baselines

Hello All, I have been looking at Seam 2.1 performance and begun profiling the wiki example. Below are my initial findings. More will follow but I wanted to provide some base information that we look at and compare with changes that we make. Environment ------------------------------ All tests were executed on my laptop (specs provide below). * Lenovo T61 - 2GB Ram - Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz - 7200RPM 200GB Hard drive - Fedora 8 with all updates as of 9-22-09 - MySQL 5.0.45 - Sun jdk1.5.0_14 - JBoss AS 4.2.3.GA * Test tools - jmeter 2.3.2 - jprofiler 5.12 Test Description ----------------------------- * Tests were all done with the wiki example - svn trunk (rev 9017) - deployed following the production deployment instructions - plus addition mysql instructions from Christian - MySQL primed with the live seamframework.org data as of July 30th. * Between all tests - AS server and test applications were restarted - 3 page hits were conducted prior to load tests to prime system * The tests described here were all focused on the first page of the users forum - http://localhost:8080/Community/SeamUsers Multi User Load Test Results ---------------------------- * These tests were primarily to provide a base line of performance for comparison. - profiling was not done as part of these tests - result times are averages of the runs completed JBoss AS 4.2.3 as is --------------------- 25 users x 25 requests each - Average request time 8.85 sec - Median request time 8.22 sec - Throughput 131.4 per min - notes: - server.log was 1.2 GB - See jboss_423_base_25u.png 50 users x 25 requests each - Average request time 22.72 sec - Median request time 18.33 sec - Throughput 96 per min - notes: - server.log is 2.4 GB - this pushed my machine to its limit - the final 10-15 threads would appear to freeze - but would finish fine (except for 1 of the runs) - See jboss_423_base_50u.png Notes: Obviously the biggest issue here was logging I reduced all logging to INFO by modifying the jboss-log4j.xml file and setting root logging level and adjusting one other category. The results were dramatic as seen below. JBoss AS 4.2.3 with reduced logging ------------------------------------ 25 users x 25 requests each - Average request time 3.08 sec - Median request time 2.94 sec - Throughput 283.5 per min - notes: - server.log was 87.6 KB - See jboss_423_rd_log_25u.png 50 users x 25 requests each - Average request time 10.10 sec - Median request time 9.30 sec - Throughput 237.6 per min - notes: - server.log is 96 KB - all thread completed correctly and without issue. - See jboss_423_rd_log_50u.png 1 user x 1000 requests - wanted to get sample with just one user and many requests - Average request time 264 ms - Median request time 255 ms - Throughput 255.5 per min - notes: - server.log is 79 KB - Very tight deviation only 42 ms as expected Notes: Obviously this is the configuration that I will use for the profiling tests to follow, and for comparison runs based on changes made to the code. I have begun profiling runs of the 25 user tests through JProfiler to identify where threads are blocking and tracking down areas for improvements. These tests with the profiler truly push my machine to the limit, but I am see some results. I will be writing a follow up email with at least some of that information soon - possibly today. Regards, Jay -- blog: http://in.relation.to/Bloggers/Jay