[seam-dev] Initial performance findings and baselines

[Rodney Russ]

Jay,

This is definitely a good start; great work!  Any plans for a sustained 
load over time (e.g. an hour)?  I know in performance and scalability 
testing I've been involved with in the past, there were products that 
performed well for short duration tests that fell over when we did our 
24 hour load test.

-Rodney

Jay Balunas wrote:
> Hello All,
>
> I have been looking at Seam 2.1 performance and begun profiling the wiki
> example.  Below are my initial findings.  More will follow but I wanted to
> provide some base information that we look at and compare with changes that
> we make.
>
> Environment
> ------------------------------
> All tests were executed on my laptop (specs provide below).
>
> * Lenovo T61
>   - 2GB Ram
>   - Intel(R) Core(TM)2 Duo CPU     T7500  @ 2.20GHz
>   - 7200RPM 200GB Hard drive
>   - Fedora 8 with all updates as of 9-22-09
>   - MySQL 5.0.45
>   - Sun jdk1.5.0_14
>   - JBoss AS 4.2.3.GA
>
> * Test tools
>   - jmeter 2.3.2
>   - jprofiler 5.12
>
> Test Description
> -----------------------------
> * Tests were all done with the wiki example
>   - svn trunk (rev 9017)
>   - deployed following the production deployment instructions
>     - plus addition mysql instructions from Christian
>   - MySQL primed with the live seamframework.org data as of July 30th.
> * Between all tests
>   - AS server and test applications were restarted
>   - 3 page hits were conducted prior to load tests to prime system
> * The tests described here were all focused on the first page of the users
> forum
>   - http://localhost:8080/Community/SeamUsers
>
> Multi User Load Test Results
> ----------------------------
> * These tests were primarily to provide a base line of performance for
> comparison.
>   - profiling was not done as part of these tests
>   - result times are averages of the runs completed
>
> JBoss AS 4.2.3 as is
> ---------------------
> 25 users x 25 requests each
>  - Average request time 8.85 sec
>  - Median request time   8.22 sec
>  - Throughput                 131.4 per min
>  - notes:
>     - server.log was 1.2 GB
>     - See jboss_423_base_25u.png
>
> 50 users x 25 requests each
>  - Average request time 22.72 sec
>  - Median request time   18.33 sec
>  - Throughput                 96 per min
>  - notes:
>     - server.log is 2.4 GB
>     - this pushed my machine to its limit
>     - the final 10-15 threads would appear to freeze
>        - but would finish fine (except for 1 of the runs)
>     - See jboss_423_base_50u.png
>
> Notes: Obviously the biggest issue here was logging I reduced all logging to
> INFO by modifying the jboss-log4j.xml file and setting root logging level
> and adjusting one other category.  The results were dramatic as seen below.
>
> JBoss AS 4.2.3 with reduced logging
> ------------------------------------
> 25 users x 25 requests each
>  - Average request time 3.08 sec
>  - Median request time   2.94 sec
>  - Throughput                 283.5 per min
>  - notes:
>     - server.log was 87.6 KB
>     - See jboss_423_rd_log_25u.png
>
> 50 users x 25 requests each
>  - Average request time 10.10 sec
>  - Median request time   9.30 sec
>  - Throughput                 237.6 per min
>  - notes:
>     - server.log is 96 KB
>     - all thread completed correctly and without issue.
>     - See jboss_423_rd_log_50u.png
>
> 1 user x 1000 requests
>  - wanted to get sample with just one user and many requests
>  - Average request time 264 ms
>  - Median request time   255 ms
>  - Throughput                 255.5 per min
>  - notes:
>     - server.log is 79 KB
>     - Very tight deviation only 42 ms as expected
>
> Notes: Obviously this is the configuration that I will use for the profiling
> tests to follow, and for comparison runs based on changes made to the code.
>
>
> I have begun profiling runs of the 25 user tests through JProfiler to
> identify where threads are blocking and tracking down areas for
> improvements.  These tests with the profiler truly push my machine to the
> limit, but I am see some results.  I will be writing a follow up email with
> at least some of that information soon - possibly today.
>
> Regards,
> Jay
>
>   
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> seam-dev mailing list
> seam-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/seam-dev
>