[jbossseam-issues] [JBoss JIRA] Closed: (JBSEAM-1332) Init.instance() causes (unecessary) lock contention under heavy load.

[Gavin King (JIRA)]

     [ http://jira.jboss.com/jira/browse/JBSEAM-1332?page=all ]

Gavin King closed JBSEAM-1332.
------------------------------

    Fix Version/s: 1.3.0.BETA1
       Resolution: Done
         Assignee: Gavin King  (was: Shane Bryzak)

Fixed in CVS (hopefully!), please run some tests, and let us know the results.

> Init.instance() causes (unecessary) lock contention under heavy load.
> ---------------------------------------------------------------------
>
>                 Key: JBSEAM-1332
>                 URL: http://jira.jboss.com/jira/browse/JBSEAM-1332
>             Project: JBoss Seam
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1.GA, 1.2.0.GA
>         Environment: Jboss AS 4.0.5GA  , Sun jdk 1.5.0_11 ,  JSF RI 1.2 b4,  Linux  FC 5  running 2.5.15.2054_FC5  SMP  , Dual Intel Xeons  w/ HT enabled. (linux sees 4 cpus);  Note that the tomcat instance is modified as described in the associated forum post.
>            Reporter: Radu Aghinii
>         Assigned To: Gavin King
>             Fix For: 1.3.0.BETA1
>
>
> Under heavy load the variation of  response time for identical seam requests gets very large.  
> For my app, for example, under heavy load some requests will take 5 seconds to finish while others take 5 minutes. 
> Check the associated forum post for a more narrative story  that motivated this investigation but  the long and the short of it is that my app ( and i suspect any seam app)   calls Init.instance()   a LOT ( ~100K times per request).
> The current implementation of it just:
> Init init = (Init) Contexts.getApplicationContext().get(Init.class);  which looks up the  seam component name for the Init class and then looks up the component by name.
> The problem is that the name lookup uses  java.lang.Class.getAnnotation() which internally calls the synchronized method  "initAnnotationsIfNecessary()".  Under heavy load i found most of the jboss request threads waiting  to  lock that lock.
> Replacing the body of Init.instance() to skip the name lookup and just do:
> Init init = (Init) Contexts.getApplicationContext().get("org.jboss.seam.core.init")  resulted in much improved behavior.
> Before this change, once  the machine was loaded to the point where requests were taking generally taking 5-10 seconds SOME connections would take a few minutes (capped at 300 seconds due to some timeout either in jboss or apache).   Now that performance is pretty bad for a web app, but even transient conditions such as GC, traffic spikes and DB locks would induce the condition.  Further, especially when this happened under moderate general load, the machine wouldn't recover, but rather the number of concurrent connections would climb, the performance per requests would fall further and eventually  the jvm would run out of memory.   With the change in place the machines recover.   Even when the average requests are taking 20-30 seconds (a very high load situation, but we see it due to the transients mentioned above)   the slowest requests are in the  1minute to 2 minute range and, most importantly, the machine recovers pretty quickly.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira