<div dir="ltr">Good news and excellent turn around from Ryan on the UIComponent issue in the JSF RI. <br><br>Ryan was able to provide a set of test jars with his fix and I reran the tests. I found ZERO UIComponent related blocking threads!!<br>
<br>I ran it through my performance benchmarks that I described in my original email with the following results:<br><br>For the 25 user tests I saw a 12-17% improvement in both average and mean response time. <br><br>Sample:<br>
25 user avg before : 3088ms<br>
25 user avg after : 2545ms<br>
<br>For the 50 user tests I saw a 4.5 - 5.3% improvement in both average and mean response time.<br><br>Sample:<br>50 user avg before : 10283ms<br>50 user avg after : 9815ms<br><br>While these are not scientifically exhaustive statistics it does show a valid increase in performance from the change.<br>
<br>Ryan created a bug to cover his change <a href="https://javaserverfaces.dev.java.net/issues/show_bug.cgi?id=806">https://javaserverfaces.dev.java.net/issues/show_bug.cgi?id=806</a> and plans on including in the next release of JSF RI 1.2_10<br>
<br>Regards,<br>Jay<br><br><div class="gmail_quote">On Thu, Sep 25, 2008 at 12:21 PM, Pete Muir <span dir="ltr"><<a href="mailto:pmuir@redhat.com">pmuir@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Thanks Ryan :-)<br>
<br>
On 25 Sep 2008, at 16:27, Ryan Lubke wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Thanks,<br>
<br>
Jim and I will look into it.<br>
<br>
Pete Muir wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">
Ryan, Ed,<br>
<br>
Jay has been working on performance analysis for Seam.<br>
<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">
UIComponent$AttributesMap<br>
-----------------------------------------------------<br>
- Issues related to this were responsible for about 20% of the blocked threads<br>
- Reported time spent blocked ranged from 30 ms - 848 ms with an average estimated to be 400ms<br>
- I need to rerun the tests to be more specific.<br>
- One of the items that was strange with this block was that it sometimes included completely unrelated actions or classes.<br>
- after adjusting the filters, and rerunning the tests several times I was able to narrow it down.<br>
<br>
* Investigation:<br></div>
Unfortunately this issue is in the RI JSF implementation itself. The UIComponentBase implements its own Map (AttributeMap) ( see <a href="http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521" target="_blank">http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521</a> <<a href="http://fisheye5cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521" target="_blank">http://fisheye5cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521</a>>). This is not an issue directly, but you will see calls to the "getAttributesThatAreSet" method in the get, put, and remove methods of the map. This is implemented in the UIComponent class ( see: <a href="http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponent.java?r=1.153#l1191" target="_blank">http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponent.java?r=1.153#l1191</a>). The real issue is the call to "this.getClass().getPackage();". This call will eventually synchronize in the "ClassLoader.getPackage" method on the "packages" object for the whole classloader. This is why other threads were also blocked on this - like org.jboss.seam.contexts.Contexts.isApplicationContextActive() which is accessing a ThreadLocal object.<div class="Ih2E3d">
<br>
<br>
We really have 3 options for this issue. - We can work with the RI to get the handling of the package check changed so that the values are some how cached in the component - how often do packages change while running ;-)<br>
- We can take a look and try to find excessive use of the components interactions with the AttributeMap.<br>
- We do nothing - but keep an eye on JSF 2 to make sure items like this are handled better.<br>
</div></blockquote><div class="Ih2E3d">
<br>
Naively, I would assume it should be possible to cache the value of this.getClass().getPackage().<br>
<br>
HTH,<br>
<br>
Pete<br>
<br>
Begin forwarded message:<br>
<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
*From: *"Jay Balunas" <<a href="mailto:tech4j@gmail.com" target="_blank">tech4j@gmail.com</a> <mailto:<a href="mailto:tech4j@gmail.com" target="_blank">tech4j@gmail.com</a>>><div class="Ih2E3d"><br>
*Date: *24 September 2008 22:05:51 BST<br></div>
*To: *"<a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a> <mailto:<a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a>>" <<a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a> <mailto:<a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a>>><div class="Ih2E3d">
<br>
*Subject: **[seam-dev] Some profiling of blocking threads*<br>
<br>
I have been executing the 25 user performance tests through the profiler to identify hot spots and areas of concern. I am initially looking at our blocking threads, and behavior around them that is causing the problems.<br>
<br>
Test Description<br>
--------------------<br>
- Use the 25 user wiki forum front page jmeter tests discussed in the previous email.<br>
- Uses same environment and procedures as defined in the previous email.<br>
- Tests are executed through the profiler with CPU recording turned on.<br>
- I have modified the default filters to allow the following packages<br>
- org.jboss.seam<br>
- javax.faces<br>
- most of the java.util collections and map classes<br>
<br>
Profile Results<br>
-------------------<br>
This is not a complete analysis of the entire application or even all of the blocking threads - I will continue to work on them and feed more information back to list.<br>
<br>
These items were detected via the thread usage history page, that shows all blocked and waiting threads and their stack traces.<br>
<br>
javax.naming.InitialContext.lookup<br>
-----------------------------------<br>
- as expected a large number of blocked threads were waiting or blocked while attempting to lookup the initial context.<br>
- One interesting thing is that not all of these were blocked on a Hashtable - others were blocked on, or were related to calls to Properties and WeakHashMap objects. This is likely caused by further method invocations that have been filtered out.<br>
- I will look further, but wanted to look at some of the other areas that were blocked.<br>
<br>
UIComponent$AttributesMap<br>
-----------------------------------------------------<br>
- Issues related to this were responsible for about 20% of the blocked threads<br>
- Reported time spent blocked ranged from 30 ms - 848 ms with an average estimated to be 400ms<br>
- I need to rerun the tests to be more specific.<br>
- One of the items that was strange with this block was that it sometimes included completely unrelated actions or classes.<br>
- after adjusting the filters, and rerunning the tests several times I was able to narrow it down.<br>
<br>
* Investigation:<br></div>
Unfortunately this issue is in the RI JSF implementation itself. The UIComponentBase implements its own Map (AttributeMap) ( see <a href="http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521" target="_blank">http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521</a> <<a href="http://fisheye5cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521" target="_blank">http://fisheye5cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponentBase.java?r=1.158#l1521</a>>). This is not an issue directly, but you will see calls to the "getAttributesThatAreSet" method in the get, put, and remove methods of the map. This is implemented in the UIComponent class ( see: <a href="http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponent.java?r=1.153#l1191" target="_blank">http://fisheye5.cenqua.com/browse/javaserverfaces-sources/jsf-api/src/javax/faces/component/UIComponent.java?r=1.153#l1191</a>). The real issue is the call to "this.getClass().getPackage();". This call will eventually synchronize in the "ClassLoader.getPackage" method on the "packages" object for the whole classloader. This is why other threads were also blocked on this - like org.jboss.seam.contexts.Contexts.isApplicationContextActive() which is accessing a ThreadLocal object.<div class="Ih2E3d">
<br>
<br>
We really have 3 options for this issue. - We can work with the RI to get the handling of the package check changed so that the values are some how cached in the component - how often do packages change while running ;-)<br>
- We can take a look and try to find excessive use of the components interactions with the AttributeMap.<br>
- We do nothing - but keep an eye on JSF 2 to make sure items like this are handled better.<br>
<br>
org.ajax4jsf.application.AjaxViewHandler<br>
--------------------------------------------<br>
- This was not a large % of the blocked threads but caught my eye as a potential trouble spot because it seemed directly related to ajax calls.<br>
- This was easier to track down and the root cause is in the org.ajax4jsf.application.ViewHandlerWrapper class.<br>
- see <a href="http://anonsvn.jboss.org/repos/richfaces/trunk/framework/api/src/main/java/org/ajax4jsf/application/ViewHandlerWrapper.java" target="_blank">http://anonsvn.jboss.org/repos/richfaces/trunk/framework/api/src/main/java/org/ajax4jsf/application/ViewHandlerWrapper.java</a><br>
- The problem is all the calls to "fillChain" which is a synchronized method in the class.<br>
- We need to find out how often the "_initialized" field is false - is this once a request, once for each component, etc...<br>
- it appears to be called many times - I will look further into this.<br>
<br>
-------------------------------------<br>
This is all I have had time for so far, but will continue to investigate some of the other aspects of the application. After completing the analysis of the primary blocking thread issues I will move on to other areas to profile. These will focus on specific issues raised by the team and the forums. I will also add performance tests and analysis for the dvd store example.<br>
<br>
Regards,<br>
Jay<br>
<br>
<br>
-- <br>
blog: <a href="http://in.relation.to/Bloggers/Jay" target="_blank">http://in.relation.to/Bloggers/Jay</a><br>
_______________________________________________<br>
seam-dev mailing list<br>
</div><a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a> <mailto:<a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a>><div class="Ih2E3d"><br>
<a href="https://lists.jboss.org/mailman/listinfo/seam-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/seam-dev</a><br>
</div></blockquote>
<br>
</blockquote>
<br>
</blockquote><div><div></div><div class="Wj3C7c">
<br>
_______________________________________________<br>
seam-dev mailing list<br>
<a href="mailto:seam-dev@lists.jboss.org" target="_blank">seam-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/seam-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/seam-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>blog: <a href="http://in.relation.to/Bloggers/Jay">http://in.relation.to/Bloggers/Jay</a><br>
</div>