<div>Hi Baq,</div><div><br></div><div>
Specify Executors.newCachedThreadPool() instead of new ThreadPoolExecutor(…).
</div><div><br></div><div>OrderedMemoryAwareThreadPoolExecutor is known to perform poor. If your sever does something that takes long time, you'd better use other thread pool implementation until OMATPE's perf issue is resolved (or it's replaced with something better)</div><div><br></div><div>HTH</div>
<div><br>-- <br><a href="http://gleamynode.net/">Trustin Lee</a><br></div>
<p style="color: #A0A0A8;">On Thursday, August 11, 2011 at 9:14 AM, Baq Haidri wrote:</p>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;">
<span><div><div>Hi,<div><br></div><div>I'm prototyping an HTTP service in Netty which accepts requests, and then proxies that request to several web applications and pulls their responses together into a single HTTP response containing an HTML payload.</div>
<div><br></div><div>The server channel factory uses the following thread pools:</div><div><br></div><div><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
<p><span>int</span> numCores = <span>4</span>*Runtime.getRuntime().availableProcessors();</p></div><div><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
<p>Executor bossExecutor = <span>new</span> ThreadPoolExecutor(<span>0</span>, Integer.MAX_VALUE,</p>
<p> <span>60L</span>, TimeUnit.MINUTES,</p>
<p> <span>new</span> SynchronousQueue<Runnable>());</p><p><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
</p><p>Executor workerExecutor = Executors.newFixedThreadPool(numCores);</p><p></p></div><div><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
<p>bootstrap.setFactory(<span>new</span> NioServerSocketChannelFactory(bossExecutor, workerExecutor, numCores));</p><p><br></p><p><meta charset="utf-8"><span style="font-family: arial; font-size: small; ">The server pipeline contains an ExecutionHandler which has an OrderedMemoryAwareThreadPoolExecutor (the thread factory only changes the thread name):</span></p>
<p><span style="font-family: arial; font-size: small; "><br></span></p><p><span style="font-family: arial; font-size: small; ">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
</span></p><p><span>final</span> ExecutionHandler embedRequestor = <span>new</span> ExecutionHandler(<span>new</span> OrderedMemoryAwareThreadPoolExecutor(numCores, </p>
<p> memory, // this is 500 Kb</p>
<p> memory,</p>
<p> <span>60L</span>, TimeUnit.MINUTES,</p>
<p> <span>new </span>FSThreadFactory(<span>"embed-requestor"</span>)));</p><p>
</p></div><div><br></div><div>The proxy HTTP clients all share a single client channel factory, which uses a cached thread pool whose threads have a long timeout (so that they're re-used):</div><div><br></div><div><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
<p> <span>private</span> <span>final</span> <span>static</span> NioClientSocketChannelFactory clientChannelFactory =</p>
<p> <span>new</span> NioClientSocketChannelFactory(<span>new</span> ThreadPoolExecutor(<span>0</span>, Integer.MAX_VALUE,</p>
<p> <span>60L</span>, TimeUnit.MINUTES,</p>
<p> <span>new</span> SynchronousQueue<Runnable>()), </p>
<p> <span>new</span> ThreadPoolExecutor(<span>0</span>, Integer.MAX_VALUE,</p>
<p> <span>60L</span>, TimeUnit.MINUTES,</p>
<p> <span>new</span> SynchronousQueue<Runnable>()));</p><p><br></p><p><meta charset="utf-8"><font face="arial" size="2">When I run this through apache bench, with concurrency levels 1 to 100 in increments of 5, here's the performance (at 50th, 90th, 99th percentiles as well as request per second). This is with the following GC settings:</font></p>
<p><font face="arial" size="2">java -server -Xms1024m -Xmx1024m -XX:MaxNewSize=768m -XX:NewSize=768m -XX:SurvivorRatio=6 -XX:+UseConcMarkSweepGC (note that Old Gen never gets full enough to actually force CMS)</font></p>
<p><font face="arial" size="2"><br></font></p><p><font face="arial">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="ProgId" content="Excel.Sheet">
<meta name="Generator" content="Microsoft Excel 14">
<link rel="Main-File" href="file://localhost/Users/bhaidri/Library/Caches/TemporaryItems/msoclip/0/clip.htm">
<link rel="File-List" href="file://localhost/Users/bhaidri/Library/Caches/TemporaryItems/msoclip/0/clip_filelist.xml">
<table border="0" cellpadding="0" cellspacing="0" width="299" style="font-size: medium; border-collapse: collapse; width: 299pt; ">
<colgroup><col width="69" style="mso-width-source:userset;mso-width-alt:2944;width:69pt">
<col width="49" style="mso-width-source:userset;mso-width-alt:2090;width:49pt">
<col width="55" style="mso-width-source:userset;mso-width-alt:2346;width:55pt">
<col width="61" style="mso-width-source:userset;mso-width-alt:2602;width:61pt">
<col width="65" style="width:65pt">
</colgroup><tbody><tr height="15" style="height:15.0pt">
<td height="15" width="69" style="height:15.0pt;width:69pt">Concurrency</td>
<td align="right" width="49" style="width:49pt">50%</td>
<td align="right" width="55" style="width:55pt">90%</td>
<td align="right" width="61" style="width:61pt">99%</td>
<td width="65" style="width:65pt">RPS</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt"><a name="RANGE!A27:D47">1</a></td>
<td align="right">70.928</td>
<td align="right">84.199</td>
<td align="right">93.12</td>
<td align="right">13.7</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">5</td>
<td align="right">74.101</td>
<td align="right">99.795</td>
<td align="right">141.391</td>
<td align="right">62.91</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">10</td>
<td align="right">88.744</td>
<td align="right">146.416</td>
<td align="right">232.333</td>
<td align="right">99.22</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">15</td>
<td align="right">108.265</td>
<td align="right">188.855</td>
<td align="right">305.268</td>
<td align="right">121.44</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">20</td>
<td align="right">130.508</td>
<td align="right">231.05</td>
<td align="right">438.312</td>
<td align="right">134.31</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">25</td>
<td align="right">158.417</td>
<td align="right">297.67</td>
<td align="right">666.237</td>
<td align="right">132.42</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">30</td>
<td align="right">170.459</td>
<td align="right">349.15</td>
<td align="right">2541.908</td>
<td align="right">110.19</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">35</td>
<td align="right">197.119</td>
<td align="right">411.041</td>
<td align="right">4129.77</td>
<td align="right">110.64</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">40</td>
<td align="right">237.777</td>
<td align="right">493.632</td>
<td align="right">3091.222</td>
<td align="right">110.69</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">45</td>
<td align="right">268.308</td>
<td align="right">475.407</td>
<td align="right">8118.155</td>
<td align="right">98.13</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">50</td>
<td align="right">306.447</td>
<td align="right">553.581</td>
<td align="right">6455.245</td>
<td align="right">118.01</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">55</td>
<td align="right">340.214</td>
<td align="right">662.951</td>
<td align="right">3274.517</td>
<td align="right">110.55</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">60</td>
<td align="right">363.786</td>
<td align="right">779.651</td>
<td align="right">5793.395</td>
<td align="right">96.38</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">65</td>
<td align="right">404.513</td>
<td align="right">659.296</td>
<td align="right">3803.272</td>
<td align="right">119.49</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">70</td>
<td align="right">439.377</td>
<td align="right">800.518</td>
<td align="right">7408.627</td>
<td align="right">105.6</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">75</td>
<td align="right">462.924</td>
<td align="right">757.701</td>
<td align="right">2264.736</td>
<td align="right">137.37</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">80</td>
<td align="right">507.593</td>
<td align="right">1117.655</td>
<td align="right">11667.996</td>
<td align="right">82.76</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">85</td>
<td align="right">521.067</td>
<td align="right">968.124</td>
<td align="right">6895.998</td>
<td align="right">111.34</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">90</td>
<td align="right">545.382</td>
<td align="right">1713.862</td>
<td align="right">9650.341</td>
<td align="right">95.52</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">95</td>
<td align="right">613.355</td>
<td align="right">1016.572</td>
<td align="right">2502.987</td>
<td align="right">119.9</td>
</tr>
<tr height="15" style="height:15.0pt">
<td height="15" align="right" style="height:15.0pt">100</td>
<td align="right">636.438</td>
<td align="right">979.166</td>
<td align="right">5237.865</td>
<td align="right">129.07</td>
</tr>
</tbody></table><br><font size="2">My question is: why is Netty's performance degrading so badly at higher concurrencies? Why is the latency so unstable? Why does the CPU get pegged at higher concurrencies? I noticed when I profiled this code through JVisualVM that 1/3 of the CPU time is spent in </font></font>LinkedTransferQueue.awaitMatch.</p>
<p><font><br></font></p><p><font>Any suggestions people might have would be greatly appreciated. We've implemented the same system in Node and as C++ module using ATS and are seeing far better performance numbers for both.<br>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Content-Style-Type" content="text/css">
<title></title>
<meta name="Generator" content="Cocoa HTML Writer">
<meta name="CocoaVersion" content="1038.35">
<br></font></p><p><font>Thanks,</font></p><p><font>Baq</font></p><p></p><p></p></div>
</div><div>_______________________________________________<br>netty-users mailing list<br><a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br><a href="https://lists.jboss.org/mailman/listinfo/netty-users">https://lists.jboss.org/mailman/listinfo/netty-users</a><br></div></div></span>
</blockquote>
<div>
<br>
</div>