<div dir="ltr"><div><div>Cody,<br><br>as we spoke on IRC few weeks back, lets do optimized version of plaintext & json tests using ServletExtension.<br><br></div>I can help you with that in coming days, just to clear few things I got piled up.<br>
<br>--<br></div>tomaz<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Feb 13, 2014 at 4:40 PM, Cody Lerum <span dir="ltr"><<a href="mailto:cody.lerum@gmail.com" target="_blank">cody.lerum@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Stuart,<br>
<br>
That does worry me as well. The benchmarks can be misread if not<br>
comparing apples to apples and properly applying filters.<br>
<br>
Raw DB vs ORM for example is about a 2x difference.<br>
(<a href="http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=4" target="_blank">http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=4</a><br>
vs <a href="http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=1" target="_blank">http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=1</a>)<br>
<br>
Someone could for example look at<br>
<a href="http://www.techempower.com/benchmarks/#section=data-r8&hw=ec2&test=db" target="_blank">http://www.techempower.com/benchmarks/#section=data-r8&hw=ec2&test=db</a><br>
and determine that undertow was the "fastest", but we all know it<br>
would be a vastly different experience writing and maintaining a full<br>
application on pure undertow vs something like say ee7.<br>
<br>
For example the single query test.<br>
<br>
<a href="https://github.com/codylerum/FrameworkBenchmarks/blob/master/undertow/src/main/java/hello/DbSqlHandler.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/master/undertow/src/main/java/hello/DbSqlHandler.java</a><br>
<br>
vs<br>
<br>
<a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java</a><br>
<br>
The the EE7 code is very minimal and concise because so much is<br>
provided by the framework, but that obviously is going to come at a<br>
cost in terms of performance.<br>
<br>
Anyway I would imagine there would eventually be a few permutations of<br>
the tests utilizing wildfly as the application server itself can be<br>
used in so many different ways.<br>
<br>
wildfly-ee7 (CDI/JAX-RS/JPA)<br>
<br>
wildfly-servlet<br>
<br>
Maybe someone could look at reusing or adapting the existing servlet<br>
test to run on wildfly instead of resin? -<br>
<a href="https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/servlet" target="_blank">https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/servlet</a><br>
<span class="HOEnZb"><font color="#888888"><br>
-C<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Thu, Feb 13, 2014 at 2:11 AM, Stuart Douglas<br>
<<a href="mailto:stuart.w.douglas@gmail.com">stuart.w.douglas@gmail.com</a>> wrote:<br>
> One thing that does worry me here is that people are going to compare JAXRS and CDI performance against resins plain servlet performance. From my testing I know that these do add overhead (and I am going to work on this when I get back from PTO).<br>
><br>
> At the very least we should also have a plain servlet version that is directly comparable.<br>
><br>
> Stuart<br>
><br>
> Sent from my iPhone<br>
><br>
> On 12/02/2014, at 22:44, Cody Lerum <<a href="mailto:cody.lerum@gmail.com">cody.lerum@gmail.com</a>> wrote:<br>
><br>
>> I've had this queued up for a bit but Stuart suggested I hold off<br>
>> until Wildfly went final so here it is.<br>
>><br>
>> Some of you are probably aware of a Web Framework Benchmark project<br>
>> that is being put on by TechEmpower<br>
>> (<a href="http://www.techempower.com/benchmarks/" target="_blank">http://www.techempower.com/benchmarks/</a>). They are currently up to the<br>
>> 8th round with 90 participating "frameworks". Undertow as been around<br>
>> for a few rounds and is routinely at or near the top of the charts.<br>
>><br>
>> My goal here is to incorporate EE7 into these results with Wildfly<br>
>> providing the implementation. There are a lot of different ways to go<br>
>> here, but my preference is to stay portable with the application so<br>
>> that if another EE7 implementation wanted to use the same code it<br>
>> would be an apples to apples comparison. IMO this means for example<br>
>> that it shouldn't dip down into undertow/hibernate/resteasy/etc<br>
>> specific features that wouldn't be available on other application<br>
>> servers. Stick to EE7 API.<br>
>><br>
>> There currently are 6 tests<br>
>><br>
>> 1. Plaintext: Simple "Hello World!" Text response. Not that<br>
>> interesting and not dynamic. Implemented via JAX-RS.<br>
>> <a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/PlainText.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/PlainText.java</a><br>
>><br>
>> 2. JSON Serialization: "Hello World" placed into a new object and then<br>
>> serialized out to JSON. Again not really a real world thing since<br>
>> nothing dynamic is happening here. Implemented via JAX-RS.<br>
>> <a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/JsonSerialization.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/JsonSerialization.java</a><br>
>><br>
>> 3. Single Query: Select a single random result from a database and<br>
>> serialize out to JSON. First real world test. Implemented via JAX-RS,<br>
>> CDI, JPA. <a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java</a><br>
>><br>
>> 4. Multiple Queries: Select a specified number of random results from<br>
>> database one by one. Add to list and serialize out to JSON.<br>
>> Implemented via JAX-RS, CDI, JPA.<br>
>> <a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/MultipleQueries.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/MultipleQueries.java</a><br>
>><br>
>> 5. Data Updates: Select a specifed number of random results from<br>
>> database one by one and update a value on each of them. Then return<br>
>> the result via JSON. Implemented via JAX-RS, CDI, JPA.<br>
>> <a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Updates.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Updates.java</a><br>
>><br>
>> 6. Fortunes: Select all from database and store in a list. Add a new<br>
>> result and sort. Display sorted results as an html table. Implemented<br>
>> via JSF, CDI, JPA.<br>
>> <a href="https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Fortunes.java" target="_blank">https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Fortunes.java</a><br>
>><br>
>> Since 1 and 2 are handled so well by undertow and really aren't<br>
>> anything someone would do real world with an application server I<br>
>> could see dropping them and just not participating in those tests.<br>
>> Unless anyone thinks differently.<br>
>><br>
>> On the other tests it would be helpful if you could review the code to<br>
>> see if there are better/faster ways to accomplish the same tasks<br>
>> without dipping into implementation specific features that would not<br>
>> be portable.<br>
>><br>
>> Thanks!<br>
>><br>
>> -C<br>
>> _______________________________________________<br>
>> wildfly-dev mailing list<br>
>> <a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>
>> <a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a><br>
_______________________________________________<br>
wildfly-dev mailing list<br>
<a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a><br>
</div></div></blockquote></div><br></div>