[wildfly-dev] Adding Wildfly to the TechEmpower Web Framework Benchmarks

[Cody Lerum]

Stuart,

That does worry me as well. The benchmarks can be misread if not
comparing apples to apples and properly applying filters.

Raw DB vs ORM for example is about a 2x difference.
(http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=4
vs http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=1)

Someone could for example look at
http://www.techempower.com/benchmarks/#section=data-r8&hw=ec2&test=db
and determine that undertow was the "fastest", but we all know it
would be a vastly different experience writing and maintaining a full
application on pure undertow vs something like say ee7.

For example the single query test.

https://github.com/codylerum/FrameworkBenchmarks/blob/master/undertow/src/main/java/hello/DbSqlHandler.java

vs

https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java

The the EE7 code is very minimal and concise because so much is
provided by the framework, but that obviously is going to come at a
cost in terms of performance.

Anyway I would imagine there would eventually be a few permutations of
the tests utilizing wildfly as the application server itself can be
used in so many different ways.

wildfly-ee7 (CDI/JAX-RS/JPA)

wildfly-servlet

Maybe someone could look at reusing or adapting the existing servlet
test to run on wildfly instead of resin? -
https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/servlet

-C

On Thu, Feb 13, 2014 at 2:11 AM, Stuart Douglas
<stuart.w.douglas at gmail.com> wrote:
> One thing that does worry me here is that people are going to compare JAXRS and CDI performance against resins plain servlet performance. From my testing I know that these do add overhead (and I am going to work on this when I get back from PTO).
>
> At the very least we should also have a plain servlet version that is directly comparable.
>
> Stuart
>
> Sent from my iPhone
>
> On 12/02/2014, at 22:44, Cody Lerum <cody.lerum at gmail.com> wrote:
>
>> I've had this queued up for a bit but Stuart suggested I hold off
>> until Wildfly went final so here it is.
>>
>> Some of you are probably aware of a Web Framework Benchmark project
>> that is being put on by TechEmpower
>> (http://www.techempower.com/benchmarks/). They are currently up to the
>> 8th round with 90 participating "frameworks". Undertow as been around
>> for a few rounds and is routinely at or near the top of the charts.
>>
>> My goal here is to incorporate EE7 into these results with Wildfly
>> providing the implementation. There are a lot of different ways to go
>> here, but my preference is to stay portable with the application so
>> that if another EE7 implementation wanted to use the same code it
>> would be an apples to apples comparison. IMO this means for example
>> that it shouldn't dip down into undertow/hibernate/resteasy/etc
>> specific features that wouldn't be available on other application
>> servers. Stick to EE7 API.
>>
>> There currently are 6 tests
>>
>> 1. Plaintext: Simple "Hello World!" Text response. Not that
>> interesting and not dynamic. Implemented via JAX-RS.
>> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/PlainText.java
>>
>> 2. JSON Serialization: "Hello World" placed into a new object and then
>> serialized out to JSON. Again not really a real world thing since
>> nothing dynamic is happening here. Implemented via JAX-RS.
>> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/JsonSerialization.java
>>
>> 3. Single Query: Select a single random result from a database and
>> serialize out to JSON. First real world test. Implemented via JAX-RS,
>> CDI, JPA.  https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java
>>
>> 4. Multiple Queries: Select a specified number of random results from
>> database one by one. Add to list and serialize out to JSON.
>> Implemented via JAX-RS, CDI, JPA.
>> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/MultipleQueries.java
>>
>> 5. Data Updates:  Select a specifed number of random results from
>> database one by one and update a value on each of them. Then return
>> the result via JSON. Implemented via JAX-RS, CDI, JPA.
>> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Updates.java
>>
>> 6. Fortunes: Select all from database and store in a list. Add a new
>> result and sort. Display sorted results as an html table. Implemented
>> via JSF, CDI, JPA.
>> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Fortunes.java
>>
>> Since 1 and 2 are handled so well by undertow and really aren't
>> anything someone would do real world with an application server I
>> could see dropping them and just not participating in those tests.
>> Unless anyone thinks differently.
>>
>> On the other tests it would be helpful if you could review the code to
>> see if there are better/faster ways to accomplish the same tasks
>> without dipping into implementation specific features that would not
>> be portable.
>>
>> Thanks!
>>
>> -C
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev