[wildfly-dev] Adding Wildfly to the TechEmpower Web Framework Benchmarks

[Cody Lerum]

Tomaz,

I think that would be a good fit for a separate wildfly-servlet
implementation that Stuart suggested. I'm liking the idea of keeping
the EE7 code container agnostic, and just dropping out the plaintext
and json test as they are really just raw throughput tests.

We should be able to copy the existing folder and create something pretty easy.

I've gone ahead and renamed the folder and tests to wildfly-ee7 (from
ee7-wildfly) so that it makes more sense with a wildfly-servlet.

Now at https://github.com/codylerum/FrameworkBenchmarks/tree/ee7-wildfly/wildfly-ee7

On Thu, Feb 13, 2014 at 1:57 PM, Tomaž Cerar <tomaz.cerar at gmail.com> wrote:
> Cody,
>
> as we spoke on IRC few weeks back, lets do optimized version of plaintext &
> json tests using ServletExtension.
>
> I can help you with that in coming days, just to clear few things I got
> piled up.
>
> --
> tomaz
>
>
> On Thu, Feb 13, 2014 at 4:40 PM, Cody Lerum <cody.lerum at gmail.com> wrote:
>>
>> Stuart,
>>
>> That does worry me as well. The benchmarks can be misread if not
>> comparing apples to apples and properly applying filters.
>>
>> Raw DB vs ORM for example is about a 2x difference.
>> (http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=4
>> vs
>> http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=1)
>>
>> Someone could for example look at
>> http://www.techempower.com/benchmarks/#section=data-r8&hw=ec2&test=db
>> and determine that undertow was the "fastest", but we all know it
>> would be a vastly different experience writing and maintaining a full
>> application on pure undertow vs something like say ee7.
>>
>> For example the single query test.
>>
>>
>> https://github.com/codylerum/FrameworkBenchmarks/blob/master/undertow/src/main/java/hello/DbSqlHandler.java
>>
>> vs
>>
>>
>> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java
>>
>> The the EE7 code is very minimal and concise because so much is
>> provided by the framework, but that obviously is going to come at a
>> cost in terms of performance.
>>
>> Anyway I would imagine there would eventually be a few permutations of
>> the tests utilizing wildfly as the application server itself can be
>> used in so many different ways.
>>
>> wildfly-ee7 (CDI/JAX-RS/JPA)
>>
>> wildfly-servlet
>>
>> Maybe someone could look at reusing or adapting the existing servlet
>> test to run on wildfly instead of resin? -
>> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/servlet
>>
>> -C
>>
>> On Thu, Feb 13, 2014 at 2:11 AM, Stuart Douglas
>> <stuart.w.douglas at gmail.com> wrote:
>> > One thing that does worry me here is that people are going to compare
>> > JAXRS and CDI performance against resins plain servlet performance. From my
>> > testing I know that these do add overhead (and I am going to work on this
>> > when I get back from PTO).
>> >
>> > At the very least we should also have a plain servlet version that is
>> > directly comparable.
>> >
>> > Stuart
>> >
>> > Sent from my iPhone
>> >
>> > On 12/02/2014, at 22:44, Cody Lerum <cody.lerum at gmail.com> wrote:
>> >
>> >> I've had this queued up for a bit but Stuart suggested I hold off
>> >> until Wildfly went final so here it is.
>> >>
>> >> Some of you are probably aware of a Web Framework Benchmark project
>> >> that is being put on by TechEmpower
>> >> (http://www.techempower.com/benchmarks/). They are currently up to the
>> >> 8th round with 90 participating "frameworks". Undertow as been around
>> >> for a few rounds and is routinely at or near the top of the charts.
>> >>
>> >> My goal here is to incorporate EE7 into these results with Wildfly
>> >> providing the implementation. There are a lot of different ways to go
>> >> here, but my preference is to stay portable with the application so
>> >> that if another EE7 implementation wanted to use the same code it
>> >> would be an apples to apples comparison. IMO this means for example
>> >> that it shouldn't dip down into undertow/hibernate/resteasy/etc
>> >> specific features that wouldn't be available on other application
>> >> servers. Stick to EE7 API.
>> >>
>> >> There currently are 6 tests
>> >>
>> >> 1. Plaintext: Simple "Hello World!" Text response. Not that
>> >> interesting and not dynamic. Implemented via JAX-RS.
>> >>
>> >> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/PlainText.java
>> >>
>> >> 2. JSON Serialization: "Hello World" placed into a new object and then
>> >> serialized out to JSON. Again not really a real world thing since
>> >> nothing dynamic is happening here. Implemented via JAX-RS.
>> >>
>> >> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/JsonSerialization.java
>> >>
>> >> 3. Single Query: Select a single random result from a database and
>> >> serialize out to JSON. First real world test. Implemented via JAX-RS,
>> >> CDI, JPA.
>> >> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java
>> >>
>> >> 4. Multiple Queries: Select a specified number of random results from
>> >> database one by one. Add to list and serialize out to JSON.
>> >> Implemented via JAX-RS, CDI, JPA.
>> >>
>> >> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/MultipleQueries.java
>> >>
>> >> 5. Data Updates:  Select a specifed number of random results from
>> >> database one by one and update a value on each of them. Then return
>> >> the result via JSON. Implemented via JAX-RS, CDI, JPA.
>> >>
>> >> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Updates.java
>> >>
>> >> 6. Fortunes: Select all from database and store in a list. Add a new
>> >> result and sort. Display sorted results as an html table. Implemented
>> >> via JSF, CDI, JPA.
>> >>
>> >> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Fortunes.java
>> >>
>> >> Since 1 and 2 are handled so well by undertow and really aren't
>> >> anything someone would do real world with an application server I
>> >> could see dropping them and just not participating in those tests.
>> >> Unless anyone thinks differently.
>> >>
>> >> On the other tests it would be helpful if you could review the code to
>> >> see if there are better/faster ways to accomplish the same tasks
>> >> without dipping into implementation specific features that would not
>> >> be portable.
>> >>
>> >> Thanks!
>> >>
>> >> -C
>> >> _______________________________________________
>> >> wildfly-dev mailing list
>> >> wildfly-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>