[wildfly-dev] Adding Wildfly to the TechEmpower Web Framework Benchmarks

[Tomaž Cerar]

Cody,

as we spoke on IRC few weeks back, lets do optimized version of plaintext &
json tests using ServletExtension.

I can help you with that in coming days, just to clear few things I got
piled up.

--
tomaz


On Thu, Feb 13, 2014 at 4:40 PM, Cody Lerum <cody.lerum at gmail.com> wrote:

> Stuart,
>
> That does worry me as well. The benchmarks can be misread if not
> comparing apples to apples and properly applying filters.
>
> Raw DB vs ORM for example is about a 2x difference.
> (http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=4
> vs
> http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=db&o=1)
>
> Someone could for example look at
> http://www.techempower.com/benchmarks/#section=data-r8&hw=ec2&test=db
> and determine that undertow was the "fastest", but we all know it
> would be a vastly different experience writing and maintaining a full
> application on pure undertow vs something like say ee7.
>
> For example the single query test.
>
>
> https://github.com/codylerum/FrameworkBenchmarks/blob/master/undertow/src/main/java/hello/DbSqlHandler.java
>
> vs
>
>
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java
>
> The the EE7 code is very minimal and concise because so much is
> provided by the framework, but that obviously is going to come at a
> cost in terms of performance.
>
> Anyway I would imagine there would eventually be a few permutations of
> the tests utilizing wildfly as the application server itself can be
> used in so many different ways.
>
> wildfly-ee7 (CDI/JAX-RS/JPA)
>
> wildfly-servlet
>
> Maybe someone could look at reusing or adapting the existing servlet
> test to run on wildfly instead of resin? -
> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/servlet
>
> -C
>
> On Thu, Feb 13, 2014 at 2:11 AM, Stuart Douglas
> <stuart.w.douglas at gmail.com> wrote:
> > One thing that does worry me here is that people are going to compare
> JAXRS and CDI performance against resins plain servlet performance. From my
> testing I know that these do add overhead (and I am going to work on this
> when I get back from PTO).
> >
> > At the very least we should also have a plain servlet version that is
> directly comparable.
> >
> > Stuart
> >
> > Sent from my iPhone
> >
> > On 12/02/2014, at 22:44, Cody Lerum <cody.lerum at gmail.com> wrote:
> >
> >> I've had this queued up for a bit but Stuart suggested I hold off
> >> until Wildfly went final so here it is.
> >>
> >> Some of you are probably aware of a Web Framework Benchmark project
> >> that is being put on by TechEmpower
> >> (http://www.techempower.com/benchmarks/). They are currently up to the
> >> 8th round with 90 participating "frameworks". Undertow as been around
> >> for a few rounds and is routinely at or near the top of the charts.
> >>
> >> My goal here is to incorporate EE7 into these results with Wildfly
> >> providing the implementation. There are a lot of different ways to go
> >> here, but my preference is to stay portable with the application so
> >> that if another EE7 implementation wanted to use the same code it
> >> would be an apples to apples comparison. IMO this means for example
> >> that it shouldn't dip down into undertow/hibernate/resteasy/etc
> >> specific features that wouldn't be available on other application
> >> servers. Stick to EE7 API.
> >>
> >> There currently are 6 tests
> >>
> >> 1. Plaintext: Simple "Hello World!" Text response. Not that
> >> interesting and not dynamic. Implemented via JAX-RS.
> >>
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/PlainText.java
> >>
> >> 2. JSON Serialization: "Hello World" placed into a new object and then
> >> serialized out to JSON. Again not really a real world thing since
> >> nothing dynamic is happening here. Implemented via JAX-RS.
> >>
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/JsonSerialization.java
> >>
> >> 3. Single Query: Select a single random result from a database and
> >> serialize out to JSON. First real world test. Implemented via JAX-RS,
> >> CDI, JPA.
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/SingleQuery.java
> >>
> >> 4. Multiple Queries: Select a specified number of random results from
> >> database one by one. Add to list and serialize out to JSON.
> >> Implemented via JAX-RS, CDI, JPA.
> >>
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/MultipleQueries.java
> >>
> >> 5. Data Updates:  Select a specifed number of random results from
> >> database one by one and update a value on each of them. Then return
> >> the result via JSON. Implemented via JAX-RS, CDI, JPA.
> >>
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Updates.java
> >>
> >> 6. Fortunes: Select all from database and store in a list. Add a new
> >> result and sort. Display sorted results as an html table. Implemented
> >> via JSF, CDI, JPA.
> >>
> https://github.com/codylerum/FrameworkBenchmarks/blob/ee7-wildfly/ee7-wildfly/src/main/java/com/techempower/ee7/tests/Fortunes.java
> >>
> >> Since 1 and 2 are handled so well by undertow and really aren't
> >> anything someone would do real world with an application server I
> >> could see dropping them and just not participating in those tests.
> >> Unless anyone thinks differently.
> >>
> >> On the other tests it would be helpful if you could review the code to
> >> see if there are better/faster ways to accomplish the same tasks
> >> without dipping into implementation specific features that would not
> >> be portable.
> >>
> >> Thanks!
> >>
> >> -C
> >> _______________________________________________
> >> wildfly-dev mailing list
> >> wildfly-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20140213/7eae6a44/attachment.html