[jboss-jira] [JBoss JIRA] (WFLY-5198) Clustering performance regression

[Paul Ferraro (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-5198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101685#comment-13101685 ] 

Paul Ferraro edited comment on WFLY-5198 at 8/25/15 3:10 PM:
-------------------------------------------------------------

# Do we have non-HA performance tests comparing 6.4.0.GA and 7.0.0.DR8 - so that we have a baseline from which to compare clustering tests?
# The read-heavy scenarios need to be updated to use immutable session attributes.  Otherwise, the session attributes are treated as mutable and result in replication following read.  This explains the drastic perf differences.  It's as if you were comparing the results from the read-heavy scenario with results from the write-heavy scenario.


was (Author: pferraro):
# Do we have non-HA performance tests comparing 6.4.0.GA and 7.0.0.DR8 - so that we have a baseline from which to compare clustering tests?
# The read-heavy scenarios need to be updated to use immutable session attributes.  Otherwise, the session attributes are treated as mutable and result in replication following read.

> Clustering performance regression
> ---------------------------------
>
>                 Key: WFLY-5198
>                 URL: https://issues.jboss.org/browse/WFLY-5198
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering
>    Affects Versions: 10.0.0.Beta1
>            Reporter: Michal Vinkler
>            Assignee: Paul Ferraro
>            Priority: Blocker
>
> Our clustering stress tests show huge performance drop of 7.0.0.DR8 bits compared to 6.4.0 bits.
> Compare these results:
> stress-heavyread-session-repl-sync
> 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-heavyread-session-repl-sync_noperf21/2/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-heavyread-session-repl-sync_noperf21/2/artifact/report/graph-reponse-times.png]
> 6.4.0.GA:  [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-heavyread-session-repl-sync_noperf21/1/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-heavyread-session-repl-sync_noperf21/1/artifact/report/graph-reponse-times.png]
> Jenkins runs (server configs + logs can be found here):
> 7.0.0.DR8: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-heavyread-session-repl-sync_noperf21/2/
> 6.4.0.GA: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-heavyread-session-repl-sync_noperf21/1/
> One can observe that when using 7.0.0.DR8 bits, throughput reaches a threshold of approximately 6000 TPS. This happens in each test using *REPL-SYNC* cache, for example:
> stress-session-repl-sync
> 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-repl-sync_noperf21/2/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-repl-sync_noperf21/2/artifact/report/graph-reponse-times.png]
> 6.4.0.GA:  [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-session-repl-sync_noperf21/1/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-session-repl-sync_noperf21/1/artifact/report/graph-reponse-times.png]
> All tests use a 4-node EAP cluster and 5 nodes that generate load. Adding another node to the EAP cluster, the threshold is lowered to 5200 TPS and the max. response time raises, see:
> stress-heavyread-session-repl-sync with 5-node cluster
> 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stress-heavyread-session-repl-sync_noperf21/1/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stress-heavyread-session-repl-sync_noperf21/1/artifact/report/graph-reponse-times.png]
> *Most of the other scenarios indicate performance regression as well*.
> stress-session-dist-sync
> 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-dist-sync_noperf21/1/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-dist-sync_noperf21/1/artifact/report/graph-reponse-times.png]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-session-dist-sync_noperf21/1/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-session-dist-sync_noperf21/1/artifact/report/graph-reponse-times.png]
> stress-session-repl-async
> 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-repl-async_noperf21/2/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-repl-async_noperf21/2/artifact/report/graph-reponse-times.png]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-session-repl-async_noperf21/1/artifact/report/graph-throughput.png] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-session-repl-async_noperf21/1/artifact/report/graph-reponse-times.png]
> This issue is related to https://issues.jboss.org/browse/JBEAP-768



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)