Clustering performance regression --------------------------------- Key: WFLY-5198 URL: https://issues.jboss.org/browse/WFLY-5198 Project: WildFly Issue Type: Bug Components: Clustering Affects Versions: 10.0.0.Beta1 Reporter: Michal Vinkler Assignee: Paul Ferraro Priority: Blocker Our clustering stress tests show huge performance drop of 7.0.0.DR8 bits compared to 6.4.0 bits. Compare these results: stress-heavyread-session-repl-sync 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-h...] 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-h...] Jenkins runs (server configs + logs can be found here): 7.0.0.DR8: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-heavyre... 6.4.0.GA: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-heavyre... One can observe that when using 7.0.0.DR8 bits, throughput reaches a threshold of approximately 6000 TPS. This happens in each test using *REPL-SYNC* cache, for example: stress-session-repl-sync 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-s...] 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-s...] All tests use a 4-node EAP cluster and 5 nodes that generate load. Adding another node to the EAP cluster, the threshold is lowered to 5200 TPS and the max. response time raises, see: stress-heavyread-session-repl-sync with 5-node cluster 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_e...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x...] *Most of the other scenarios indicate performance regression as well*. stress-session-dist-sync 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-s...] 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-s...] stress-session-repl-async 7.0.0.DR8: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-s...] 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...] + [response times|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-s...] This issue is related to https://issues.jboss.org/browse/JBEAP-768