[jboss-jira] [JBoss JIRA] (WFLY-5822) Clustering performance regression in ejbremote-dist-sync scenario

[Richard Achmatowicz (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148279#comment-13148279 ] 

Richard Achmatowicz commented on WFLY-5822:
-------------------------------------------

I did a quick check on the speed at which invocations are being processed in servers, once they arrive. When an invocation is unpacked, a call is made to MethodInvocationMessageHandler to perform the invocation, which is performed in a separate thread in a runnable. I instrumented start ms and stop ms and printed out (stop-start) ms line, both for EAP 6.4.0 and EAP 7.0.0. Here are the results:

{noformat}
// EAP 6.4.0 run (https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/rachmatowicz___byteman___eap-6x-stress-ejbremote-dist-sync-custom-testfix/1/)
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf18.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7401445 1.7345
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf18.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7401445 1.7345
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf19.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7330766 1.77929
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf20.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7165113 1.80155

// EAP 7.0.0 run, number 6
[nrla at lenovo 6]$ grep MethodInvocationMessageHandler perf18.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
8589348 2.20701
[nrla at lenovo 6]$ grep MethodInvocationMessageHandler perf19.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
6827447 2.96184
[nrla at lenovo 6]$ grep MethodInvocationMessageHandler perf20.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
8718950 2.17151
{noformat}// EAP 6.4.0 run, number 1
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf18.log | awk '{sum+=$16;recs+=1} END {print recs sum/recs}' 
74014451.7345
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf18.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7401445 1.7345
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf19.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7330766 1.77929
[nrla at lenovo 1]$ grep MethodInvocationMessageHandler perf20.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
7165113 1.80155

// EAP 7.0.0 run ()
[nrla at lenovo 6]$ grep MethodInvocationMessageHandler perf18.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
8589348 2.20701
[nrla at lenovo 6]$ grep MethodInvocationMessageHandler perf19.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
6827447 2.96184
[nrla at lenovo 6]$ grep MethodInvocationMessageHandler perf20.log | awk '{sum+=$16;recs+=1} END {print recs " " sum/recs}' 
8718950 2.17151

 
There doesn't seem to be a great deal of difference in the time it takes to process an invocation once it has arrived: rougfhly 1.7 ms for EAP 6 and 2.4 for EAP 7. Given that EAP 7 has a few mote cache operations per invocation, this makes sense. However, looking at the corresponding throughput data, EAP 6 seems to be handling many more requests than EAP 7. 

 

> Clustering performance regression in ejbremote-dist-sync scenario 
> ------------------------------------------------------------------
>
>                 Key: WFLY-5822
>                 URL: https://issues.jboss.org/browse/WFLY-5822
>             Project: WildFly
>          Issue Type: Bug
>          Components: Clustering, EJB
>    Affects Versions: 10.0.0.CR5
>            Reporter: Michal Vinkler
>            Assignee: Richard Achmatowicz
>            Priority: Critical
>
> Compared to EAP 6, all SYNC scenarios have the same/better performance except of this one, wonder why?
> Compare these results:
> stress-ejbremote-dist-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-ejbremote-dist-sync/4/artifact/report/graph-throughput.png]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-ejbremote-dist-sync_noperf21/1/artifact/report/graph-throughput.png]
> ---------------------------------------
> Just for comparison: ejbremote REPL_SYNC scenario *performs well* on the other hand:
> stress-ejbremote-repl-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-stress-ejbremote-repl-sync/3/artifact/report/graph-throughput.png]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-stress-ejbremote-repl-sync_noperf21/2/artifact/report/graph-throughput.png]



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)