[JBoss JIRA] (WFLY-5822) Clustering performance regression in ejbremote-dist-sync scenario
by Richard Achmatowicz (JIRA)
[ https://issues.jboss.org/browse/WFLY-5822?page=com.atlassian.jira.plugin.... ]
Richard Achmatowicz edited comment on WFLY-5822 at 2/2/16 12:44 PM:
--------------------------------------------------------------------
I've done an informal measurement of individual invocation response times from the client side by:
(i) instrumenting each EJBClient invocation to measure the time taken to process invoke() and writing that value to sysout on the client node
(ii) collecting up all such measurements on each client and averaging them
Here are the results:
[TIMING] method invocation took 359 ms
[TIMING] method invocation took 333 ms
[TIMING] method invocation took 365 ms
[TIMING] method invocation took 378 ms
[TIMING] method invocation took 388 ms
[TIMING] method invocation took 360 ms
[TIMING] method invocation took 403 ms
[TIMING] method invocation took 316 ms
[TIMING] method invocation took 342 ms
[TIMING] method invocation took 360 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
.....
-bash-4.2$ cat eap-7x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
150.916 15786362
-bash-4.2$ cat eap-7x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
148.895 15676367
----------------------------------
[TIMING] method invocation took 198 ms
[TIMING] method invocation took 197 ms
[TIMING] method invocation took 195 ms
[TIMING] method invocation took 211 ms
[TIMING] method invocation took 209 ms
[TIMING] method invocation took 201 ms
[TIMING] method invocation took 213 ms
[TIMING] method invocation took 214 ms
[TIMING] method invocation took 215 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
-bash-4.2$ cat eap-6x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.394 8957198
-bash-4.2$ cat eap-6x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.85 8814924
This is both with JDK 8 using a 2 node server configuration.
>From these results, which take measurements from inside the EJB client code, lit looks as though EAP 7 is on average faster. This seems to be corroborated by the response time charts:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
It would be helpful to see the SmartFrog per invocation response times to compare; these would be measures from the client itself.
Also, why is the same test processing double the number of invocations?
Both tests take 44 minutes to run...
Throughput is also better (which may explain the above):
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
was (Author: rachmato):
I've done an informal measurement of individual invocation response times from the client side by:
(i) instrumenting each EJBClient invocation to measure the time taken to process invoke() and writing that value to sysout on the client node
(ii) collecting up all such measurements on each client and averaging them
Here are the results:
[TIMING] method invocation took 359 ms
[TIMING] method invocation took 333 ms
[TIMING] method invocation took 365 ms
[TIMING] method invocation took 378 ms
[TIMING] method invocation took 388 ms
[TIMING] method invocation took 360 ms
[TIMING] method invocation took 403 ms
[TIMING] method invocation took 316 ms
[TIMING] method invocation took 342 ms
[TIMING] method invocation took 360 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
.....
-bash-4.2$ cat eap-7x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
150.916 15786362
-bash-4.2$ cat eap-7x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
148.895 15676367
----------------------------------
[TIMING] method invocation took 198 ms
[TIMING] method invocation took 197 ms
[TIMING] method invocation took 195 ms
[TIMING] method invocation took 211 ms
[TIMING] method invocation took 209 ms
[TIMING] method invocation took 201 ms
[TIMING] method invocation took 213 ms
[TIMING] method invocation took 214 ms
[TIMING] method invocation took 215 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
-bash-4.2$ cat eap-6x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.394 8957198
-bash-4.2$ cat eap-6x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.85 8814924
This is both with JDK 8 using a 2 node server configuration.
>From these results, which take measurements from inside the EJB client code, lit looks as though EAP 7 is on average faster. This seems to be corroborated by the response time charts:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
It would be helpful to see the SmartFrog per invocation response times to compare; these would be measures from the client itself.
Also, why is the same test processing double the number of invocations?
Both tests take 44 minutes to run...
Throughput is also better:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
> Clustering performance regression in ejbremote-dist-sync scenario
> ------------------------------------------------------------------
>
> Key: WFLY-5822
> URL: https://issues.jboss.org/browse/WFLY-5822
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB
> Affects Versions: 10.0.0.CR5
> Reporter: Michal Vinkler
> Assignee: Richard Achmatowicz
> Priority: Critical
>
> Compared to EAP 6, all SYNC scenarios have the same/better performance except of this one, wonder why?
> Compare these results:
> stress-ejbremote-dist-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
> ---------------------------------------
> Just for comparison: ejbremote REPL_SYNC scenario *performs well* on the other hand:
> stress-ejbremote-repl-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (WFLY-5822) Clustering performance regression in ejbremote-dist-sync scenario
by Richard Achmatowicz (JIRA)
[ https://issues.jboss.org/browse/WFLY-5822?page=com.atlassian.jira.plugin.... ]
Richard Achmatowicz edited comment on WFLY-5822 at 2/2/16 12:42 PM:
--------------------------------------------------------------------
I've done an informal measurement of individual invocation response times from the client side by:
(i) instrumenting each EJBClient invocation to measure the time taken to process invoke() and writing that value to sysout on the client node
(ii) collecting up all such measurements on each client and averaging them
Here are the results:
[TIMING] method invocation took 359 ms
[TIMING] method invocation took 333 ms
[TIMING] method invocation took 365 ms
[TIMING] method invocation took 378 ms
[TIMING] method invocation took 388 ms
[TIMING] method invocation took 360 ms
[TIMING] method invocation took 403 ms
[TIMING] method invocation took 316 ms
[TIMING] method invocation took 342 ms
[TIMING] method invocation took 360 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
.....
-bash-4.2$ cat eap-7x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
150.916 15786362
-bash-4.2$ cat eap-7x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
148.895 15676367
----------------------------------
[TIMING] method invocation took 198 ms
[TIMING] method invocation took 197 ms
[TIMING] method invocation took 195 ms
[TIMING] method invocation took 211 ms
[TIMING] method invocation took 209 ms
[TIMING] method invocation took 201 ms
[TIMING] method invocation took 213 ms
[TIMING] method invocation took 214 ms
[TIMING] method invocation took 215 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
-bash-4.2$ cat eap-6x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.394 8957198
-bash-4.2$ cat eap-6x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.85 8814924
This is both with JDK 8 using a 2 node server configuration.
>From these results, which take measurements from inside the EJB client code, lit looks as though EAP 7 is on average faster. This seems to be corroborated by the response time charts:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
It would be helpful to see the SmartFrog per invocation response times to compare; these would be measures from the client itself.
Also, why is the same test processing double the number of invocations?
Both tests take 44 minutes to run...
Throughput is also better:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
was (Author: rachmato):
I've done an informal measurement of individual invocation response times from the client side by:
(i) instrumenting each EJBClient invocation to measure the time taken to process invoke() and writing that value to sysout on the client node
(ii) collecting up all such measurements on each client and averaging them
Here are the results:
[TIMING] method invocation took 359 ms
[TIMING] method invocation took 333 ms
[TIMING] method invocation took 365 ms
[TIMING] method invocation took 378 ms
[TIMING] method invocation took 388 ms
[TIMING] method invocation took 360 ms
[TIMING] method invocation took 403 ms
[TIMING] method invocation took 316 ms
[TIMING] method invocation took 342 ms
[TIMING] method invocation took 360 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
.....
-bash-4.2$ cat eap-7x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
150.916 15786362
-bash-4.2$ cat eap-7x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
148.895 15676367
----------------------------------
[TIMING] method invocation took 198 ms
[TIMING] method invocation took 197 ms
[TIMING] method invocation took 195 ms
[TIMING] method invocation took 211 ms
[TIMING] method invocation took 209 ms
[TIMING] method invocation took 201 ms
[TIMING] method invocation took 213 ms
[TIMING] method invocation took 214 ms
[TIMING] method invocation took 215 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
-bash-4.2$ cat eap-6x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.394 8957198
-bash-4.2$ cat eap-6x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.85 8814924
This is both with JDK 8.
>From these results, which take measurements from inside the EJB client code, lit looks as though EAP 7 is on average faster. This seems to be corroborated by the response time charts:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
It would be helpful to see the SmartFrog per invocation response times to compare; these would be measures from the client itself.
Also, why is the same test processing double the number of invocations?
Both tests take 44 minutes to run...
> Clustering performance regression in ejbremote-dist-sync scenario
> ------------------------------------------------------------------
>
> Key: WFLY-5822
> URL: https://issues.jboss.org/browse/WFLY-5822
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB
> Affects Versions: 10.0.0.CR5
> Reporter: Michal Vinkler
> Assignee: Richard Achmatowicz
> Priority: Critical
>
> Compared to EAP 6, all SYNC scenarios have the same/better performance except of this one, wonder why?
> Compare these results:
> stress-ejbremote-dist-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
> ---------------------------------------
> Just for comparison: ejbremote REPL_SYNC scenario *performs well* on the other hand:
> stress-ejbremote-repl-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (WFCORE-1352) Embedded logged leaving filehandles open after stop
by Ken Wills (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1352?page=com.atlassian.jira.plugi... ]
Ken Wills updated WFCORE-1352:
------------------------------
Description: After stop-embedded-server or stop-embedded-host-controller commands, logs in standalone/logs/ or domain/logs/ are still held open. (was: After stop-embedded-server or stop-embedded-host-controller commands, )
> Embedded logged leaving filehandles open after stop
> ---------------------------------------------------
>
> Key: WFCORE-1352
> URL: https://issues.jboss.org/browse/WFCORE-1352
> Project: WildFly Core
> Issue Type: Bug
> Components: Logging
> Environment: Windows
> Reporter: Ken Wills
> Assignee: Ken Wills
> Priority: Minor
>
> After stop-embedded-server or stop-embedded-host-controller commands, logs in standalone/logs/ or domain/logs/ are still held open.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (WFCORE-1352) Embedded logged leaving filehandles open after stop
by Ken Wills (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1352?page=com.atlassian.jira.plugi... ]
Ken Wills updated WFCORE-1352:
------------------------------
Description: After stop-embedded-server or stop-embedded-host-controller commands, (was: JBEAP-1779 makes sure that installed EAP distribution is aligned with the zip as much as possible by removing all post-installation configuration task leftovers.
Following directories are not removed by the installer on Windows (server.log file inside of them), once some post-installation configuration task is included (e.g. logging level change).
{noformat}
EAP-7.0.0\domain\log
EAP-7.0.0\standalone\log
{noformat}
*reproduce*
\- Install EAP on Windows, change the level of console logger to Warning)
> Embedded logged leaving filehandles open after stop
> ---------------------------------------------------
>
> Key: WFCORE-1352
> URL: https://issues.jboss.org/browse/WFCORE-1352
> Project: WildFly Core
> Issue Type: Bug
> Components: Logging
> Environment: Windows
> Reporter: Ken Wills
> Assignee: Ken Wills
> Priority: Minor
>
> After stop-embedded-server or stop-embedded-host-controller commands,
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (WFCORE-1352) Embedded logged leaving filehandles open after stop
by Ken Wills (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1352?page=com.atlassian.jira.plugi... ]
Ken Wills moved JBEAP-3157 to WFCORE-1352:
------------------------------------------
Project: WildFly Core (was: JBoss Enterprise Application Platform)
Key: WFCORE-1352 (was: JBEAP-3157)
Workflow: GIT Pull Request workflow (was: CDW with loose statuses v1)
Component/s: Logging
(was: Packaging and Installation)
Target Release: (was: 7.0.0.GA)
Affects Version/s: (was: 7.0.0.ER4)
> Embedded logged leaving filehandles open after stop
> ---------------------------------------------------
>
> Key: WFCORE-1352
> URL: https://issues.jboss.org/browse/WFCORE-1352
> Project: WildFly Core
> Issue Type: Bug
> Components: Logging
> Environment: Windows
> Reporter: Ken Wills
> Assignee: Ken Wills
> Priority: Minor
>
> JBEAP-1779 makes sure that installed EAP distribution is aligned with the zip as much as possible by removing all post-installation configuration task leftovers.
> Following directories are not removed by the installer on Windows (server.log file inside of them), once some post-installation configuration task is included (e.g. logging level change).
> {noformat}
> EAP-7.0.0\domain\log
> EAP-7.0.0\standalone\log
> {noformat}
> *reproduce*
> \- Install EAP on Windows, change the level of console logger to Warning
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months
[JBoss JIRA] (WFLY-5822) Clustering performance regression in ejbremote-dist-sync scenario
by Richard Achmatowicz (JIRA)
[ https://issues.jboss.org/browse/WFLY-5822?page=com.atlassian.jira.plugin.... ]
Richard Achmatowicz edited comment on WFLY-5822 at 2/2/16 12:37 PM:
--------------------------------------------------------------------
I've done an informal measurement of individual invocation response times from the client side by:
(i) instrumenting each EJBClient invocation to measure the time taken to process invoke() and writing that value to sysout on the client node
(ii) collecting up all such measurements on each client and averaging them
Here are the results:
[TIMING] method invocation took 359 ms
[TIMING] method invocation took 333 ms
[TIMING] method invocation took 365 ms
[TIMING] method invocation took 378 ms
[TIMING] method invocation took 388 ms
[TIMING] method invocation took 360 ms
[TIMING] method invocation took 403 ms
[TIMING] method invocation took 316 ms
[TIMING] method invocation took 342 ms
[TIMING] method invocation took 360 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
.....
-bash-4.2$ cat eap-7x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
150.916 15786362
-bash-4.2$ cat eap-7x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
148.895 15676367
----------------------------------
[TIMING] method invocation took 198 ms
[TIMING] method invocation took 197 ms
[TIMING] method invocation took 195 ms
[TIMING] method invocation took 211 ms
[TIMING] method invocation took 209 ms
[TIMING] method invocation took 201 ms
[TIMING] method invocation took 213 ms
[TIMING] method invocation took 214 ms
[TIMING] method invocation took 215 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
-bash-4.2$ cat eap-6x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.394 8957198
-bash-4.2$ cat eap-6x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.85 8814924
This is both with JDK 8.
>From these results, which take measurements from inside the EJB client code, lit looks as though EAP 7 is on average faster. This seems to be corroborated by the response time charts:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-stre...
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-6x-stre...
It would be helpful to see the SmartFrog per invocation response times to compare; these would be measures from the client itself.
Also, why is the same test processing double the number of invocations?
Both tests take 44 minutes to run...
was (Author: rachmato):
I've done an informal measurement of individual invocation response times from the client side by:
(i) instrumenting each EJBClient invocation to measure the time taken to process invoke() and writing that value to sysout on the client node
(ii) collecting up all such measurements on each client and averaging them
Here are the results:
[TIMING] method invocation took 359 ms
[TIMING] method invocation took 333 ms
[TIMING] method invocation took 365 ms
[TIMING] method invocation took 378 ms
[TIMING] method invocation took 388 ms
[TIMING] method invocation took 360 ms
[TIMING] method invocation took 403 ms
[TIMING] method invocation took 316 ms
[TIMING] method invocation took 342 ms
[TIMING] method invocation took 360 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
.....
-bash-4.2$ cat eap-7x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
150.916 15786362
-bash-4.2$ cat eap-7x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
148.895 15676367
----------------------------------
[TIMING] method invocation took 198 ms
[TIMING] method invocation took 197 ms
[TIMING] method invocation took 195 ms
[TIMING] method invocation took 211 ms
[TIMING] method invocation took 209 ms
[TIMING] method invocation took 201 ms
[TIMING] method invocation took 213 ms
[TIMING] method invocation took 214 ms
[TIMING] method invocation took 215 ms
[NOTE: these measurements above are not indicative of what happens later on in the test]
-bash-4.2$ cat eap-6x-perf30-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.394 8957198
-bash-4.2$ cat eap-6x-perf31-timing.txt | awk '{print $5}' | awk '{count+=$1;recs+=1} END {print count/recs " " recs}'
346.85 8814924
This is both with JDK 8.
> Clustering performance regression in ejbremote-dist-sync scenario
> ------------------------------------------------------------------
>
> Key: WFLY-5822
> URL: https://issues.jboss.org/browse/WFLY-5822
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB
> Affects Versions: 10.0.0.CR5
> Reporter: Michal Vinkler
> Assignee: Richard Achmatowicz
> Priority: Critical
>
> Compared to EAP 6, all SYNC scenarios have the same/better performance except of this one, wonder why?
> Compare these results:
> stress-ejbremote-dist-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
> ---------------------------------------
> Just for comparison: ejbremote REPL_SYNC scenario *performs well* on the other hand:
> stress-ejbremote-repl-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 5 months