[jboss-jira] [JBoss JIRA] (AS7-3158) Requests around cluster membership change with REPL+SYNC cache could take up to 17 seconds to respond
Radoslav Husar (JIRA)
jira-events at lists.jboss.org
Fri Feb 24 11:08:36 EST 2012
[ https://issues.jboss.org/browse/AS7-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670906#comment-12670906 ]
Radoslav Husar commented on AS7-3158:
-------------------------------------
Yes, still seeing this with little higher load, so the occurence is a little less than before. But needless to say the overall performance for resilience/failover use cases is miserable.
{noformat}
...
2012/02/24 06:36:46:332 EST [DEBUG][Runner - 148] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:332 EST [DEBUG][Runner - 60] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:420 EST [DEBUG][Runner - 88] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:420 EST [DEBUG][Runner - 97] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:420 EST [DEBUG][Runner - 40] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:420 EST [DEBUG][Runner - 33] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:420 EST [DEBUG][Runner - 18] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 4 ms.
2012/02/24 06:36:46:448 EST [DEBUG][Runner - 35] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 3 ms.
2012/02/24 06:36:46:462 EST [DEBUG][Runner - 3] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 3 ms.
2012/02/24 06:36:46:483 EST [DEBUG][Runner - 107] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 3 ms.
2012/02/24 06:36:46:503 EST [DEBUG][Runner - 44] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 3 ms.
2012/02/24 06:36:47:241 EST [DEBUG][Runner - 119] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:241 EST [DEBUG][Runner - 26] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:241 EST [DEBUG][Runner - 31] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:241 EST [DEBUG][Runner - 13] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:241 EST [DEBUG][Runner - 34] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:244 EST [DEBUG][Runner - 32] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:245 EST [DEBUG][Runner - 147] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17503 ms.
2012/02/24 06:36:47:256 EST [DEBUG][Runner - 145] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:256 EST [DEBUG][Runner - 81] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:258 EST [DEBUG][Runner - 19] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17502 ms.
2012/02/24 06:36:47:259 EST [DEBUG][Runner - 80] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17503 ms.
2012/02/24 06:36:47:259 EST [DEBUG][Runner - 46] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:264 EST [DEBUG][Runner - 82] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:264 EST [DEBUG][Runner - 92] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
2012/02/24 06:36:47:264 EST [DEBUG][Runner - 93] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17504 ms.
...
{noformat}
https://hudson.qa.jboss.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-http-session-shutdown-repl-sync/16/
> Requests around cluster membership change with REPL+SYNC cache could take up to 17 seconds to respond
> -----------------------------------------------------------------------------------------------------
>
> Key: AS7-3158
> URL: https://issues.jboss.org/browse/AS7-3158
> Project: Application Server 7
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 7.1.0.CR1, 7.1.0.CR1b
> Reporter: Radoslav Husar
> Assignee: Paul Ferraro
> Priority: Critical
> Labels: eap6_prd_req, failover_testing, performance
> Fix For: 7.1.1.Final
>
>
> 10:59:22:800 one of the 4 nodes was brought up
> 10:59:23:879 requested url
> 10:59:41:390 clustered replied 17.511 seconds later
> nothing in the logs
> {noformat}
> 2012/01/02 10:59:15:865 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 11 ms.
> 2012/01/02 10:59:15:866 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Response size is 2. Processing Response Body.
> 2012/01/02 10:59:19:866 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Requesting URL: http://perf17:8080/clusterbench/session
> 2012/01/02 10:59:19:878 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 12 ms.
> 2012/01/02 10:59:19:879 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Response size is 2. Processing Response Body.
> 2012/01/02 10:59:19:900 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Bringing back node 1 (perf19)
> 2012/01/02 10:59:22:800 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Node 1 (perf19) is up.
> 2012/01/02 10:59:22:800 EST [INFO ][TestController] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Cluster state: [UP, UP, UP, UP].
> 2012/01/02 10:59:23:879 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Requesting URL: http://perf17:8080/clusterbench/session
> 2012/01/02 10:59:24:212 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Total: Sessions: 1, active: 1, samples: 2, throughput 0.2 samples/s, 0.0 MB/s, mean response: 11 ms, sampling errors: 0, invalid samples: 0, valid samples: 2 (100%)
> 2012/01/02 10:59:24:213 EST [DEBUG][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Updated totals: Sessions: 0, active: 30, samples: 76, throughput 7.6 samples/s, 0.0 MB/s, mean response: 11 ms, sampling errors: 0, invalid samples: 0, valid samples: 76 (100%)
> 2012/01/02 10:59:24:213 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - perf18: Sessions: 1, active: 0, samples: 0, throughput 0.0 samples/s, 0.0 MB/s, mean response: 0 ms, sampling errors: 0, invalid samples: 0, valid samples: 0 (0%)
> 2012/01/02 10:59:24:213 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - perf19: Sessions: 1, active: 0, samples: 0, throughput 0.0 samples/s, 0.0 MB/s, mean response: 0 ms, sampling errors: 0, invalid samples: 0, valid samples: 0 (0%)
> 2012/01/02 10:59:24:213 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - perf20: Sessions: 1, active: 1, samples: 2, throughput 0.2 samples/s, 0.0 MB/s, mean response: 11 ms, sampling errors: 0, invalid samples: 0, valid samples: 2 (100%)
> 2012/01/02 10:59:34:213 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Total: Sessions: 1, active: 0, samples: 0, throughput 0.0 samples/s, 0.0 MB/s, mean response: 0 ms, sampling errors: 0, invalid samples: 0, valid samples: 0 (0%)
> 2012/01/02 10:59:34:214 EST [DEBUG][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Updated totals: Sessions: 1, active: 30, samples: 76, throughput 7.6 samples/s, 0.0 MB/s, mean response: 11 ms, sampling errors: 0, invalid samples: 0, valid samples: 76 (100%)
> 2012/01/02 10:59:34:214 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - perf18: Sessions: 1, active: 0, samples: 0, throughput 0.0 samples/s, 0.0 MB/s, mean response: 0 ms, sampling errors: 0, invalid samples: 0, valid samples: 0 (0%)
> 2012/01/02 10:59:34:214 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - perf19: Sessions: 1, active: 0, samples: 0, throughput 0.0 samples/s, 0.0 MB/s, mean response: 0 ms, sampling errors: 0, invalid samples: 0, valid samples: 0 (0%)
> 2012/01/02 10:59:34:214 EST [INFO ][StatsRunner] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - perf20: Sessions: 1, active: 0, samples: 0, throughput 0.0 samples/s, 0.0 MB/s, mean response: 0 ms, sampling errors: 0, invalid samples: 0, valid samples: 0 (0%)
> 2012/01/02 10:59:41:390 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Request took 17511 ms.
> 2012/01/02 10:59:41:390 EST [DEBUG][Runner - 0] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Response size is 2. Processing Response Body.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
More information about the jboss-jira
mailing list