[
https://issues.jboss.org/browse/MODCLUSTER-390?page=com.atlassian.jira.pl...
]
Michal Babacek edited comment on MODCLUSTER-390 at 9/11/14 7:35 AM:
--------------------------------------------------------------------
@[~jfclere] Hmm, the forced failover doesn't happen at once, it takes several
requests...is that the correct behaviour? Take a look:
h3. Setup
* Apache HTTP Server: 2.2.26 from EAP 6.3.0 distro
* Two EAP 6.3 servers, one called _jboss-eap-6.3_ and the other being _jboss-eap-6.3-2_
* mod_cluster modules build from the latest 1.2.x upstream
This is my mod_cluster.conf:
{code}
MemManagerFile
"/dev/shm/mod_cluster-eapx/jboss-ews-2.1/httpd/cache/mod_cluster"
ProxyPass / balancer://balancerxxx stickysession=JSESSIONID|jsessionid nofailover=on
failonstatus=203,204
ProxyPassReverse / balancer://balancerxxx
ProxyPreserveHost on
ServerName 192.168.122.204:2181
<IfModule manager_module>
Listen 192.168.122.204:8847
LogLevel debug
<VirtualHost 192.168.122.204:8847>
ServerName 192.168.122.204:8847
<Directory />
Order deny,allow
Deny from all
Allow from all
</Directory>
KeepAliveTimeout 60
MaxKeepAliveRequests 0
ServerAdvertise on
AdvertiseFrequency 5
ManagerBalancerName balancerxxx
AdvertiseGroup 224.0.5.79:65009
EnableMCPMReceive
CreateBalancers 1
<Location /mcm>
SetHandler mod_cluster-manager
Order deny,allow
Deny from all
Allow from all
</Location>
</VirtualHost>
</IfModule>
{code}
EAP config...
{code}
<subsystem xmlns="urn:jboss:domain:modcluster:1.2">
<mod-cluster-config advertise-socket="modcluster"
sticky-session="true"
sticky-session-remove="false"
sticky-session-force="false"
connector="ajp">
<dynamic-load-provider>
<load-metric type="busyness"/>
</dynamic-load-provider>
</mod-cluster-config>
</subsystem>
{code}
h3. Test
This dead simple
[
servlet|https://github.com/Karm/clusterbench/blob/simplified-and-pure/clu...]
sets an arbitrary response return code according to the given parameter. The output is the
following:
Warming up, {{http_code=200}} is not one of the defined error codes, so everything should
be normal. The session is created on the first request, the subsequent ones are the same,
no failover, cool...
{code}
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:02:22 GMT
Set-Cookie: JSESSIONID=gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3; Path=/clusterbench
Content-Type: text/plain;charset=UTF-8
Content-Length: 122
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: true
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:02:24 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:02:27 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
{code}
Now, let's try {{http_code=203}}, one of the predefined error codes. What I would
expect is an immediate failover from _jboss-eap-6.3_ to _jboss-eap-6.3-2_, although this
is what happens:
{code}
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:02:39 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:03 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:16 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:19 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:21 GMT
Set-Cookie: JSESSIONID=gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2; Path=/clusterbench
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
{code}
There were four requests and four _HTTP 203_ responses before the failover finally
occurred. Is this correct? Continuing with {{http_code=200}} looks normal, it stays on the
new server:
{code}
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:03:34 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:03:38 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:03:40 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
{code}
This is the complete [^error_log.zip]. It's noteworthy that message: {{proxy:
BALANCER: (balancer://balancerxxx). Forcing recovery for worker
(ajp://192.168.122.204:8009), failonstatus 203}} seems to by duly present...
WDYT?
was (Author: mbabacek):
@[~jfclere] Hmm, the forced failover doesn't happen at once, it takes several
requests...is that the correct behaviour? Take a look:
h3. Setup
* Apache HTTP Server: 2.2.26 from EAP 6.3.0 distro
* Two EAP 6.3 servers, one called _jboss-eap-6.3_ and the other being _jboss-eap-6.3-2_
* mod_cluster modules build from the latest 1.2.x upstream
This is my mod_cluster.conf:
{code}
MemManagerFile
"/dev/shm/mod_cluster-eapx/jboss-ews-2.1/httpd/cache/mod_cluster"
ProxyPass / balancer://balancerxxx stickysession=JSESSIONID|jsessionid nofailover=on
failonstatus=203,204
ProxyPassReverse / balancer://balancerxxx
ProxyPreserveHost on
ServerName 192.168.122.204:2181
<IfModule manager_module>
Listen 192.168.122.204:8847
LogLevel debug
<VirtualHost 192.168.122.204:8847>
ServerName 192.168.122.204:8847
<Directory />
Order deny,allow
Deny from all
Allow from all
</Directory>
KeepAliveTimeout 60
MaxKeepAliveRequests 0
ServerAdvertise on
AdvertiseFrequency 5
ManagerBalancerName balancerxxx
AdvertiseGroup 224.0.5.79:65009
EnableMCPMReceive
CreateBalancers 1
<Location /mcm>
SetHandler mod_cluster-manager
Order deny,allow
Deny from all
Allow from all
</Location>
</VirtualHost>
</IfModule>
{code}
EAP config...
{code}
<subsystem xmlns="urn:jboss:domain:modcluster:1.2">
<mod-cluster-config advertise-socket="modcluster"
sticky-session="true"
sticky-session-remove="false"
sticky-session-force="false"
connector="ajp">
<dynamic-load-provider>
<load-metric type="busyness"/>
</dynamic-load-provider>
</mod-cluster-config>
</subsystem>
{code}
h3. Test
This dead simple
[
servlet|https://github.com/Karm/clusterbench/blob/simplified-and-pure/clu...]
sets an arbitrary response return code according to the given parameter. The output is the
following:
Warming up, {{http_code=200}} is not one of the defined error codes, so everything should
be normal. The session is created on the first request, the subsequent ones are the same,
no failover, cool...
{code}
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:02:22 GMT
Set-Cookie: JSESSIONID=gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3; Path=/clusterbench
Content-Type: text/plain;charset=UTF-8
Content-Length: 122
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: true
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:02:24 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:02:27 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
{code}
Now, let's try {{http_code=203}}, one of the predefined error codes. What I would
expect is an immediate failover from _jboss-eap-6.3_ to _jboss-eap-6.3-2_, although this
is what happens:
{code}
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:02:39 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:03 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:16 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:19 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 123
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=203 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 203 Non-Authoritative Information
Date: Thu, 11 Sep 2014 11:03:21 GMT
Set-Cookie: JSESSIONID=gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2; Path=/clusterbench
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 203
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
{code}
There were four requests and four _HTTP 203_ responses before the failover finally
occurred. Is this correct? Continuing with {{http_code=200}} looks normal, it stays on the
new server:
{code}
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:03:34 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:03:38 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
mbabacek@localhost:~$ curl
http://rhel6x86-64:8847/clusterbench/httpcode/?http_code=200 -c
cookie.txt -b cookie.txt -i
HTTP/1.1 200 OK
Date: Thu, 11 Sep 2014 11:03:40 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 127
Connection: close
Done.
HTTP Code was: 200
JVM route: jboss-eap-6.3-2
Session ID: gMPIq7FfCeiPwzM2iZLyJiR4.jboss-eap-6.3-2
Session isNew: false
{code}
This is the complete [^error_log]. It's noteworthy that message: {{proxy: BALANCER:
(balancer://balancerxxx). Forcing recovery for worker (ajp://192.168.122.204:8009),
failonstatus 203}} seems to by duly present...
WDYT?
add failonstatus to mod_cluster
-------------------------------
Key: MODCLUSTER-390
URL:
https://issues.jboss.org/browse/MODCLUSTER-390
Project: mod_cluster
Issue Type: Enhancement
Affects Versions: 1.2.6.Final, 1.3.0.Final
Reporter: Jean-Frederic Clere
Assignee: Jean-Frederic Clere
Fix For: 1.2.8.Final, 1.3.1.Alpha2
The feature was added in httpd-2.2.17 in the mod_proxy_balancer we just need to copy that
code and find a way to configure it,
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)