[JBoss JIRA] (WFWIP-248) Avoid Route unavailability caused by load balancer recreation on CRD change
12:04 a.m.
[
https://issues.jboss.org/browse/WFWIP-248?page=com.atlassian.jira.plugin....
]
Petr Kremensky commented on WFWIP-248:
--------------------------------------
[~jmesnil] Scale up/down scenarios no longer shows the route unavailability. I'll few
more tests, but this could be resolved for now. Thanks.
Avoid Route unavailability caused by load balancer recreation on CRD
change
---------------------------------------------------------------------------
Key: WFWIP-248
URL: https://issues.jboss.org/browse/WFWIP-248
Project: WildFly WIP
Issue Type: Bug
Components: OpenShift
Reporter: Petr Kremensky
Assignee: Jeff Mesnil
Priority: Blocker
Labels: operator
Tracked by https://github.com/wildfly/wildfly-operator/issues/117
Reproduce:
{code:sh}
### REPRODUCER TERMINAL
# deploy quickstart project
oc new-project wildfly-test
wildfly-operator]$ ./build/run-openshift.sh
oc apply -f deploy/crds/quickstart-cr.yaml
wildfly-operator]$ oc get pods
NAME READY STATUS RESTARTS AGE
quickstart-0 1/1 Running 0 34s
quickstart-1 1/1 Running 0 34s
wildfly-operator-7f555b86d5-jkctc 1/1 Running 0 99s
# verify the app is responsive
ROUTE=quickstart-route-wildfly-test.apps-crc.testing
curl $ROUTE
{"ip":"10.128.1.43"}
# watch pods
oc get pods -w &
# verify route readiness
ROUTE=quickstart-route-wildfly-test.apps-crc.testing
# I've seen some weird bash optimization using `while true` in past which I'd
like to avoid here...
for i in {0..100000} ; do
echo '----------------------------------------'
date
curl -sI $ROUTE | grep HTTP
curl -s $ROUTE | head -n 1
echo ''
oc describe service/quickstart-loadbalancer | grep Endpoints
sleep 0.5
done
### ADMIN TERMINAL - scale the application
oc patch wildflyserver quickstart -p '[{"op":"replace",
"path":"/spec/replicas", "value":3}]' --type json
{code}
output:
{noformat}
----------------------------------------
Mon 14 Oct 2019 01:12:09 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 ContainerCreating 0 0s
quickstart-2 0/1 ContainerCreating 0 0s
Error from server (NotFound): services "quickstart-loadbalancer" not found
----------------------------------------
Mon 14 Oct 2019 01:12:09 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:10 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:11 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:12 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:13 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:13 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:14 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:15 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:16 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
quickstart-2quickstart-2 0/10/1
ContainerCreatingContainerCreating 00 8s8s
----------------------------------------
Mon 14 Oct 2019 01:12:17 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
quickstart-2 1/1 Running 0 8s
quickstart-2 1/1 Running 0 8s
----------------------------------------
Mon 14 Oct 2019 01:12:17 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:18 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:19 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:20 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:24 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:24 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:25 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:26 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:27 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:28 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:28 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:29 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:30 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:31 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:32 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:32 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:33 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:34 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:35 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.43"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:36 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.47"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
{noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
2475
days inactive
2475
days old
0 comments
1 participants
participants (1)
-
Petr Kremensky (Jira)