7:26 a.m.
Petr Kremensky created WFWIP-248:
------------------------------------
Summary: Avoid Route unavailability caused by load balancer recreation
Key: WFWIP-248
URL: https://issues.jboss.org/browse/WFWIP-248
Project: WildFly WIP
Issue Type: Bug
Components: OpenShift
Reporter: Petr Kremensky
Assignee: Jeff Mesnil
Tracked by https://github.com/wildfly/wildfly-operator/issues/117
Reproduce:
{code:sh}
### REPRODUCER TERMINAL
# deploy quickstart project
oc new-project wildfly-test
wildfly-operator]$ ./build/run-openshift.sh
oc apply -f deploy/crds/quickstart-cr.yaml
wildfly-operator]$ oc get pods
NAME READY STATUS RESTARTS AGE
quickstart-0 1/1 Running 0 34s
quickstart-1 1/1 Running 0 34s
wildfly-operator-7f555b86d5-jkctc 1/1 Running 0 99s
# verify the app is responsive
ROUTE=quickstart-route-wildfly-test.apps-crc.testing
curl $ROUTE
{"ip":"10.128.1.43"}
# watch pods
oc get pods -w &
# verify route readiness
ROUTE=quickstart-route-wildfly-test.apps-crc.testing
# I've seen some weird bash optimization using `while true` in past which I'd like
to avoid here...
for i in {0..100000} ; do
echo '----------------------------------------'
date
curl -sI $ROUTE | grep HTTP
curl -s $ROUTE | head -n 1
echo ''
oc describe service/quickstart-loadbalancer | grep Endpoints
sleep 0.5
done
### ADMIN TERMINAL - scale the application
oc patch wildflyserver quickstart -p '[{"op":"replace",
"path":"/spec/replicas", "value":3}]' --type json
{code}
output:
{noformat}
----------------------------------------
Mon 14 Oct 2019 01:12:09 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 Pending 0 0s
quickstart-2 0/1 ContainerCreating 0 0s
quickstart-2 0/1 ContainerCreating 0 0s
Error from server (NotFound): services "quickstart-loadbalancer" not found
----------------------------------------
Mon 14 Oct 2019 01:12:09 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:10 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:11 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:12 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:13 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:13 PM CEST
HTTP/1.0 503 Service Unavailable
<html>
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:14 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:15 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
----------------------------------------
Mon 14 Oct 2019 01:12:16 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
quickstart-2quickstart-2 0/10/1
ContainerCreatingContainerCreating 00 8s8s
----------------------------------------
Mon 14 Oct 2019 01:12:17 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080
quickstart-2 1/1 Running 0 8s
quickstart-2 1/1 Running 0 8s
----------------------------------------
Mon 14 Oct 2019 01:12:17 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:18 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:19 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:20 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:24 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:24 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:25 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:26 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:27 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:28 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:28 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:29 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:30 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:31 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:32 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:32 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:33 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:34 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.44"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:35 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.43"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
----------------------------------------
Mon 14 Oct 2019 01:12:36 PM CEST
HTTP/1.1 200 OK
{"ip":"10.128.1.47"}
Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
{noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)