[jboss-jira] [JBoss JIRA] (WFWIP-248) Avoid Route unavailability caused by load balancer recreation on CRD change
Petr Kremensky (Jira)
issues at jboss.org
Tue Oct 15 08:41:01 EDT 2019
[ https://issues.jboss.org/browse/WFWIP-248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Petr Kremensky closed WFWIP-248.
--------------------------------
Closing this one, the issues is now fixed.
> Avoid Route unavailability caused by load balancer recreation on CRD change
> ---------------------------------------------------------------------------
>
> Key: WFWIP-248
> URL: https://issues.jboss.org/browse/WFWIP-248
> Project: WildFly WIP
> Issue Type: Bug
> Components: OpenShift
> Reporter: Petr Kremensky
> Assignee: Jeff Mesnil
> Priority: Blocker
> Labels: operator
>
> Tracked by https://github.com/wildfly/wildfly-operator/issues/117
> Reproduce:
> {code:sh}
> ### REPRODUCER TERMINAL
> # deploy quickstart project
> oc new-project wildfly-test
> wildfly-operator]$ ./build/run-openshift.sh
> oc apply -f deploy/crds/quickstart-cr.yaml
> wildfly-operator]$ oc get pods
> NAME READY STATUS RESTARTS AGE
> quickstart-0 1/1 Running 0 34s
> quickstart-1 1/1 Running 0 34s
> wildfly-operator-7f555b86d5-jkctc 1/1 Running 0 99s
> # verify the app is responsive
> ROUTE=quickstart-route-wildfly-test.apps-crc.testing
> curl $ROUTE
> {"ip":"10.128.1.43"}
> # watch pods
> oc get pods -w &
> # verify route readiness
> ROUTE=quickstart-route-wildfly-test.apps-crc.testing
> # I've seen some weird bash optimization using `while true` in past which I'd like to avoid here...
> for i in {0..100000} ; do
> echo '----------------------------------------'
> date
> curl -sI $ROUTE | grep HTTP
> curl -s $ROUTE | head -n 1
> echo ''
> oc describe service/quickstart-loadbalancer | grep Endpoints
> sleep 0.5
> done
> ### ADMIN TERMINAL - scale the application
> oc patch wildflyserver quickstart -p '[{"op":"replace", "path":"/spec/replicas", "value":3}]' --type json
> {code}
> output:
> {noformat}
> ----------------------------------------
> Mon 14 Oct 2019 01:12:09 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> quickstart-2 0/1 Pending 0 0s
> quickstart-2 0/1 Pending 0 0s
> quickstart-2 0/1 Pending 0 0s
> quickstart-2 0/1 Pending 0 0s
> quickstart-2 0/1 ContainerCreating 0 0s
> quickstart-2 0/1 ContainerCreating 0 0s
> Error from server (NotFound): services "quickstart-loadbalancer" not found
> ----------------------------------------
> Mon 14 Oct 2019 01:12:09 PM CEST
> HTTP/1.0 503 Service Unavailable
> <html>
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:10 PM CEST
> HTTP/1.0 503 Service Unavailable
> <html>
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:11 PM CEST
> HTTP/1.0 503 Service Unavailable
> <html>
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:12 PM CEST
> HTTP/1.0 503 Service Unavailable
> <html>
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:13 PM CEST
> HTTP/1.0 503 Service Unavailable
> <html>
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:13 PM CEST
> HTTP/1.0 503 Service Unavailable
> <html>
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:14 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:15 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:16 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> quickstart-2quickstart-2 0/10/1 ContainerCreatingContainerCreating 00 8s8s
> ----------------------------------------
> Mon 14 Oct 2019 01:12:17 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080
> quickstart-2 1/1 Running 0 8s
> quickstart-2 1/1 Running 0 8s
> ----------------------------------------
> Mon 14 Oct 2019 01:12:17 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:18 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:19 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:20 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:24 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:24 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:25 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:26 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:27 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:28 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:28 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:29 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:30 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:31 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:32 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:32 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:33 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:34 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.44"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:35 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.43"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> ----------------------------------------
> Mon 14 Oct 2019 01:12:36 PM CEST
> HTTP/1.1 200 OK
> {"ip":"10.128.1.47"}
> Endpoints: 10.128.1.43:8080,10.128.1.44:8080,10.128.1.47:8080
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
More information about the jboss-jira
mailing list