[jboss-jira] [JBoss JIRA] (WFWIP-252) Error log messages in operator logs on CRD edit after scale down
Petr Kremensky (Jira)
issues at jboss.org
Wed Oct 16 05:00:00 EDT 2019
Petr Kremensky created WFWIP-252:
------------------------------------
Summary: Error log messages in operator logs on CRD edit after scale down
Key: WFWIP-252
URL: https://issues.jboss.org/browse/WFWIP-252
Project: WildFly WIP
Issue Type: Bug
Components: OpenShift
Reporter: Petr Kremensky
Assignee: Jeff Mesnil
Attachments: operator-logs.txt
Error messages can be found in operator logs after user performs a scale down task (WFWIP-251) and than edit the CRD.
*reproduce*
* deploy operator {noformat}./build/run-openshift.sh{noformat}
* change the default number of initial replicas from 2 to 3 {noformat}sed -i 's/2/3/' deploy/crds/quickstart-cr.yaml{noformat}
* deploy quickstart project from operator repository {noformat}oc apply -f deploy/crds/quickstart-cr.yaml{noformat}
* wait for both replicas to be ready {noformat}oc get pods
NAME READY STATUS RESTARTS AGE
quickstart-0 1/1 Running 0 31s
quickstart-1 1/1 Running 0 31s
quickstart-2 1/1 Running 0 31s
wildfly-operator-5bcb545b46-j788h 1/1 Running 0 41s{noformat}
* scale down to 2 {noformat}oc patch wildflyserver quickstart -p '[{"op":"replace", "path":"/spec/replicas", "value":2}]' --type json{noformat}
* wait for scale down to perform {noformat} Pods:
Name: quickstart-0
Pod IP: 10.128.1.193
State: ACTIVE
Name: quickstart-1
Pod IP: 10.128.1.192
State: ACTIVE
Replicas: 2
Scalingdown Pods: 0
{noformat}
* edit the CRD (e.g. add an env variable) {noformat}oc edit wildflyserver quickstart
...
replicas: 2
env:
- name: FOO
value: BAR{noformat}
* wait for both pods to be reloaded
* see the [^operator-logs.txt]
** apart WFWIP-251, there are 2 extra error messages
There might be other ways to reproduce the issue, but these should make it 100% appearance (at least for me).
*actual*
Error messages in operator log
{noformat}
{"level":"error","ts":1571215869.7030337,"logger":"wildlfyserver_resources","msg":"Failed to update resource","WildFlyServer.Namespace":"wildfly-test","WildFlyServer.Name":"quickstart","Resource.Name":"quickstart","Resource.Type":"*v1.StatefulSet","error":"Operation cannot be fulfilled on statefulsets.apps \"quickstart\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr at v0.1.1/zapr.go:128\ngithub.com/wildfly/wildfly-operator/pkg/resources.Update\n\t/go/src/github.com/wildfly/wildfly-operator/pkg/resources/actions.go:72\ngithub.com/wildfly/wildfly-operator/pkg/controller/wildflyserver.(*ReconcileWildFlyServer).checkStatefulSet\n\t/go/src/github.com/wildfly/wildfly-operator/pkg/controller/wildflyserver/wildflyserver_controller.go:399\ngithub.com/wildfly/wildfly-operator/pkg/controller/wildflyserver.(*ReconcileWildFlyServer).Reconcile\n\t/go/src/github.com/wildfly/wildfly-operator/pkg/controller/wildflyserver/wildflyserver_controller.go:187\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime at v0.1.12/pkg/internal/controller/controller.go:215\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime at v0.1.12/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:88"}
{"level":"error","ts":1571215869.7032435,"logger":"wildflyserver_controller","msg":"Failed to update StatefulSet.","StatefulSet.Namespace":"wildfly-test","StatefulSet.Name":"quickstart","error":"Operation cannot be fulfilled on statefulsets.apps \"quickstart\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr at v0.1.1/zapr.go:128\ngithub.com/wildfly/wildfly-operator/pkg/controller/wildflyserver.(*ReconcileWildFlyServer).checkStatefulSet\n\t/go/src/github.com/wildfly/wildfly-operator/pkg/controller/wildflyserver/wildflyserver_controller.go:400\ngithub.com/wildfly/wildfly-operator/pkg/controller/wildflyserver.(*ReconcileWildFlyServer).Reconcile\n\t/go/src/github.com/wildfly/wildfly-operator/pkg/controller/wildflyserver/wildflyserver_controller.go:187\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime at v0.1.12/pkg/internal/controller/controller.go:215\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime at v0.1.12/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:88"}
{"level":"error","ts":1571215869.7034042,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"wildflyserver-controller","request":"wildfly-test/quickstart","error":"Operation cannot be fulfilled on statefulsets.apps \"quickstart\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr at v0.1.1/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime at v0.1.12/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime at v0.1.12/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery at v0.0.0-20190221213512-86fb29eff628/pkg/util/wait/wait.go:88"}
{noformat}
*expected*
No error messages in operator log.
Operator image build from [0a221ea|https://github.com/wildfly/wildfly-operator/commit/0a221eac5eb4628bd1cba27448e389bdbda8e813]
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
More information about the jboss-jira
mailing list