OpenShift 3.11: Prometheus operator update

Question

According to the docs the Prometheus operator on a OpenShift 3.11 cluster is self-upgrading. However, i have upgraded the cluster to 3.11.141 yesterday, but the operator is still stuck on 3.11.117. There are prometheus images for 3.11.141 available, so i am wondering when this self-update will take place. Can i somehow trigger it manually, maybe deleting the old pods?

Answer 1

Any chance your inventory would mention something like:

openshift_cluster_monitoring_operator_install: False

Have you tried applying the playbook deploying Prometheus?

/usr/share/ansible/openshift-ansible/playbooks/openshift-monitoring/config.yaml

The operator should get updated while applying the control plane upgrade playbook (don't wait for something to hopefully change, eventually, ... it won't).

Deleting Pods at random won't help. The cluster-monitoring-operator Deployment definition should point to specific tags:

containers:
- args:
  - -namespace=openshift-monitoring
  - -configmap=cluster-monitoring-operator
  - -logtostderr=true
  - -v=4
  - tags=prometheus-operator=v3.11.141
  - tags=prometheus-config-reloader=v3.11.141
  - tags=config-reloader=v3.11.141
  - tags=prometheus=v3.11.141
  - tags=alertmanager=v3.11.141
  - tags=grafana=v3.11.141
  - tags=oauth-proxy=v3.11.141
  - tags=node-exporter=v3.11.141
  - tags=kube-state-metrics=v3.11.141
  - tags=kube-rbac-proxy=v3.11.141
  image: xxx/openshift3/ose-cluster-monitoring-operator:v3.11.141

You could try to edit that Deployment manually, though the upgrade process should have done it (if cluster monitoring operator deployment is enabled, which should be the default)