2

We have a system that currently uses docker-compose for deployment of containers (effectively docker-compose pull && docker-compose up) and are in the process of converting the system to Kubernetes.

We have CI/CD that deploys to production automatically based on git tags pushed to the git repository. For staging, we keeping advancing a branch. The docker images are built and tagged with either the git production tag or git staging branch name.

At deployment, docker-compose is intelligent enough to not restart the container if the image is actually the same, i.e. two production tags point to the same image.

I'm looking for a way to do the same thing in Kubernetes - if at deployment the actual image used for the container of the pod is the same as what's already running (based on the image checksum, not just the image tag), then it shouldn't have to restart the pod. Is this possible?

Thanks

Steve Folly
  • 545
  • 3
  • 12
  • Hey @Steve Folly, what is your `imagePullPolicy` setting? According to [documentation](https://kubernetes.io/docs/concepts/containers/images/#updating-images) if the `imagePullPolicy` is set to `IfNotPresent`, it should tell the kubelet to skip pulling an image if it already exists, so the pod shouldn't be restarded. – Jakub Jan 13 '21 at 10:25
  • @Jakub thanks, it's set to `Always`. I thought the `IfNotPresent` behaviour would be "I already have an image with this tag, so I won't pull". So, as the same tag is always moving to a new image, I thought it wouldn't pull. I guess I'll just have to try it...! – Steve Folly Jan 15 '21 at 07:23
  • Hi again @Steve Folly, I've found this [documentation](https://kubernetes.io/docs/concepts/configuration/overview/#container-images), as previous one wasn't widely described, if I understand correctly it should actually work as you want out of the box with `Always` image policy, `the kubelet queries the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet download the image with the resolved digest, and uses that image to launch the container. ` – Jakub Jan 19 '21 at 14:49
  • Also there is a tool called [imago](https://github.com/philpep/imago), worth to take a look, it checks running pods sha256 and just restart resource that need to use newer images. – Jakub Jan 19 '21 at 14:52
  • Hi @SteveFolly, did you manage to resolve your problem? I'm dealing with a very similar issue, a deploying with an image with is updated, same image (same sha256) with different tag and that is triggering the restart of the pods. Indeed my image references include both tag + sha256 (as build by skaffold) and I'm thinking on removing tags and keep only sha256, but wonder if you found some solution that can allow to keep both – agascon Aug 26 '22 at 09:57

1 Answers1

1

As I mentioned in comments, it should work out of the box with imagePullPolicy:Always.

According to documentation

Container Images

The imagePullPolicy and the tag of the image affect when the kubelet attempts to pull the specified image.

imagePullPolicy: Always: every time the kubelet launches a container, the kubelet queries the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet downloads (pulls) the image with the resolved digest, and uses that image to launch the container.

Note: To make sure the container always uses the same version of the image, you can specify its digest; replace : with @ (for example, image@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2). The digest uniquely identifies a specific version of the image, so it is never updated by Kubernetes unless you change the digest value.

Also there is a tool called imago, worth to take a look on it. Imago checks running pods sha256 and just restart resource that need to use newer images.

Jakub
  • 365
  • 1
  • 9