How to automatically deploy new docker images from dockerhub to kubernetes?

Question

I'm looking for a CD solution for my k8s cluster. Now after pushing a commit with dev-* tag, dockerhub creates new image tagged as dev-latest. Here's my deployment config:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-image-backend-local-deployment
  labels:
    app: my-image
    type: web
spec:
  replicas: 2
  minReadySeconds: 15
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 1
  selector:
    matchLabels:
      app: my-image
  template:
    metadata:
      labels:
        app: my-image
    spec:
      imagePullSecrets:
        - name: regcred
      containers:
      - name: backend
        image: cosmicfruit/my-image:dev-latest
        imagePullPolicy: IfNotPresent
        envFrom:
        - secretRef:
            name: my-image-backend-local-secret
        ports:
        - containerPort: 8000
        readinessProbe:
          httpGet:
             path: /flvby
             port: 8000
          initialDelaySeconds: 10
          periodSeconds: 5
      - name: celery
        image: cosmicfruit/my-image:dev-latest
        imagePullPolicy: IfNotPresent
        workingDir: /code
        command: ["/code/run/celery.sh"]
        envFrom:
        - secretRef:
            name: my-image-backend-local-secret
      - name: redis
        image: redis:latest
        imagePullPolicy: IfNotPresent

I want new images to be automatically deployed into the pods but cant find a relevant solution.

UPD I'd like my images to be automatically deployed after new image with dev-0.1.* (where * is a number) tag was pushed to the registry. I want to do nothing after code was pushed and built into the image. I've discovered some project of CD Foundation: Jenkins X, Spinnaker and Tekton and I guess some of them can help me to implement my idea but I would be great to implement it without one more pet in my zoo.

Answer 1

Image reference and manifests

Kubernetes manage deployments, e.g. using a rolling deployment strategy whenever your kind: Deployment manifest is changed.

Now after pushing a commit with dev-* tag, dockerhub creates new image tagged as dev-latest

The main problem with this is that you use the same tag name for every image. This means that your kind: Deployment manifest is not updated when your image is updated, so Kubernetes does not initiate a new rolling deployment, since there are no changes in the manifest.

A good practice is to use an unique image tag every time you push an image, and then update the kind: Deployment manifest with this new image tag name, so Kubernetes automatically trigger a new rolling deployment.

Some image registries can be configured to use "immutable tag names", this means that you are enforced to always use a unique tag name for every built image. Alternatively, you can use the full Image Digest in the kind: Deployment manifest, so that you can have an unique image reference even if the tag name is the same (Image Digest is a hash of the content).

In summary:

• Use unique image tag names for every build
• Update the kind: Deployment manifest with the new tag name
• Keep the kind: Deployment manifest (and other yaml manifest) in version control, e.g. Git
• Consider use full Image Digest in the manifest when referring to the Image

Process and automation

Updating the manifest and applying it to the cluster can be done in several different ways, but it is recommended to always store the updated manifest in a Git repository after every change, before it is applied to the cluster. In this way, you get good traceability about what has changed and it is easier to get back to a working version if something fail.

With the steps needed as described above, another important practice is to use automation for this. After every change to the Git repository, trigger an automated process to do the work, preferably without any manual steps. Historically, Jenkins has been a popular tool for this, but it has its age and is not designed to run well in a container environment. I now recommend to use a tool like GitHub Actions, Google Cloud Build or a modern system within the Kubernetes cluster, like Tekton Pipelines

Build and Deploy Pipeline using Tekton

If you choose to use Tekton in your Kubernetes cluster, you may structure the project like this:

1. Git push of code change to a code repository
2. A Tekton Trigger receives an event from the Git system and initiate a new PipelineRun.
3. The Tekton Pipeline contains steps for git-clone, build the code, run tests and to build and push an image. The result is an image digest or label. Then you have a last task in the pipeline that, if all previous tasks succeeded, updates the kind: Deployment manifest with the new image-digest and git-push to a repo with the manifests.
4. A Deployment Pipeline is trigged that apply the manifests to the cluster (perhaps using a deployment strategy of choice, rolling deployment or gradual deployment or canary deployment)

Book recommendations

• Kubernetes Up & Running 2nd edition (from 2019) - it contains a new chapter: 18. Organizing Your Application that describes how to manage deployments with manifests and version control.

• Continuous Delivery - a classic book about how to use build and deploy pipelines and automation.

Alternatives

I've discovered some project of CD Foundation: Jenkins X, Spinnaker and Tekton and I guess some of them can help me to implement my idea but I would be great to implement it without one more pet in my zoo.

Deployments on Kubernetes can be done without using declarative manifests and instead be done with imperative commands without any version controlled changes but this is really discouraged for a professional environment. It is important to do this in a declarative and reproducible way although automated pipelines takes some time to setup and configure.

Answer 2

Finally I’ve discovered awesome tool “keel.sh” that does exactly what I expect from the CD for kubernetes