Kubernetes - imagePullSecrets deployment authorization failure

Question

I'm trying to deploy a Kubernetes container by pulling an image from a remote registry (quay.io). While it works perfectly when deploying a pod, there is an unexpected problem with quay authorization when deploying it as a deployment. Don't know what misconfiguration in the yaml is causing the issue. Anyway, here's the working pod yaml:

apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: quay.io/xxx/yyy
    imagePullPolicy: Always
  imagePullSecrets:
  - name: test-secret

and here's the not working deployment yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: random-name-deployment
  namespace: test-ns
spec:
  selector:
    matchLabels:
      app: random-name
  replicas: 1
  template:
    metadata:
      labels:
        app: random-name
    spec:
      restartPolicy: Always
      containers:
      - name: random-name
        image: quay.io/xxx/yyy
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: 10Mi
            cpu: 0.1
          limits:
            memory: 10Mi
            cpu: 0.1
        volumeMounts:
        - mountPath: /etc/config/config.yaml
          name: random-name-config
          subPath: config.yaml
      imagePullSecrets:
      - name: test-secret
      volumes:
      - name: random-name-config
        configMap:
          name: random-name-config

It fails with an error:

kubectl describe:
Failed to pull image "quay.io/xxx/yyy": rpc error: code = Unknown desc = Error response from daemon: unauthorized: access to the requested resource is not authorized

However the secret seems to be loaded properly (and the secret has not changed in between deploying the pod and deployment - in the first scenario authorization works like a charm):

kubectl get -o yaml:
(...)
imagePullSecrets:
  - name: test-secret

What do I do wrong? Any sugestions will be greatly appreciated, thanks!

After some testing found that it's namespace that's causing the issue. So when "namespace: test-ns" is deleted, the deployment succeeds. However I'm still not sure why is that so and how can I use namespace other than default. — theo, Apr 12 '19 at 07:22

score 0 · Accepted Answer · answered Apr 12 '19 at 08:45

Found the problem - my secret was deployed in DEFAULT namespace, not TEST-NS, as it should be. If anyone wanted to check for all secrets and their namespaces:

kubectl get secrets --all-namespaces

Funny though, that Kubectl never logged that the secret cannot be found - such info would be very helpful :(

PjoterS · Answer 2 · 2019-04-15T13:10:19.710

Unfortunately you did not provide commands you used. You probably forgot to add --namespace flag during POD deploying.

In Kubernetes any command like create, run, explain, describe, get ... etc automatically refers to default namespace, until you will specify another.

So if you will use kubectl apply -f <pod.yaml> pod will be created in default namespace. But if you will use kubectl apply -f <pod.yaml> -n test pod will be created in test namespace.

You can always verify what is contained in specified namespace

kubectl get all --namespace <namespace_name>

Here you can find for what namespaces are.

In addition, to debuggig very helpful commands are kubectl describe pod <pod_name> -n <namespace> kubectl describe deployment <name> -n <namespace> or kubeclt get events -n <namespace>

Kubernetes - imagePullSecrets deployment authorization failure

2 Answers2