20

Without changes in infrastructure If I execute any kubectl command, ie:

kubectl get nodes

I got error

You must be logged in to the server (Unauthorized)

And I had working kubernetes cluster and did no changes to it... Any ideas how to debug this? kubectl has no -vv od debug flag to give more information.

If i try

kubectl version

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0",
(...) 
error: You must be logged in to the server (the server has asked for the client to provide credentials)
Wojtas.Zet
  • 321
  • 1
  • 2
  • 8

10 Answers10

13

In my case the issue started after renewing kubernates certificates, this caused the existing ~/.kube/config to have outdated keys and certificate values in it.

The solution was to replace the values client-certificate-data and client-key-data in file ~/.kube/config with the values from the updated file in /etc/kubernetes/kubelet.conf of the same name.

Japheth Ongeri - inkalimeva
  • 231
  • 1
  • 5
  • 2
    For me the new keys in `/etc/kubernetes/kubelet.conf` had paths to the pem files. I had to do `sudo base64 -w 0 /var/lib/kubelet/pki/kubelet-client-current.pem` and paste the output of that into `~/.kube/config` for the matching keys. – cstack Feb 19 '21 at 18:14
10

You can copy the client-certificate-data and client-key-data from /etc/kubernetes/admin.conf to your ~/.kube/config file as of more recent versions of Kubernetes. See this answer for determining when your certificates expire.

koehn
  • 241
  • 2
  • 9
6

I have faced the similar issue today and the above comments helped me to fix the issue. I am adding more details with my scenario because it might be helpful for the people which have similar settings.

I have a separate user for connecting to my k8s cluster (It's a normal cluster in EC2 instances). I had created the user arunlal with limited access by adding ClusterRoleBindings.

If you get the following error while running API to cluster (in my case Kubectl):

error: You must be logged in to the server (Unauthorized)

Go through the following order.

- First check the cert used in your config file (local client)

I had a separate configuration on my local system, because the main config had the details about the other k8s & okd cluster credentials. So I had created second configuration on my Laptop (/Users/arunlal/.kube/config_two). In this case I have the following aliases:

alias kctl="kubectl --kubeconfig=/Users/arunlal/.kube/config_two"

- From this file you will get the cert that we are using.

[arunlal.as@crybit.com ~] cat /Users/arunlal/.kube/config_two| grep -A 5 users
users:
- name: arunlal
  user:
    client-certificate: /Users/arunlal/.arunlal-keys/arunlal.crt
    client-key: /Users/arunlal/.arunlal-keys/arunlal.key

- Once you get the cert in your client configuration you can check the validity using the openssl command.

    [arunlal.as@crybit.com ~] openssl x509 -noout -dates -in /Users/arunlal/.arunlal-keys/arunlal.crt
    notBefore=Jun 22 23:43:22 2021 GMT
    notAfter=Sep 30 23:43:22 2021 GMT

- Validate the expiry

While creating the user I passed the days as 5, that was the issue. How I created user?

openssl genrsa -out arunlal.key 2048
openssl req -new -key arunlal.key -out arunlal.csr -subj "/CN=arunlal/O=crybit"
openssl x509 -req -in arunlal.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out arunlal.crt -days 5
kubectl config set-credentials arunlal --client-certificate=/root/arunlal-keys/arunlal.crt  --client-key=/root/arunlal-keys/arunlal.key

- To fix, I recreated the cert with more number of days

openssl x509 -req -in arunlal.csr -CA /etc/kubernetes/pki/ca.crt
-CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out arunlal.crt -days 100

- This we need to run from the k8s cluster.

- Replaced the cert locally.

Modified /Users/arunlal/.arunlal-keys/arunlal.crt with new cert.

Hope this will help someone. Thanks!

~ arun

Arunlal Ashok
  • 111
  • 2
  • 4
6

I got the same after updating certificates:

kubeadm alpha certs renew all

And then I had to follow

$ cd ~/.kube

# Archive the old config file containing the out of date certificates
$ mv config conf.archive.2021

# Copy the new configuration file created using kubeadm
$ cp /etc/kubernetes/admin.conf config

# apply permissions to your current admin user and group
$ sudo chown $(id -u):$(id -g) config

Kubernetes version : 1.19

Reference

Sachith Muhandiram
  • 193
  • 3
  • 8
3

The kubeconfig certificate may have changed. If you deployed your cluster using terraform. Do terraform apply to generate a new kubeconfig file.

Azeez Adeniji
  • 31
  • 1
1

In my case I was using a cluster created by kOps and cluster admin user with credentials (~/.kube/config) generated by: kops export kubeconfig --admin

By default the credentials expire after 18 hours. So another:

kops export kubeconfig --admin

Did it for me.

Eliandro
  • 11
  • 2
0

Fixed - after all the cert has changed. Check your ~/.kube/config if you have this

Wojtas.Zet
  • 321
  • 1
  • 2
  • 8
0

I'm getting "error: You must be logged in to the server (the server has asked for the client to provide credentials)" in Azure Kubernetes service.

I fixed this with trying to connect and executed the command mentioned on the Connect to ClusterName.

az account set --subscription de81a6e3-1784-4732-9282-XXXXXXXX7
az aks get-credentials --resource-group resourceGroupName --name clusterName
Glorfindel
  • 1,213
  • 3
  • 15
  • 22
kaushal kishore
  • 1
  • 1
0

I fixed the issue by deleting minikube

minikube delete 
minikube start --vm-driver=none
scoulomb
  • 105
  • 3
0

After renewing your certs you need to apply the new certs to the admin config for kubectl to work.

The following takes a backup of your existing config, and applies the new admin config.

cp /root/.kube/config /root/.kube/.old-$(date --iso)-config
cp /etc/kubernetes/admin.conf /root/.kube/config
kruserr
  • 101