1

I'm really struggling to follow the AWS documentation for "Enabling IAM user and role access to your cluster".

When I run kubectl edit -n kube-system configmap/aws-auth, I see this:

  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes

The documentation tells me to run kubectl get roles -A and kubectl get clusterroles to see my roles and clusterroles. Then kubectl describe role role-name -n kube-system and kubectl describe clusterrole cluster-role-name.

I wanted to set up my AWS IAM user as an admin or cluster-admin, but it didn't work when I used this config:

  mapUsers: |
    - userarn: arn:aws:iam::**********:user/nathan
      username: nathan
      groups:
      - cluster-admin

I figured out that I could get it to work when I used system:masters. But then I looked through all the roles, clusterroles, rolebindings, and clusterrolebindings, and I can't see any of these "system" ones that are used in configmap/aws-auth:

  • system:bootstrappers
  • system:nodes
  • system:masters

However, here are some that I can see:

  • clusterrolebindings - system:node (singular)
  • clusterroles - system:node-bootstrapper

I'm really lost! What am I misunderstanding here? What's a "group" in configmap/aws-auth, and how does it relate to these roles / clusterroles? If I want to use cluster-admin, do I need to prefix it with something, or add a "binding" somewhere?

system:masters is working fine and gives me access in the EKS web console, but I just want to understand how this works.

Here's all my roles and clusterroles:

$ kubectl get roles -A
NAMESPACE          NAME                                             CREATED AT
europe-v3-system   ingress-nginx                                    2022-04-26T01:21:05Z
kube-public        system:controller:bootstrap-signer               2022-04-26T01:13:11Z
kube-system        cert-manager-cainjector:leaderelection           2022-04-26T01:21:34Z
kube-system        cert-manager:leaderelection                      2022-04-26T01:21:34Z
kube-system        cluster-autoscaler                               2022-04-26T01:18:18Z
kube-system        eks-vpc-resource-controller-role                 2022-04-26T01:13:15Z
kube-system        eks:addon-manager                                2022-04-26T01:13:13Z
kube-system        eks:certificate-controller                       2022-04-26T01:13:12Z
kube-system        eks:fargate-manager                              2022-04-26T01:13:12Z
kube-system        eks:node-manager                                 2022-04-26T01:13:12Z
kube-system        extension-apiserver-authentication-reader        2022-04-26T01:13:10Z
kube-system        system::leader-locking-kube-controller-manager   2022-04-26T01:13:11Z
kube-system        system::leader-locking-kube-scheduler            2022-04-26T01:13:11Z
kube-system        system:controller:bootstrap-signer               2022-04-26T01:13:10Z
kube-system        system:controller:cloud-provider                 2022-04-26T01:13:10Z
kube-system        system:controller:token-cleaner                  2022-04-26T01:13:11Z
kube-system        vpc-resource-controller-leader-election-role     2022-04-26T01:13:14Z

$ kubectl get clusterroles
NAME                                                                   CREATED AT
admin                                                                  2022-04-26T01:13:10Z
atom                                                                   2022-04-26T01:21:03Z
aws-node                                                               2022-04-26T01:13:12Z
cert-manager-cainjector                                                2022-04-26T01:21:34Z
cert-manager-controller-certificates                                   2022-04-26T01:21:34Z
cert-manager-controller-challenges                                     2022-04-26T01:21:34Z
cert-manager-controller-clusterissuers                                 2022-04-26T01:21:34Z
cert-manager-controller-ingress-shim                                   2022-04-26T01:21:34Z
cert-manager-controller-issuers                                        2022-04-26T01:21:34Z
cert-manager-controller-orders                                         2022-04-26T01:21:34Z
cert-manager-edit                                                      2022-04-26T01:21:34Z
cert-manager-view                                                      2022-04-26T01:21:34Z
cert-manager-webhook:webhook-requester                                 2022-04-26T01:21:34Z
cloudwatch-agent-role                                                  2022-06-05T02:20:04Z
cluster-admin                                                          2022-04-26T01:13:10Z
cluster-autoscaler                                                     2022-04-26T01:18:18Z
edit                                                                   2022-04-26T01:13:10Z
eks-console-dashboard-full-access-clusterrole                          2022-04-26T02:03:47Z
eks:addon-manager                                                      2022-04-26T01:13:12Z
eks:fargate-manager                                                    2022-04-26T01:13:12Z
eks:node-bootstrapper                                                  2022-04-26T01:13:13Z
eks:node-manager                                                       2022-04-26T01:13:12Z
eks:podsecuritypolicy:privileged                                       2022-04-26T01:13:13Z
europe-v3-api                                                          2022-04-26T01:21:03Z
europe-v3-fluentd                                                      2022-04-26T01:21:04Z
ingress-nginx                                                          2022-04-26T01:21:04Z
resolver                                                               2022-04-26T01:21:03Z
system:aggregate-to-admin                                              2022-04-26T01:13:10Z
system:aggregate-to-edit                                               2022-04-26T01:13:10Z
system:aggregate-to-view                                               2022-04-26T01:13:10Z
system:aggregated-metrics-reader                                       2022-04-26T01:21:04Z
system:auth-delegator                                                  2022-04-26T01:13:10Z
system:basic-user                                                      2022-04-26T01:13:10Z
system:certificates.k8s.io:certificatesigningrequests:nodeclient       2022-04-26T01:13:10Z
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient   2022-04-26T01:13:10Z
system:certificates.k8s.io:kube-apiserver-client-approver              2022-04-26T01:13:10Z
system:certificates.k8s.io:kube-apiserver-client-kubelet-approver      2022-04-26T01:13:10Z
system:certificates.k8s.io:kubelet-serving-approver                    2022-04-26T01:13:10Z
system:certificates.k8s.io:legacy-unknown-approver                     2022-04-26T01:13:10Z
system:controller:attachdetach-controller                              2022-04-26T01:13:10Z
system:controller:certificate-controller                               2022-04-26T01:13:10Z
system:controller:clusterrole-aggregation-controller                   2022-04-26T01:13:10Z
system:controller:cronjob-controller                                   2022-04-26T01:13:10Z
system:controller:daemon-set-controller                                2022-04-26T01:13:10Z
system:controller:deployment-controller                                2022-04-26T01:13:10Z
system:controller:disruption-controller                                2022-04-26T01:13:10Z
system:controller:endpoint-controller                                  2022-04-26T01:13:10Z
system:controller:endpointslice-controller                             2022-04-26T01:13:10Z
system:controller:endpointslicemirroring-controller                    2022-04-26T01:13:10Z
system:controller:expand-controller                                    2022-04-26T01:13:10Z
system:controller:generic-garbage-collector                            2022-04-26T01:13:10Z
system:controller:horizontal-pod-autoscaler                            2022-04-26T01:13:10Z
system:controller:job-controller                                       2022-04-26T01:13:10Z
system:controller:namespace-controller                                 2022-04-26T01:13:10Z
system:controller:node-controller                                      2022-04-26T01:13:10Z
system:controller:persistent-volume-binder                             2022-04-26T01:13:10Z
system:controller:pod-garbage-collector                                2022-04-26T01:13:10Z
system:controller:pv-protection-controller                             2022-04-26T01:13:10Z
system:controller:pvc-protection-controller                            2022-04-26T01:13:10Z
system:controller:replicaset-controller                                2022-04-26T01:13:10Z
system:controller:replication-controller                               2022-04-26T01:13:10Z
system:controller:resourcequota-controller                             2022-04-26T01:13:10Z
system:controller:route-controller                                     2022-04-26T01:13:10Z
system:controller:service-account-controller                           2022-04-26T01:13:10Z
system:controller:service-controller                                   2022-04-26T01:13:10Z
system:controller:statefulset-controller                               2022-04-26T01:13:10Z
system:controller:ttl-controller                                       2022-04-26T01:13:10Z
system:coredns                                                         2022-04-26T01:13:12Z
system:discovery                                                       2022-04-26T01:13:10Z
system:heapster                                                        2022-04-26T01:13:10Z
system:kube-aggregator                                                 2022-04-26T01:13:10Z
system:kube-controller-manager                                         2022-04-26T01:13:10Z
system:kube-dns                                                        2022-04-26T01:13:10Z
system:kube-scheduler                                                  2022-04-26T01:13:10Z
system:kubelet-api-admin                                               2022-04-26T01:13:10Z
system:metrics-server                                                  2022-04-26T01:21:04Z
system:node                                                            2022-04-26T01:13:10Z
system:node-bootstrapper                                               2022-04-26T01:13:10Z
system:node-problem-detector                                           2022-04-26T01:13:10Z
system:node-proxier                                                    2022-04-26T01:13:10Z
system:persistent-volume-provisioner                                   2022-04-26T01:13:10Z
system:public-info-viewer                                              2022-04-26T01:13:10Z
system:volume-scheduler                                                2022-04-26T01:13:10Z
view                                                                   2022-04-26T01:13:10Z
vpc-resource-controller-role                                           2022-04-26T01:13:14Z

Thanks!

ndbroadbent
  • 276
  • 2
  • 11

1 Answers1

2

You will not find a kubectl get resource for any authn principals, since they don't require pre-allocation. That system:masters group is one of a few well known names that are hard-coded into the source code

What's a "group" in configmap/aws-auth

If one frames it from the CN= and OU= terms from k8s x.509 auth, the username: is the singular principal name and the groups: are the arbitrary number of authz containers that can be used for (Cluster)RoleBindings

and how does it relate to these roles / clusterroles? If I want to use cluster-admin, do I need to prefix it with something, or add a "binding" somewhere?

The RoleBinding and ClusterRoleBinding merely associate the presented authn names (which, aside from the few hard-coded ones, are arbitrary) to the Role and ClusterRole objects in the k8s api

I can't tell if you're also asking how the IAM User/Role gets mapped into a k8s principal, but if so it's because aws eks get-token (spawned by the exec: stanza of $KUBECONFIG) generates a JWT which encodes the sub: and claims: signed by an AWS IAM OpenID Connect provider that is then trusted by the apiserver just like a Google or GitLab or your favorite OIDC provider would be

Reading the aws-iam-authenticator repo goes into the gory details about that setup

mdaniel
  • 2,338
  • 1
  • 8
  • 13