What is the correct way to manage EKS node access to ECR via the IAM policy documents when I want to limit who can start jobs using different images?
In a simplified example, I have users A and B and ECR repositories C and D. User A has access to both repositories and user B should only have access to C. I want both users to be able to use the same EKS cluster (and the same set of nodes), and A should be able to run jobs on EKS using images from either C or D while B should be able to run only jobs using images from C but not D.
In my current setup, all EC2 nodes in the EKS cluster use a common IAM role which has AmazonEC2ContainerRegistryReadOnly attached to it so that it can read from all ECR repositories. Repository C gives user A and the EC2 node role read access, and repository D gives both users A and B as well as the EC2 node role read access. However, this means that both user A and user B can start jobs that read from either repository C or D, which is not what I want (user B shouldn't be able to start jobs reading from repository D).
I understand that one possibility would be to partition the set of nodes and assign them different EC2 IAM instance profiles and set up the ECR IAM policy documents so that the first set of nodes has access to repositories C and D while a second set of nodes only has access to C, but is there an alternative to this that avoids having to set up a set of nodes for each unique set of user permissions?
