1

I’m currently trying to get Keycloak to run in EKS behind ALB and for the life of me, I can’t get it to work. I get the redirect to a login screen and after I log in - I instantly get presented with 401 Authorization required.

Infrastructure as follows: Browser (HTTPS) → ALB (HTTP between pods w/ OIDC config) → Target Group → Application in EKS

Keycloak with debug logs enabled doesn’t give me any errors at all:

keycloak logs

ALB on the other side gives me an AuthInvalidIdToken error after I sign in and it tries to redirect to the original URI:

2022-07-27T13:20:12.184736Z app/k8s-ingr-35696e2f02/3e62855cad3d2877 ip_address:53578 - -1 -1 -1 302 - 384 554 "GET https://application.domain.com:443/favicon.ico HTTP/2.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:769668795579:targetgroup/k8s-application-367cc81867/9fbca3b5630aecf1 "Root=1-62e13b8c-14d1e5670bc7d8d86056ddef" "application.domain.com" "arn:aws:acm:eu-west-1:769668795579:certificate/83bd567a-a624-4ff8-86e7-744ebff68575" 6 2022-07-27T13:20:12.184000Z "authenticate" "-" "-" "-" "-" "-" "-"
2022-07-27T13:20:12.314098Z app/k8s-ingr-35696e2f02/3e62855cad3d2877 ip_address:53578 10.0.2.19:8080 0.000 0.010 0.000 302 302 1509 3271 "GET https://keycloak.domain.com:443/realms/master/protocol/openid-connect/auth?client_id=oauth2-proxy&redirect_uri=https%3A%2F%2Fapplication.domain.com%2Foauth2%2Fidpresponse&response_type=code&scope=email%20openid&state=Giz6L1tOQP4KTWIT%2BvrEvnRV3tnHFhCmswD%2B2tattLy8%2FRThp1hYyCrNLQQdG%2FIWq2JjO1p8rjDMHawgK0JW3S%2BszvY6JrZ9%2BeMzBmp05iTDXRre2IWaYnYxlH86i1FVCYrkIONZ0bdxu0UydsLuXEhx57FGsu6fbjKMNxAKO2cKPvDn7xZYLzCQuJWKKx3NZXUSEVNGgQLtCLMTM9YRBA%3D%3D HTTP/2.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:769668795579:targetgroup/k8s-keycloak-1f3317ccac/0ac746adee6d293e "Root=1-62e13b8c-73b50ba974b867b243093ce4" "application.domain.com" "arn:aws:acm:eu-west-1:769668795579:certificate/83bd567a-a624-4ff8-86e7-744ebff68575" 11 2022-07-27T13:20:12.303000Z "authenticate,forward" "-" "-" "10.0.2.19:8080" "302" "-" "-"
2022-07-27T13:20:12.479301Z app/k8s-ingr-35696e2f02/3e62855cad3d2877 ip_address:53578 - -1 -1 -1 401 - 674 616 "GET https://application.domain.com:443/oauth2/idpresponse?state=Giz6L1tOQP4KTWIT%2BvrEvnRV3tnHFhCmswD%2B2tattLy8%2FRThp1hYyCrNLQQdG%2FIWq2JjO1p8rjDMHawgK0JW3S%2BszvY6JrZ9%2BeMzBmp05iTDXRre2IWaYnYxlH86i1FVCYrkIONZ0bdxu0UydsLuXEhx57FGsu6fbjKMNxAKO2cKPvDn7xZYLzCQuJWKKx3NZXUSEVNGgQLtCLMTM9YRBA%3D%3D&session_state=f0c7b40f-df70-40a4-a4ec-ae674eddf704&code=5e30afcf-2121-4c3f-a2c5-6be9ccce569e.f0c7b40f-df70-40a4-a4ec-ae674eddf704.398968b2-cc51-4f41-8680-92e32051d6b0 HTTP/2.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 - "Root=1-62e13b8c-466c6d524d1bd9a745476851" "application.domain.com" "arn:aws:acm:eu-west-1:769668795579:certificate/83bd567a-a624-4ff8-86e7-744ebff68575" -1 2022-07-27T13:20:12.459000Z "authenticate" "-" "AuthInvalidIdToken" "-" "-" "-" "-"

ALB annotations for OIDC auth (Keycloak) is nothing fancy and/or custom:

auth_path = f"https://{self.keycloak_domain}/realms/{self.keycloak.realm}"
auth_annotations = base_annotations | {
    "alb.ingress.kubernetes.io/auth-type": "oidc",
    "alb.ingress.kubernetes.io/auth-idp-oidc": (
        f'{{"issuer":"https://{self.keycloak_domain}/",'
        f'"authorizationEndpoint":"{auth_path}/protocol/openid-connect/auth",'
        f'"tokenEndpoint":"{auth_path}/protocol/openid-connect/token",'
        f'"userInfoEndpoint":"{auth_path}/protocol/openid-connect/userinfo",'
        f'"secretName":"{self.keycloak.secrets_name}"}}'
    ),
    "alb.ingress.kubernetes.io/auth-on-unauthenticated-request": "authenticate",
}

ALB route in AWS

Keycloak config:

fullnameOverride: "{{ full_name }}"
extraStartupArgs: "--spi-login-protocol-openid-connect-legacy-logout-redirect-uri=true"

service:
  type: "ClusterIP"

auth:
  createAdminUser: true
  adminUser: "{{ admin_username }}"
  adminPassword: "{{ admin_password }}"

postgresql:
  enabled: false

externalDatabase:
  host: "{{ postgresql_host }}"
  database: "{{ postgresql_db }}"
  user: "{{ postgresql_user }}"
  password: "{{ postgresql_password }}"

extraEnvVars:
  - name: KEYCLOAK_HOSTNAME
    value: "{{ hostname }}"
  - name: PROXY_ADDRESS_FORWARDING
    value: "true"

What I’ve tried:

  1. KEYCLOAK_FRONTEND_URL
  2. Adding a trailing / to URI’s
  3. Setting ALB session cookie names (KEYCLOAK_SESSION for example)
  4. Enforcing HTTPS for ALB internal routing (alb_backend_protocol annotation)
  5. Running everything with redirect_uri in Keycloak as *
  6. List item

Some options I’ve come up while trying to debug this:

AWS ALB parses keycloak auth response somehow wrong? Misconfiguration from my side? (even tho keycloak is happy with everything?)

Versions:

Keycloak (bitnami) Helm Chart: 9.2.2 chart w/ 18.0.0 Keycloak

AWS ALB Helm Chart: 1.4.2 chart w/ 2.4.2 ALB

iotanum
  • 21
  • 5
  • I can't answer your question, but wanted to comment that there are lots of problems like this since Quarkus. I hope someone here can resolve this, but you may need to post over on their GitHub discussions or even an issue, if you feel so inclined, as these problem seem unanswered and unresolved everywhere I find them. Also, I noticed you seem to be authenticating against the Master realm, which is intended to be non-globally reachable, and the project makes it hard to go against this at least with my (very different) hosting configuration. – Paul Jul 28 '22 at 12:27
  • Yeah, regarding the realm - currently I'm trying to get the whole infra to work first before I get to the nitty-gritty here. Thanks for the input tho! I'm also writing to aws support regarding this because, at least atm, it seems like an issue at their end (unless there will be a super specific client configuration that you need to have for ALB) – iotanum Jul 28 '22 at 22:20

1 Answers1

1

Turns out that according to openid connect you have to validate authorization_code according to these steps: https://openid.net/specs/openid-connect-basic-1_0.html#IDTokenValidation

What my issue here was is that the issuer in my OIDC config != iss claim from id_token. Basically changing:

f'{{"issuer":"https://{self.keycloak_domain}/",' -> f'{{"issuer":"https://{self.keycloak_domain}/realms/master",'

fixed the issue!

also turns out that oauth2-proxy doesn't validate user claims from id_token

iotanum
  • 21
  • 5