400 Error with nginx-ingress to Kubernetes Dashboard

Question

I have an ingress that connects to Kubernetes Dashboard, but I'm getting a 400 error when trying to access it.

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "false"
    nginx.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/16"
spec:
  tls:
    - hosts:
        - kube.example.com
      secretName: dashboard-tls  # confirmed is valid LE cert
  rules:
    - host: kube.example.com
      http:
        paths:
          - backend:
              serviceName: kubernetes-dashboard
              servicePort: 443

That gives me a 400 error in the nginx pod.

2020/08/28 01:25:58 [error] 2609#2609: *795 readv() failed (104: Connection reset by peer) while reading upstream, client: 10.0.0.25, server: kube.example.com, request: "GET / HTTP/1.1", upstream: "http://10.42.0.2:8443/", host: "kube.example.com"

10.0.0.25 - - [28/Aug/2020:01:25:58 +0000] "GET / HTTP/1.1" 400 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0" "-"

And in case it is relevant, my nginx configuration, deployed through the helm chart nginx-stable/nginx-ingress

  ## nginx configuration
  ## Ref: https://github.com/kubernetes/ingress/blob/master/controllers/nginx/configuration.md
  ##
  controller:
    config:
      entries:
        hsts-include-subdomains: "false"
        ssl-ciphers: "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4"
        ssl-protocols: "TLSv1.1 TLSv1.2"
    ingressClass: nginx
    service:
      externalTrafficPolicy: Local
      annotations:
        metallb.universe.tf/address-pool: default
  defaultBackend:
    enabled: true
  tcp:
    22: "gitlab/gitlab-gitlab-shell:22"

Seems like you're trying to use HTTP to access an HTTPS endpoint. — Spooler, Aug 28 '20 at 02:14
@Spooler but doesn't `backend-protocol=HTTPS` make it use HTTPS instead? — cclloyd, Aug 28 '20 at 02:18
That makes the reverse proxy communicate with the backend services via https, but the client initiating requests to the ingress controller must not use 'http', or it will be classified as an invalid request. You have not defined a redirect from http to https. — Spooler, Aug 28 '20 at 02:26
@Spooler I removed the `ssl-redirect` annotation because the docs stated that it's on by default if https is available. But even adding it back and setting it to true, and doing `curl https://kube.example.com` returns `Client sent an HTTP request to an HTTPS server.` — cclloyd, Aug 28 '20 at 02:31

score 13 · Answer 1 · answered Sep 01 '20 at 20:04

13

All you need is this annotation on your service ingress

annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/configuration-snippet: |-
      proxy_ssl_server_name on;
      proxy_ssl_name $host;

answered Sep 01 '20 at 20:04

Iyddaz

131
3

You saved my life (: – Barış Velioğlu Feb 13 '21 at 19:09
I had to use this syntax, https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-certificate-authentication – oxr463 Mar 09 '21 at 22:06
1

OMG - you saved my life too. Though it seems I just needed to use the first annotation `backend-protocol`. What additional stuff does the `configuration-snippet` enable? – AnthonyK Apr 10 '21 at 15:07
1

This solved my problem too, but besides the configuration snippet, an explanation of what is going on would be needed for the answer to be complete. – istepaniuk Sep 11 '21 at 22:48

400 Error with nginx-ingress to Kubernetes Dashboard

1 Answers1

Linked