ERR_SSL_PROTOCOL_ERROR on otherwise working kubernetes ingress/service

Question

I have a basic nginx deployment and an existing certificate issued by let's encrypt via cert-manager. I thought everything was in place to start using the certificate but I'm unable to connect on https.

Connecting to the LoadBalancer IP and the domain works. Connecting to the domain with https is unable to connect. Chrome says ERR_SSL_PROTOCOL_ERROR, Firefox says SSL_ERROR_RX_RECORD_TOO_LONG and SSL Labs says Assessment failed: No secure protocols supported. It's all the same issue.

Here's the service:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: example
  labels:
    app: example
spec:
  type: LoadBalancer
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
  - name: https
    protocol: TCP
    port: 443
    targetPort: 80
  selector:
    app: example

Here's the ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  namespace: example
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-production
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
  tls:
    - hosts:
      - 'example.com'
      secretName: example-production-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx
          servicePort: 443

The certificate is populated:

kubectl describe secret
...
Data
====
tls.crt:  3574 bytes
tls.key:  1675 bytes
ca.crt:   0 bytes

And the certificate resource is owned by the correct ingress. I've replaced my domain with "example" above.

It seems everything is in place but I'm not sure why I'm unable to connect through https. What can I run to troubleshoot this?

UPDATE: I found I had some missing configurations for my deployment and nginx image. I've followed all steps here: https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/#securing-the-service

Like before, I can connect to the LoadBalancer IP on 443 and 80, but the https connection is failing. Working with http:

curl http://<EXTERNAL-IP> -k
<html>
<h1>Hello!</h1>
  <p>Stay tuned for launch!</p>
</html>

Failing with https:

curl https://<EXTERNAL-IP> -k
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

score 2 · Answer 1 · answered Jan 03 '20 at 03:40

The issue in this case was the combination of the deployment and the service. I was routing https traffic to port 80 with this line in the service yaml:

- name: https
  protocol: TCP
  port: 443
  targetPort: 80

This was unable to complete an SSL handshake which is what the curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number was about.

I thought this would be ok because on a previous service with working SSL, I was routing port 80 and 443 to 3000 on puma. I'm not sure what would happen if routing 443 to 80 on puma, but it certainly broke with nginx. Changing the above service yaml this solved it (after ensuring nginx was listening on 443 and had SSL enabled with listen 443 ssl; in default.conf):

- name: https
  protocol: TCP
  port: 443
  targetPort: 443

ERR_SSL_PROTOCOL_ERROR on otherwise working kubernetes ingress/service

1 Answers1