1

i successfully installed cert-manager and nginx-ingress in my kubernetes cluster. Ingress is working as expected and was tested. Certificate creation via cert-manager is also working. I created a test deployment, but the service is not accessible via https.

Deployment file looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80

Service file looks like this: 

apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: nginx

And finally the ingress definition:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/issuer: "letsencrypt-staging"
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx
          servicePort: 80

The certificate is not getting issued and i am getting the following error in the cert-manager logs:

cert-manager/controller/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://example.com/.well-known/acme-challenge/ETLbSNl2WHi3jbkc0S8HeYuu5uwKvuQxExn9k54z7dQ': Get \"https://example.com:443/.well-known/acme-challenge/ETLbSNl2WHi3jbkc0S8HeYuu5uwKvuQxExn9k54z7dQ\": remote error: tls: handshake failure" "dnsName"="example.com" "resource_kind"="Challenge" "resource_name"="example-tls-2091549504-1001399895-2322937816" "resource_namespace"="default" "type"="http-01"

Curling the challenged address:

curl -v https://example.com/.well-known/acme-challenge/ETLbSNl2WHi3jbkc0S8HeYuu5uwKvuQxExn9k54z7dQ
Trying 123.45.67.89...
* TCP_NODELAY set
* Connected to example.com (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS alert, Server hello (2):
* error:14004410:SSL routines:CONNECT_CR_SRVR_HELLO:sslv3 alert handshake failure
* stopped the pause stream!
* Closing connection 0
curl: (35) error:14004410:SSL routines:CONNECT_CR_SRVR_HELLO:sslv3 alert handshake failure

I am completed confused why this is happening. Any ideas?

OYPS
  • 58
  • 7
  • 1
    Where do you run `curl https://example.com` from ? Do you have this domain configured only locally so it points to the address that ingress is listening on ? Could you provide more details about your **kubernetes cluster** setup ? Is the "example.com" just an example and you're using your own domain which you have registered ? – mario Apr 02 '20 at 15:26
  • Thanks four your reply! I am running curl on a client locally, within the same network of the kubernetes cluster. The cluster is a bare-metal installation (version 1.18.0). The domain is configured with a real domain (example.com just being a placeholder here). The domain is pointing to the nginx-ingress controller IP (LoadBalancer IP configured by MetalLB) for both port 80 and 443. The domain setup is correct, because calling `curl http://example.com` gives a **308 Permanent redirect** response to https as expected. – OYPS Apr 02 '20 at 15:51
  • 1
    Could you check a few more things ? First of all please run `curl -v https://example.com` so we can see more verbose output. Then try to test the following scenario: 1. Temporarily remove from your ingress definition `cert-manager.io/issuer: "letsencrypt-prod"` annotation. 2. Change in your ingress definition `secretName` to non-existing one e.g. `example-tls-tmp`. This way Ingress will load its own default self-signed certificate. 3. Run `curl -v https://example.com` again. What is the output ? – mario Apr 03 '20 at 20:04
  • I reinstalled both components (nginx-ingress and cert-manager) to ensure everything worked fine on that end. When i try to request a certificate now i can see the following error in the cert-manager logs: **remote error: tls: handshake failure**. Its the same output when calling `curl` with all the modifications you described. – OYPS Apr 06 '20 at 09:27
  • @mario I have updated my question with the output of the curl commands. Please see my edits above. – OYPS Apr 06 '20 at 10:01
  • If you tested it with the default self-signed certificate and it didn't work either, it can possibly mean that your **ingress controller** isn't configured properly and it is passing the encrypted **https** data to **http** backend. Take a look at [this issue](https://github.com/kubernetes/ingress-nginx/issues/3556) or more specifically [this comment](https://github.com/kubernetes/ingress-nginx/issues/3556#issuecomment-446586277). – mario Apr 06 '20 at 14:59

1 Answers1

1

The issue was a incompatibility of my used components. Removing all three main parts - MetalLB, Nginx ingress, Cert-Manager - and reinstalling them with Helm did the trick. It was also crucial to define a default certificate for the ingress .

OYPS
  • 58
  • 7