I have tried and tried many different things, and I'm unable to land into a solution. The certificate was issued once using letsencrypt, but was never refreshed. Could you people give me a hint on what I'm doing wrong? This is my current setup:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: <mi email>
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: external-ingress
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
ingress.kubernetes.io/secure-backends: "true"
spec:
tls:
- hosts:
- example.com
secretName: example-tls
rules:
- host: example.com
http:
paths:
- path: /
backend:
serviceName: web-service
servicePort: 4000
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: example-tls
spec:
secretName: example-tls
issuerRef:
name: letsencrypt-prod
commonName: example.com
dnsNames:
- www.example.com
- example.com
---
apiVersion: v1
kind: Service
metadata:
name: web-service
spec:
type: ClusterIP
selector:
pod: web
ports:
- protocol: TCP
port: 4000
targetPort: 8000
[EDIT] Here are the latest logs which include the output of 2 or 3 changes I made:
W0226 19:21:59.418601 1 reflector.go:299] external/io_k8s_client_go/tools/cache/reflector.go:96: watch of *v1alpha2.Certificate ended with: too old resource version: 20102319 (57035298)
W0226 19:22:00.706904 1 reflector.go:299] external/io_k8s_client_go/tools/cache/reflector.go:96: watch of *v1alpha2.Challenge ended with: too old resource version: 20102318 (57035302)
W0226 19:22:02.208128 1 reflector.go:299] external/io_k8s_client_go/tools/cache/reflector.go:96: watch of *v1alpha2.ClusterIssuer ended with: too old resource version: 20102319 (57035310)
W0226 19:22:03.492014 1 reflector.go:299] external/io_k8s_client_go/tools/cache/reflector.go:96: watch of *v1alpha2.Issuer ended with: too old resource version: 20102318 (57035315)
I0226 19:28:32.827986 1 controller.go:129] cert-manager/controller/clusterissuers "level"=0 "msg"="syncing item" "key"="letsencrypt-prod"
I0226 19:28:32.834873 1 setup.go:86] cert-manager/controller/clusterissuers "level"=0 "msg"="generating acme account private key" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod-key" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 19:28:33.004479 1 controller.go:129] cert-manager/controller/webhook-bootstrap "level"=0 "msg"="syncing item" "key"="cert-manager/letsencrypt-prod-key"
I0226 19:28:33.004517 1 controller.go:135] cert-manager/controller/webhook-bootstrap "level"=0 "msg"="finished processing work item" "key"="cert-manager/letsencrypt-prod-key"
I0226 19:28:33.005861 1 setup.go:167] cert-manager/controller/clusterissuers "level"=0 "msg"="ACME server URL host and ACME private key registration host differ. Re-checking ACME account registration" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod-key" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 19:28:33.005988 1 logger.go:88] Calling GetAccount
I0226 19:28:33.406134 1 logger.go:83] Calling CreateAccount
I0226 19:28:33.500282 1 setup.go:229] cert-manager/controller/clusterissuers "level"=0 "msg"="verified existing registration with ACME server" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod-key" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 19:28:33.500466 1 conditions.go:92] Setting lastTransitionTime for Issuer "letsencrypt-prod" condition "Ready" to 2020-02-26 19:28:33.500447206 +0000 UTC m=+6081889.996740166
I0226 19:28:33.510355 1 controller.go:135] cert-manager/controller/clusterissuers "level"=0 "msg"="finished processing work item" "key"="letsencrypt-prod"
I0226 19:28:33.510519 1 controller.go:129] cert-manager/controller/clusterissuers "level"=0 "msg"="syncing item" "key"="letsencrypt-prod"
I0226 19:28:33.510996 1 setup.go:161] cert-manager/controller/clusterissuers "level"=0 "msg"="skipping re-verifying ACME account as cached registration details look sufficient" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod-key" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 19:28:33.512002 1 controller.go:135] cert-manager/controller/clusterissuers "level"=0 "msg"="finished processing work item" "key"="letsencrypt-prod"
I0226 19:28:33.689098 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="default/example-tls"
E0226 19:28:34.305985 1 pki.go:128] cert-manager/controller/certificates "msg"="error decoding x509 certificate" "error"="error decoding cert PEM block" "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "secret_key"="tls.crt"
I0226 19:28:34.306020 1 conditions.go:155] Setting lastTransitionTime for Certificate "example-tls" condition "Ready" to 2020-02-26 19:28:34.306016164 +0000 UTC m=+6081890.802309099
I0226 19:28:34.314767 1 controller.go:135] cert-manager/controller/certificates "level"=0 "msg"="finished processing work item" "key"="default/example-tls"
I0226 19:28:34.314803 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="default/example-tls"
I0226 19:28:34.315105 1 sync.go:361] cert-manager/controller/certificates "level"=0 "msg"="no existing CertificateRequest resource exists, creating new request..." "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default"
I0226 19:28:34.337317 1 sync.go:373] cert-manager/controller/certificates "level"=0 "msg"="created certificate request" "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "request_name"="example-tls-3355383384"
E0226 19:28:34.338431 1 pki.go:128] cert-manager/controller/certificates "msg"="error decoding x509 certificate" "error"="error decoding cert PEM block" "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "secret_key"="tls.crt"
I0226 19:28:34.338617 1 conditions.go:155] Setting lastTransitionTime for Certificate "example-tls" condition "Ready" to 2020-02-26 19:28:34.33861082 +0000 UTC m=+6081890.834903757
I0226 19:28:34.339270 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-vault "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.339369 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-ca "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.339604 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-tls-3355383384" condition "Ready" to 2020-02-26 19:28:34.339596714 +0000 UTC m=+6081890.835889670
I0226 19:28:34.339905 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-selfsigned "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.340089 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-tls-3355383384" condition "Ready" to 2020-02-26 19:28:34.340084454 +0000 UTC m=+6081890.836377378
I0226 19:28:34.340122 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-venafi "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.340485 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-tls-3355383384" condition "Ready" to 2020-02-26 19:28:34.340480478 +0000 UTC m=+6081890.836773397
I0226 19:28:34.340152 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-acme "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.341293 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-tls-3355383384" condition "Ready" to 2020-02-26 19:28:34.341288083 +0000 UTC m=+6081890.837581015
I0226 19:28:34.339630 1 conditions.go:200] Setting lastTransitionTime for CertificateRequest "example-tls-3355383384" condition "Ready" to 2020-02-26 19:28:34.339591879 +0000 UTC m=+6081890.835884796
E0226 19:28:34.361771 1 controller.go:131] cert-manager/controller/certificaterequests-issuer-venafi "msg"="re-queuing item due to error processing" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"example-tls-3355383384\": the object has been modified; please apply your changes to the latest version and try again" "key"="default/example-tls-3355383384"
I0226 19:28:34.361829 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-venafi "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.362171 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-venafi "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
E0226 19:28:34.362545 1 controller.go:131] cert-manager/controller/certificaterequests-issuer-selfsigned "msg"="re-queuing item due to error processing" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"example-tls-3355383384\": the object has been modified; please apply your changes to the latest version and try again" "key"="default/example-tls-3355383384"
I0226 19:28:34.362587 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-selfsigned "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.362744 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-selfsigned "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
E0226 19:28:34.363722 1 controller.go:131] cert-manager/controller/certificates "msg"="re-queuing item due to error processing" "error"="Operation cannot be fulfilled on certificates.cert-manager.io \"example-tls\": the object has been modified; please apply your changes to the latest version and try again" "key"="default/example-tls"
I0226 19:28:34.363765 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="default/example-tls"
I0226 19:28:34.364148 1 sync.go:379] cert-manager/controller/certificates "level"=0 "msg"="validating existing CSR data" "related_resource_kind"="CertificateRequest" "related_resource_name"="example-tls-3355383384" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default"
I0226 19:28:34.364427 1 sync.go:479] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest is not in a final state, waiting until CertificateRequest is complete" "related_resource_kind"="CertificateRequest" "related_resource_name"="example-tls-3355383384" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "state"="Pending"
E0226 19:28:34.364645 1 pki.go:128] cert-manager/controller/certificates "msg"="error decoding x509 certificate" "error"="error decoding cert PEM block" "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "secret_key"="tls.crt"
E0226 19:28:34.365169 1 controller.go:131] cert-manager/controller/certificaterequests-issuer-vault "msg"="re-queuing item due to error processing" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"example-tls-3355383384\": the object has been modified; please apply your changes to the latest version and try again" "key"="default/example-tls-3355383384"
I0226 19:28:34.365210 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-vault "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.365402 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-vault "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:34.365624 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-ca "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:34.365663 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-ca "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
E0226 19:28:34.366005 1 controller.go:131] cert-manager/controller/certificaterequests-issuer-acme "msg"="re-queuing item due to error processing" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"example-tls-3355383384\": the object has been modified; please apply your changes to the latest version and try again" "key"="default/example-tls-3355383384"
I0226 19:28:34.366156 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-ca "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:34.366217 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-acme "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:34.366416 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-acme "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:34.380457 1 controller.go:135] cert-manager/controller/certificates "level"=0 "msg"="finished processing work item" "key"="default/example-tls"
I0226 19:28:34.380509 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="default/example-tls"
I0226 19:28:34.380988 1 sync.go:379] cert-manager/controller/certificates "level"=0 "msg"="validating existing CSR data" "related_resource_kind"="CertificateRequest" "related_resource_name"="example-tls-3355383384" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default"
I0226 19:28:34.381222 1 sync.go:479] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest is not in a final state, waiting until CertificateRequest is complete" "related_resource_kind"="CertificateRequest" "related_resource_name"="example-tls-3355383384" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "state"="Pending"
E0226 19:28:34.381431 1 pki.go:128] cert-manager/controller/certificates "msg"="error decoding x509 certificate" "error"="error decoding cert PEM block" "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "secret_key"="tls.crt"
I0226 19:28:34.381679 1 controller.go:135] cert-manager/controller/certificates "level"=0 "msg"="finished processing work item" "key"="default/example-tls"
I0226 19:28:38.003631 1 controller.go:129] cert-manager/controller/clusterissuers "level"=0 "msg"="syncing item" "key"="letsencrypt-prod"
I0226 19:28:38.003935 1 setup.go:161] cert-manager/controller/clusterissuers "level"=0 "msg"="skipping re-verifying ACME account as cached registration details look sufficient" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod-key" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 19:28:38.004082 1 controller.go:135] cert-manager/controller/clusterissuers "level"=0 "msg"="finished processing work item" "key"="letsencrypt-prod"
I0226 19:28:39.362069 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-venafi "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:39.362358 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-venafi "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:39.362732 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-selfsigned "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:39.362895 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-selfsigned "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:39.363917 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="default/example-tls"
I0226 19:28:39.364256 1 sync.go:379] cert-manager/controller/certificates "level"=0 "msg"="validating existing CSR data" "related_resource_kind"="CertificateRequest" "related_resource_name"="example-tls-3355383384" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default"
I0226 19:28:39.364992 1 sync.go:479] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest is not in a final state, waiting until CertificateRequest is complete" "related_resource_kind"="CertificateRequest" "related_resource_name"="example-tls-3355383384" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "state"="Pending"
E0226 19:28:39.365241 1 pki.go:128] cert-manager/controller/certificates "msg"="error decoding x509 certificate" "error"="error decoding cert PEM block" "related_resource_kind"="Secret" "related_resource_name"="example-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="example-tls" "resource_namespace"="default" "secret_key"="tls.crt"
I0226 19:28:39.365404 1 controller.go:135] cert-manager/controller/certificates "level"=0 "msg"="finished processing work item" "key"="default/example-tls"
I0226 19:28:39.365457 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-vault "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:39.365595 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-vault "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 19:28:39.366141 1 controller.go:129] cert-manager/controller/certificaterequests-issuer-acme "level"=0 "msg"="syncing item" "key"="default/example-tls-3355383384"
I0226 19:28:39.366255 1 controller.go:135] cert-manager/controller/certificaterequests-issuer-acme "level"=0 "msg"="finished processing work item" "key"="default/example-tls-3355383384"
I0226 21:06:24.117890 1 controller.go:129] cert-manager/controller/ingress-shim "level"=0 "msg"="syncing item" "key"="default/external-ingress"
E0226 21:06:24.118633 1 sync.go:57] cert-manager/controller/ingress-shim "msg"="failed to determine issuer to be used for ingress resource" "error"="failed to determine issuer name to be used for ingress resource" "resource_kind"="Ingress" "resource_name"="external-ingress" "resource_namespace"="default"
I0226 21:06:24.118876 1 controller.go:135] cert-manager/controller/ingress-shim "level"=0 "msg"="finished processing work item" "key"="default/external-ingress"
I0226 21:15:27.660117 1 controller.go:129] cert-manager/controller/clusterissuers "level"=0 "msg"="syncing item" "key"="letsencrypt-prod"
I0226 21:15:27.660248 1 setup.go:86] cert-manager/controller/clusterissuers "level"=0 "msg"="generating acme account private key" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 21:15:28.153028 1 setup.go:167] cert-manager/controller/clusterissuers "level"=0 "msg"="ACME server URL host and ACME private key registration host differ. Re-checking ACME account registration" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 21:15:28.153059 1 logger.go:88] Calling GetAccount
I0226 21:15:28.153331 1 controller.go:129] cert-manager/controller/webhook-bootstrap "level"=0 "msg"="syncing item" "key"="cert-manager/letsencrypt-prod"
I0226 21:15:28.153497 1 controller.go:135] cert-manager/controller/webhook-bootstrap "level"=0 "msg"="finished processing work item" "key"="cert-manager/letsencrypt-prod"
I0226 21:15:28.413415 1 logger.go:83] Calling CreateAccount
I0226 21:15:28.469758 1 setup.go:229] cert-manager/controller/clusterissuers "level"=0 "msg"="verified existing registration with ACME server" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 21:15:28.475847 1 controller.go:135] cert-manager/controller/clusterissuers "level"=0 "msg"="finished processing work item" "key"="letsencrypt-prod"
I0226 21:15:28.476076 1 controller.go:129] cert-manager/controller/clusterissuers "level"=0 "msg"="syncing item" "key"="letsencrypt-prod"
I0226 21:15:28.476426 1 setup.go:161] cert-manager/controller/clusterissuers "level"=0 "msg"="skipping re-verifying ACME account as cached registration details look sufficient" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 21:15:28.476584 1 controller.go:135] cert-manager/controller/clusterissuers "level"=0 "msg"="finished processing work item" "key"="letsencrypt-prod"
I0226 21:15:33.153209 1 controller.go:129] cert-manager/controller/clusterissuers "level"=0 "msg"="syncing item" "key"="letsencrypt-prod"
I0226 21:15:33.153499 1 setup.go:161] cert-manager/controller/clusterissuers "level"=0 "msg"="skipping re-verifying ACME account as cached registration details look sufficient" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-prod" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-prod" "resource_namespace"=""
I0226 21:15:33.153537 1 controller.go:135] cert-manager/controller/clusterissuers "level"=0 "msg"="finished processing work item" "key"="letsencrypt-prod"
