tl;dr - letsencrypt is awesome, ployst/docker-letsencrypt makes it easy to use with Kubernetes (feel free to check out the blog post that describes it). There are even easier ways to do it these days that I haven’t tried:
kube-lego which looks pretty amazing.
After going through figuring out how to run HTTP applications on Kubernetes, as well as how to run databases on Kubernetes, the next natural step is to figure out how to gear up to running HTTPS applications on Kubernetes. The first step in that, of course, is getting letsencrypt working.
Normally the non-kubernetes process of using letsencrypt to enable HTTPS apps looks like this for me:
certbotstandalone with the appropriate options to obtain a cert for all domain names I need
Kuberentes obviously changes this flow a little bit, but it’s not too bad with the help of
ployst/docker-letsencrypt. After reading the documentation in the README, along with a very helpful blog post by the author, the steps look to be:
kubectl createthe given resource configuration
kubectl execto run the commands noted in the ployst/docker-letsencrypt README (
One thing that will be a little different from what I can see on the README is that I have Kubernetes ingress set up and working, I think I’ll also have to create an ingress resource to help out with the ACME DNS challenge for letsencrypt’s validations step.
Welp, with this general idea of what I need to do, let’s jump into my notes on what actually happened.
After a bit of experimenting with the example from the blog post, I figured out what the appropriate resource configuration should look like, with lots of help from the README (I had to remove the
role annotations, they’re not a thing anymore evidently):
--- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: letsencrypt-helper-ing annotationsn: ingress.kubernetes.io/class: "nginx" ingress.kubernetes.io/ssl-redirect: "false" ingress.kubernetes.io/limit-rps: "20" spec: rules: - http: paths: - path: /.well-known/acme-challenge backend: serviceName: letsencrypt-helper-svc servicePort: 80 --- kind: Service apiVersion: v1 metadata: name: letsencrypt-helper-svc spec: type: LoadBalancer selector: app: letsencrypt-helper ports: - name: http protocol: TCP port: 80 --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: letsencrypt-helper-deployment labels: name: letsencrypt-helper spec: replicas: 1 selector: matchLabels: app: letsencrypt-helper template: metadata: name: letsencrypt labels: name: letsencrypt app: letsencrypt-helper spec: containers: - name: letsencrypt image: ployst/letsencrypt:0.3.0 env: - name: EMAIL value: <your email> - name: DOMAINS value: <adddress 1> <address 2> <...> - name: SECRET_NAME value: letsencrypt-certs-all - name: DEPLOYMENTS value: <deployment 1> <deployment 2> <...> ports: - name: ssl-proxy-http containerPort: 80
So there’s a bit to unpack there, let’s go top to bottom – I’ll try and explain what exactly is happening so it doesn’t seem so big and scary.
First we have the letsencrypt-helper Ingress Resource – it’s job is to make the letsencrypt helper available to the outside world, in particular exposing the
/.well-known/acme-challenge path that letsencrypt will be using to do domain validation.
Next, is the letsencrypt-helper Service and the letsencrypt-helper Deployment for – they’re your normal garden variety Service and Deployment, ensuring that the helper is available and accessible.
You can use the usual commands to ensure that everything is working –
kubectl get ing (list the ingresses),
kubectl get deployments (list the deployments), etc. Checking the logs with a command like
kubectl logs letsencrypt-helper-deployment-<random string> also was very helpful, it will show requests hitting NGINX if you happen to visit the page in your browser (@
I was able to successfully
kubectl create this resource configuration so the next step was to run the commands from the README and obtain the certificates.
Just as the docs suggest, the first command I run is
kubectl exec -it letsencrypt-helper-deployment-<random string> -- bash -c './fetch_certs.sh'. Since I’ve already specified the domains through the ENV, the command is nice and simple. After running this command you can go through and run the
save_certs.sh command as well to save them to the secret.
And just like that, the certificates are fetched and available! You can check out the Kubernetes Dashboard if you have it installed or do a
kubectl get secrets to confirm that a secret with the name
SECRET_NAME that you specified earlier was made.
If you run multiple apps with different domains through your ingress, you need to direct the ACME DNS challenge URL to
letsencrypt-helper as well. Here’s an example (sneak peak of the next post on running HTTPS applications):
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: example-ing annotations: ingress.kubernetes.io/class: "nginx" ingress.kubernetes.io/ssl-redirect: "true" ingress.kubernetes.io/limit-rps: "20" spec: tls: - hosts: - "example.com" secretName: letsencrypt-certs-all rules: - host: "example.com" http: paths: - path: "/.well-known/acme-challenge" backend: serviceName: letsencrypt-helper-svc servicePort: 80 - path: "/" backend: serviceName: example-svc servicePort: 80 - path: "/api" backend: serviceName: example-svc servicePort: 4000 - host: "getexample.com" http: paths: - path: "/.well-known/acme-challenge" backend: serviceName: letsencrypt-helper-svc servicePort: 80 - path: "/" backend: serviceName: example-svc servicePort: 80 - path: "/api" backend: serviceName: example-svc servicePort: 4000
This is what the configuration looks like for an Ingress with HTTPS enabled. Note the
tls group of options – the Ingress documentation is pretty straigh forward so it was easy to write this.
Here’s what the secret looks like in Kubernetes Dashboard once everythings set up and done:
So this is a fairly manual way to retrieve the certificates – note that I’m actually running
exec comamnds here.
After getting it set up this way, I came across this fully automated way of obtaining the certificates, called
kube-lego. I haven’t tried it yet, but it might absolutely be worth looking to if you’re approaching this for the first time and want a less manual way of getting all this done.
After going through this, I was pretty happy to have a pretty simple process for getting the certs though it’s a bit manual for my liking. Now it’s time to port some of my applications to that require HTTPS (basically all of them) – stay tuned for the next post, in which I’ll go through what it took to set up a HTTPS-enabled app (we’re already 80% there, being that regular HTTP apps were covered, and I just showed you what the ingress resource looks like).