Awesome FOSS Logo
Discover awesome open source software
Launched 🚀🧑‍🚀

cert-manager moved from Bazel to Make

cert manager logo

tl;dr - cert-manager and moved from Bazel to Make (i.e. Makefile) based build in 1.8.0. I’m into Makefiles (+ kustomize) for builds and deploys so I’m taking an undeserved victory lap.

A welcome surprise during a long round of updates

Recently I’ve been upgrading some of the critical services in my cluster and like everyone else, cert-manager is one of the best parts of my cluster.

The cert-manager maintainers and contributors have saved what I can only imagine is millions of person years at this point through great effort, silently and without fanfare. Shout out to Venafi as well for acquiring Jetstack and being great stewards of cert-manager so far.

Anyway, as I was reading lots of release notes (I was quite a few versions back), I noticed that v1.8.0 included a move to Make from Bazel. The reasoning is reproduced below (thanks for documenting your thoughts, cert-manager maintainers/contributors!):

From Bazel to Make

A common theme when someone tries to make a change to cert-manager for the first time is that they ask for help with navigating Bazel, which cert-manager used as its build tool. Helping people with Bazel isn’t easy; it’s an incredibly powerful tool, but that power also brings a lot of complications which can seriously get in the way of being able to make even simple changes to the code base. Even developers who are familiar with contributing to open source projects in Go can find it daunting to make changes thanks to Bazel.

The problem isn’t limited to open-source contributors; many of cert-manager’s maintainers also struggle with configuring and changing Bazel, too.

cert-manager 1.8 is the first release which is built and tested using a newly written make-based build system. We believe that this new build system should make it much simpler to understand and change the commands which are being run behind the scenes to build and test cert-manager. In time, we’ll fully document the new build system, ensure it’s at full feature-parity with Bazel and then remove all references to Bazel across the codebase.

A neat side effect of this change is that our build times have significantly improved. Bazel took around 14 minutes to build every cert-manager artifact for every platform during a release, while the new make build system can do the same (and more) in under 5 minutes.

If you’re a regular reader, you know I’m into Makefiles + kustomize for deploys to k8s, so I was happy to see this!

make and Makefiles are underrated tooling for projects today (whether development or infra), and I’m glad to see them used more! Now I’ll just patiently wait for them to remove Helm… Just kidding – the important drawback with kustomize is that they don’t do variable substitution (and don’t plan to).

For variable substitution, I use envsubst in a pinch, but most of the time I make sure I build my apps in a way that doesn’t require variable substitution at the resource level!.

One concrete example of how you might remove the need for variable templating is by using secrets (with the same name for each environment, but differing content) – your different evnironments should be in different namespaces, so there should be no risk of mixing!

Anyway, I wanted to take a victory lap since I see this as validation of my rambling – Hopefully more people move to Make across projects.

Sidebar: This isn’t meant to start a flamewar

This post does not mean to disparage Bazel or imply that Bazel is bad in any way! I don’t even mean this post to be clickbaity or provocative which I do from time to time.

Bazel is awesome, Make is awesome – nothing to see there.

Sidebar: Cool Makefile tricks

While you’re here, here’s some cool make tips and tricks I’ve honed over the last few projects.

Ensuring ENV vars and binaries are present

AKA the poor man’s configure – here’s how one might check for and ensure some ENV vars and tools are present:

.PHONY: don't forget to put your non-file/folder based makefile targets here!

DOCKER ?= docker
KUBECTL ?= kubectl
BASE64 ?= base64
ENVSUBST ?= envsubst

PROJECT_NAME ?= waaard
ENVIRONMENT ?= staging


all: secrets registry-login deploy

ifeq (,$(ENVIRONMENT))
	$(error "ENVIRONMENT not specified")

ifeq (,$(shell which $(BASE64)))
	$(error "base64 binary must be installed")

clean: clean-generated-secrets

Per-project (folder) docker credentials

Here’s how you might do per-project (folder) docker credentials (nice and easy per-project creds for use with docker login, docker push, etc):

## EXAMPLE: Per-project (folder) docker credentials
## (this is best used with direnv and git-crypt!)



RSS_API_REGISTRY_PASSWORD_SECRET_PATH ?= ./secrets/gitlab/rss-api/deploy-tokens/registry-password.secret
RSS_API_REGISTRY_USERNAME_SECRET_PATH ?= ./secrets/gitlab/rss-api/deploy-tokens/registry-username.secret


registry-login: secret-folders registry-folders registry-rss-api-login

	@if [ ! -f "$(RSS_API_REGISTRY_USERNAME_SECRET_PATH)" ] ; then \
		echo -e "\n[info] Please enter a username for pulling the rss-api docker container:"; \
			echo "[info] registry username to => [$$REGISTRY_USERNAME]" && \
			echo "[info] Writing registry username to relevant secret files..." && \

	@if [ ! -f "$(RSS_API_REGISTRY_PASSWORD_SECRET_PATH)" ] ; then \
		echo -e "\n[info] Please enter a password for pulling the rss-api docker container:"; \
			echo "[info] registry password to => [$$REGISTRY_PASSWORD]" && \
			echo "[info] Writing registry password to relevant secret files..." && \

	@if [ -f "$(RSS_API_REGISTRY_DOCKER_JSON_PATH)" ] ; then \
		echo "[info] file @ [$(RSS_API_REGISTRY_DOCKER_JSON_PATH)] already exists, skipping..."; \
	else \
		echo "\n[info] logging into docker..."; \
				| $(DOCKER) login $(CONTAINER_REGISTRY) -u $$DOCKER_USERNAME --password-stdin; \


Random-OK secret generation

Here’s how you might generate some secrets that are OK to randomize:

## EXAMPLE: Generating random secrets

## Random secret generation!
	@cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1 | tr -d \\n

## Files that can be randomly generated
	secrets/pulumi/password-encryption.secret \
	secrets/$(ENVIRONMENT)/db/users.readysetsaas.PASSWORD.secret \

	@echo -e "[info] Removing generated random-ok secrets..."
			[ ! -f $$SECRET_PATH ] && echo -e "[info] file @ [$$SECRET_PATH] does not exist" && continue; \
			echo -e "[info] removing random secret @ [$$SECRET_PATH]..."; \
			rm -rf $$SECRET_PATH; \

secrets-generated: secret-folders
	@echo -e "[info] Generating known random-ok secrets..."
			[ -f $$SECRET_PATH ] && echo -e "[info] file @ [$$SECRET_PATH] already exists" && continue; \
			echo -e "[info] Generating random secret @ [$$SECRET_PATH]..."; \
			$(MAKE) -s --no-print-directory secret-gen-random-alpha > $$SECRET_PATH; \

Manual secret prompt and save

Here’s how you might enable people to enter manual secrets while setting up the project (or a new environment) for the first time:

## Example: Manual secrets that a user needs to enter

secrets-manual: secret-folders check-environment

# Administrator email
	@if [ ! -f $(SECRET_ADMIN_EMAIL_ADDRESS_PATH) ] ; then \
		echo -e "\n[info] Please enter the enter the email address of the administrator:"; \
		read ADMIN_EMAIL && \
			echo "[info] Administrator email set to => [$$ADMIN_EMAIL]" && \
			echo "[info] Writing email to relevant secret files..." && \
			echo $$ADMIN_EMAIL | tr -d '[:space:]' > $(SECRET_ADMIN_EMAIL_ADDRESS_PATH) && \
			echo $$ADMIN_EMAIL | tr -d '[:space:]' > secrets/$(ENVIRONMENT)/analytics/FATHOM_ROOT_USER_NAME.secret && \
			echo $$ADMIN_EMAIL | tr -d '[:space:]' > secrets/$(ENVIRONMENT)/rss-api/ROOT_ADMIN_EMAIL_ADDRESS.secret; \

DKIM key generation

These days the cool kids (me?) generate their DKIM secrets and feed them to SES instead of the other way around. Funnily enough, this is how it was done in the old days. Time is a flat circle, etc.

Here’s how you might generate some DKIM secrets.

## Example: generating DKIM secrets

	@echo -e "\n[info] generating DKIM keys for use with email..."

	@if [ ! -f "$(DKIM_PRIVATE_KEY_PATH)" ] ; then \
		echo "[info] DKIM private key not found @  [$(DKIM_PRIVATE_KEY_PATH)], creating new key..."; \
		$(OPENSSL) genrsa -f4 -out $(DKIM_PRIVATE_KEY_PATH); \
	@echo -e "[info] private key @ [$(DKIM_PRIVATE_KEY_PATH)]"

	@if [ ! -f "$(DKIM_PUBLIC_KEY_PATH)" ] ; then \
		echo "[info] DKIM public key not found @  [$(DKIM_PUBLIC_KEY_PATH)], creating new key..."; \
		$(OPENSSL) rsa -in $(DKIM_PRIVATE_KEY_PATH) -outform PEM -pubout -out $(DKIM_PUBLIC_KEY_PATH); \
	@echo -e "[info] public key @ [$(DKIM_PUBLIC_KEY_PATH)]"

That’s it for now! I should put out all these tricks someday (maybe compile them all in some sort of Makefile tip site?), but for now I’ll leave this for the curious out there.