Better K8s Monitoring Part 1: Adding Prometheus

Adding better monitoring for applications running in my k8s cluster using Prometheus.


10 minute read

It’s been a while since I learned of the wonders (and cleared up my misconceptions) of dedicated hosting and set up a “Baremetal” CoreOS single-node k8s cluster. For a while now I’ve maintained a single large (by my standards) machine that has been running Kubernetes, and purring right along – outside of the occasional restart or operator error, it hasn’t gone down and has kept my applications running. While most of the applications don’t get much…

Fun with Gitlab CI

Some fun infrastructure-related things you can do with Gitlab CI


11 minute read

tl;dr If you/your team aren’t on to the wonders of CI yet, you should check it out. A nice easy way to get started is with Gitlab, which is self-hostable and has a free tier @ There is a lot of cool stuff you can do automatically with CI, all you need is some scripting chops and some patience to figure out what works and what doesn’t.

Ansible Is Awesome

Ansible is just the tool I was looking for, for my current level of ops mastery.


14 minute read

tl;dr - There are lots of ways to get smarter about how you deploy. Ansible is one choice, it’s not the fanciest, but it’s amazing.

Quick And Dirty Forward-Only DB Migrations

Quick and dirty forward-only migrations, in just a few lines of make & bash


5 minute read

While working on the The Start (a now defunct project)’s jobs backend, I found myself at the age-old problem of how to migrate my database (which is surprisingly SQLite). I looked at tools like flyway, migrate, but they all kind of seemed like more than I needed. If anything I was leaning towards migrate, but flyway is a very well-known and often used solution that it was also pretty appealing. In the end, I decided to just write a quick makefile target (basically a bash script) for a few…

Research Idea: Maintaining A Write Ahead Log For Application Runtime State

Exploring the stepladder of features that lead to well-engineered backends


13 minute read

tl;dr:What if there was a write-ahead-logging layer above/outside the database? Then you could get CQRS+ES capabilities and assurnaces from your database without rewriting, and even abstract out the database as a whole, going as far as to create your own DSL for application-level changes, ADD_USER instead of INSERT ... INTO USERS ...*

Stop Worrying And Use Gitlab

Worried you (or your org) are not 'doing it right'? Get (or at least start following practices used by) Gitlab


6 minute read

tldr;instead of now installing/using JIRA, Trello, Github, Jenkins/TravisCI, and Teamcity, I can just install one thing – Gitlab, spend a little time configuring it, and be ready to take your development process to the next 5 levels (if you’re not already there)

LetsEncrypt Systemd Recipes

Some useful SystemD unit files


2 minute read

If you’re unfamiliar with Let’s Encrypt, it’s a project (I believe originally sponsored by the EFF) that creates a first of it’s kind free automated and open certificate authority. This means administrators who run websites can get free access to SSL certificates. In the past I’ve had to go to sites like StartSSL or purchase a certificate from my hosting provider (and of course, some still do), but Let’s Encrypt has been wonderful for me (I highly recommend…

Using dockerized Ghost with local SMTP

Getting Ghost (the open source publishing platform) working with local SMTP


2 minute read

When handling mail for a ghost instance, the official recommendation of the Ghost team is to use Mailgun. Since I have email set up on the server on which I’m running Ghost, despite the fact that Mailgun offers a pretty good free tier of services, it seems pretty extraneous/unnecessary to use mailgun just to send email from my own server. Of course, this ignores the effort that it takes to set up things like postfix, dovecot, and opendkim, but I digress.

Issues with Thunderbird and Dovecot

Issues encountered setting up dovecot, with proper TLS & thunderbird support


4 minute read

Recently while doing some server-administration tasks (setting up the server, enabling email, TLS, etc) on a Ubuntu 14.04 system, I ran into some problems with dovecot. For some reason, no matter what I tried, I could not get Thunderbird to auto-configure properly with the newly stood up web server’s SMTP server (postfix + dovecot).