DDG search drop in powered by ddg.patdryburgh.com

Awesome FOSS Logo
Discover awesome open source software
Launched 🚀🧑‍🚀
Sun, Apr 10, 2022 by vados

Thunderbird Autoconfig for your self-hosted email server with MTA-STS

Categories
email devops sysadmin thunderbird tools howto guide
email icon + thunderbird retro logo

tl;dr - Host some Thunderbird autoconfig formatted XML, and text files at the right places (thanks to MTA-STS) so clients can have easy-peasy email setup. This was a fun chance to flex my Traefik installation, and kcup.

You’re hosting your own email server right? Cool, me too. Good thing we ignored all the naysayers out there.

In the past I’ve run postfix + dovecot setups and now I run many small maddy instances (for various projects), but I never looked into autoconfig. I’ve looked into it and set it up, and now you can too (it’s not like anything was stopping you before, maybe stop being so lazy?).

What are you on about?

For those who have no idea what I’m talking about, we’re trying to make this particular screen of Thunderbird go by faster:

thunderbird initial setup

If you don’t have autoconfig set up, you may have to click that “configure manually” button link:

thunderbird manual setup

We want to make sure people don’t have to “configure manually”, unless they’re into that sort of thing.

RTFM

As always, the first step is to give a good faith attempt to RTFM. How is this magical autoconfiguration supposed to work? Is there a standard?

A quick search turns up the Mozilla Wiki page on Thunderbird’s Autoconfiguration.

Internet Archive saved the day, so I donated

If you try and follow the links on that page you’ll be met with 404s @ Mozilla Developer Network (MDN), so instead here are some links to the internet archive:

It’s amazing that I could put in those links and be able to find an older version of the page cached the Internet Archive is doing God’s work. I made a donation to them and you should too:

weird flex, but ok

Back on task: What we actually need to do

Anyway, getting back on track there are a few options listed for how to get autoconfig working:

  • Add an entry to the ISPDB (you need to file a bug in mozilla’s bug tracker in a specific place)
  • Make some XML available @ autoconfig.yourdomain.tld/mail/config-v1.1.xml?emailaddress=<some email>
  • Make some XML available @ yourdomain.tld/.well-known/autoconfig/mail/config-v1.1.xml

I’m going to avoid the ISPDB option here (cool that it’s there though), and go for the second option. The third one is probably the most reasonable option, but that’s not what we’re all here for – it’d be too easy. Making the query path stuff work will give me a good chance to flex my Traefik muscles – I’ll do both 2 and 3.

I’ve been working on a project I launched recently called Waaard lately, so I’ll get it working on that (the operative domain will be waaard.com). Some emails like hello@waaard.com will be expected to work properly.

Just to summarize, to get it working, we are going to:

  • Write the XML file
  • Make sure the XML file is available at [http|https]://autoconfig.example.com/mail/config-v1.1.xml?emailaddress=<some email address>
  • Make sure the XML file is available at [http|https]://example.com/.well-known/autoconfig/mail/config-v1.1.xml

Step 1: Create the XML file

You can do this by copying the resources from before:

Here’s the example “real-world” example XML file they provided (I modified this to get to my setup):

<?xml version="1.0" encoding="UTF-8"?>

<clientConfig version="1.1">
  <emailProvider id="freenet.de">
    <domain>freenet.de</domain>
    <displayName>Freenet Mail</displayName>
    <displayShortName>Freenet</displayShortName>
    <incomingServer type="imap">
      <hostname>imap.freenet.de</hostname>
      <port>993</port>
      <socketType>SSL</socketType>
      <authentication>password-encrypted</authentication>
      <username>%EMAILADDRESS%</username>
    </incomingServer>
    <incomingServer type="imap">
      <hostname>imap.freenet.de</hostname>
      <port>143</port>
      <socketType>STARTTLS</socketType>
      <authentication>password-encrypted</authentication>
      <username>%EMAILADDRESS%</username>
    </incomingServer>
    <incomingServer type="pop3">
      <hostname>pop.freenet.de</hostname>
      <port>995</port>
      <socketType>SSL</socketType>
      <authentication>password-cleartext</authentication>
      <username>%EMAILADDRESS%</username>
    </incomingServer>
    <incomingServer type="pop3">
      <hostname>pop.freenet.de</hostname>
      <port>110</port>
      <socketType>STARTTLS</socketType>
      <authentication>password-cleartext</authentication>
      <username>%EMAILADDRESS%</username>
    </incomingServer>
    <outgoingServer type="smtp">
      <hostname>smtp.freenet.de</hostname>
      <port>465</port>
      <socketType>SSL</socketType>
      <authentication>password-encrypted</authentication>
      <username>%EMAILADDRESS%</username>
    </outgoingServer>
    <outgoingServer type="smtp">
      <hostname>smtp.freenet.de</hostname>
      <port>587</port>
      <socketType>STARTTLS</socketType>
      <authentication>password-encrypted</authentication>
      <username>%EMAILADDRESS%</username>
    </outgoingServer>
    <documentation url="https://web.archive.org/web/20210116164029/http://kundenservice.freenet.de/hilfe/email/programme/config/index.html">
      <descr lang="de">Allgemeine Beschreibung der Einstellungen</descr>
      <descr lang="en">Generic settings page</descr>
    </documentation>
    <documentation url="https://web.archive.org/web/20210116164029/http://kundenservice.freenet.de/hilfe/email/programme/config/thunderbird/imap-thunderbird/imap/index.html">
      <descr lang="de">TB 2.0 IMAP-Einstellungen</descr>
      <descr lang="en">TB 2.0 IMAP settings</descr>
    </documentation>
  </emailProvider>
</clientConfig>

Obviously, you’re going to want to replace the freenet.de parts, unless you’re from freenet.de…

NOTE: A more thorough walk-through of the options is in the other link.

Step 2: Host the file on your server

For those with simple setups

Hopefully most of you have reasonably simple hosting setups that make sense and you should be able to update some lines in Caddy or NGINX configurations and you’ll be done.

Please skip this section if you’ve got a nice reasonable setup like that, because what follows will look like chaos if you haven’t yet sipped the kubernetes kool-aid.

For the yak shavers with needlessly complicated setups (me)

For the yak shavers in the house hosting their own compute with over-engineered setups, here’s how you do it on your needlessly multi-node HA k8s cluster.

kcup Deployment

You’ll need something to serve just one file – I’m going to use the small binary I wrote called kcup. I’ve already got a maddy deployment so I’m nestling it in there:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: maddy
spec:
  revisionHistoryLimit: 2
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: waaard
      component: maddy
      tier: email
  template:
    metadata:
      labels:
        app: waaard
        component: maddy
        tier: email
    spec:
      containers:
        # MTA-STS (https://maddy.email/tutorials/setting-up/#mta-sts)
        - name: kcup-mta-sts
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 3000
          readinessProbe:
            tcpSocket:
              port: 3000
            initialDelaySeconds: 5
            periodSeconds: 2
          livenessProbe:
            tcpSocket:
              port: 3000
            initialDelaySeconds: 15
            periodSeconds: 2
          env:
            - name: HOST
              value: "0.0.0.0"
            - name: PORT
              value: "3000"
            - name: FILE
              value: /etc/maddy/mta-sts.txt
          volumeMounts:
            - name: maddy-config
              mountPath: /etc/maddy

        # Thunderbird Autoconfig
        - name: kcup-thunderbird-autoconfig
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 3001
          readinessProbe:
            tcpSocket:
              port: 3001
            initialDelaySeconds: 5
            periodSeconds: 2
          livenessProbe:
            tcpSocket:
              port: 3001
            initialDelaySeconds: 15
            periodSeconds: 2
          env:
            - name: HOST
              value: "0.0.0.0"
            - name: PORT
              value: "3001"
            - name: FILE
              value: /etc/maddy/autoconfig.xml
          volumeMounts:
            - name: maddy-config
              mountPath: /etc/maddy

        # Maddy
        - name: maddy
          # 0.5.2 as of 11/18/2021
          image: foxcpp/maddy@sha256:8fa2bd8f683011501c059e866d19f7f602658a7198a6ba313d5fede42690a224
          imagePullPolicy: IfNotPresent
          # args:
          #   - -debug
          ports:
            # SMTP
            - containerPort: 2525
            - containerPort: 465
            - containerPort: 587
            # IMAP
            - containerPort: 143
            - containerPort: 993
            # Monitoring
            - containerPort: 9749
          readinessProbe:
            tcpSocket:
              port: 2525
            initialDelaySeconds: 5
            periodSeconds: 2
          livenessProbe:
            tcpSocket:
              port: 2525
            initialDelaySeconds: 15
            periodSeconds: 2
          volumeMounts:
            # TLS
            - name: maddy-tls
              mountPath: /data/tls
            # Config
            - name: maddy-config
              mountPath: /data/maddy.conf
              subPath: maddy.conf
            - name: maddy-config
              mountPath: /etc/maddy/aliases
              subPath: aliases
            # Data
            - name: maddy-data
              mountPath: /data

      volumes:
        - name: maddy-config
          configMap:
            name: maddy-config
        - name: maddy-tls
          secret:
            secretName: web-tls
        - name: maddy-data
          persistentVolumeClaim:
            claimName: maddy-data
        - name: maddy-dkim
          secret:
            secretName: maddy-dkim

(NOTE: I use kustomize so not all the details are in here, for example resource requests and limits!)

HTTPS Ingress

You’re hopefully using my personal recommendation for best Ingress, Traefik, so you can use an IngressRoute (unless you’re using the Ingress or k8s Gateway API integrations), and you’re probably running cert-manager as well to manage your certs, so you’ll need a Certificate first:

---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: web
spec:
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  secretName: web-tls
  commonName: waaard.com
  dnsNames:
    # ... other endpoints ...
    - autoconfig.waaard.com

I won’t get into it here, but managing your Certificate objects manually is often better than having them created automatically – I do this all the time now.

And you’ll want of course the actual IngressRoute:

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: maddy
spec:
  tls:
    secretName: web-tls
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`mta-sts.waaard.com`)
      services:
        - name: maddy
          port: 3000

    - kind: Rule
      match: Host(`autoconfig.waaard.com`) && Path(`/mail/config-v1.1.xml`) && Query(`emailaddress=admin@waaard.com`)
      services:
        - name: maddy
          port: 3001

    - kind: Rule
      match: Host(`waaard.com`) && Path(`/.well-known/autoconfig/mail/config-v1.1.xml`)
      services:
        - name: maddy
          port: 3001

What about HTTP?

So the spec says to support HTTP as well but I’m not going to bother. I get the feeling most good mail clients will be giving HTTPS a try where possible. If you’re running an important mail client/server and disagree, email me and let me know how wrong I am.

…And we’re done

Here’s what it looks like:

Wrapup

Well that was easy! Hope this article helps people out there who may be wanting this last piece of their mail hosting puzzle.