Self-Hosting Email, Step by Step: Postfix, Dovecot, and OpenDKIM
A complete, explained walkthrough of installing and configuring a mail server from a bare Ubuntu VPS — including the TLS-permissions gotcha that catches almost everyone the first time.
In the first post in this series we covered what Postfix, Dovecot, and OpenDKIM each actually do. This post is the hands-on part: installing and configuring all three on a bare Ubuntu server, with every step explained — not just the commands to copy, but why each one matters.
Everything below assumes a fresh Ubuntu VPS with root or sudo access, and a domain you control DNS for. Replace yourdomain.com with your actual domain throughout.
1. Set the server's mail hostname first
Mail servers identify themselves by hostname during every SMTP conversation, and receiving servers use that hostname as one signal when deciding whether to trust you. Set it to a real, DNS-resolvable name before installing anything:
sudo hostnamectl set-hostname mail.yourdomain.com
Then add two DNS records: an A record pointing that hostname at your server's IP address, and an MX record on your bare domain pointing at it:
mail.yourdomain.com. A YOUR_SERVER_IP
yourdomain.com. MX 10 mail.yourdomain.com.
Here's a mistake worth learning from before you make it yourself: when adding that MX record in your DNS provider's dashboard, the "Name" or "Host" field needs to be @ (meaning "the root domain itself"), not the word "mail". If you set the Name to "mail", you've actually created an MX record for the subdomain mail.yourdomain.com, which does nothing for mail addressed to you@yourdomain.com — the domain people actually email. The bare domain would still have no valid MX record at all, and mail sent to it would bounce or fail silently. You can always verify what's actually live with:
dig +short MX yourdomain.com
If that command returns nothing, your MX record isn't set up correctly yet — go back and check the Name field.
2. Install the packages
apt is Ubuntu's package manager — the tool that downloads and installs software from a trusted repository. apt update refreshes its list of what's available; apt install then installs the packages you name:
sudo apt update
sudo apt install -y postfix dovecot-core dovecot-imapd opendkim opendkim-tools
Postfix's installer will interrupt this process to ask you to choose a configuration type. Pick "Internet Site" and give it your mail hostname when prompted.
3. Configure Postfix's basics
Postfix's main configuration file is /etc/postfix/main.cf. The settings that matter most to get right early:
myhostname = mail.yourdomain.com
mydestination = $myhostname, yourdomain.com, localhost
home_mailbox = Maildir/
inet_protocols = ipv4
A quick explanation of each: myhostname is the identity Postfix announces itself with. mydestination lists which domains Postfix should treat as "local" — i.e., deliver to a mailbox on this server rather than relay elsewhere. home_mailbox = Maildir/ tells Postfix to store incoming mail using the Maildir format (one file per message, in a folder), which works much better with Dovecot than the older single-file mbox format. And restricting inet_protocols to IPv4 avoids a common trap: a half-configured IPv6 listener with no matching reverse-DNS record, which causes mysterious, hard-to-diagnose delivery failures.
4. Set up OpenDKIM to sign your outgoing mail
Generate a signing key pair for your domain:
sudo mkdir -p /etc/opendkim/keys/yourdomain.com
cd /etc/opendkim/keys/yourdomain.com
sudo opendkim-genkey -s default -d yourdomain.com
sudo chown opendkim:opendkim default.private
This produces two files. default.private is the key OpenDKIM uses to sign your outgoing mail — it stays on the server and nowhere else. default.txt contains the matching public key, which you publish in DNS so anyone receiving your mail can verify the signature. Open that file and publish its contents as a TXT record at default._domainkey.yourdomain.com — it'll be a long string starting with v=DKIM1; h=sha256; k=rsa; p=....
Configure /etc/opendkim.conf with a KeyTable and SigningTable (these tell OpenDKIM which key to use for which domain) and a TrustedHosts file listing 127.0.0.1 and your server's hostname. Then connect OpenDKIM to Postfix as a "milter" — a mail filter plugin — by adding this to main.cf:
milter_default_action = accept
milter_protocol = 6
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301
5. Configure Dovecot for IMAP and Maildir
Dovecot's configuration lives across several files in /etc/dovecot/conf.d/. The two settings you'll change first, mostly in 10-mail.conf and 10-ssl.conf:
mail_location = maildir:~/Maildir
ssl = required
mail_location tells Dovecot where to find the Maildir folders Postfix is writing into — note it matches the home_mailbox setting from Postfix's config. Requiring SSL means mail clients must connect over an encrypted connection to read mail, not send credentials in plain text.
The TLS permissions gotcha that catches almost everyone
This is the one part of the setup that isn't obvious even if you've read the documentation carefully, so it's worth understanding rather than just copying the fix.
Once you have a TLS certificate (Let's Encrypt via certbot is the standard free option), you'll naturally want to point both Postfix and Dovecot at it. Dovecot will just work. Postfix will silently fail to read the private key, and you'll see cryptic TLS errors with no obvious cause.
Here's why: Let's Encrypt's private key files are created with permissions 600 root:root, inside a directory that's 700 root:root — readable only by the root user. Dovecot's master process starts as root and reads the certificate before dropping down to an unprivileged user for its actual mail-handling work, so it never runs into a permissions wall. Postfix's smtpd service, by contrast, runs entirely as the unprivileged postfix user from the moment it starts — it never has root access at all, so it simply cannot open a file it doesn't have permission to read.
The fix is not to loosen the permissions on the actual Let's Encrypt directory — that weakens a private key other services may depend on, and certbot's automatic renewal can reset permissions you've changed anyway. The clean solution is a small script, triggered automatically every time the certificate renews, that copies the certificate files somewhere Postfix-readable with the correct ownership:
sudo mkdir -p /etc/postfix/certs
sudo chgrp postfix /etc/postfix/certs
sudo chmod 750 /etc/postfix/certs
Then a script in /etc/letsencrypt/renewal-hooks/deploy/ that runs, roughly:
install -o root -g postfix -m 640 privkey.pem /etc/postfix/certs/privkey.pem
install -o root -g root -m 644 fullchain.pem /etc/postfix/certs/fullchain.pem
systemctl reload postfix
systemctl reload dovecot
(systemctl reload tells a running service to pick up its new configuration without fully restarting — the standard way to apply changes to a service that's already running.) Run the script once by hand to populate the initial copy, then point Postfix's smtpd_tls_cert_file and smtpd_tls_key_file settings at those copies — not directly at the Let's Encrypt path. Because it's a renewal hook, it reruns automatically every time the certificate renews, and you never have to think about it again.
What you have at this point
With all three services configured and talking to each other, you have a mail server capable of genuinely sending and receiving email. What you don't yet have is any guarantee that mail you send actually reaches an inbox instead of a spam folder — that's an entirely separate problem, solved almost completely in DNS, and it's the whole subject of the final post in this series.