I run dovecot on a VPS for remote IMAP access, with postfix as the underlying mail server. Originally, it handled mail for just one of the domains whose blogs are hosted there. But recently I decided to add email accounts for JumpForJoySoftware, a site I use to publish open-source apps. How hard could it be?
I really, really, really need to stop thinking that. Because every time I do it ends up being the preamble to a nightmare experience.
This one wasn’t as bad as some (I did end up disabling the pre-existing email accounts but was able to fix that without too much aggravation). But a combination of ignorance1, an unfamiliar configuration style2 and some inconsistent and confusing documentation made for a fun time. If you enjoy self-flagellation, that is.
What’s In a Name?
From what I could figure out the main problem stems from the way the various password authentication modules in dovecot interact with the overall system. The PAM module — the one you need to authenticate a system user against system credentials — does not allow you specify the format of the user name passed to it. It requires “just” the user name. No domain part allowed.
That makes perfect sense for PAM authentication. System accounts don’t include the concept of a domain. Or rather the “domain” is the system itself, which needs no additional information to identify it.
But it won’t work with setting up a virtual mail user. Unless you assign a unique name to each virtual user that looks like a system name. That would be arbitrary, and therefore hard to maintain. No thanx! Instead, you’ll almost certainly want to use an email username which includes a domain.
Unfortunately, so far as I can tell, if you use dovecot’s PAM module for system users3 you have to configure dovecot to respond to requests for a user name with just the name part (no domain). Globally.
This is done in the 10-auth.conf file via the following entry:
auth_username_format = %n
The ‘%n’ is dovecot-speak for “just the name part of the fully-qualified name”.
Since this is a global setting it applies to all the dovecot passwd modules, including passwd-file, which is a simple way of adding virtual users to a dovecot system4. Which means passwd-file will never be able to match a virtual user against a fully-qualified email account.
The solution, which I stumbled across after about five hours of work5 is to use dovecot’s conditional settings feature:
auth_username_format = %{if;%d;eq;theboilingfrog.net;%n;%u}Translated, this means “if the domain part (%d) matches the base domain for the mail server (theboilingfrog.net, in my case), user names should just be the name part, otherwise use the fully-qualified email account name (%u)”.
But intuitively obvious to the casual observer it ain’t6.
Interestingly, there have been a number of suggestions to the dovecot team to fix this by allowing the PAM module to accept a specific user name format7. I just posted yet another such suggestion to the dovecot mailing list. It’ll be interesting to see what their response is.
Configuring Postfix
Other problems I ran into involved figuring out how to configure postfix to handle virtual users. The documentation on how to do so is complete8 but hard to understand.
The basic steps are:
- Create a user and a group for postfix to use to manage the virtual mailboxes. Traditionally this user is called vmail and has a userid (uid) and groupid (gid) of 5000/5000.
- Create the directory postfix will use to store mail for the virtual users. It needs a special place because since the users are virtual they have no user accounts on the system itself. I created mine at /var/mail/vhosts.
- Make sure your virtual storage folder is owned by the virtual user you set up and belongs to the virtual group you set up (again, vmail in my case)
- Make sure you’ve got TLS set up and your SSL certificate available (I use Let’s Encrypt and certbot, which are fabulous). Since you can only reference a single certificate it will need to cover all the mail domains (consult the certbot documentation on how to do that). Here’s what my settings look like in main.cf:
#Enable TLS Encryption when Postfix receives incoming emails
smtpd_tls_cert_file=/etc/letsencrypt/live/theboilingfrog.net/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/theboilingfrog.net/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_loglevel = 1
#smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache- Comment out myhostname in main.cf. It’ll be replaced by the next step.
- Add the following lines to main.cf to set up virtual mailboxes (substitute your own domains you want to provide mail for in the virtual_mailbox_domains field value).
virtual_mailbox_domains = theboilingfrog.net jumpforjoysoftware.com virtual_mailbox_base = /var/mail/vhosts virtual_mailbox_maps = hash:/etc/postfix/vmailbox virtual_minimum_uid = 100 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/virtual
- Create the virtual mailbox map file (vmailbox in my case). Dont forget to run postmap on the file to create the db file postfix expects (vmailbox.db in my case)
support@jumpforjoysoftware.com jumpforjoysoftware.com/mark/ mark@jumpforjoysoftware.com jumpforjoysoftware.com/mark/
- Restart postfix. With your fingers crossed, of course.
If everything works you should have a system capable of supporting both system and virtual users.
Of course, it likely won’t. Hello, Murphy’s Law! But that’s what the internet is for.
Note for Outlook Users
If, like me, you use Outlook as an email client you’ll need to know what settings to use when configuring IMAP access to dovecot. For reasons I don’t know these are not well, or consistently, documented. What’s worse, you can no longer see the settings on a functioning Outlook IMAP account once you’ve configured it. All the interesting bits are hidden9.
Here’s what I use:
- Incoming (receiving email) port: 143
- Outgoing (sending email) port: 587
- Encryption: STARTTLS (for both incoming and outgoing)
Remember, you have to open these ports on your firewall for them to work.
on my part ↩
open source software is great…except that there are no standards for how to configure it, so every app is its own learning curve ↩
You can run everything off a database, but that would be overkill for my situation ↩
It’s just a key/value pair list of user names and their encrypted passwords ↩
admittedly not all related to this particular problem ↩
I would never have figured this out myself. I greatly appreciate hitd’s answer over on ServerFault ↩
most of the other dovecot modules allow this ↩
it solved my problem, at least ↩
You can find them by going to the old Mail control panel applet but that’s not advertised and is, I suspect, on its way to being deprecated ↩