Background: Every few months CERT announces Yet Another Security Hole In Sendmail -- something that lets local or even remote users take complete control of the machine. I'm sure there are many more holes waiting to be discovered; sendmail's design means that any minor bug in 46000 lines of code is a major security risk. Other popular mailers, such as Smail, and even mailing-list managers, such as Majordomo, seem nearly as bad.
Note added in 1998: I wrote the above paragraph in December 1995, when the latest version of sendmail was 8.6.12 (with 41000 lines of code). Fourteen security holes were discovered from sendmail 8.6.12 through 8.8.5. See https://cr.yp.to/maildisasters/sendmail.html.
I started working on qmail because I was sick of this cycle of doom. Here are some of the things I did to make sure that qmail will never let an intruder into your machine.
- Programs and files are not addresses. Don't treat them as addresses.
sendmail treats programs and files as addresses. Obviously random people can't be allowed to execute arbitrary programs or write to arbitrary files, so sendmail goes through horrendous contortions trying to keep track of whether a local user was "responsible" for an address. This has proven to be an unmitigated disaster.
In qmail, programs and files are not addresses. The local delivery agent, qmail-local, can run programs or write to files as directed by ~user/.qmail, but it's always running as that user. (The notion of "user" is configurable, but root is never a user. To prevent silly mistakes, qmail-local makes sure that neither ~user nor ~user/.qmail is group-writable or world-writable.)
Security impact: .qmail, like .cshrc and .exrc and various other files, means that anyone who can write arbitrary files as a user can execute arbitrary programs as that user. That's it.
- Do as little as possible in setuid programs.
A setuid program must operate in a very dangerous environment: a user is under complete control of its fds, args, environ, cwd, tty, rlimits, timers, signals, and more. Even worse, the list of controlled items varies from one vendor's UNIX to the next, so it is very difficult to write portable code that cleans up everything.
Of the twenty most recent sendmail security holes, eleven worked only because the entire sendmail system is setuid.
Only one qmail program is setuid: qmail-queue. Its only purpose is to add a new mail message to the outgoing queue.
- Do as little as possible as root.
The entire sendmail system runs as root, so there's no way that its mistakes can be caught by the operating system's built-in protections. In contrast, only two qmail programs, qmail-start and qmail-lspawn, run as root.
- Move separate functions into mutually untrusting programs.
Five of the qmail programs -- qmail-smtpd, qmail-send, qmail-rspawn, qmail-remote, and tcp-env -- are not security-critical. Even if all of these programs are completely compromised, so that an intruder has control over the qmaild, qmails, and qmailr accounts and the mail queue, he still can't take over your system. None of the other programs trust the results from these five.
In fact, these programs don't even trust each other. They are in three groups: tcp-env and qmail-smtpd, which run as qmaild; qmail-rspawn and qmail-remote, which run as qmailr; and qmail-send, the queue manager, which runs as qmails. Each group is immune from attacks by the others.
(From root's point of view, as long as root doesn't send any mail, only qmail-start and qmail-lspawn are security-critical. They don't write any files or start any other programs as root.)
- Don't parse.
I have discovered that there are two types of command interfaces in the world of computing: good interfaces and user interfaces.
The essence of user interfaces is parsing -- converting an unstructured sequence of commands, in a format usually determined more by psychology than by solid engineering, into structured data.
When another programmer wants to talk to a user interface, he has to quote: convert his structured data into an unstructured sequence of commands that the parser will, he hopes, convert back into the original structured data.
This situation is a recipe for disaster. The parser often has bugs: it fails to handle some inputs according to the documented interface. The quoter often has bugs: it produces outputs that do not have the right meaning. Only on rare joyous occasions does it happen that the parser and the quoter both misinterpret the interface in the same way.
When the original data is controlled by a malicious user, many of these bugs translate into security holes. Some examples: the Linux login -froot security hole; the classic find | xargs rm security hole; the Majordomo injection security hole. Even a simple parser like getopt is complicated enough for people to screw up the quoting.
In qmail, all the internal file structures are incredibly simple: text0 lines beginning with single-character commands. (text0 format means that lines are separated by a 0 byte instead of line feed.) The program-level interfaces don't take options.
All the complexity of parsing RFC 822 address lists and rewriting headers is in the qmail-inject program, which runs without privileges and is essentially part of the UA.
- Keep it simple, stupid.
See BLURB.md for some of the reasons that qmail is so much smaller than sendmail. There's nothing inherently complicated about writing a mailer. (Except RFC 822 support; but that's only in qmail-inject.) Security holes can't show up in features that don't exist.
- Write bug-free code.
I've mostly given up on the standard C library. Many of its facilities, particularly stdio, seem designed to encourage bugs. A big chunk of qmail is stolen from a basic C library that I've been developing for several years for a variety of applications. The stralloc concept and getln() make it very easy to avoid buffer overruns, memory leaks, and artificial line length limits.