2 Background

  This section provides an in-depth discussion of why available methods for delivering mail to home directories are not as good as the one used by hlfsd.

2.1 Single-Host Mail Spool Directory

  The most common method for mail delivery is for mail to be appended to a mailbox file in a standard spool directory on the designated ``mail home'' machine of the user. The greatest advantage of this method is that it is the default method most vendors provide with their systems, thus very little (if any) configuration is required on the SA's part. All they need to set up are mail aliases directing mail to the host on which the user's mailbox file is assigned. (Otherwise, mail is delivered locally, and users find mailboxes on many machines.)

As users become more sophisticated, and aided by windowing systems, they find themselves logging in on multiple hosts at once, performing several tasks concurrently. They ask to be able to read their mail on any host on the network, not just the one designated as their ``mail home.''

2.2 Centralized Mail Spool Directory

  A popular method for providing mail readability from any host is to have all mail delivered to a mail spool directory on a designated ``mail-server'' which is exported via NFS to all of the hosts on the network. Configuring such a system is relatively easy. On most systems, the bulk of the work is a one-time addition to one or two configuration files in /etc. The file-server's spool directory is then hard-mounted across every machine on the local network. In small environments with only a handful of hosts this can be an acceptable solution. In our department, with a couple of hundred active hosts and thousands of mail messages processed daily, this was deemed completely unacceptable, as it introduced several types of problems:

• Scalability and Performance: as more and more machines get added to the network, more mail traffic has to go over NFS to and from the mail-server. Users like to run mail-watchers[2,7] and read their mail often. The stress on the shared infrastructure increases with every user and host added; loads on the mail server would most certainly be high since all mail delivery goes through that one machine. This leads to lower reliability and performance. To reduce the number of concurrent connections between clients and the server host, some SAs have resorted to automounting the mail-spool directory. But this solution only makes things worse: since users often run mail watchers, and many popular applications such as trn, emacs, csh or ksh check periodically for new mail, the automounted directory would be effectively permanently mounted. If it gets unmounted automatically by the automounter program[3], it is most likely to get mounted shortly afterwards, consuming more I/O resources by the constant cycle of mount and umount calls.
• Reliability: the mail-server host and its network connectivity must be very reliable. Worse, since the spool directory has to be hard-mounted, many processes which access the spool directory (various shells, login, emacs, etc.) would be hung as long as connectivity to the mail-server is severed. To improve reliability, SAs may choose to backup the mail-server's spool partition several times a day. This may make things worse since reading or delivering mail while backups are in progress may cause backups to be inconsistent; more backups consume more backup-media resources, and increase the load on the mail-server host.

2.3 Distributed Mail Spool Service

  Despite the existence of a few systems that support delivery to users' home directories, mail delivery to home directories hasn't caught on. We believe the main reason is that there are too many programs that ``know'' where mailbox files reside. Besides the obvious (the delivery program /bin/mail and mail readers like /usr/ucb/Mail, mush, mm, etc.), other programs that know mailbox location are login, from, almost every shell, xbiff, xmailbox, and even some programs not directly related to mail, such as emacs and trn. Although some of these programs can be configured to look in different directories with the use of environment variables and other resources, many of them cannot. The overall porting work is significant.

Other methods that have yet to catch on require the use of a special mail-reading server, such as IMAP[16] or POP[17]. The main disadvantage of these systems is that UAs need to be modified to use these services -- a long and involved task. That is why they are not popular at this time. See Section 6.1 for more details.

Several other ideas have been proposed and even used in various environments. None of them is robust. They are mostly very specialized, inflexible, and do not extend to the general case. Some of the ideas are plain bad, potentially leading to lost or corrupt mail:

• automounters: using an automounter such as amd[13] to provide a set of symbolic links from the normal spool directory to user home directories is not sufficient. UAs rename, unlink, and recreate the mailbox as a regular file, therefore it must be a real file, not a symbolic link. Furthermore, it must reside in a real directory which is writable by the UAs and MTAs. This method may also require populating /var/spool/mail with symbolic links and making sure they are updated. Making amd manage that directory directly fails, since many various lock files need to be managed as well (see Section 4.6). Also, amd does not provide all of the NFS operations which are required to write mail such as write, create, remove, and unlink.
• $MAIL: setting this variable to an automounted directory pointing to the user's mail spool host only solves the problem for those programs which know and use $MAIL. Many programs don't, therefore this solution is partial and of limited flexibility. Also, it requires the SAs or the users to set it themselves -- an added level of inconvenience and possible failures.
• /bin/mail: using a different mail delivery agent could be the solution. One such example is hdmail[6]. However, hdmail still requires modifying all UAs, the MTA's configuration, installing new daemons, and changing login scripts. This makes the system less upgradable or compatible with others, and adds one more complicated system for SAs to deal with. It is not a complete solution because it still requires each user have their $MAIL variable setup correctly, and that every program use this variable.

2.3.1 Why Deliver Into the Home Directory?

  There are several major reasons why SAs might want to deliver mail directly into the users' home directories:

• Location: many mail readers need to move mail from the spool directory to the user's home directory. It speeds up this operation if the two are on the same filesystem. If for some reason the user's home directory is inaccessible, it isn't that useful to be able to read mail, since there is no place to move it to. In some cases, trying to move mail to a non-existent or hung filesystem may result in mail loss.
• Distribution: having all mail spool directories spread among the many more filesystems minimizes the chances that complete environments will grind to a halt when a single server is down. It does increase the chance that there will be someone who is not able to read their mail when a machine is down, but that is usually preferred to having no one be able to read their mail because a centralized mail server is down. The problem of losing some mail due to the (presumably) higher chances that a user's machine is down is minimized in HLFS as described in Sections 4.3 and 4.4.
• Security: delivering mail to users' home directories has another advantage -- enhanced security and privacy. Since a shared system mail spool directory has to be world-readable and searchable, any user can see whether other users have mail, when they last received new mail, or when they last read their mail. Programs such as finger display this information, which some consider an infringement of privacy. While it is possible to disable this feature of finger so that remote users cannot see a mailbox file's status, this doesn't prevent local users from getting the information. Furthermore, there are more programs which make use of this information. In shared environments, disabling such programs has to be done on a system-wide basis, but with mail delivered to users' home directories, users less concerned with privacy who do want to let others know when they last received or read mail can easily do so using file protection bits. Lastly, on systems that do not export their NFS filesystem with anon=0, superusers are less likely to snoop around others' mail, as they become ``nobodies'' across NFS.

In summary, delivering mail to home directories provides users the functionality sought, and also avoids most of the problems discussed in Section 2.2.