Transport Agents

ZMailer: The Manual; v1.99.20
Prev |  | Next

# Chapter 23. Transport Agents

The delivery agent programs normally form the final stage of message delivery.

These programs vary in function and facilities based on what they are doing to the messages, and what kind of channels they handle.

# 23.1. mailbox

The mailbox is a ZMailer transport agent which is usually only run by the scheduler(8) program to deliver mail to local user mailbox files. The mailbox program must be run with root privileges and invoked with the same current directory as the scheduler, namely: `$`POSTOFFICE`/transport/`.

Recipient addresses are processed as follows:

- Strip doublequotes around the address, if any.

- Strip prefixing backslashes, if any.

- If the address starts with a ``|'', the rest of the recipient address string is interpreted as a shell command to be run.

- If the address starts with a ``/'', the recipient address is a filename to append the message to.

- Otherwise the recipient address must be a local user id.

- If user is not found, and the first character of the address is a capital letter, the entire address is folded to lowercase and the user lookup is retried.

If delivering to a user mailbox (`$`MAILBOX`/userid`) which does not exist, mailbox will try to create it. If the `$`MAILBOX`/` directory is mounted from a remote system this will succeed if the directory is group writable.

Some sanity checks are done on deliveries to files and mailboxes:

- The file being delivered to must have one link only, and must be either ```/dev/null`'' or a regular file.

- The file lock must be held. (See below for a chapter about locks.)

There is a further sanity check on mailbox deliveries, namely if the mailbox is not empty the mailbox program will enforce 2 newlines as a separator before the message to be delivered. This guarantees that User Agents, like Mail(1), can find the about-to-be delivered message even if the current contents of the mailbox is corrupt.

When delivering to a process (by starting a Bourne shell to execute a specified command line), the environment is set up to contain `$PATH`, `$SHELL`, `$HOME`, `$USER`, `$SENDER`, `$UID` environment variables. The `$HOME` and `$USER` values are the recipient user's home directory and login id respectively. The `$SENDER` value is the sender address for the message (as it would appear in a ```From` ''-line), and the UID value is the owner id of the process. The `SIGINT` and `SIGHUP` signals are ignored, but `SIGTERM` is treated normally. If the process dumps core, it will be retried later. Otherwise any non-zero exit status is taken as a permanent failure, and will result in an error message back to the sender. The actual data delivered to a file, mailbox, or process are identical. It consists of the concationation of a UUCP style separator line, the message header specified in the message control file, and the message body from the original message file. The separator line starts with ```From` '' and is followed by the sender address and a timestamp.

After all deliveries and just before exiting, the mailbox process will poke comsat(8C) in case recipients have turned on biff(1). The program may be compiled to look in the rwho files on the system for recipient names logged onto neighbouring hosts, in which case the comsat on the remote host will be poked. Even if this compile-time option is enabled, this will only be done for users that have a ```.rbiff`'' file in their home directory. (Unless an ```-DRBIFF_ALWAYS`'' compile option is used.)

Usage:

  mailbox [-8] [-M] [-c `channel`] [-h `localpart`] [-l `logfile`] [-VabrH]

Parameters:

  `-c "channel"`

    specifies which channel name should be keyed on. The default is "local".

  `-h "localpart"`

    specifies which of the possible multiple recipients is to be picked this time. Default is ``none'', which selects all local channel recipients, however when the routing is done with scripts storing some tokens (other than ``-'') into the ``host''-part, it is possible to process ``host-wise'', i.e. so that each user has his or her own lock-state, and not just everybody hang on the same lock(s)...

  `-l "logfile"`

    specifies a logfile. Each entry is a line containing message id, pre-existing mailbox size in bytes, number of bytes appended, and the file name or command line delivered to.

  `-V`

    prints a version message and exits.

  `-a`

    the access time on mailbox files is, by default, preserved across delivery, so that programs such as login(1) can determine, if new mail has arrived. This option disables the above action.

  `-b`

    disables biff notification.

  `-r`

    disables remote biff notification (if supported).

  `-8`

    enables the MIME-QP-decoder to decode incoming MIME-email with Quoted-Printable encoded characters.

  `-M`

    enables the creation of MMDF-style mail-folder in the incoming mail folder. The default is ``classic'' UNIX-style folder.

Interface:

  As with all transport agents, the program reads relative pathnames of message control files from standard input (terminated with two linefeeds), and produces diagnostic output on the standard output. Normal diagnostic output is of the form:

      id/offset<TAB>notify-data<TAB>status message

  where id is the inode number of the message file, offset is a byte offset within its control file where the address being reported on is kept, status is one of ok, error, or deferred, and the message is descriptive text associated with the report. The text is terminated by a linefeed. Any other format (as might be produced by subprocesses) is passed to standard output for logging in the scheduler log. The exit status is a code from `<sysexits.h>`.
Locks:

  The locking scheme used on the system is configurable at the runtime, and has separate parameters for mailboxes and files. The data is configurable with zenv variable `$MBOXLOCKS` in which the following characters have the meanings:

  `:`

    Separates mailbox locks, and file-locks at the string. The left side has mailbox locks, and the right side has locks for other regular files. (Files with explicit paths defined.)

  `.`

    For mailboxes only: Does ``dotlock'' (userid.lock), or (on Sun Solaris) the `maillock()` mechanism.

  `F`

    If the system has `flock()` system call, uses it to lock the entire file. (Ignored on systemswithout `flock()`)

  `L`

    If the system has `lockf()` system call, uses it to lock the entire file. (Ignored on systems without `lockf()`)

  Locks are acquired in the same order as the key characters are listed.

  The default for `lockf()` capable systems is: `MBOXLOCKS=".L:L"`.

  You can choose insane combinations of lock mechanisms, which on some systems cause locks to fail always, like on Linux-2.0 series where programs must not use both `lockf()` and `flock()` locks. It is extremely important that selected locking methods are consistent throughout the system with all programs trying to acquire locks on mail spools.

Environment:

  The default location for user mailbox files is currently `/var/mail/`. This may be modified by setting the variable `$MAILBOX` in `/etc/zmailer.conf` to the directory containing user mailbox files, for example `/usr/spool/mail/`. This is best done in the ZMailer Config file. The variable `$MBOXLOCKS` is used to define locking schemes used for mailbox spool files, and separately for other regular files.

Security:

  Like all parts of ZMailer, the mailbox chooses to err on the overly cautious side. In thecase of pipes being run under the mailbox, the program in the pipe is started through `/bin/sh` with severely sanitized environment variables, and with only the file descriptors `STDIN`, `STDOUT`, and `STDERR`. Programs are refused from running, if address analysis has found suspicuous data; external messages cannot directly run programs, nor those addresses that have had a security breach detected during `~/.forward`- or other aliasing analysis. (Same applies also with writing into explicitely named files.)

  The pipe subprogram is run with user-id it gets thru the address privilege analysis during message routing, and it gets the groupid through lookup of `getpwuid(uid)`. That is, if you have multiple usernames with same uid, there are no guarantees as to which of them is used for the gid entry.

Subprogram Envonmrm´e:

  The mailbox sets the following eight environment variables for the subprograms it runs in the pipes:

  `HOME`

    The home directory path is taken from abovementioned `getpwuid()` lookup.

  `USER`

    Likewise the textual username.

  `SENDER`

    is the incoming ``MAIL FROM:<..>'' address without brackets. For an incoming error message, value ``<>'' is used.

  `ORCPT`

    when present, is the XTEXT encoded ORCPT value received at the message injection into this system. See RFC 1891 for details.

  `ENVID`

    when present, is the XTEXT encoded ENVID value received at the message injection into this system. See RFC 1891 for details.

  `ZCONFIG`

    is the location of the ZMailer ZENV file.

  `MAILBIN`

    is the value from ZENV.

  `MAILSHARE`

    is the value from ZENV.

# 23.2. hold

hold - zmailer deferred processing transport agent

Description:

  hold is a ZMailer transport agent which is usually only run by the scheduler(8) program to test conditions for reprocessing of previously deferred message addresses.

  The hold program must be run with the same current directory as the scheduler, namely: `$`POSTOFFICE`/transport/`.

  The program will interpret the host part of an address destined for its channel as a condition that must be met before the original address (in the user part) can be reprocessed by the router. The condition specification contains a general condition class name followed by colon followed by a parameter string. The currently supported condition classes are:

  `ns`

    succeeds when the nameserver lookup indicated by the parameter does not produce a temporary nameserver error. The parameter is a domain name followed by a slash followed by a standard Internet nameserver Resource Record type name.

  `timeout`

    succeeds when the time given by the parameter (in normal seconds-since-epoch format) has passed.

  `io`

    succeeds 10% of the time, to allow retry of temporary I/O failures.

  `script`

    runs the named shell script with the optional given argument. The parameter is a simple name, the shell script name within the `$`MAILBIN`/bin/` directory, optionally followed by a slash followed by an argument to be passed to the shell script.

  For example:

        NS:nic.ddn.mil/cname
        TIMEOUT:649901432
        IO:error
        SCRIPT:homedir/joe

  The condition class name is case-insensitive but is capitalised by convention. The parameter strings are case-preserved for condition class-specific interpretation. Whitespace is not permitted.

  The envelope of the resubmitted message is created from the sender and (no longer deferred) recipient addresses, and a ``via suspension'' header.

Description:

  {\bf Usage} \begin{verbatim} hold [ -c channel ] [ -V ] \end{verbatim}

Description:

  {\bf Parameters} {\tt -c channel} specifies which channel name should be keyed on. The default is hold. {\tt -V} prints a version message and exits.

Interface:

  As all transport agents, the program reads relative path-names of message control files from standard input (terminated with two linefeeds), and produces diagnostic output on the standard output. Normal diagnostic output is of the form:

        id/offset/status message

  where id is the inode number of the message file, offset is a byte offset within its control file where the address being reported on is kept, status is one of ok, error, or deferred, and the message is descriptive text associated with the report. The text is terminated by a linefeed. Any other format (as might be produced by subprocesses) is passed to standard output for logging in the scheduler log.

  The exit status is a code from `<sysexits.h>`;.

# 23.3. smtp

smtp - zmailer SMTP client transport agent

smtp is a ZMailer transport agent which is usually only run by the scheduler(8) to transfer messages to a remote Internet host using the SMTP protocol. The smtp program must be run with the same current directory as the scheduler, namely `$`POSTOFFICE`/transport/`.

The program scans the message control files named on `STDIN` for addresses destined for its channel and the host given on the command line. If any are found, all matching addresses and messages are transferred in a single SMTP conversation. The destination host might in fact be served by any available mail exchanger for that host.

Usage:

      smtp [ -78deEHrPsVxW ] [ -c channel ] [ -h heloname ] [ -l logfile ]
      [ -p remote-port ] [ -T timeout ] [ -F forcedest] [ -L localidentity ] host

Parameters:

  `-7`

    forces SMTP channel to be 7-bit, and thus forcing all 8-bit texts to be MIME-QP-encoded for the transport.

  `-8`

    forces SMTP channel to be 8-bit-clean, and as such, to decode the message while transporting it (is it is MIME QP encoded).

  `-c channel`

    specifies which channel name should be keyed on. The default is smtp.

  `-d`

    turns on debugging output.

  `-e`

    asks that for every destination address specification with a matching channel name, an MX lookup is done on the hostname to see whether the currently connected host can provide service for that destination. The default is to just do a textual name comparison with the destination hostname as given on the command line.

  `-e`

    use the ``EHLO''-greeting only if the remote server initial banner reports ``ESMTP'' on it.

  `-h host`

    specifies the hostname for the SMTP HELO greeting. The default is the hostname of the local system, as returned by `gethostname(2)` or `uname(2)`.

  `-F forcedest`

    overrides delivery destination by forcing all email to be sent to given forcedest IP-number/hostname.

  `-H`

    Disable the per default active forced 8-bit headers conversion into MIME-2-format.

  `-L localident`

    specifies (for multi-homed machines) that they should use specified identity when connecting to the destination. Think of server with multiple IP numbers due to virtual hosting, for example. At such systems there may be situation when virtual identity needs to be used for reaching the destination system.

  `-l logfile`

    specifies a log file where the complete SMTP command transaction will be copied. Each line in the log will be prefixed with the process id of the transport agent process, so the same log file can be used by all SMTP clients.

  `-r`

    asks to set up SMTP connections using a source TCP port number under 1024. This is in the range of port numbers only available to a privileged process on some UNIX systems, which has led to some misguided attempts at mail security based on this mechanism.

  `-s`

    asks to report the progress of the SMTP conversation and data transfer on the command line in a way that will be visible to ps(1).

  `-x`

    turns off MX lookups on delivery connections. This may be used ignore public MX knowledge and do exactly what the router says in cases where delivering to an explicit IP address is inappropriate.

  `-P`

    disable SMTP-PIPELINING usage (ESMTP keyword: PIPELINING)

  `-T timeout`

    specifies the timeout, in seconds, when waiting for a response to an SMTP command. The timeout applies to all SMTP command-response exchanges except for the acknowledgement after terminating the DATA portion of a message transaction (after sending the ``.'' CRLF sequence). The default timeout is 10 minutes, the minimum acceptable value is 5 seconds. The timeout on the DATA acknowledgement is large, at least 10 minutes.

  `-V`

    prints a version message and exits.

  `-W`

    turns on the DNS WKS checking, and if the remote system does not have SMTP in its WKS-bits, email delivery to such address is aborted with an error message.

Interface:

  As all transport agents, the program reads relative path names of message control files from standard input (terminated with two linefeeds), and produces diagnostic output on the standard output. Normal diagnostic output is of the form:

        id/offset<TAB>notify-data<TAB>status message

  where id is the inode number of the message file, offset is a byte offset within its control file where the address being reported on is kept, status is one of ok, error, or deferred, and the message is descriptive text associated with the report. The text is terminated by a linefeed. Any other format (as might be produced by subprocesses) is passed to standard output for logging in the scheduler log.

  The exit status is a code from `<sysexits.h>`.

Extended SMTP:

  When a user sends out 8-bit mail with the proper headers, this module can send it out to conforming servers either in 8-bit transparent manner, or down-converting ``Content-Transfer-Encoding: 8BIT'' to ``Content-Transfer-Encoding: 7BIT'' or to ``Content-Transfer-Encoding: QUOTED-PRINTABLE'' depending on what is the mail contents. This works only with ``Content-Type: TEXT/PLAIN'' thus no fancy MULTIPART/ALTERNATE et.al. schemes.. When ``Content-Transfer-Encoding:''-header is not present in the headers, and recipient has not declared 8-bit SMTP capability, mail contents are treated with old 7-bit stripping method.

# 23.4. sm - zmailer Sendmail compatible transport agent

sm is a ZMailer transport agent which is usually only run by the scheduler(8), to deliver messages by invoking a program with facilities and in a way compatible with a sendmail MTA. The sm program must be run with the same current directory as the scheduler, namely `$`POSTOFFICE`/transport/`.

The program scans the message control files named on `STDIN` for addresses destined for the channel and/or the host given on the command line. If any are found, all matching addresses and messages are processed according to the specifications for the mailer in the configuration file.

The exit status of a mailer should be one of the standard values specified in `<sysexits.h>`. Of these, `EX_OK` indicates successful delivery, and `EX_DATAERR`, `EX_NOUSER`, `EX_NOHOST`, `EX_UNAVAILABLE`, and `EX_NOPERM` indicate permanent failure. All other exit codes will be treated as a temporary failure and the delivery will be retried.

Usage:

  sm [-8] [-H] [-Q] [-V] [-f `configfile`] -c `channel` -h `host` `mailer`

Parameters:

  `-8`

    tells that the output is 8-bit clean, and for any MIME message with QUOTED-PRINTABLE encoding the coding can be decoded.

  `-Q`

    tells that the transport channel will likely treat poorly control characters like TAB, and possibly SPACE too.. This encodes them all by using QUOTED-PRINTABLE encoding.

  `-f configfile`

    specifies the name of a configuration file containing specifications of the various known sendmail compatible mailer programs: how to invoke them and how to process messages for them. The default is `$`MAILSHARE`/sm.cf`.

  `-c channel`

    specifies which channel name should be keyed on. There is no default. If this option is not specified, the `-h` option must be.

  `-h host`

    specifies which host name should be keyed on. There is no default. If this option is not specified, the `-c` option must be.

  `-h host`

    prints a version message and exits.

## 23.4.1. configuration of sm

sm is a ZMailer's sendmail(8) compatible transport agent to deliver messages by invoking a program with facilities and in a way compatible with a sendmail(8) MTA.

The program scans the message control files named on stdin for addresses destined for the channel and/or the host given on the command line. If any are found, all matching addresses and messages are processed according to the specifications for the mailer in the configuration file.

The exit status of a mailer should be one of the standard values specified in #include <`sysexits.h`>. Of these, EX_OK indicates successful deliver, and EX_DATAERR, EX_NOUSER, EX_NOHOST, EX_UNAVAILABLE, and EX_NOPERM indicate permanent failure. All other exit codes will be treated as a temporary failure and the delivery will be retried.

Usage:

sm [-8] [-H] [-Q] [-V] [-f `configfile`] -c `channel` -h `host` `mailer`

Configuration:

The configuration file `$`MAILSHARE`/sm.conf` associates the mailer keyword from the command line with a specification of a delivery program. This is very similar to the way the definition of a ``mailer'' in sendmail(8). It requires flags, a program name, and a command line specification. These are in fact the fields of the entries of the configuration file. Lines starting with whitespace or a ``#'' are ignored, and all others are assumed to follow format shown in figure Figure 23-1.

Figure 23-1. Sample `sm.conf` file

    #mailer flags program        argument list
    #------ ----- -------------- --------------------------------
    local   mS    sm/localm      localm -r $g $u
    prog    -     /bin/sh        sh -c $u
    tty     rs    /usr/local/to  to $u
    uucp    U     /usr/bin/uux   uux - -r -a$g -gC $h!rmail ($u)
    usenet  m     sm/usenet      usenet $u
    test    n     sm/test        test $u

The mailer field extends from the beginning of the line to the first whitespace. It is used simply as a key index to the configuration file contents. One or more whitespace is used as the field separator for all the fields.

The flags field contains a concatenation of one-letter flags. If no flags are desired, a ``-'' character should be used to indicate presence of the field. All normal sendmail (of 5.x era..) flags are recognized, but the ones that do not make sense in the context of ZMailer will produce an error (or some are ignored). The flags that change the behaviour of sm are:

`b`

  will activate BSMTP-type wrapping with a ``hidden-dot'' algorithm; e.g. quite ordinary SMTP stream, but in ``batch mode''.

`B`

  The first ```B`'' turns on similar BSMTP wrapping as ```b`'', but adds SIZE and, if the sm is started with option ```-8`'', also 8BITMIME options. The second ```B`'' adds there also DSN (Delivery Status Notification) parameters.

`E`

  will prepend ```>`'' to any message body line starting with ``From ''. (Read: ``From-space'')

`f`

  adds ```-f sender`'' arguments to the delivery program.

`n`

  will not prepend a ``From ''-line (normal mailbox separator line) to the message.

`r`

  adds ```-r sender`'' arguments to the delivery program.

`S`

  will run the delivery program with the same real and effective uid as the sm process. If this flag is not set, the delivery program will be run with the real uid of the sm process. This may be useful if sm is setuid.

`m`

  informs sm that each instance of the delivery program can deliver to many destinations. This affects `$u` expansion in the argument list, see below.

`P`

  prepends a ``Return-Path:'' header to the message.

`U`

  will prepend a ``From ''-line, with a ``remote from myuucpname'' at the end, to the message. This is what is expected by remote rmail(1) programs for incoming UUCP mail.

`R`

  use CRLF sequence as end-of-line sequence. Without it, will use LF-only end-of-line sequence.

`X`

  does SMTP-like ``hidden-dot'' algorithm of doubling all dots that are at the start of the line.

`7`

  will strip (set to 0) the 8th bit of every character in the message.

The path field specifies the location of the delivery program. Relative pathnames are allowed and are relative to the `$`MAILBIN`/` directory.

The arguments field extends to the end of the line. It contains whitespace separated `argv` parameters which may contain one of the following sequences:

`$g`

  which is replaced by the sender address.

`$h`

  which is replaced by the destination host.

`$u`

  which is replaced by the recipient address. If the ```m`'' mailer flag is set and there are several recipients for this message, the argument containing the ```$u`'' will be replicated as necessary for each recipient.

# 23.5. libta - Transport Agent Support Library

This is the library that all transport agents use, and several of its functions are intended to aid message processing.

## 23.5.1. Function groupings

Transport agent support library function groups are:

- Message file manipulation routines.

- Diagnostics routines.

## 23.5.2. Function listings

Text to be inserted here.

## 23.5.3. Function usage examples

Text to be inserted here.

# 23.6. Security Issues

Text to be inserted here.

Prev | Home | Next
Scheduler | Up | ZMailer Utilities