70.1.12. pepsi-stage-relay-to-maildir

deliver a message to local users’ Maildirs

Manual section:

1

70.1.12.1.1. Name

pepsi-stage-relay-to-maildir - the local-delivery stage of the Pepsi pipeline.

70.1.12.1.2. Synopsis

pepsi-stage-relay-to-maildir [GLOBAL-OPTIONS] worker

70.1.12.1.3. Description

pepsi-stage-relay-to-maildir is a stage program run by pepsi-dispatch(1). It loads the pepsi.ingress row (refusing to act unless its status is running), reads its [stage-<stage>] section, and delivers the message to local system users by writing it into their Maildir/new/. Recipients that are not local are routed onward.

For each envelope recipient the stage decides whether it is local: its domain must be listed in LOCAL_DOMAINS, its mailbox name (the local-part, with any sub-address after RECIPIENT_DELIMITER removed, lower-cased) must resolve to a passwd entry, and that user’s uid must be permitted by TARGETS. A local recipient is delivered into its Maildir.

Every other recipient is routed on, and where depends on why it is not local:

• A recipient on someone else’s domain is a forward — typically an alias that expands off-site (cz@example.org → dold@other.net). It goes to NEXT_STAGE, which should reach a relay. Route it through pepsi-stage-srs(1) first, so the forwarded envelope sender still passes the next hop’s SPF check.

• A recipient on one of our own LOCAL_DOMAINS that resolves to no permitted account is an unknown mailbox. It goes to UNKNOWN_MAILBOX_STAGE, which should reach a bounce stage.

Keeping these apart matters: sending an unknown local mailbox to a relay would look up our own domain’s MX, find this very host, and post the message straight back to ourselves — a loop that only ends when the relay’s MAX_HOP_COUNT cuts it off, a dozen needless trips through the whole pipeline after the message should simply have been rejected.

A single message can therefore be partially delivered: the local recipients are written to their Maildirs and the remaining recipients are split onto new pending row(s) at their destination. Two addresses that resolve to the same local user are delivered only once.

Delivery itself is performed by the privileged helper pepsi-helper-maildir-writer(1): the stage spawns it once per local recipient with the target uid as its argument and the message piped on standard input, and the helper — running setuid-root — drops to that user to create the file. The message handed to the helper has a Return-Path:, a Delivered-To: (loop guard) and a Received: trace header prepended.

What happens when the helper reports a failure for a recipient depends on whether retrying can ever help, which the helper states in its exit status (see pepsi-helper-maildir-writer(1)):

Transient failures (exit 1 — a full disk, an I/O error, a helper not yet installed setuid-root) are paused and retried with an exponential backoff (re-queued by pepsi-dispatch(1) when the timeout elapses) until MAX_LIFETIME, and only then routed to BOUNCE_STAGE: an administrator can repair such a fault underneath the queued message, so the message waits.

Permanent failures (exit 2 — the mailbox itself is unusable: a Maildir that is not a directory, ownership or a mode that locks the user out, a missing home directory, no passwd entry for the uid) are routed to BOUNCE_STAGE immediately, and logged at ERROR naming the recipient and the helper’s diagnostic. No retry could deliver such a message, and keeping it queued for MAX_LIFETIME (days) would hide a broken mailbox from the sender and the postmaster. The bounce carries the helper’s own diagnostic, so the DSN states what is actually wrong with the account.

Over quota (exit 3) is neither, and is handled by site policy rather than by this stage’s judgement: see below.

Either way the DSN honours the sender’s NOTIFY, and a message that is itself a bounce (null sender) is never re-bounced. When the global [pepsi] ORIGINATE_SUCCESS_DSN is set and a recipient asked for NOTIFY=SUCCESS, a positive DSN is originated on successful local delivery.

70.1.12.1.4. Mailbox quota

When a quota applies to a recipient — from [pepsi] MAILBOX_QUOTA, or from that account’s own pepsi.mailbox_quota row — the stage decides, per message and per recipient, whether the helper must measure the mailbox before writing into it. It does that from the running estimate in the same table: the last measurement plus everything Pepsi has delivered since. Because nothing ever subtracts from that estimate (Pepsi is not told when a user deletes mail over IMAP), it is an upper bound — so an estimate that says there is room is a proof that there is, and only when it says the account may be close does the helper walk the tree. A mailbox nowhere near its limit costs one small append per delivery.

A helper that reports the mailbox full (exit 3) leaves the recipient’s fate to [pepsi] MAILBOX_OVER_QUOTA:

defer (the default)

The message stays queued and is retried exactly as a transient failure is, until MAX_LIFETIME and only then bounced. This is what Pepsi has always done with a kernel EDQUOT, and it gives the account time to make room — a mailbox that is briefly full is far more common than one that stays full.

bounce

The recipient is routed at once to QUOTA_LIMIT_STAGE with an RFC 3463 5.2.2 (mailbox full), so the sender learns immediately instead of days later.

Every measurement the helper reports is written back, which is what lets pepsi-ingress(1) refuse an over-quota recipient at RCPT — the sending MTA is still on the line there, so it tells its own user and Pepsi generates no backscatter. See pepsi-quota(1) for the operator’s view and for the reconciliation sweep that keeps those measurements fresh.

A message that is itself a bounce is not exempt: it is refused like any other and, because a bounce is never re-bounced, dropped with a warning. That is correct — the alternative is a loop — but it is mail disappearing, so the log line naming it is the one to watch.

70.1.12.1.5. Arguments and sub-commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input and writing one status line each. This is how the dispatcher runs the stage in production.

70.1.12.1.6. Configuration

Read from the message’s [stage-<stage>] section:

PROGRAM

Must be pepsi-stage-relay-to-maildir.

SERVER_NAME (required)

The host name recorded in the prepended Received: trace header.

TARGETS

Which local accounts may receive mail, as a comma/space separated list of: a user name (alice); a uid (452); an inclusive uid range (1000-1100); or an open-ended uid range (10000-, every uid at or above the bound). A token is read as a range only when both hyphen-separated parts are numeric, so a hyphenated user name (pepsi-maildir) is unambiguous. When unset it defaults to the regular (non-system) uid range from /etc/login.defs (UID_MIN..``UID_MAX``), falling back to 1000-60000.

LOCAL_DOMAINS

Whitespace/comma separated domains whose recipients are eligible for local delivery. Defaults to [pepsi-ingress] ACCEPTED_DOMAINS.

RECIPIENT_DELIMITER

The sub-address separator (alice+lists delivers to mailbox alice). Defaults to [pepsi] RECIPIENT_DELIMITER and, failing that, to +; set it to none to disable sub-address stripping. Prefer setting the [pepsi] one: pepsi-ingress(1) uses that value to work out whose mailbox quota a recipient belongs to, and has no stage section to read. pepsi-setup(1) warns when a stage overrides it, because sub-addressed mail then escapes the RCPT-time refusal.

HELPER

The privileged delivery helper to run. Defaults to pepsi-helper-maildir-writer (resolved on $PATH); set an absolute path to override.

NEXT_STAGE

Where recipients on other domains are forwarded. Required whenever a message may have such recipients. The wizard wires it to the SRS relay tail (srs → the smarthost, or srs → direct-to-MX internet delivery when no smarthost is configured), so an alias that expands off-site is actually delivered.

UNKNOWN_MAILBOX_STAGE

Where a recipient on one of our own LOCAL_DOMAINS goes when it resolves to no permitted account. Defaults to NEXT_STAGE, which is the historical behaviour and correct when NEXT_STAGE is itself a bounce stage or a smarthost that owns the domain’s accounts. Set it explicitly (the wizard sets it to the bounce stage) whenever NEXT_STAGE reaches a relay, or mail for an unknown local user is relayed back to this host’s own MX and loops until MAX_HOP_COUNT.

BOUNCE_STAGE

The DSN-generation stage a recipient is routed to when its mailbox is permanently unusable, or once a transient failure has been retried past MAX_LIFETIME.

QUOTA_LIMIT_STAGE

Where a recipient goes when its mailbox is full and [pepsi] MAILBOX_OVER_QUOTA is bounce. Defaults to BOUNCE_STAGE, so an existing configuration still produces a bounce without being changed. Named and used exactly like pepsi-stage-relay-to-lmtp(1)’s option of the same name, which routes the x.2.2 an MDA reports for the same condition: whichever local delivery path a site uses, the operator configures one thing.

RETRY_INITIAL, RETRY_MAX_INTERVAL, RETRY_FACTOR, MAX_LIFETIME

The retry/give-up timing for a transient failed local write (same meaning as the relay stages); a permanently unusable mailbox is bounced at once and never waits for these. Durations use h/m/s units.

70.1.12.1.7. Installation and privileges

To reach the setuid-root helper, pepsi-stage-relay-to-maildir must itself be installed set-group-id to the pepsi-maildir group (mode 2755, owner root:pepsi-maildir). pepsi-dispatch(1) runs the stage as the unprivileged pepsi service user; the setgid bit gives the worker an effective gid of pepsi-maildir, which is exactly what lets it execute the group-restricted pepsi-helper-maildir-writer(1) — and nothing else on the host gains that ability. make install sets this up (its install-maildir-stage step), provided it is run as root and the pepsi-maildir group exists; otherwise it prints the exact commands to run by hand.

70.1.12.1.8. See Also

pepsi-helper-maildir-writer(1), pepsi-quota(1), pepsi-stage-relay-to-smarthost(1), pepsi-stage-relay-to-internet(1), pepsi-stage-bounce(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), maildir(5)

70.1.12.1.9. Bugs

Report bugs to the Pepsi issue tracker.