95.1.13. pepsi-stage-relay-to-maildir

deliver a message to local users’ Maildirs

Manual section:

1

95.1.13.1.1. Name

pepsi-stage-relay-to-maildir - the local-delivery stage of the Pepsi pipeline.

95.1.13.1.2. Synopsis

pepsi-stage-relay-to-maildir [GLOBAL-OPTIONS] worker

95.1.13.1.3. Description

pepsi-stage-relay-to-maildir is a stage program run by pepsi-dispatch(1). It loads the pepsi.workqueue row (refusing to act unless its status is running), reads its [stage-<stage>] section, and delivers the message to local system users by writing it into their Maildir/new/. Recipients that are not local are routed onward.

For each envelope recipient the stage decides whether it is local: its domain must be listed in LOCAL_DOMAINS, its mailbox name (the local-part, with any sub-address after RECIPIENT_DELIMITER removed, lower-cased) must resolve to a passwd entry, and that user’s uid must be permitted by TARGETS. A local recipient is delivered into its Maildir. A passwd lookup that fails (an LDAP or sssd backend that is down) is not “no such user”: the whole message is retried before anything is delivered (one minute, doubling to an hour, until MAX_LIFETIME; see pepsi-dispatch(1)), rather than bouncing local users as unknown mailboxes.

Every other recipient is routed on, and where depends on why it is not local:

  • A recipient on someone else’s domain is a forward, typically an alias that expands off-site (alice@example.org → alice@example.net). It goes to NEXT_STAGE, which should reach a relay. Route it through pepsi-stage-srs(1) first, so that the forwarded envelope sender still passes the next hop’s SPF check.

  • A recipient on one of our own LOCAL_DOMAINS that resolves to no permitted account is an unknown mailbox. It goes to UNKNOWN_MAILBOX_STAGE, which should reach a bounce stage.

Keeping these apart matters: sending an unknown local mailbox to a relay would look up our own domain’s MX, find this very host, and post the message straight back to ourselves. That loop only ends when the relay’s MAX_HOP_COUNT cuts it off, after a dozen needless trips through the whole pipeline for a message that should simply have been rejected.

A single message can therefore be partially delivered: the local recipients are written to their Maildirs and the remaining recipients are split onto new pending row(s) at their destination. Two addresses that resolve to the same local user are delivered only once.

Everything a pass decides is written in one statement after the deliveries: the non-local recipients split off, the success and failure notifications created, and the message either deleted or reduced to the recipients still to be retried. Delivery is at least once: when that statement cannot be written (the database is unreachable), or the worker dies between a Maildir write and that statement, the retry delivers again to the recipients already written, and the user sees the message twice. The Maildir format has no way to take a delivery back, and a duplicate is preferred to a lost message.

Delivery itself is performed by the privileged helper pepsi-helper-maildir-writer(1). The stage spawns it once per local recipient, with the target uid as its argument and the message piped on standard input. When a quota applies, the uid is preceded by --quota-bytes/--quota-count and, if the helper must measure, --measure (see below); under [pepsi] MAILBOX_FS_QUOTA it is also preceded by --fs-quota. The helper runs setuid-root and drops to that user to create the file. The message handed to the helper has a Return-Path:, a Delivered-To: (loop guard) and a Received: trace header prepended.

What happens when the helper reports a failure for a recipient depends on whether retrying can ever help, which the helper states in its exit status (see pepsi-helper-maildir-writer(1)):

Transient failures (exit 1) are paused and retried with an exponential backoff until MAX_LIFETIME, and only then routed to BOUNCE_STAGE; pepsi-dispatch(1) re-queues the message when the timeout elapses. Transient failures are a full disk, an I/O error, a helper that is not yet installed setuid-root or whose switch to the user the kernel refused, and a passwd lookup that fails or finds no entry for the uid. A helper that could not be started, was killed by a signal or exited with any other status counts the same way. An administrator can repair such a fault underneath the queued message, so the message waits. Only a fault of the target account or its mailbox is ever taken as permanent.

Permanent failures (exit 2) mean that the mailbox itself is unusable: a Maildir that is not a directory, ownership or a mode that locks the user out, a missing or relative home directory, or a uid below UID_MIN. They are routed to BOUNCE_STAGE immediately, and logged at ERROR naming the recipient and the helper’s diagnostic. No retry could deliver such a message, and keeping it queued for MAX_LIFETIME (days) would hide a broken mailbox from the sender and the postmaster. The bounce carries the helper’s own diagnostic, so the DSN states what is actually wrong with the account.

Over quota (exit 3) is neither, and is handled by site policy rather than by this stage’s judgement: see below.

Either way the DSN honours the sender’s NOTIFY, and a message that is itself a bounce (null sender) is never re-bounced. A stage with no BOUNCE_STAGE leaves such a recipient on a row with the status failed instead, with the diagnostic in state.last_error, for pepsi-failure-bouncer(1) or an operator to find. The same holds for a full mailbox when QUOTA_LIMIT_STAGE is unset as well. When the global [pepsi] ORIGINATE_SUCCESS_DSN is set and a recipient asked for NOTIFY=SUCCESS, a positive DSN is originated on successful local delivery.

95.1.13.1.4. Recycled addresses (RFC 7293)

Before writing anything, the stage records the login and uid of every local recipient in pepsi.login_assignment (see pepsi-setup(1), The login assignment record). A login that comes back under another uid, or after it was found gone, has a new owner: the database drops the previous owner’s per-address state at that moment, and logs it.

A recipient may ask for continuous ownership since a date: with an RRVS= parameter at RCPT TO (see pepsi-ingress(1)), or, when it has none, with a Require-Recipient-Valid-Since: <address>; <date> header field naming that recipient exactly. Such a recipient is not delivered when its login has a recorded change of owner after the date: it is bounced with 550 5.7.17 via BOUNCE_STAGE, honouring NOTIFY like any other bounce. A login with one owner as far as Pepsi knows passes for any date, and role addresses (postmaster, abuse and the rest of RFC 2142) are never tested. Several fields for one recipient must all pass. A database error while a recipient has such a request retries the message (the test cannot be completed); without a request it only skips the recording. Require-Recipient-Valid-Since fields are removed from the copy written to the Maildir (RFC 7293 section 5.2), since they name the recipient and a date by which the sender knew them.

95.1.13.1.5. Mailbox quota

A quota applies to a recipient through [pepsi] MAILBOX_QUOTA or through that account’s own pepsi.mailbox_quota row. When one applies, the stage decides, per message and per recipient, whether the helper must measure the mailbox before writing into it. It does that from the running estimate in the same table: the last measurement plus everything Pepsi has delivered since. Because nothing ever subtracts from that estimate (Pepsi is not told when a user deletes mail over IMAP), it is an upper bound, so an estimate that says there is room is a proof that there is. The helper is asked to walk the tree only when the account has never been measured, its last measurement is older than [pepsi] MAILBOX_QUOTA_MAX_AGE (default 15 m), its message count is at a MAILBOX_QUOTA_COUNT limit, or the messages delivered since the measurement, each counted at this message’s size, could fill what is left. Otherwise a delivery costs one small append.

A helper that reports the mailbox full (exit 3: over the limit that it was given, or a kernel EDQUOT) leaves the recipient’s fate to [pepsi] MAILBOX_OVER_QUOTA:

defer (the default)

The message stays queued and is retried exactly as a transient failure is, until MAX_LIFETIME and only then bounced. Deferring gives the account time to make room: a mailbox that is briefly full is far more common than one that stays full.

bounce

The recipient is routed at once to QUOTA_LIMIT_STAGE with an RFC 3463 5.2.2 (mailbox full), so the sender learns immediately instead of days later.

Every measurement that the helper reports is written back, which is what lets pepsi-ingress(1) refuse an over-quota recipient at RCPT. The sending MTA is still on the line there, so it tells its own user, and Pepsi generates no backscatter. See pepsi-quota(1) for the operator’s view and for the reconciliation sweep that keeps those measurements fresh.

A message that is itself a bounce is not exempt: it is refused like any other and, because a bounce is never re-bounced, dropped with a warning. That is correct, because the alternative is a loop, but it means that mail disappears, so the log line naming it is the one to watch.

95.1.13.1.6. Arguments and sub-commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input and writing one status line each. This is how the dispatcher runs the stage in production.

95.1.13.1.7. Configuration

Read from the message’s [stage-<stage>] section:

PROGRAM

Must be pepsi-stage-relay-to-maildir.

SERVER_NAME (required)

The host name recorded in the prepended Received: trace header.

TARGETS

Which local accounts may receive mail, as a comma/space separated list of: a user name (alice); a uid (452); an inclusive uid range (1000-1100); or an open-ended uid range (10000-, every uid at or above the bound). A token is read as a range only when both hyphen-separated parts are numeric, so a hyphenated user name (pepsi-maildir) is unambiguous. When unset it defaults to the regular (non-system) uid range from /etc/login.defs (UID_MIN..``UID_MAX``), falling back to 1000-60000. Independently of TARGETS, pepsi-helper-maildir-writer(1) refuses every uid below UID_MIN, so a token naming a system account (such as the 452 above, on a host with UID_MIN 1000) makes delivery to it fail permanently; pepsi-setup(1) warns about such a token.

LOCAL_DOMAINS

Whitespace/comma separated domains whose recipients are eligible for local delivery. Defaults to [pepsi-ingress] ACCEPTED_DOMAINS.

RECIPIENT_DELIMITER

The sub-address separator (alice+lists delivers to mailbox alice). Defaults to [pepsi] RECIPIENT_DELIMITER and, failing that, to +; set it to none to disable sub-address stripping. Prefer setting the [pepsi] one: pepsi-ingress(1) uses that value to work out whose mailbox quota a recipient belongs to, and has no stage section to read. pepsi-setup(1) warns when a stage overrides it, because sub-addressed mail then escapes the RCPT-time refusal.

HELPER

The privileged delivery helper to run. Defaults to pepsi-helper-maildir-writer (resolved on $PATH); set an absolute path to override.

NEXT_STAGE

Where recipients on other domains are forwarded. Required whenever a message may have such recipients. The wizard wires it to the SRS relay tail (srs → the smarthost, or srs → direct-to-MX internet delivery when no smarthost is configured), so an alias that expands off-site is actually delivered.

UNKNOWN_MAILBOX_STAGE

Where a recipient on one of our own LOCAL_DOMAINS goes when it resolves to no permitted account. Defaults to NEXT_STAGE, which is correct when NEXT_STAGE is itself a bounce stage or a smarthost that owns the domain’s accounts. Set it explicitly (the wizard sets it to the bounce stage) whenever NEXT_STAGE reaches a relay, or mail for an unknown local user is relayed back to this host’s own MX and loops until MAX_HOP_COUNT.

BOUNCE_STAGE

The stage that generates DSNs. A recipient is routed there when its mailbox is permanently unusable, or once a transient failure has been retried past MAX_LIFETIME. Without it, that recipient’s row is left failed.

QUOTA_LIMIT_STAGE

Where a recipient goes when its mailbox is full and [pepsi] MAILBOX_OVER_QUOTA is bounce. Defaults to BOUNCE_STAGE. Named and used exactly like pepsi-stage-relay-to-lmtp(1)’s option of the same name, which routes the x.2.2 that an MDA reports for the same condition: whichever local delivery path a site uses, the operator configures one thing.

RETRY_INITIAL, RETRY_MAX_INTERVAL, RETRY_FACTOR, MAX_LIFETIME

The retry/give-up timing for a transient failed local write (same meaning as the relay stages); a permanently unusable mailbox is bounced at once and never waits for these. Durations use h/m/s units. Defaults: 5 m, 2 h, 2 and 120 h.

95.1.13.1.8. Installation and privileges

To reach the setuid-root helper, pepsi-stage-relay-to-maildir must itself be installed set-group-id to the pepsi-maildir group, with mode 2550 and owner pepsi:pepsi-maildir. The mode allows owner-execute and not world-execute, because the setgid bit is the gate on the setuid-root helper. The right to execute cannot come from the group bits either, since pepsi is deliberately not a member of the group, so it comes from the owner. pepsi-dispatch(1) runs the stage as the unprivileged pepsi service user. The setgid bit gives the worker an effective gid of pepsi-maildir, which is exactly what lets it execute the group-restricted pepsi-helper-maildir-writer(1); nothing else on the host gains that ability. make install sets this up (its install-maildir-stage step), provided it is run as root and the pepsi-maildir group exists; otherwise it prints the exact commands to run by hand.

Because the setgid bit is that gate, the program enforces the same rule itself. When it runs with that bit and the real user is neither root nor the pepsi service account, it exits with an error before any configuration is read. Like the setuid crypto stages, it also strips the environment variables that steer configuration loading (HOME, XDG_CONFIG_HOME, PG*, TALER_*, PEPSI_*) and pins PATH to a safe default. The file mode belongs to the deployment and this check belongs to the program, so neither has to provide the protection on its own.

The Security chapter of the Pepsi manual lists every program carrying a setuid or setgid bit and why; its Threat model chapter states what a local account can and cannot gain through them.

95.1.13.1.9. See Also

pepsi-helper-maildir-writer(1), pepsi-quota(1), pepsi-stage-relay-to-smarthost(1), pepsi-stage-relay-to-internet(1), pepsi-stage-bounce(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), maildir(5)

95.1.13.1.10. Bugs

Report bugs to the Pepsi issue tracker.