95.1.13. pepsi-stage-relay-to-maildir¶
deliver a message to local users’ Maildirs
- Manual section:
1
95.1.13.1.1. Name¶
pepsi-stage-relay-to-maildir - the local-delivery stage of the Pepsi pipeline.
95.1.13.1.2. Synopsis¶
pepsi-stage-relay-to-maildir [GLOBAL-OPTIONS] worker
95.1.13.1.3. Description¶
pepsi-stage-relay-to-maildir is a stage program run by pepsi-dispatch(1).
It loads the pepsi.workqueue row (refusing to act unless its status is
running), reads its [stage-<stage>] section, and delivers the message to
local system users by writing it into their Maildir/new/. Recipients that
are not local are routed onward.
For each envelope recipient the stage decides whether it is local: its domain
must be listed in LOCAL_DOMAINS, its mailbox name (the local-part, with any
sub-address after RECIPIENT_DELIMITER removed, lower-cased) must resolve to a
passwd entry, and that user’s uid must be permitted by TARGETS. A local
recipient is delivered into its Maildir. A passwd lookup that fails (an LDAP or
sssd backend that is down) is not “no such user”: the whole message is
retried before anything is delivered (one minute, doubling to an hour, until
MAX_LIFETIME; see pepsi-dispatch(1)), rather than bouncing local users
as unknown mailboxes.
Every other recipient is routed on, and where depends on why it is not local:
A recipient on someone else’s domain is a forward, typically an alias that expands off-site (
alice@example.org→alice@example.net). It goes to NEXT_STAGE, which should reach a relay. Route it through pepsi-stage-srs(1) first, so that the forwarded envelope sender still passes the next hop’s SPF check.A recipient on one of our own LOCAL_DOMAINS that resolves to no permitted account is an unknown mailbox. It goes to UNKNOWN_MAILBOX_STAGE, which should reach a bounce stage.
Keeping these apart matters: sending an unknown local mailbox to a relay would look up our own domain’s MX, find this very host, and post the message straight back to ourselves. That loop only ends when the relay’s MAX_HOP_COUNT cuts it off, after a dozen needless trips through the whole pipeline for a message that should simply have been rejected.
A single message can therefore be partially delivered: the local recipients
are written to their Maildirs and the remaining recipients are split onto new
pending row(s) at their destination. Two addresses that resolve to the same
local user are delivered only once.
Everything a pass decides is written in one statement after the deliveries: the non-local recipients split off, the success and failure notifications created, and the message either deleted or reduced to the recipients still to be retried. Delivery is at least once: when that statement cannot be written (the database is unreachable), or the worker dies between a Maildir write and that statement, the retry delivers again to the recipients already written, and the user sees the message twice. The Maildir format has no way to take a delivery back, and a duplicate is preferred to a lost message.
Delivery itself is performed by the privileged helper
pepsi-helper-maildir-writer(1). The stage spawns it once per local
recipient, with the target uid as its argument and the message piped on
standard input. When a quota applies, the uid is preceded by
--quota-bytes/--quota-count and, if the helper must measure,
--measure (see below); under [pepsi] MAILBOX_FS_QUOTA it is also
preceded by --fs-quota. The helper runs setuid-root and drops to that user
to create the file. The message handed to the helper has a Return-Path:, a
Delivered-To: (loop guard) and a Received: trace header prepended.
What happens when the helper reports a failure for a recipient depends on whether retrying can ever help, which the helper states in its exit status (see pepsi-helper-maildir-writer(1)):
Transient failures (exit 1) are paused and retried with an exponential
backoff until MAX_LIFETIME, and only then routed to BOUNCE_STAGE;
pepsi-dispatch(1) re-queues the message when the timeout elapses.
Transient failures are a full disk, an I/O error, a helper that is not yet
installed setuid-root or whose switch to the user the kernel refused, and a
passwd lookup that fails or finds no entry for the uid. A helper that could not
be started, was killed by a signal or exited with any other status counts the
same way. An administrator can repair such a fault underneath the queued
message, so the message waits. Only a
fault of the target account or its mailbox is ever taken as permanent.
Permanent failures (exit 2) mean that the mailbox itself is unusable: a
Maildir that is not a directory, ownership or a mode that locks the user
out, a missing or relative home directory, or a uid below UID_MIN. They are
routed to BOUNCE_STAGE immediately, and logged at ERROR naming the recipient and the helper’s
diagnostic. No retry could deliver such a message, and keeping it queued for
MAX_LIFETIME (days) would hide a broken mailbox from the sender and the
postmaster. The bounce carries the helper’s own diagnostic, so the DSN states
what is actually wrong with the account.
Over quota (exit 3) is neither, and is handled by site policy rather than by this stage’s judgement: see below.
Either way the DSN honours the sender’s NOTIFY, and a message that is itself
a bounce (null sender) is never re-bounced. A stage with no BOUNCE_STAGE
leaves such a recipient on a row with the status failed instead, with the
diagnostic in state.last_error, for pepsi-failure-bouncer(1) or an
operator to find. The same holds for a full mailbox when
QUOTA_LIMIT_STAGE is unset as well.
When the global [pepsi] ORIGINATE_SUCCESS_DSN is set and a recipient asked
for NOTIFY=SUCCESS, a positive DSN is originated on successful local delivery.
95.1.13.1.4. Recycled addresses (RFC 7293)¶
Before writing anything, the stage records the login and uid of every local
recipient in pepsi.login_assignment (see pepsi-setup(1), The login
assignment record). A login that comes back under another uid, or after it
was found gone, has a new owner: the database drops the previous owner’s
per-address state at that moment, and logs it.
A recipient may ask for continuous ownership since a date: with an RRVS=
parameter at RCPT TO (see pepsi-ingress(1)), or, when it has none,
with a Require-Recipient-Valid-Since: <address>; <date> header field naming
that recipient exactly. Such a recipient is not delivered when its login
has a recorded change of owner after the date: it is bounced with 550 5.7.17 via
BOUNCE_STAGE, honouring NOTIFY like any other bounce. A login with one
owner as far as Pepsi knows passes for any date, and role addresses
(postmaster, abuse and the rest of RFC 2142) are never tested. Several
fields for one recipient must all pass. A database error while a recipient has
such a request retries the message (the test cannot be completed); without a
request it only skips the recording. Require-Recipient-Valid-Since fields
are removed from the copy written to the Maildir (RFC 7293 section 5.2), since
they name the recipient and a date by which the sender knew them.
95.1.13.1.5. Mailbox quota¶
A quota applies to a recipient through [pepsi] MAILBOX_QUOTA or through
that account’s own pepsi.mailbox_quota row. When one applies, the stage
decides, per message and per recipient, whether the helper must measure the mailbox before writing
into it. It does that from the running estimate in the same table: the last
measurement plus everything Pepsi has delivered since. Because nothing ever
subtracts from that estimate (Pepsi is not told when a user deletes mail over
IMAP), it is an upper bound, so an estimate that says there is room is a
proof that there is. The helper is asked to walk the tree only when the account has
never been measured, its last measurement is older than [pepsi]
MAILBOX_QUOTA_MAX_AGE (default 15 m), its message count is at a
MAILBOX_QUOTA_COUNT limit, or the messages delivered since the measurement,
each counted at this message’s size, could fill what is left. Otherwise a
delivery costs one small append.
A helper that reports the mailbox full (exit 3: over the limit that it was
given, or a kernel EDQUOT) leaves the recipient’s fate to
[pepsi] MAILBOX_OVER_QUOTA:
defer(the default)The message stays queued and is retried exactly as a transient failure is, until MAX_LIFETIME and only then bounced. Deferring gives the account time to make room: a mailbox that is briefly full is far more common than one that stays full.
bounceThe recipient is routed at once to QUOTA_LIMIT_STAGE with an RFC 3463
5.2.2(mailbox full), so the sender learns immediately instead of days later.
Every measurement that the helper reports is written back, which is what lets
pepsi-ingress(1) refuse an over-quota recipient at RCPT. The sending
MTA is still on the line there, so it tells its own user, and Pepsi generates no
backscatter. See pepsi-quota(1) for the operator’s view and for the
reconciliation sweep that keeps those measurements fresh.
A message that is itself a bounce is not exempt: it is refused like any other and, because a bounce is never re-bounced, dropped with a warning. That is correct, because the alternative is a loop, but it means that mail disappears, so the log line naming it is the one to watch.
95.1.13.1.6. Arguments and sub-commands¶
- worker
Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input and writing one status line each. This is how the dispatcher runs the stage in production.
95.1.13.1.7. Configuration¶
Read from the message’s [stage-<stage>] section:
- PROGRAM
Must be
pepsi-stage-relay-to-maildir.- SERVER_NAME (required)
The host name recorded in the prepended
Received:trace header.- TARGETS
Which local accounts may receive mail, as a comma/space separated list of: a user name (
alice); a uid (452); an inclusive uid range (1000-1100); or an open-ended uid range (10000-, every uid at or above the bound). A token is read as a range only when both hyphen-separated parts are numeric, so a hyphenated user name (pepsi-maildir) is unambiguous. When unset it defaults to the regular (non-system) uid range from/etc/login.defs(UID_MIN..``UID_MAX``), falling back to1000-60000. Independently of TARGETS, pepsi-helper-maildir-writer(1) refuses every uid belowUID_MIN, so a token naming a system account (such as the452above, on a host withUID_MIN1000) makes delivery to it fail permanently; pepsi-setup(1) warns about such a token.- LOCAL_DOMAINS
Whitespace/comma separated domains whose recipients are eligible for local delivery. Defaults to
[pepsi-ingress] ACCEPTED_DOMAINS.- RECIPIENT_DELIMITER
The sub-address separator (
alice+listsdelivers to mailboxalice). Defaults to[pepsi] RECIPIENT_DELIMITERand, failing that, to+; set it tononeto disable sub-address stripping. Prefer setting the[pepsi]one: pepsi-ingress(1) uses that value to work out whose mailbox quota a recipient belongs to, and has no stage section to read. pepsi-setup(1) warns when a stage overrides it, because sub-addressed mail then escapes theRCPT-time refusal.- HELPER
The privileged delivery helper to run. Defaults to
pepsi-helper-maildir-writer(resolved on$PATH); set an absolute path to override.- NEXT_STAGE
Where recipients on other domains are forwarded. Required whenever a message may have such recipients. The wizard wires it to the SRS relay tail (
srs→ the smarthost, orsrs→ direct-to-MX internet delivery when no smarthost is configured), so an alias that expands off-site is actually delivered.- UNKNOWN_MAILBOX_STAGE
Where a recipient on one of our own LOCAL_DOMAINS goes when it resolves to no permitted account. Defaults to NEXT_STAGE, which is correct when NEXT_STAGE is itself a bounce stage or a smarthost that owns the domain’s accounts. Set it explicitly (the wizard sets it to the bounce stage) whenever NEXT_STAGE reaches a relay, or mail for an unknown local user is relayed back to this host’s own MX and loops until MAX_HOP_COUNT.
- BOUNCE_STAGE
The stage that generates DSNs. A recipient is routed there when its mailbox is permanently unusable, or once a transient failure has been retried past MAX_LIFETIME. Without it, that recipient’s row is left
failed.- QUOTA_LIMIT_STAGE
Where a recipient goes when its mailbox is full and
[pepsi] MAILBOX_OVER_QUOTAisbounce. Defaults to BOUNCE_STAGE. Named and used exactly like pepsi-stage-relay-to-lmtp(1)’s option of the same name, which routes thex.2.2that an MDA reports for the same condition: whichever local delivery path a site uses, the operator configures one thing.- RETRY_INITIAL, RETRY_MAX_INTERVAL, RETRY_FACTOR, MAX_LIFETIME
The retry/give-up timing for a transient failed local write (same meaning as the relay stages); a permanently unusable mailbox is bounced at once and never waits for these. Durations use
h/m/sunits. Defaults:5 m,2 h,2and120 h.
95.1.13.1.8. Installation and privileges¶
To reach the setuid-root helper, pepsi-stage-relay-to-maildir must itself be
installed set-group-id to the pepsi-maildir group, with mode 2550
and owner pepsi:pepsi-maildir. The mode allows owner-execute and not
world-execute, because the setgid bit is the gate on the setuid-root helper.
The right to execute cannot come from the group bits either, since pepsi is
deliberately not a member of the group, so it comes from the owner.
pepsi-dispatch(1) runs the stage as the unprivileged pepsi service
user. The setgid bit gives the worker an effective gid of pepsi-maildir,
which is exactly what lets it execute the group-restricted
pepsi-helper-maildir-writer(1); nothing else on the host gains that
ability. make install sets this up (its
install-maildir-stage step), provided it is run as root and the
pepsi-maildir group exists; otherwise it prints the exact commands to run by
hand.
Because the setgid bit is that gate, the program enforces the same rule itself.
When it runs with that bit and the real user is neither root nor the
pepsi service account, it exits with an error before any configuration is
read. Like the setuid crypto stages, it also strips the environment variables that steer configuration loading (HOME,
XDG_CONFIG_HOME, PG*, TALER_*, PEPSI_*) and pins PATH to a
safe default.
The file mode belongs to the deployment and this check belongs to the program,
so neither has to provide the protection on its own.
The Security chapter of the Pepsi manual lists every program carrying a setuid or setgid bit and why; its Threat model chapter states what a local account can and cannot gain through them.
95.1.13.1.9. See Also¶
pepsi-helper-maildir-writer(1), pepsi-quota(1), pepsi-stage-relay-to-smarthost(1), pepsi-stage-relay-to-internet(1), pepsi-stage-bounce(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), maildir(5)
95.1.13.1.10. Bugs¶
Report bugs to the Pepsi issue tracker.