85.1.24. pepsi-stage-check-whitelist

mark trusted senders’ mail as non-spam

Manual section:

1

85.1.24.1.1. Name

pepsi-stage-check-whitelist - the sender-whitelist stage of the Pepsi pipeline.

85.1.24.1.2. Synopsis

pepsi-stage-check-whitelist [GLOBAL-OPTIONS] worker

85.1.24.1.3. Description

This stage defaults to FUSION = yes: when stage fusion is enabled ([pepsi] ALLOW_FUSION, the default) and this stage is folded into the unified pepsi binary, a predecessor may run it in its own worker process instead of dispatching it separately. Because this stage needs the message’s header block (for List-Id:), only a predecessor that loaded the headers or the whole message can fuse it; a predecessor that loaded envelope metadata alone dispatches it normally. See pepsi-dispatch(1) and pepsi.conf(5).

pepsi-stage-check-whitelist is a stage program run by pepsi-dispatch(1) as a persistent worker reading message ids on standard input. It loads that pepsi.workqueue row (refusing to act unless its status is running), reads its [stage-<stage>] section, and consults a named whitelist of trusted senders. If the message matches the whitelist, the stage records state.spam = false so a later pepsi-stage-anti-spam(1) lets it through without payment. Either way the message then advances to NEXT_STAGE — this stage never drops, bounces or pauses a message.

The whitelist lives in the pepsi.whitelist table, populated out of band by the operator (pepsi-whitelist(1)), by the outbound pepsi-stage-auto-whitelist(1), or by a user importing their own mailbox (pepsi-whitelist import). Each row pairs a whitelist_name (a group label) with a whitelist_regex, the match_field naming what that expression is matched against, and the conditions that gate the match. The stage’s WHITELIST_NAME option selects the group or groups; whether any of their rows matches — subject to its conditions — is decided by PostgreSQL in one query. The whitelist_regex is an extended POSIX regular expression, matched case-insensitively (it is evaluated by PostgreSQL’s ~* operator, so the dialect is exactly POSIX ERE). The expression is unanchored; anchor it with ^/$ to match the whole subject.

A row whose expression does not compile matches nothing; it does not fail the check. Every tool that writes the table validates an expression first, but a row that got in anyway (a hand-written INSERT) would otherwise make PostgreSQL refuse the whole query – and since every applicable name is checked in that one query, one user’s bad row would hold up the mail of everybody sharing a whitelist name with them. The stage logs a warning naming the whitelists concerned; pepsi-whitelist list finds the row.

85.1.24.1.3.1. What is matched

match_field selects the subject of the match:

  • from (the default): the addr-spec parsed out of the message’s From: header — not the raw header value, so a whitelisted address written into the display-name position cannot spoof a match, and RFC 5322 comments and folding whitespace are removed. A message with no From: header matches nothing, and so does one whose From: names more than one mailbox (the DKIM/DMARC verdict dkim_required consults was computed for the first address, so matching any other would decide two questions about two different senders).

  • list-id: the RFC 2919 list identifier of the message’s List-Id: header, reduced to the bracketed token, trimmed and lower-cased (List-Id: Rust users <users.rust-lang.org> matches against users.rust-lang.org). A message with no List-Id: matches nothing.

A list-id row exists because a mailing list’s postings carry arbitrary From: addresses. No pattern over From: can say “anything that came through this list”, so a subscriber would otherwise have to whitelist every individual poster — or pay the gate for every message the list relays. Matching List-Id: says it once. Because that header is unauthenticated text anyone can copy, pair such a row with sealer_domain (below).

85.1.24.1.3.2. Several groups, and per-user whitelists

WHITELIST_NAME is a comma-separated list, and an entry may be a template containing {localpart} or {login}, which is expanded per envelope recipient:

WHITELIST_NAME = correspondents, {localpart}/correspondents

That consults the operator’s shared list and each recipient’s own. The <login>/... namespace is the one pepsi-whitelist(1) lets an ordinary user manage and seed from their mailbox, so this is what makes a per-user whitelist take effect — without it, such a name is only reachable through a per-address pepsi.settings override (pepsi-settings(1)).

{localpart} expands to the recipient’s local part with any sub-address stripped (alice+lists@ → alice). {login} first resolves the recipient to a passwd login, which is what lines up with the namespace when the address is not itself a login (first.last@ → alice); it therefore needs the recipient to resolve locally, so place the stage after pepsi-stage-aliases(1) when using it. A recipient a template cannot be expanded for is skipped, as is any expansion outside the whitelist-name grammar (letters, digits, _, . and - per /-separated segment, and no segment made only of dots).

Only a recipient at a served domain names a whitelist. Both placeholders are expanded solely for recipients whose domain is in LOCAL_DOMAINS (defaulting to [pepsi-ingress] ACCEPTED_DOMAINS); a recipient elsewhere is skipped. This matters because a match in any consulted list clears the whole message: the local part of a foreign address is chosen by whoever wrote the envelope, so without the test a sender already in one user’s whitelist could add RCPT TO:<thatuser@anything.invalid> to a message for somebody else and have it cleared.

All applicable names are checked in one query, and a match in any of them marks the message non-spam — the verdict is per message, and a queue row is one message. Note that this pools trust between the served recipients of one message: a message addressed to two of your users is non-spam if it matches either one’s list. The locality options LOCAL_DOMAINS, TARGETS and RECIPIENT_DELIMITER are read only to expand these placeholders.

A row may further restrict when it counts as a match:

  • dkim_required: the row matches only if the message’s From: domain is authenticated — that is, ingress recorded state.auth.dkim = pass, or pepsi-stage-arc(1) recorded state.auth.arc = pass and DMARC also passed (state.auth.dmarc = pass). An ARC pass on its own is only chain integrity — a sender controlling any single domain can mint a valid one-hop ARC chain over a forged From: — so it is accepted only when DMARC confirms the From: alignment.

  • signature_required: the row matches only if state.signature_verified is true, which pepsi-stage-decrypt(1) sets for a message carrying a signature it verified against a trusted key.

  • sealer_domain: the row matches only if the message arrived carrying an ARC chain that validated (state.auth.arc = pass) and the named ADMD is one of that chain’s sealers (state.auth.arc_sealers, recorded by pepsi-stage-arc(1)).

sealer_domain is what makes a rule about an intermediary safe to write, and neither half suffices alone. List-Id: proves nothing: it is text a sender copies out of a genuine posting. A valid ARC chain proves nothing either — RFC 8617 §8.4 is explicit that it conveys no trust, only that the ADMDs named in it really did handle these bytes, which is exactly why anyone can mint one over a forged message. Naming the sealer is the local policy the protocol deliberately leaves to the receiver: the operator (or the user) states which intermediary they accept, and the chain then proves that intermediary handled the message. It applies to from rows too, where it narrows a sender pattern to mail relayed through a known forwarder.

Note that the domains in state.auth.arc_sealers are recorded whether or not the chain validated, because they are a record of what was claimed; the stage discards all of them unless state.auth.arc is pass, so a forged chain can never satisfy a sealer_domain row.

If any row matches (subject to its conditions), the stage merges spam: false into the message state and advances. Otherwise it advances the message unchanged.

85.1.24.1.4. Configuration

Options live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-check-whitelist): the required WHITELIST_NAME list of groups to consult, a NEXT_STAGE (also required — the stage always hands the message on, so pepsi-setup rejects a section without one), and — only when a WHITELIST_NAME entry uses a placeholder — the locality options LOCAL_DOMAINS / TARGETS / RECIPIENT_DELIMITER. They are documented in pepsi.conf(5); the whitelist rows themselves are managed with pepsi-whitelist(1).

85.1.24.1.5. State

Inputs: state.auth.dkim / state.auth.arc / state.auth.dmarc (for dkim_required rows), state.signature_verified (for signature_required rows), which pepsi-stage-decrypt(1) sets — and only for its valid signature verdict, so a sound signature by a key that could not be anchored (valid-untrusted) does not open the gate — and state.auth.arc / state.auth.arc_sealers (for sealer_domain rows), both written by pepsi-stage-arc(1). The From: header comes from the from_header column, not from state; List-Id: is read from the message’s header block, which is why this stage loads it (the body is never loaded).

Outputs: on a match the stage merges spam: false into state (the key pepsi-stage-anti-spam(1) reads to skip the payment gate) and advances; on no match state is left untouched. The state layout is described in pepsi.state(7).

Transitions: always advances to NEXT_STAGE, whether or not the sender matched the whitelist. A match is committed with the reroute helper, which merges spam: false and moves the stage in the same single UPDATE; a non-match is a plain advance. There is no branch; the stage never pauses, fails, bounces or finishes — the actual payment/drop decision is left to a later stage (pepsi-stage-anti-spam(1)).

85.1.24.1.6. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

85.1.24.1.7. Global Options

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

85.1.24.1.8. Exit Status

0

The message was processed (advanced, with or without a whitelist match).

1

An error occurred (message not found or not running, misconfigured stage — e.g. a missing WHITELIST_NAME — or a database error, including a malformed whitelist_regex). The reason is written to the log.

85.1.24.1.9. Examples

Check message 42 against the whitelist (it must be running):

echo 42 | pepsi-stage-check-whitelist -c /etc/pepsi/pepsi.conf worker

A pipeline section that whitelists trusted senders before the payment gate:

[stage-check-whitelist]
PROGRAM = pepsi-stage-check-whitelist
NEXT_STAGE = anti-spam
WHITELIST_NAME = trusted-senders

Consulting the shared list and each recipient’s own, so that a whitelist a user seeded with pepsi-whitelist import is honoured:

[stage-check-whitelist]
PROGRAM = pepsi-stage-check-whitelist
NEXT_STAGE = anti-spam
WHITELIST_NAME = trusted-senders, {localpart}/correspondents

Whitelisting a partner domain, but only when its mail is DKIM/ARC-signed:

INSERT INTO pepsi.whitelist (whitelist_name, whitelist_regex, dkim_required)
VALUES ('trusted-senders', '@partner\.example\.org$', TRUE);

dkim_required defaults to TRUE when an INSERT leaves it out, as it does for pepsi-whitelist add; a row trusting unsigned mail must say dkim_required = FALSE explicitly.

Letting a mailing list past the payment gate, whoever posted to it, but only when the list’s own ADMD sealed the message’s ARC chain:

pepsi-whitelist add-list alice/lists users.rust-lang.org \
    --sealer mail.rust-lang.org

Deliberately trusting the header alone, for a list that does not ARC-seal — this is what an attacker only has to copy a List-Id: to satisfy, so scope it to one user’s own whitelist rather than a shared one:

pepsi-whitelist add-list alice/lists announce.example.org --no-dkim-required

85.1.24.1.10. See Also

pepsi-config(1), pepsi-whitelist(1), pepsi-stage-anti-spam(1), pepsi-stage-arc(1), pepsi-settings(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

85.1.24.1.11. Bugs

Report bugs to the Pepsi issue tracker.