95.1.26. pepsi-stage-auto-whitelist

whitelist recipients of outgoing mail

Manual section:

1

95.1.26.1.1. Name

pepsi-stage-auto-whitelist - the recipient-recording stage of the Pepsi pipeline.

95.1.26.1.2. Synopsis

pepsi-stage-auto-whitelist [GLOBAL-OPTIONS] worker

95.1.26.1.3. Description

This stage defaults to FUSION = yes: when stage fusion is enabled ([pepsi] ALLOW_FUSION, the default) and this stage is folded into the unified pepsi binary, a predecessor may run it in its own worker process instead of dispatching it separately. See pepsi-dispatch(1) and pepsi.conf(5).

pepsi-stage-auto-whitelist is a stage program run by pepsi-dispatch(1) as a persistent worker that reads message ids on standard input. For each id it loads the pepsi.workqueue row (refusing to act unless its status is running) and reads its [stage-<stage>] section. It is the write-side counterpart of pepsi-stage-check-whitelist(1): as an outgoing message passes through, it records every envelope recipient (rcpt_to) in a named whitelist, so that when that person later replies, pepsi-stage-check-whitelist(1) recognises their From: header and lets the reply skip the anti-spam payment gate. Place it on the outbound path (for example, ahead of a relay stage).

The whitelist lives in the pepsi.whitelist table, shared with pepsi-stage-check-whitelist(1); this stage only INSERTs into it (and refreshes the mailing-list rows that it learnt). Each recipient address is stored as a whitelist_regex: the address is lower-cased, every non-alphanumeric character is backslash-escaped so it matches literally, and the result is anchored on ^/$ so it matches that address and nothing else. The check stage matches that regex case-insensitively (PostgreSQL’s ~* operator) against the addr-spec parsed out of a reply’s From: header, not against the raw header value. A whitelisted address written into the display-name position (From: "bob@example.com" <evil@attacker.example>) therefore cannot spoof a match. The boundary anchoring is what additionally stops bob@example.com.evil from matching.

Each inserted row carries three condition flags that govern when a future reply counts as whitelisted:

  • dkim_required is set from the stage’s DKIM_REQUIRED option (default yes): when true, the reply is only trusted if its From: domain is authenticated: by a DKIM pass, or by an ARC pass that DMARC also confirms (see pepsi-stage-check-whitelist(1)).

  • dmarc_required is set from the stage’s DMARC_REQUIRED option (default yes): when true, the reply is only trusted if it passed DMARC on an aligned identifier, an aligned SPF pass included. A dkim_required pass implies it; with DKIM_REQUIRED = no (for correspondents whose domains do not sign) it is what still binds the row to an authenticated sender, so a forged From: naming a correspondent does not match.

  • signature_required is set to true when the outgoing message’s state has encrypted: true (the message was end-to-end encrypted), otherwise false. That key is written by pepsi-stage-encrypt(1), so this stage must run after the encrypt stage (and, since it changes no content, may run between it and pepsi-stage-dkim-sign(1), which is where pepsi-setup(1) --wizard puts it). If the stage is placed earlier, no row ever carries the flag. With the right placement, a row for an encrypted correspondent matches a reply only when the reply arrives with a verified signature (state.signature_verified, which pepsi-stage-decrypt(1) sets).

Every recipient row is written with match_field = 'from': what is recorded is an address from which a reply will arrive.

95.1.26.1.3.1. Mailing lists

A mailing list’s postings carry arbitrary From: addresses, so the recipient rows cannot cover them. With LIST_WHITELIST_NAME set, the stage also learns the lists that its users subscribe to, as match_field = 'list-id' rows matched against a posting’s List-Id:. It learns from one signal only: the user answering a list’s subscription confirmation request. That is a deliberate act. The stage does not learn from a reply to just any message that carried a List-Id:, since anybody who can get one message answered could then plant a rule.

The confirmation request is inbound mail and this stage sees only outbound mail, so the two are correlated through pepsi.list_confirmation: pepsi-stage-check-whitelist(1) with RECORD_LIST_CONFIRMATIONS records each request that it sees (by its Message-ID:, its recipient, its list and its control addresses), and this stage learns the list when all of the following hold for an outgoing message:

  • it is locally submitted (state.local_origin);

  • a recorded request, still within LIST_CONFIRMATION_DAYS, was delivered to exactly this envelope sender;

  • the message is addressed to one of that request’s control addresses (the -request / -confirm+<token> address that the reply goes to); and

  • its In-Reply-To: names the request’s Message-ID:.

The first three are answered from the envelope; the header block, which this stage otherwise never loads, is read only for a message that already looks like a confirmation reply. The row is written into the sender’s own namespace: LIST_WHITELIST_NAME must begin with {login}/, which is expanded in the same way as in pepsi-stage-check-whitelist(1), so a list that one user joined is never whitelisted for anybody else. The row has an exact ^…$ pattern of the list identifier, dkim_required from DKIM_REQUIRED (default yes: a List-Id: is text that anybody can copy, so a posting must still be authenticated), dmarc_required from DMARC_REQUIRED, and no sealer_domain. (Pinning whatever sealed the request would be trust on first use, and would silently break the rule when the list changes its infrastructure.) Leaving a list sends no signal, so a learnt row expires. It carries last_seen and max_idle (LIST_MAX_IDLE_DAYS, default 180). The check stage ignores the row once last_seen + max_idle has passed, and moves last_seen forward whenever an authenticated posting of the list matches it; pepsi-queue gc deletes it. A row for the same list that never expires (written with pepsi-whitelist add-list or confirmed at a pepsi-whitelist import prompt) is left as it is.

How a request is recognised, and therefore what it takes to plant one, is described under “Mailing-list confirmation requests” in pepsi-stage-check-whitelist(1).

Existing rows are left untouched (the insert is ON CONFLICT DO NOTHING on the UNIQUE(whitelist_name, match_field, whitelist_regex) constraint), so reprocessing a message is harmless. The stage then advances the message to NEXT_STAGE. It never modifies the message’s own state and never drops, bounces or pauses a message; a message with no recipients simply advances.

95.1.26.1.4. Configuration

Options live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-auto-whitelist): the required WHITELIST_NAME, which names the group to fill (one literal whitelist name; the stage expands no {login} or {localpart} placeholder and refuses a value that contains one), DKIM_REQUIRED and DMARC_REQUIRED (both default yes), the mailing-list options LIST_WHITELIST_NAME (unset: no lists are learnt) and LIST_MAX_IDLE_DAYS (default 180, 1 to 3650), and a NEXT_STAGE. The NEXT_STAGE is also required: the stage records the recipients and then always hands the message on, so pepsi-setup rejects a section without one. The options are documented in pepsi.conf(5).

95.1.26.1.4.1. Whose whitelist

The stage runs on the outbound path, so the per-address layers of the configuration are looked up for the envelope sender: that address’s pepsi.config_override rows at domain:/address: scope and its pepsi.settings row decide which list its recipients are written to.

The operator may name any whitelist, a shared one included, in each of its own layers: the INI file, pepsi.config_override at any scope, and a pepsi.settings row written with pepsi-settings(1). The stage writes to the effective value as it finds it.

An account owner who may edit this stage by mail (it is in pepsi-stage-edit-settings(1)’s EDITABLE_STAGES) may only choose a whitelist of their own: a name in the <login>/... namespace of the account that their address resolves to (alice/sent, say), or the whitelist that the operator’s configuration already names for them. pepsi-stage-edit-settings(1) refuses anything else, because it would let them fill that list with addresses of their choosing: a shared list, another user’s list, or any value for an address that is no local account. The login is resolved as pepsi-stage-check-whitelist(1) resolves {login}: the address’s domain must be one of LOCAL_DOMAINS (default [pepsi-ingress] ACCEPTED_DOMAINS), its local part, with the sub-address after RECIPIENT_DELIMITER removed, must be a passwd login, and that account must be permitted by TARGETS. These three options are read from the operator’s stage section, never from the owner’s own override.

A business may want every correspondent to be recognised whoever they reply to, for example a customer who is told to write to a colleague rather than to the employee who first mailed them. Such a business shares one whitelist among all local users, written by this stage and read by the check stage:

[stage-auto-whitelist]
PROGRAM = pepsi-stage-auto-whitelist
NEXT_STAGE = dkim-sign
WHITELIST_NAME = correspondents

[stage-check-whitelist]
PROGRAM = pepsi-stage-check-whitelist
NEXT_STAGE = anti-spam
WHITELIST_NAME = correspondents

95.1.26.1.5. State

Inputs: the envelope recipients come from the rcpt_to column; state.encrypted sets the stored signature_required flag. pepsi-stage-encrypt(1) writes that key, earlier on the same outbound path, when the outgoing message really was encrypted to a recipient key. A correspondent reached under encryption is thus later held to the same standard.

Outputs: none on the message itself; the message state is left untouched. The side effect is one pepsi.whitelist row per recipient, plus one learnt list-id row per list whose confirmation request the message answers (see “Mailing lists”). The state layout is described in pepsi.state(7).

Transitions: always advances to NEXT_STAGE: after inserting one whitelist row per envelope recipient, or unchanged when there are none. There is no branch and no reroute or finish. Only a per-address override that breaks the stage’s section fails the message (permanently, like any stage’s configuration error); a failed recording does not (see “Failures”).

95.1.26.1.6. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

95.1.26.1.7. Failures

Recording is fail-open. When the INSERT fails (a database error, a missing grant), or the mailing-list check cannot complete, the stage logs a warning and advances the message anyway: the whitelist is a convenience for the recipient’s reply, and holding the user’s outbound mail until it can be recorded would be the wrong way round. A pattern that PostgreSQL could not compile is never written (the patterns are escaped addresses, so this is a safeguard, not an expected case).

95.1.26.1.8. Global Options

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default [pepsi] LOG, else info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

95.1.26.1.9. Exit Status

The outcome of each message is reported to pepsi-dispatch(1) on the worker’s status line, not as the exit status.

0

The worker ran until its standard input was closed.

1

A fatal error occurred (unreadable configuration, the database could not be opened, or standard input/output failed). The reason is written to the log.

78

The worker refused to start: a [stage-*] section it serves does not parse (e.g. a missing WHITELIST_NAME, a placeholder in it, a LIST_WHITELIST_NAME not starting with {login}/, or LIST_MAX_IDLE_DAYS outside 1 to 3650). The dispatcher requeues what it handed over and retries the stage later.

95.1.26.1.10. Examples

Record the recipients of message 42 (it must be running):

echo 42 | pepsi-stage-auto-whitelist -c /etc/pepsi/pepsi.conf worker

An outbound pipeline section that whitelists everyone we mail, requiring their replies to come from an authenticated From: domain:

[stage-auto-whitelist]
PROGRAM = pepsi-stage-auto-whitelist
NEXT_STAGE = srs
WHITELIST_NAME = trusted-senders
DKIM_REQUIRED = yes

95.1.26.1.11. See Also

pepsi-config(1), pepsi-stage-check-whitelist(1), pepsi-stage-anti-spam(1), pepsi-stage-arc(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

95.1.26.1.12. Bugs

Report bugs to the Pepsi issue tracker.