95.1.26. pepsi-stage-auto-whitelist¶
whitelist recipients of outgoing mail
- Manual section:
1
95.1.26.1.1. Name¶
pepsi-stage-auto-whitelist - the recipient-recording stage of the Pepsi pipeline.
95.1.26.1.2. Synopsis¶
pepsi-stage-auto-whitelist [GLOBAL-OPTIONS] worker
95.1.26.1.3. Description¶
This stage defaults to FUSION = yes: when stage fusion is enabled ([pepsi]
ALLOW_FUSION, the default) and this stage is folded into the unified pepsi
binary, a predecessor may run it in its own worker process instead of
dispatching it separately. See pepsi-dispatch(1) and pepsi.conf(5).
pepsi-stage-auto-whitelist is a stage program run by pepsi-dispatch(1)
as a persistent worker that reads message ids on standard input. For each id it
loads the pepsi.workqueue row (refusing to act unless its status is
running) and reads its
[stage-<stage>] section. It is the write-side counterpart of
pepsi-stage-check-whitelist(1): as an outgoing message passes through,
it records every envelope recipient (rcpt_to) in a named whitelist, so that
when that person later replies, pepsi-stage-check-whitelist(1)
recognises their From: header and lets the reply skip the anti-spam payment
gate. Place it on the outbound path (for example, ahead of a relay stage).
The whitelist lives in the pepsi.whitelist table, shared with
pepsi-stage-check-whitelist(1); this stage only INSERTs into it (and
refreshes the mailing-list rows that it learnt). Each recipient address is stored as a
whitelist_regex: the address is lower-cased, every non-alphanumeric character
is backslash-escaped so it matches literally, and the result is anchored on
^/$ so it matches that address and nothing else. The check stage matches
that regex case-insensitively
(PostgreSQL’s ~* operator) against the addr-spec parsed out of a reply’s
From: header, not against the raw header value. A whitelisted address
written into the display-name position
(From: "bob@example.com" <evil@attacker.example>) therefore cannot spoof a
match. The boundary anchoring is what additionally stops
bob@example.com.evil from matching.
Each inserted row carries three condition flags that govern when a future reply counts as whitelisted:
dkim_requiredis set from the stage’s DKIM_REQUIRED option (default yes): when true, the reply is only trusted if itsFrom:domain is authenticated: by a DKIM pass, or by an ARC pass that DMARC also confirms (see pepsi-stage-check-whitelist(1)).dmarc_requiredis set from the stage’s DMARC_REQUIRED option (default yes): when true, the reply is only trusted if it passed DMARC on an aligned identifier, an aligned SPF pass included. Adkim_requiredpass implies it; with DKIM_REQUIRED= no(for correspondents whose domains do not sign) it is what still binds the row to an authenticated sender, so a forgedFrom:naming a correspondent does not match.signature_requiredis set to true when the outgoing message’s state hasencrypted: true(the message was end-to-end encrypted), otherwise false. That key is written by pepsi-stage-encrypt(1), so this stage must run after the encrypt stage (and, since it changes no content, may run between it and pepsi-stage-dkim-sign(1), which is where pepsi-setup(1)--wizardputs it). If the stage is placed earlier, no row ever carries the flag. With the right placement, a row for an encrypted correspondent matches a reply only when the reply arrives with a verified signature (state.signature_verified, which pepsi-stage-decrypt(1) sets).
Every recipient row is written with match_field = 'from': what is recorded
is an address from which a reply will arrive.
95.1.26.1.3.1. Mailing lists¶
A mailing list’s postings carry arbitrary From: addresses, so the recipient
rows cannot cover them. With LIST_WHITELIST_NAME set, the stage also learns
the lists that its users subscribe to, as match_field = 'list-id' rows
matched against a posting’s List-Id:. It learns from one signal only: the
user answering a list’s subscription confirmation request. That is a
deliberate act. The stage does not learn from a reply to just any message that
carried a List-Id:, since anybody who can get one message answered could
then plant a rule.
The confirmation request is inbound mail and this stage sees only outbound mail,
so the two are correlated through pepsi.list_confirmation:
pepsi-stage-check-whitelist(1) with RECORD_LIST_CONFIRMATIONS records
each request that it sees (by its Message-ID:, its recipient, its list and its
control addresses), and this stage learns the list when all of the following
hold for an outgoing message:
it is locally submitted (
state.local_origin);a recorded request, still within LIST_CONFIRMATION_DAYS, was delivered to exactly this envelope sender;
the message is addressed to one of that request’s control addresses (the
-request/-confirm+<token>address that the reply goes to); andits
In-Reply-To:names the request’sMessage-ID:.
The first three are answered from the envelope; the header block, which this
stage otherwise never loads, is read only for a message that already looks like a
confirmation reply. The row is written into the sender’s own namespace:
LIST_WHITELIST_NAME must begin with {login}/, which is expanded in the
same way as in pepsi-stage-check-whitelist(1), so a list that one user
joined is never whitelisted for anybody else. The row has an exact ^…$
pattern of the list identifier, dkim_required from DKIM_REQUIRED
(default yes: a List-Id: is text that anybody can copy, so a posting must
still be authenticated), dmarc_required from DMARC_REQUIRED, and no
sealer_domain. (Pinning whatever sealed the request would be trust on first
use, and would silently break the rule when the list changes its
infrastructure.) Leaving a list sends no signal, so a learnt row expires. It
carries last_seen and max_idle (LIST_MAX_IDLE_DAYS, default 180).
The check stage ignores the row once last_seen + max_idle has passed, and
moves last_seen forward whenever an authenticated posting of the list
matches it; pepsi-queue gc deletes it. A row for the same list that never expires
(written with pepsi-whitelist add-list or confirmed at a
pepsi-whitelist import prompt) is left as it is.
How a request is recognised, and therefore what it takes to plant one, is described under “Mailing-list confirmation requests” in pepsi-stage-check-whitelist(1).
Existing rows are left untouched (the insert is ON CONFLICT DO NOTHING on the
UNIQUE(whitelist_name, match_field, whitelist_regex) constraint), so
reprocessing a message is harmless. The stage then advances the message to
NEXT_STAGE. It never modifies the message’s own state and never drops,
bounces or pauses a message; a message with no recipients simply advances.
95.1.26.1.4. Configuration¶
Options live in the stage’s own [stage-<name>] section
(PROGRAM = pepsi-stage-auto-whitelist): the required WHITELIST_NAME,
which names the group to fill (one literal whitelist name; the stage expands no
{login} or {localpart} placeholder and refuses a value that contains
one), DKIM_REQUIRED and DMARC_REQUIRED (both default yes), the
mailing-list options LIST_WHITELIST_NAME (unset: no lists are learnt) and
LIST_MAX_IDLE_DAYS (default 180, 1 to 3650), and a NEXT_STAGE. The
NEXT_STAGE is also required: the stage records the recipients and then
always hands the message on, so pepsi-setup rejects a section without one.
The options are documented in pepsi.conf(5).
95.1.26.1.4.1. Whose whitelist¶
The stage runs on the outbound path, so the per-address layers of the
configuration are looked up for the envelope sender: that address’s
pepsi.config_override rows at domain:/address: scope and its
pepsi.settings row decide which list its recipients are written to.
The operator may name any whitelist, a shared one included, in each of its
own layers: the INI file, pepsi.config_override at any scope, and a
pepsi.settings row written with pepsi-settings(1). The stage writes to
the effective value as it finds it.
An account owner who may edit this stage by mail (it is in
pepsi-stage-edit-settings(1)’s EDITABLE_STAGES) may only choose a
whitelist of their own: a name in the <login>/... namespace of the account
that their address resolves to (alice/sent, say), or the whitelist that the
operator’s configuration already names for them. pepsi-stage-edit-settings(1)
refuses anything else, because it would let them fill that list with addresses
of their choosing: a shared list, another user’s list, or any value for an
address that is no local account. The login is resolved as
pepsi-stage-check-whitelist(1) resolves {login}: the address’s domain
must be one of LOCAL_DOMAINS (default [pepsi-ingress] ACCEPTED_DOMAINS),
its local part, with the sub-address after RECIPIENT_DELIMITER removed, must
be a passwd login, and that account must be permitted by TARGETS. These
three options are read from the operator’s stage section, never from the
owner’s own override.
A business may want every correspondent to be recognised whoever they reply to, for example a customer who is told to write to a colleague rather than to the employee who first mailed them. Such a business shares one whitelist among all local users, written by this stage and read by the check stage:
[stage-auto-whitelist]
PROGRAM = pepsi-stage-auto-whitelist
NEXT_STAGE = dkim-sign
WHITELIST_NAME = correspondents
[stage-check-whitelist]
PROGRAM = pepsi-stage-check-whitelist
NEXT_STAGE = anti-spam
WHITELIST_NAME = correspondents
95.1.26.1.5. State¶
Inputs: the envelope recipients come from the rcpt_to column;
state.encrypted sets the stored signature_required flag.
pepsi-stage-encrypt(1) writes that key, earlier on the same outbound path,
when the outgoing message really was encrypted to a recipient key. A
correspondent reached under encryption is thus later held to the same standard.
Outputs: none on the message itself; the message state is left
untouched. The side effect is one pepsi.whitelist row per recipient, plus
one learnt list-id row per list whose confirmation request the message
answers (see “Mailing lists”). The
state layout is described in pepsi.state(7).
Transitions: always advances to NEXT_STAGE: after inserting one whitelist row per envelope recipient, or unchanged when there are none. There is no branch and no reroute or finish. Only a per-address override that breaks the stage’s section fails the message (permanently, like any stage’s configuration error); a failed recording does not (see “Failures”).
95.1.26.1.6. Commands¶
- worker
Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.
95.1.26.1.7. Failures¶
Recording is fail-open. When the INSERT fails (a database error, a
missing grant), or the mailing-list check cannot complete, the stage logs a
warning and advances the message anyway: the
whitelist is a convenience for the recipient’s reply, and holding the user’s
outbound mail until it can be recorded would be the wrong way round. A pattern
that PostgreSQL could not compile is never written (the patterns are escaped
addresses, so this is a safeguard, not an expected case).
95.1.26.1.8. Global Options¶
- -c FILE, –config FILE
Read the configuration from FILE instead of searching the default locations.
- -L LOGLEVEL, –log LOGLEVEL
Set the logging verbosity (default
[pepsi] LOG, elseinfo).- -v, –verbose
Show log messages from all sources.
- -h, –help; -V, –version
Print a usage summary / the version and exit.
95.1.26.1.9. Exit Status¶
The outcome of each message is reported to pepsi-dispatch(1) on the worker’s status line, not as the exit status.
- 0
The worker ran until its standard input was closed.
- 1
A fatal error occurred (unreadable configuration, the database could not be opened, or standard input/output failed). The reason is written to the log.
- 78
The worker refused to start: a
[stage-*]section it serves does not parse (e.g. a missing WHITELIST_NAME, a placeholder in it, a LIST_WHITELIST_NAME not starting with{login}/, or LIST_MAX_IDLE_DAYS outside 1 to 3650). The dispatcher requeues what it handed over and retries the stage later.
95.1.26.1.10. Examples¶
Record the recipients of message 42 (it must be running):
echo 42 | pepsi-stage-auto-whitelist -c /etc/pepsi/pepsi.conf worker
An outbound pipeline section that whitelists everyone we mail, requiring their
replies to come from an authenticated From: domain:
[stage-auto-whitelist]
PROGRAM = pepsi-stage-auto-whitelist
NEXT_STAGE = srs
WHITELIST_NAME = trusted-senders
DKIM_REQUIRED = yes
95.1.26.1.11. See Also¶
pepsi-config(1), pepsi-stage-check-whitelist(1), pepsi-stage-anti-spam(1), pepsi-stage-arc(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)
95.1.26.1.12. Bugs¶
Report bugs to the Pepsi issue tracker.