70.1.34. pepsi-whitelist

manage the sender whitelist, by hand or by mailbox

Manual section:

1

70.1.34.1.1. Name

pepsi-whitelist - add, list, remove and import sender whitelist entries.

70.1.34.1.2. Synopsis

pepsi-whitelist [GLOBAL-OPTIONS] add [–no-dkim-required] [–signature-required] [–sealer DOMAIN] NAME REGEX

pepsi-whitelist [GLOBAL-OPTIONS] add-list [–pattern] [–no-dkim-required] [–signature-required] [–sealer DOMAIN] NAME LIST-ID

pepsi-whitelist [GLOBAL-OPTIONS] list [NAME] [–json]

pepsi-whitelist [GLOBAL-OPTIONS] remove NAME REGEX

pepsi-whitelist [GLOBAL-OPTIONS] remove-list [–pattern] NAME LIST-ID

pepsi-whitelist [GLOBAL-OPTIONS] import [–name NAME] [–auto-wildcard] [–dry-run] [PATH…]

70.1.34.1.3. Description

pepsi-whitelist is the operator’s tool for the pepsi.whitelist table — the named sets of patterns that mark a message as trusted (non-spam). Each row has a whitelist_name (the group it belongs to), a whitelist_regex (a POSIX extended regular expression matched case-insensitively, via the ~* operator), a match_field naming what that expression is matched against — the message’s From: addr-spec or its List-Id: — and three conditions: dkim_required, signature_required and sealer_domain. They are described in pepsi-stage-check-whitelist(1), which is what evaluates them.

The whitelist is consulted by pepsi-stage-check-whitelist(1) and populated automatically from outgoing recipients by pepsi-stage-auto-whitelist(1); this tool lets an administrator manage rows directly, and lets each user seed a whitelist of their own from the mail they have already sent (import). It is not a stage. It connects to the same database as the other components, through the shared [pepsi-postgres] section, and reads the optional [pepsi-whitelist] section described in pepsi.conf(5).

Unlike pepsi-stage-auto-whitelist(1), which escapes a recipient address into a boundary-anchored pattern, add stores the whitelist_regex exactly as given, so the operator controls the matching directly. import escapes the addresses it finds, exactly as the stage does.

70.1.34.1.3.1. Mailing lists

add-list and remove-list manage the list-id rows that whitelist a mailing list as a whole — every posting it relays, whoever wrote it, which a From: pattern cannot express because a list’s postings carry arbitrary sender addresses.

They are separate subcommands rather than a flag on add because they take a different kind of argument. add stores its REGEX verbatim, which is right for a pattern an operator composed deliberately; a list identifier typed into that slot would be an unanchored regular expression, so users.example.org would also match users.example.org.attacker.example and whitelist a list of the attacker’s choosing. add-list takes the identifier as it appears in the header — brackets optional, case and surrounding phrase ignored — and anchors it itself. –pattern is the escape hatch for the deliberate wildcard (every list at a domain), and then the argument is a regular expression again, as with add.

A List-Id: header is unauthenticated text that any sender can copy out of a genuine posting, so a rule resting on it alone is a rule anyone can satisfy. –sealer DOMAIN is the other half: the row then also requires that the message arrived with an ARC chain that validated and that DOMAIN was one of its sealers, as recorded by pepsi-stage-arc(1). DOMAIN is normalised (lower-cased, trailing dot removed) to the same form the ARC stage records, so the two cannot disagree. It works on add rows too, where it restricts a sender pattern to mail relayed through a known forwarder.

70.1.34.1.3.2. Whitelist names and who may use them

A whitelist_name is either

global — a single segment such as correspondents, which only the operator

may manage; or

owned by a user — <login>/<segment> such as alice/correspondents, and

alice/lists/rust-lang for further sub-lists.

A segment consists of letters, digits, _ and -; the separator is /. A dot is deliberately not allowed, which makes .. unrepresentable, so a whitelist name can never be a relative path. / is the separator because no POSIX-ish system permits it in a login name, whereas -, _, . and a trailing $ are all legal in one and so could not delimit unambiguously.

Because this program is installed setuid (see below), any local user can run it. An unprivileged caller may only read and write names in their own namespace — the login of the real uid, taken from the passwd database, never from $USER or an argument. That restriction covers list as well: the operator’s shared whitelist is a record of who the organisation corresponds with, and this program must not hand it to every account on the host. root, and the pepsi, pepsi-owner and pepsi-whitelist accounts, may manage every whitelist.

For a per-user whitelist to have any effect, some stage must consult it; see the {localpart} and {login} placeholders of WHITELIST_NAME in pepsi-stage-check-whitelist(1).

70.1.34.1.3.3. The setuid model

Managing the shared pepsi.whitelist table needs a database identity that ordinary users do not have, while scanning a user’s mailbox needs their identity. The program is therefore installed setuid to the pepsi-whitelist account (mode 4755) and immediately lowers its effective ids back to the invoking user, keeping the owner only in the saved set-user-id. It raises them again for the few milliseconds it talks to PostgreSQL — peer authentication keys off the effective uid — and for nothing else.

The same mechanism is what lets root use the tool. The other operator commands — pepsi-queue(1), pepsi-status(1), pepsi-settings(1), pepsi-tlsrpt(1), pepsi-failure-bouncer(1) — simply become the pepsi service account when started as root, permanently, because they have nothing to do as root. This one does: import –user and –all-users run the scan helper under another user’s credentials, which only root can arrange. It therefore keeps the invoking identity, root included, and switches only its effective ids — to the pepsi-whitelist account, whose database role is granted the whitelist table alone — for the duration of each database call, returning to root afterwards.

That account is deliberately not the pepsi service account: pepsi-setup(1) grants its database role SELECT, INSERT and DELETE on the pepsi.whitelist table and nothing more, so a compromise of a binary every local user may execute cannot reach the message queue, the statistics or the signing keys.

The mailbox parsing — the one part that reads attacker-controlled data, namely every message anybody ever sent the user — does not happen in this process at all. It runs in pepsi-helper-mailbox-scan(1), spawned with setresuid setting the real, effective and saved ids to the target user, so that process has no privileged id left anywhere in its credentials and cannot become pepsi-whitelist even in principle.

Because a setuid program must not be pointed at a configuration file of the caller’s choosing, -c/–config is rejected for unprivileged callers, and the environment variables that would otherwise redirect the configuration search (HOME, XDG_CONFIG_HOME), the expansion of ${DATADIR} (PATH) or the database connection (PG*) are removed before the configuration is read.

70.1.34.1.4. Commands

add [–no-dkim-required] [–signature-required] [–sealer DOMAIN] NAME REGEX

Insert a from entry storing REGEX under whitelist NAME. The insert is idempotent on the UNIQUE(whitelist_name, match_field, whitelist_regex) constraint: re-adding an existing triple is a no-op and leaves its conditions unchanged.

–no-dkim-required

Clear the row’s dkim_required flag. By default it is set, so a match counts only if the sender’s DKIM or ARC signature verified.

–signature-required

Set the row’s signature_required flag. By default it is clear. When set, a match counts only if the message’s signature verified.

–sealer DOMAIN

Set the row’s sealer_domain. By default it is empty (any path). When set, a match counts only if the message arrived with an ARC chain that validated and DOMAIN was one of its sealers. DOMAIN is lower-cased and stripped of a trailing dot, matching what pepsi-stage-arc(1) records.

add-list [–pattern] [CONDITION-OPTIONS] NAME LIST-ID

Insert a list-id entry under whitelist NAME, matching every message whose List-Id: header carries the identifier LIST-ID. The identifier may be given bracketed or bare, in any case, with or without a descriptive phrase; it is reduced to the bare lower-cased form and anchored, so it matches that list and no other. Takes the same condition options as add, and pairing it with –sealer is strongly advised — see Mailing lists above.

–pattern

Treat LIST-ID as a POSIX extended regular expression to store verbatim, rather than as one identifier to anchor. For the deliberate wildcard case (every list at a domain); an unanchored expression matches any identifier containing it, so write the anchors yourself.

list [NAME] [–json]

List entries ordered by name, match field, then regex. With NAME, only that whitelist group is shown.

–json

Emit a JSON array of objects (whitelist_id, whitelist_name, whitelist_regex, match_field, dkim_required, signature_required, sealer_domain) instead of the human-readable table.

remove NAME REGEX

Delete the from entry whose whitelist_name is NAME and whose whitelist_regex is exactly REGEX. A list-id row with the same pattern text is a different entry and is left alone; use remove-list for it.

remove-list [–pattern] NAME LIST-ID

Delete the list-id entry add-list would have created for LIST-ID, so the identifier that added a list also removes it. –pattern has the same meaning as in add-list, and must be given if it was given then.

import [OPTIONS] [PATH…]

Seed a whitelist from the recipients of the mail the user has sent.

A mailbox is scanned for messages whose From: (or Sender:) is at one of the local domains — those are messages we sent, wherever they happen to be filed — and the To:, Cc: and Bcc: addresses of those messages become whitelist entries, so that when those people write back their replies are recognised. Received mail is ignored: whitelisting the sender of anything that ever arrived would defeat the purpose. Recipients at a local domain are skipped too — local senders never pass through the inbound gate.

Only the header block of each message is read (never the body), and a message is matched by address, not by folder, so ~/Maildir including its sub-folders, an mbox spool, or both, can be scanned in one run.

PATH… are mailbox files or Maildir directories. With none, ~/Maildir is scanned if it exists, otherwise /var/mail/<login>; [pepsi-whitelist] MAILBOX overrides that default.

–name NAME

Whitelist to seed. Defaults to <login>/correspondents (or the global correspondents for a privileged caller).

–format auto|mbox|maildir

Force the mailbox format instead of detecting it per path (a directory is a Maildir, a file an mbox).

–auto-wildcard

Offer to replace the individual addresses at a domain with one *@domain entry when at least N distinct addresses were seen there (N is --threshold, default [pepsi-whitelist] WILDCARD_THRESHOLD, 10).

Proposals are printed and confirmed one at a time on a terminal; declining one simply keeps that domain’s addresses individual. With –yes all are accepted; with no terminal and no –yes all are declined, so an unattended run can never widen a whitelist to a whole domain by itself.

Public e-mail hosters are never proposed, however many correspondents the user has there: “everyone at gmail.com” is not a set of people the user knows. The list of those domains is [pepsi-whitelist] HOSTERS_FILE (/usr/share/pepsi/hosters.txt).

–threshold N

Distinct addresses at one domain before a wildcard is proposed.

–hosters FILE

Use FILE as the public-hoster list instead of the configured one.

–no-hosters

Do not exclude any domain from wildcarding.

–dry-run

Report what would be added and change nothing.

-y, –yes

Accept every wildcard proposal without asking.

–max-entries N

Most rows this import may add (default [pepsi-whitelist] MAX_ENTRIES, 5000). Every row of a whitelist is matched against the From: of every inbound message that consults it, so an unbounded import is a permanent per-message cost.

–max-messages N

Stop after examining N messages. Useful on a very large archive.

–own-address ADDRESS

Treat only mail sent from this exact address as the user’s own, instead of any sender at a local domain. Repeatable.

–imap URL

Scan a mail store that is not on this filesystem, e.g. imaps://alice@mail.example.com/. imaps:// is implicit TLS (port 993), imap:// is STARTTLS (port 143); a path component is a mailbox pattern (default *, every mailbox). Mailboxes are opened read-only (EXAMINE) and only header blocks are fetched, so a scan does not mark anything \Seen.

LMTP cannot do this — it is a delivery protocol with no verb that lists or fetches anything — which is why IMAP is the network option.

–imap-user NAME

IMAP login, if it is not in the URL.

–imap-password-file FILE

Read the password from the first line of FILE (which must not be readable by other users) instead of prompting. The password is never passed on a command line, where every other user could read it out of ps.

–doveadm

Read a local Dovecot store through doveadm, for formats no file scanner can parse (mdbox/sdbox). doveadm is run by this program (it needs privilege) and its output is piped into the unprivileged scan helper.

–doveadm-path PATH

The doveadm binary to use.

–user LOGIN

Import for another user, into their namespace. Requires a privileged caller; the scan helper is dropped to that user’s credentials.

–all-users

Import for every local account that [pepsi-whitelist] TARGETS permits, each into its own <login>/correspondents. Requires a privileged caller. Only accounts in the local passwd file are found; name a directory-service account with –user.

–no-dkim-required, –signature-required

As for add, applied to every row the import adds.

70.1.34.1.5. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations. Rejected for unprivileged callers when this program is installed setuid: the configuration decides the database connection, so a setuid process reads only a root-owned one.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace (default: info).

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

70.1.34.1.6. Exit Status

0

Successful completion.

1

An error occurred: a malformed configuration file or a failed database connection or query. The reason is written to the log.

70.1.34.1.7. Examples

Whitelist a sender (DKIM/ARC required by default):

pepsi-whitelist -c /etc/pepsi/pepsi.conf add trusted-senders '(^|[^[:alnum:]._%+@-])alice@example\.com([^[:alnum:]._%+@-]|$)'

Whitelist a whole domain without requiring a signature:

pepsi-whitelist -c /etc/pepsi/pepsi.conf add trusted-senders --no-dkim-required '@example\.org'

List the trusted-senders group as JSON:

pepsi-whitelist -c /etc/pepsi/pepsi.conf list trusted-senders --json

Remove an entry:

pepsi-whitelist -c /etc/pepsi/pepsi.conf remove trusted-senders '@example\.org'

As an ordinary user, whitelist a mailing list you subscribe to, so its postings reach you whoever wrote them, but only when the list’s own ADMD sealed them:

pepsi-whitelist add-list alice/lists users.rust-lang.org \
    --sealer mail.rust-lang.org

Every list run at one domain, which needs the anchors written by hand:

pepsi-whitelist add-list alice/lists --pattern '^[a-z0-9-]+\.lists\.example\.org$' \
    --sealer lists.example.org

Unsubscribed; drop the rule again with the identifier that created it:

pepsi-whitelist remove-list alice/lists users.rust-lang.org

As an ordinary user, seed your own whitelist from your mailbox, seeing first what it would do:

pepsi-whitelist import --dry-run
pepsi-whitelist import

Same, but collapsing the domains you write to widely into one entry each (public hosters are never collapsed):

pepsi-whitelist import --auto-wildcard

Scan only your Sent folder, and a remote account over IMAP:

pepsi-whitelist import ~/Maildir/.Sent
pepsi-whitelist import --imap imaps://alice@mail.example.com/Sent

As root, seed every local account’s own whitelist from a Dovecot store:

pepsi-whitelist import --all-users --doveadm --auto-wildcard --yes

70.1.34.1.8. See Also

pepsi-config(1), pepsi-helper-mailbox-scan(1), pepsi-stage-check-whitelist(1), pepsi-stage-auto-whitelist(1), pepsi-queue(1), pepsi.conf(5), pepsi-setup(1)

70.1.34.1.9. Bugs

Report bugs to the Pepsi issue tracker.