85.1.43. pepsi-whitelist

manage the sender whitelist, by hand or by mailbox

Manual section:

1

85.1.43.1.1. Name

pepsi-whitelist - add, list, remove and import sender whitelist entries.

85.1.43.1.2. Synopsis

pepsi-whitelist [GLOBAL-OPTIONS] add [–no-dkim-required] [–signature-required] [–sealer DOMAIN] NAME REGEX

pepsi-whitelist [GLOBAL-OPTIONS] add-list [–pattern] [–no-dkim-required] [–signature-required] [–sealer DOMAIN] NAME LIST-ID

pepsi-whitelist [GLOBAL-OPTIONS] list [NAME] [–json]

pepsi-whitelist [GLOBAL-OPTIONS] remove NAME REGEX

pepsi-whitelist [GLOBAL-OPTIONS] remove-list [–pattern] NAME LIST-ID

pepsi-whitelist [GLOBAL-OPTIONS] import [IMPORT-OPTIONS] [PATH…]

85.1.43.1.3. Description

pepsi-whitelist is the operator’s tool for the pepsi.whitelist table — the named sets of patterns that mark a message as trusted (non-spam). Each row has a whitelist_name (the group it belongs to), a whitelist_regex (a POSIX extended regular expression matched case-insensitively, via the ~* operator), a match_field naming what that expression is matched against — the message’s From: addr-spec or its List-Id: — and three conditions: dkim_required, signature_required and sealer_domain. They are described in pepsi-stage-check-whitelist(1), which is what evaluates them.

Every pattern is checked before it is stored, by asking PostgreSQL – the engine that will run it – to compile it: add, add-list and import refuse an expression that does not compile, with PostgreSQL’s reason, and an import writes nothing if any of its patterns is refused.

The whitelist is consulted by pepsi-stage-check-whitelist(1) and populated automatically from outgoing recipients by pepsi-stage-auto-whitelist(1); this tool lets an administrator manage rows directly, and lets each user seed a whitelist of their own from the mail they have already sent (import). It is not a stage. It connects to the same database as the other components, through the shared [pepsi-postgres] section, and reads the optional [pepsi-whitelist] section described in pepsi.conf(5).

Unlike pepsi-stage-auto-whitelist(1), which escapes a recipient address into a boundary-anchored pattern, add stores the whitelist_regex exactly as given, so the operator controls the matching directly. import escapes the addresses it finds, exactly as the stage does.

85.1.43.1.3.1. Mailing lists

add-list and remove-list manage the list-id rows that whitelist a mailing list as a whole — every posting it relays, whoever wrote it, which a From: pattern cannot express because a list’s postings carry arbitrary sender addresses.

They are separate subcommands rather than a flag on add because they take a different kind of argument. add stores its REGEX verbatim, which is right for a pattern an operator composed deliberately; a list identifier typed into that slot would be an unanchored regular expression, so users.example.org would also match users.example.org.attacker.example and whitelist a list of the attacker’s choosing. add-list takes the identifier as it appears in the header — brackets optional, case and surrounding phrase ignored — and anchors it itself. –pattern is the escape hatch for the deliberate wildcard (every list at a domain), and then the argument is a regular expression again, as with add.

A List-Id: header is unauthenticated text that any sender can copy out of a genuine posting, so a rule resting on it alone is a rule anyone can satisfy. –sealer DOMAIN is the other half: the row then also requires that the message arrived with an ARC chain that validated and that DOMAIN was one of its sealers, as recorded by pepsi-stage-arc(1). DOMAIN is normalised (lower-cased, trailing dot removed) to the same form the ARC stage records, so the two cannot disagree. It works on add rows too, where it restricts a sender pattern to mail relayed through a known forwarder.

85.1.43.1.3.2. Whitelist names and who may use them

A whitelist_name is either

global — a single segment such as correspondents, which only the operator

may manage; or

owned by a user — <login>/<segment> such as alice/correspondents, and

alice/lists/rust-lang for further sub-lists.

A segment consists of letters, digits, _, . and -; the separator is /. A segment made of nothing but dots is refused, so . and .. cannot be segments and a whitelist name can never be a relative path. A dot is otherwise allowed because the owning segment is the account’s passwd login, and alice.smith is an ordinary login on any LDAP/SSSD/FreeIPA deployment — as is a {localpart} of first.last in pepsi-stage-check-whitelist(1). / is the separator because no POSIX-ish system permits it in a login name, whereas -, _, . and a trailing $ are all legal in one and so could not delimit unambiguously. (A login holding a character outside the segment grammar still owns no namespace; import –all-users skips such an account with a message instead of failing the whole run.)

Because this program is installed setuid (see below), any local user can run it. An unprivileged caller may only read and write names in their own namespace — the login of the real uid, taken from the passwd database, never from $USER or an argument. That restriction covers list as well: the operator’s shared whitelist is a record of who the organisation corresponds with, and this program must not hand it to every account on the host. root, and the pepsi, pepsi-owner and pepsi-whitelist accounts, may manage every whitelist.

For a per-user whitelist to have any effect, some stage must consult it; see the {localpart} and {login} placeholders of WHITELIST_NAME in pepsi-stage-check-whitelist(1).

85.1.43.1.3.3. The setuid model

Managing the shared pepsi.whitelist table needs a database identity that ordinary users do not have, while scanning a user’s mailbox needs their identity. The program is therefore installed setuid to the pepsi-whitelist account (mode 4755) and immediately lowers its effective ids back to the invoking user, keeping the owner only in the saved set-user-id. It raises them again for the few milliseconds it talks to PostgreSQL — peer authentication keys off the effective uid — and for nothing else.

The same mechanism is what lets root use the tool. The other operator commands — pepsi-queue(1), pepsi-status(1), pepsi-settings(1), pepsi-tlsrpt(1), pepsi-failure-bouncer(1) — simply become the pepsi service account when started as root, permanently, because they have nothing to do as root. This one does: import –user and –all-users run the scan helper under another user’s credentials, which only root can arrange. It therefore keeps the invoking identity, root included, and switches only its effective ids — to the pepsi-whitelist account, whose database role is granted the whitelist table alone — for the duration of each database call, returning to root afterwards.

That account is deliberately not the pepsi service account: pepsi-setup(1) grants its database role SELECT, INSERT and DELETE on the pepsi.whitelist table and nothing more, so a compromise of a binary every local user may execute cannot reach the message queue, the statistics or the signing keys.

The mailbox parsing — the one part that reads attacker-controlled data, namely every message anybody ever sent the user — does not happen in this process at all. It runs in pepsi-helper-mailbox-scan(1), spawned with setresuid setting the real, effective and saved ids to the target user, so that process has no privileged id left anywhere in its credentials and cannot become pepsi-whitelist even in principle.

Because a setuid program must not be pointed at a configuration file of the caller’s choosing, -c/–config is rejected for unprivileged callers, and the environment variables that would otherwise redirect the configuration search (HOME, XDG_CONFIG_HOME), the expansion of ${DATADIR} (PATH) or the database connection (PG*) are removed before the configuration is read.

85.1.43.1.4. Commands

add [–no-dkim-required] [–signature-required] [–sealer DOMAIN] NAME REGEX

Insert a from entry storing REGEX under whitelist NAME. The insert is idempotent on the UNIQUE(whitelist_name, match_field, whitelist_regex) constraint: re-adding an existing triple is a no-op and leaves its conditions unchanged.

–no-dkim-required

Clear the row’s dkim_required flag. By default it is set, so a match counts only if the sender’s DKIM or ARC signature verified.

–signature-required

Set the row’s signature_required flag. By default it is clear. When set, a match counts only if the message’s signature verified.

–sealer DOMAIN

Set the row’s sealer_domain. By default it is empty (any path). When set, a match counts only if the message arrived with an ARC chain that validated and DOMAIN was one of its sealers. DOMAIN is lower-cased and stripped of a trailing dot, matching what pepsi-stage-arc(1) records.

add-list [–pattern] [CONDITION-OPTIONS] NAME LIST-ID

Insert a list-id entry under whitelist NAME, matching every message whose List-Id: header carries the identifier LIST-ID. The identifier may be given bracketed or bare, in any case, with or without a descriptive phrase; it is reduced to the bare lower-cased form and anchored, so it matches that list and no other. Takes the same condition options as add, and pairing it with –sealer is strongly advised — see Mailing lists above.

–pattern

Treat LIST-ID as a POSIX extended regular expression to store verbatim, rather than as one identifier to anchor. For the deliberate wildcard case (every list at a domain); an unanchored expression matches any identifier containing it, so write the anchors yourself.

list [NAME] [–json]

List entries ordered by name, match field, then regex. With NAME, only that whitelist group is shown.

–json

Emit a JSON array of objects (whitelist_id, whitelist_name, whitelist_regex, match_field, dkim_required, signature_required, sealer_domain) instead of the human-readable table.

remove NAME REGEX

Delete the from entry whose whitelist_name is NAME and whose whitelist_regex is exactly REGEX. A list-id row with the same pattern text is a different entry and is left alone; use remove-list for it.

remove-list [–pattern] NAME LIST-ID

Delete the list-id entry add-list would have created for LIST-ID, so the identifier that added a list also removes it. –pattern has the same meaning as in add-list, and must be given if it was given then.

import [OPTIONS] [PATH…]

Seed a whitelist from the recipients of the mail the user has sent.

A mailbox is scanned for messages that were sent by the user, and the To:, Cc: and Bcc: addresses of those messages become whitelist entries, so that when those people write back their replies are recognised. Recipients at a local domain are skipped — local senders never pass through the inbound gate.

A message counts as sent by the user when its From: (or Sender:) is at one of the local domains and its header block carries no Received:, Return-Path:, Delivered-To: or X-Original-To: field. Both halves are needed: From: is unauthenticated, so on its own it is a claim anybody can make, and a stranger who sends you one message with your own address in From: and their own in To: would otherwise have that address inserted into your whitelist by the next import — which is exactly the gate the whitelist opens. Requiring a signature does not close it, because the forger signs their own domain’s mail. The four fields above are written by a receiving MTA or MDA and never by a mail user agent composing a message, so their absence is the evidence that the message was filed here rather than delivered here.

Received mail is therefore ignored, and so is anything else that arrived. –include-delivered turns that test off; see below before using it.

Where a message is filed still does not matter: a Maildir scan walks the whole tree, so ~/Maildir covers .Sent and every archive folder in one run. Only the header block of each message is read, never the body.

PATH… are mailbox files or Maildir directories. With none, ~/Maildir is scanned if it exists, otherwise /var/mail/<login>; [pepsi-whitelist] MAILBOX overrides that default. Note that /var/mail/<login> is a delivery spool and holds nothing but received mail, so on a host with no Maildir the default finds nothing and a Sent folder must be named explicitly.

–name NAME

Whitelist to seed. Defaults to <login>/correspondents (or the global correspondents for a privileged caller).

–include-delivered

Count a message as sent on its From: alone, dropping the delivery-trace test above. Unsafe on any mailbox that holds received mail — a forged From: is then enough to plant a whitelist entry. It exists for the unusual store whose sent copies really did go through delivery, and should be pointed at that Sent folder only, never at an inbox.

–format auto|mbox|maildir

Force the mailbox format instead of detecting it per path (a directory is a Maildir, a file an mbox).

–auto-wildcard

Offer to replace the individual addresses at a domain with one *@domain entry when at least N distinct addresses were seen there (N is --threshold, default [pepsi-whitelist] WILDCARD_THRESHOLD, 10).

Proposals are printed and confirmed one at a time on a terminal; declining one simply keeps that domain’s addresses individual. With –yes all are accepted; with no terminal and no –yes all are declined, so an unattended run can never widen a whitelist to a whole domain by itself.

Public e-mail hosters are never proposed, however many correspondents the user has there: “everyone at gmail.com” is not a set of people the user knows. The list of those domains is [pepsi-whitelist] HOSTERS_FILE (/usr/share/pepsi/hosters.txt).

–threshold N

Distinct addresses at one domain before a wildcard is proposed.

–hosters FILE

Use FILE as the public-hoster list instead of the configured one.

–no-hosters

Do not exclude any domain from wildcarding.

–dry-run

Report what would be added and change nothing.

-y, –yes

Accept every wildcard proposal without asking.

–max-entries N

Most rows this import may add (default [pepsi-whitelist] MAX_ENTRIES, 5000). Every row of a whitelist is matched against the From: of every inbound message that consults it, so an unbounded import is a permanent per-message cost.

–max-messages N

Examine at most N messages. This caps how many are parsed and classified, not how much of the mailbox is read: the file, tree or IMAP account is still walked to the end.

–own-address ADDRESS

Treat only mail sent from this exact address as the user’s own, instead of any sender at a local domain. Repeatable.

–imap URL

Scan a mail store that is not on this filesystem, e.g. imaps://alice@mail.example.com/. imaps:// is implicit TLS (port 993), imap:// is STARTTLS (port 143); a path component is a mailbox pattern (default *, every mailbox). Mailboxes are opened read-only (EXAMINE) and only header blocks are fetched, so a scan does not mark anything \Seen.

LMTP cannot do this — it is a delivery protocol with no verb that lists or fetches anything — which is why IMAP is the network option.

–imap-user NAME

IMAP login, if it is not in the URL.

–imap-password-file FILE

Read the password from the first line of FILE instead of prompting. Keep it mode 0600: this path reads the file itself and does not check its permissions, unlike pepsi-helper-mailbox-scan(1)’s option of the same name, which refuses a file any other user can read. The password is never passed on a command line, where every other user could read it out of ps.

–doveadm

Read a local Dovecot store through doveadm, for formats no file scanner can parse (mdbox/sdbox). doveadm is run by this program (it needs privilege) and its output is piped into the unprivileged scan helper.

–doveadm-path PATH

The doveadm binary to use.

–user LOGIN

Import for another user, into their namespace. Requires a privileged caller; the scan helper is dropped to that user’s credentials.

–all-users

Import for every local account that [pepsi-whitelist] TARGETS permits, each into its own <login>/correspondents. Requires a privileged caller. Only accounts in the local passwd file are found; name a directory-service account with –user.

–no-dkim-required, –signature-required

As for add, applied to every row the import adds.

85.1.43.1.5. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations. Rejected for unprivileged callers when this program is installed setuid: the configuration decides the database connection, so a setuid process reads only a root-owned one.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace (default: info).

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

85.1.43.1.6. Exit Status

0

Successful completion.

1

An error occurred: a malformed configuration file or a failed database connection or query. The reason is written to the log.

85.1.43.1.7. Files

When –config is not given, the first existing file from the following list is used. Every Pepsi component shares the same configuration file, so this is the same list each one searches:

• $XDG_CONFIG_HOME/pepsi.conf

• $HOME/.config/pepsi.conf

• /etc/pepsi/pepsi.conf

• /etc/pepsi.conf

${DATADIR}/hosters.txt

The public-hoster list import –auto-wildcard never proposes a wildcard for, installed by make install from contrib/hosters.txt (/usr/share/pepsi/hosters.txt on a default installation). Overridden by [pepsi-whitelist] HOSTERS_FILE or, for one run, by –hosters.

~/Maildir, /var/mail/login

The mailboxes import scans when no PATH is given, in that order; [pepsi-whitelist] MAILBOX overrides the pair.

85.1.43.1.8. Examples

Whitelist a sender (DKIM/ARC required by default):

pepsi-whitelist -c /etc/pepsi/pepsi.conf add trusted-senders '^alice@example\.com$'

The pattern is stored verbatim, so anchoring it is the operator’s job. An unanchored alice@example\.com also matches alice@example.com.attacker.example and "alice@example.com!"@attacker.example, both of which an attacker can put in a From: header. The generated patterns (import and pepsi-stage-auto-whitelist(1)) always anchor on ^/$ for this reason.

Whitelist a whole domain without requiring a signature:

pepsi-whitelist -c /etc/pepsi/pepsi.conf add trusted-senders --no-dkim-required '@example\.org$'

List the trusted-senders group as JSON:

pepsi-whitelist -c /etc/pepsi/pepsi.conf list trusted-senders --json

Remove an entry:

pepsi-whitelist -c /etc/pepsi/pepsi.conf remove trusted-senders '@example\.org$'

As an ordinary user, whitelist a mailing list you subscribe to, so its postings reach you whoever wrote them, but only when the list’s own ADMD sealed them:

pepsi-whitelist add-list alice/lists users.rust-lang.org \
    --sealer mail.rust-lang.org

Every list run at one domain, which needs the anchors written by hand:

pepsi-whitelist add-list alice/lists --pattern '^[a-z0-9-]+\.lists\.example\.org$' \
    --sealer lists.example.org

Unsubscribed; drop the rule again with the identifier that created it:

pepsi-whitelist remove-list alice/lists users.rust-lang.org

As an ordinary user, seed your own whitelist from your mailbox, seeing first what it would do:

pepsi-whitelist import --dry-run
pepsi-whitelist import

Same, but collapsing the domains you write to widely into one entry each (public hosters are never collapsed):

pepsi-whitelist import --auto-wildcard

Scan only your Sent folder, and a remote account over IMAP:

pepsi-whitelist import ~/Maildir/.Sent
pepsi-whitelist import --imap imaps://alice@mail.example.com/Sent

As root, seed every local account’s own whitelist from a Dovecot store:

pepsi-whitelist import --all-users --doveadm --auto-wildcard --yes

85.1.43.1.9. See Also

pepsi-config(1), pepsi-helper-mailbox-scan(1), pepsi-stage-check-whitelist(1), pepsi-stage-auto-whitelist(1), pepsi-queue(1), pepsi.conf(5), pepsi-setup(1)

85.1.43.1.10. Bugs

Report bugs to the Pepsi issue tracker.