95.1.63. pepsi-helper-mailbox-scan¶
list the addresses a mailbox’s owner has written to
- Manual section:
1
95.1.63.1.1. Name¶
pepsi-helper-mailbox-scan - extract correspondent addresses from a mailbox.
95.1.63.1.2. Synopsis¶
pepsi-helper-mailbox-scan (–local-domain DOMAIN | –own-address ADDRESS)… [OPTIONS] PATH…
pepsi-helper-mailbox-scan (–local-domain DOMAIN | –own-address ADDRESS)… –imap URL –imap-password-stdin
pepsi-helper-mailbox-scan (–local-domain DOMAIN | –own-address ADDRESS)… –doveadm-stdin
95.1.63.1.3. Description¶
pepsi-helper-mailbox-scan reads a mailbox and prints, one per line, the
addresses that its owner has sent mail to. It is the scanning half of
pepsi-whitelist(1)’s import subcommand, which spawns it and turns its
output into pepsi.whitelist rows. It can also be run by hand, because its
output is a protocol of tagged lines (see Output below) that is trivial to
parse and readable as it stands.
A message counts as “sent by us” when both of the following hold.
Its From: (or its Sender:, for mail submitted on somebody’s behalf) is at
one of the –local-domain domains; or, when –own-address is given, it is
exactly one of those addresses. (The two are alternatives, not a union: once you
name any own address, the sender test uses that list alone, and the domains are
then used only to decide which recipients are dropped as local.) And its
header block carries no Received:, Return-Path:, Delivered-To: or
X-Original-To: field.
The second condition is what makes the first one mean anything. From: is
unauthenticated: anybody may put your address in it, and such a message sits in
your INBOX looking exactly like mail you sent, so scanning an inbox on the
From: test alone lets a stranger choose the addresses that end up in your
whitelist. A receiving MTA or MDA writes those four fields, and a mail client does
not write them when it composes a message, so their presence shows that the
message arrived here, whatever its From: says. A sent copy that carries
one of them anyway is skipped, which can only leave a correspondent out of the
list, never let a stranger in. What is left is the copy that your
client filed after composing it, which is what a scan is meant to read. Where that copy
lives does not matter: a Maildir scan walks the whole tree, so ~/Maildir
covers .Sent and every archive folder. A plain /var/mail/ spool, holding
only received mail, yields nothing.
The To:, Cc:
and Bcc: addresses of such messages are the correspondents; addresses at a
local domain are omitted, because a local sender never passes through the inbound
whitelist gate. Addresses are deduplicated, lower-cased and syntax-checked, and
each is printed at most once.
It also reports the mailing lists that its owner took part in: every message
carrying a List-Id: and a Message-ID: is remembered, received mail
included, and a list is reported when a message sent by us answers one of them
(its In-Reply-To: names that message). Receiving list mail alone reports
nothing (anybody can write any List-Id:), and pepsi-whitelist(1) asks
about each list before adding a row for it.
Only the header block of each message is read: every reader stops at the blank line that ends the headers, and message bodies are never touched. Both the block and any single line of it are capped at –max-header-bytes, so one absurd message cannot make the scan allocate without bound.
Two things do grow with the work rather than being constant. The set of distinct correspondent addresses is held in memory (that is what deduplication means), so its size is proportional to the number of people written to, not to the size of the mailbox. And –doveadm-stdin is the one input that may buffer its whole source: Dovecot may emit its result as a single top-level JSON array, which is parsed as one value.
95.1.63.1.3.1. Why this is a separate, unprivileged program¶
This is the only standalone pepsi-helper-* program that carries no
setuid or setgid bit (mode 0755). The other three standalone helpers,
pepsi-helper-maildir-writer(1), pepsi-helper-dot-forward(1) and
pepsi-helper-auto-pay(1), are installed setuid-root and group-restricted
(mode 4750). (pepsi-helper-token-refresh(1) holds no privilege either, but
it is folded into the unified pepsi binary rather than installed as a
program of its own.)
The reason is the direction of privilege. pepsi-whitelist(1) is setuid
because it needs a database identity, and a setuid process keeps its owner in the
saved set-user-id and can regain it at any point. Parsing a mailbox is the opposite
kind of work: it processes megabytes of RFC 5322 text that attackers control
(every message that anybody ever sent the user), and it needs no privilege
beyond the user’s own. So the parsing runs here, in a process that
pepsi-whitelist spawns with setresuid, which sets the real, effective
and saved ids to the target user. With no privileged id left anywhere in its
credentials, this program cannot become the whitelist account, cannot reach the
database, and can read nothing that the user could not read anyway.
95.1.63.1.4. Options¶
- –local-domain DOMAIN
A domain whose senders count as “us”. Repeatable. At least one –local-domain or –own-address is required: without one, no message can be recognised as sent by us.
- –own-address ADDRESS
Count only mail from this exact address as ours, rather than any sender at a local domain. Repeatable.
- –include-delivered
Drop the delivery-trace test and count a message as ours on its
From:alone.This is unsafe on any mailbox that holds received mail: a forged
From:is then enough to make an inbound message look sent, and itsTo:addresses become whitelist entries. The option exists for the unusual store whose sent copies really did go through delivery, such as a provider that files its own copy, withReceived:fields, inSent. Point it at that folder only, never at an inbox.- –format
auto|mbox|maildir Format of the given paths.
auto(the default) treats a directory as a Maildir tree and a file as an mbox.An mbox is split on
From `` lines that stand at the start of the file or directly after a blank line. A Maildir is read from ``cur/andnew/(nevertmp/, whose files may be half-written), and every directory below the one given that itself containscur/ornew/is scanned too, so the Maildir++ layout (.Sent,.Archive.2025) is covered without naming each folder.- –imap URL
Scan a remote store instead of files.
imaps://user@host[:port]/[PATTERN]uses implicit TLS (port 993),imap://usesSTARTTLS(port 143), andimap+plain://disables TLS entirely (which sends the password in the clear, and is only sensible against a loopback test server). PATTERN is an IMAP mailbox pattern,*by default.The server certificate is always verified against the system trust store. Mailboxes are opened with
EXAMINE(read-only) and onlyBODY.PEEK[HEADER]is fetched, so a scan neither marks messages\Seennor changes anything else.- –imap-user NAME
IMAP login, if it is not in the URL.
- –imap-password-file FILE
Read the password from the first line of FILE, which must grant no access to its group or to other users (for example mode
0600).- –imap-password-stdin
Read the password from the first line of standard input. This is how pepsi-whitelist(1) passes it, so it never appears in a command line where other users could read it out of
ps.- –doveadm-stdin
Read a
doveadm -f json fetch -uLOGINhdr mailbox '*' allstream from standard input instead of opening files. Used for Dovecot’s own mail formats (mdbox/sdbox): the caller runs the privilegeddoveadm, and only its output crosses into this unprivileged process. JSON is required because a header block contains blank lines and colons of its own, which doveadm’s human-readable formats do not escape.- –max-header-bytes BYTES
The maximum number of header bytes kept per message (default 65536).
- –max-messages N
Examine at most N messages (0, the default, means no limit).
This limits how many messages are parsed and classified, which is the expensive part of each message; it does not abort the walk. The mbox file is still read to its end, the Maildir tree is still walked, and an IMAP session still fetches every mailbox’s headers. Use it to bound the work of a scan, not the time it takes on a very large store.
- -h, –help
Print a usage summary and exit.
- -V, –version
Print the version and exit.
95.1.63.1.5. Output¶
One tab-separated record per line on standard output:
a<TAB>ADDRESSA correspondent address, emitted at most once.
l<TAB>LIST-IDA mailing list that the user took part in, emitted at most once, after the addresses: a message sent by us (by the rule above) whose
In-Reply-To:names theMessage-ID:of a message in the scanned mailbox that carried thisList-Id:. LIST-ID is the bare, lower-cased identifier. Merely receiving list mail produces no record. At most 200000 list messages and as many replies are remembered for this; past that awrecord says so.m<TAB>COUNTHow many messages were examined.
s<TAB>COUNTHow many of them were sent by us.
w<TAB>MESSAGEA non-fatal problem: an unreadable folder, a skipped record, a mailbox with no
cur/new. Warnings are emitted after the addresses and are capped.
A reader ignores record types that it does not know, so later versions may add some.
95.1.63.1.6. Exit Status¶
- 0
The scan completed. Any
wrecords are non-fatal.- 1
A fatal error: no –local-domain or –own-address, an unreadable mailbox path, an IMAP login refused, a password file that other users can read. The diagnostic is on standard error.
- 2
The command line could not be parsed (an unknown option or a missing value).
95.1.63.1.7. Examples¶
Scan your own mail:
pepsi-helper-mailbox-scan --local-domain example.com ~/Maildir /var/mail/alice
Only mail you sent from one particular address:
pepsi-helper-mailbox-scan --own-address alice@example.com ~/Maildir
A remote account over IMAP, with the password in a file:
pepsi-helper-mailbox-scan --local-domain example.com \
--imap imaps://alice@mail.example.com/ --imap-password-file ~/.imap-pw
A Dovecot store, as root:
doveadm -f json fetch -u alice hdr mailbox '*' all \
| pepsi-helper-mailbox-scan --local-domain example.com --doveadm-stdin
95.1.63.1.8. See Also¶
pepsi-whitelist(1), pepsi-stage-check-whitelist(1), pepsi-stage-auto-whitelist(1), pepsi.conf(5); the privilege split and the local-user adversary are described in the Pepsi manual’s security and threat-model chapters.
95.1.63.1.9. Bugs¶
Report bugs to the Pepsi issue tracker.