85.1.26. pepsi-stage-autocrypt-learn

learn correspondents’ keys from the mail they send

Manual section:

1

85.1.26.1.1. Name

pepsi-stage-autocrypt-learn - the key-learning stage of the Pepsi pipeline.

85.1.26.1.2. Synopsis

pepsi-stage-autocrypt-learn [GLOBAL-OPTIONS] worker

85.1.26.1.3. Description

pepsi-stage-autocrypt-learn is a stage program run by pepsi-dispatch(1) as a persistent worker reading message ids on standard input. It loads that pepsi.workqueue row (refusing to act unless its status is running) and reads its [stage-<stage>] section.

It is where an inbound message teaches Pepsi a public key. Four sources, all of them things a correspondent put in a message they sent us:

• the sender’s Autocrypt: header (Autocrypt Level 1 §2.1);

• an attached application/pgp-keys part;

• the certificates carried by an S/MIME signature;

• the Autocrypt-Gossip: fields of a message that arrived encrypted, which introduce the other recipients’ keys (Level 1 §5.3).

Everything it learns is written to pepsi.peer_key through the same keydisc_resolve path pepsi-keydisc(1) uses, so a key learnt here also releases any message parked waiting for that address’s key.

The stage never drops, pauses, bounces or rewrites a message. It learns and advances, so NEXT_STAGE is mandatory.

It also records the opposite direction: that a correspondent provably holds one of our keys (see Proof that a correspondent holds our key).

85.1.26.1.4. Where to put it

Placement is what makes this a stage of its own rather than part of pepsi-stage-decrypt(1), and getting it wrong quietly undoes it:

• After pepsi-stage-decrypt. Gossip fields live inside the ciphertext, so there is nothing to read until the message has been opened.

• After every stage that can decide a message is junk — pepsi-stage-check-whitelist(1) and any other stage that sets state.spam. Autocrypt Level 1 §5.3 says peer state SHOULD be ignored for a message the client believes to be spam, and a stage placed before that belief exists cannot honour it. pepsi-stage-decrypt(1) runs first on the inbound path by design, so that belief does not yet exist there.

• Before anything that answers or delivers — pepsi-stage-vacation(1), the relay stages, the local-delivery stages. A key learnt after the reply has been sent is a key learnt too late to encrypt it with.

pepsi-setup --wizard places it correctly when end-to-end cryptography is enabled.

85.1.26.1.5. What it will not learn

The gates are Autocrypt Level 1’s, plus Pepsi’s own where Pepsi is deliberately stricter. A field or key that fails any of them is skipped; the message is never failed for it.

• No key at all when LEARN_KEYS is off: the stage learns nothing, gossip included — gossip is a widening of what a message may teach, so it cannot be on while the narrower rung is off. (The proof described under Proof that a correspondent holds our key is still recorded.)

• Nothing from a message scored as spam, unless LEARN_FROM_SPAM says otherwise.

• Only the sender’s own key from the first three sources: a message may legitimately carry a keyring of third parties, and storing those as though the sender vouched for them would let one correspondent seed the store for everybody. Introducing a third party is what gossip is for, and it lands on a rung of its own.

• Gossip only from a message that arrived encrypted, and only when the arriving header block carried no Autocrypt-Gossip: field of its own — anything on the delivery path can write one, so such a message teaches no gossip at all. Neither fact is visible from the committed plaintext, so pepsi-stage-decrypt(1) records both under state.crypto.in and this stage reads them there. A message that never met a decrypt stage has no such record, and absent means no: such a deployment learns headers and attached keys, and no gossip.

• No gossip for an address the message does not visibly name: a field’s addr must appear in the message’s own To:, Cc: or Reply-To: (Level 1 §5.3’s receiver rule). Without it a sender could introduce a key for any address in the world by mentioning it in a header nobody reads.

• Never a gossiped key for an address this host serves. A peer key for a local address is what our own outbound mail to that user would be encrypted with, so a gossip field naming one is a key substitution, not an introduction. The test is LOCAL_DOMAINS, and it applies to gossip alone: the sender’s own material is filtered by address rather than by locality (see below for what stops a forged From: from mattering).

• Never the sender’s own address by gossip (their own key belongs on the rung above), never two fields naming one address (nothing here can arbitrate between two claims about a stranger), and nothing from a message with more than 50 gossip fields (all of them are ignored), each field carrying at most 16 KiB of keydata= (a minimal transferable key is under 4 kB base64 even for RSA-4096, so that is generous by a factor of four; an oversized field is skipped, not fatal). The separate 10 KiB limit some readers will have met is the Autocrypt Level 1 §2.1 cap on the Autocrypt: header itself, and is never applied to a gossip field.

85.1.26.1.6. What a learnt key is worth

Little, on purpose. Harvested keys sit at rank inbound and gossiped ones at gossip, the bottom two rungs of the trust ladder (see pepsi-keydisc(1)):

• such a key can never displace one from discovery or from an operator;

• a signature checked against one is reported valid-untrusted at best, never valid, so it never sets state.signature_verified — the key pepsi-stage-check-whitelist(1) gates a signature_required row on;

• a peer_key row is ignored entirely for an address this host holds a crypto_identity for — the lookup the crypto stages use replaces such an address’s cached keys with the public half of the identity — so a message forging From: a local user can leave a row behind but can never have it used for them.

Within those two rungs the conflict rule is newest wins on the message’s Autocrypt effective date, so a correspondent who reinstalls their client stops receiving mail encrypted to a key they no longer hold. That is a deliberate trade: it is protection against a passive adversary only, which is all Autocrypt claims. An operator who wants opportunistic encryption but no third-party introductions sets [pepsi-keydiscovery] MIN_TRUST = harvested.

85.1.26.1.7. When a correspondent’s key changes

Newest-wins is keyed on the From: header, which the sender writes, and inbound authentication is fail-open — so a message that merely claims to come from a correspondent can replace the key we encrypt to them with. Autocrypt accepts that; Pepsi accepts it too, but never silently:

• An audit row. The database writes a key.peer.rotate row to pepsi.event_log (severity notice, actor autocrypt:inbound or autocrypt:gossip, subject the correspondent’s address) in the same statement as the replacement, with the protocol, the displaced and the new fingerprints and both effective dates. There is no rotation without its row. It is listed by GET /api/v1/events and the console’s event log, and summarised by pepsi-status(1) under Correspondent key changes.

• A notice to the people it affects. With RESPONSE_STAGE set, each local envelope recipient of the message that caused the change — the people this correspondent just wrote to, whose reply will be encrypted to the new key — is mailed a notice naming the correspondent and both fingerprints, and whether the new key was the correspondent’s own or introduced by gossip (and by whom). It is null-sender, Auto-Submitted: auto-generated, From: postmaster@<recipient's domain>, rendered from the key-rotation.<lang>.body template under [pepsi] TEMPLATE_DIR for the message’s detected state.language with an English fallback (which pepsi-setup(1) requires to exist), and injected at RESPONSE_STAGE. “Local” is a recipient at a LOCAL_DOMAINS domain; nobody elsewhere is ever mailed, and nobody is told about their own address. Local users who correspond with the address but did not receive this message are not told: Pepsi keeps no record of who corresponds with whom, and building one for this would be the worse trade. The audit row covers them. A notice that cannot be queued (a template that cannot be read, a database error) is logged at warn and not retried: a second pass would find the new key already stored and have no change to report, so the audit row is then the only record.

• A log line and a telemetry count (autocrypt-rotation).

A replacement by a strictly better source — a WKD or DANE answer displacing a harvested key, or the owner’s own Autocrypt: header displacing a gossiped one — is the ladder working as designed and is not a rotation.

A gossiped key its owner confirms is promoted. When the sender’s own Autocrypt: header (or an attached key naming them) carries the same key that gossip already stored for them, the row moves from the gossip rung to inbound — Autocrypt Level 1 §5.3’s promotion. Because the key did not change it is neither a rotation nor held, and nobody is sent a notice; the database writes one key.peer.promote row (severity info, actor autocrypt:inbound, detail the protocol, the fingerprint, from_source gossip, source inbound and both effective dates) in the same statement, and the stage logs it. The row’s effective date becomes the sender’s own, so a later rotation by them is judged against their own claim and not the date a third party’s message carried. The verdict a signature can reach is unchanged — valid-untrusted at best, as for any harvested key — but the row now defends at the inbound rank, so a later gossip field offering a different key for the address is refused, and MIN_TRUST = harvested admits it. ACCEPT_ROTATION plays no part: nothing is displaced.

ACCEPT_ROTATION = expired narrows the rule for an operator who prefers refusing a genuine rotation to accepting a forged one: the newer key is taken only when every stored key for that address and protocol is dead — its validity revoked or expired, or its expires_at in the past. Otherwise the stored key is kept, and the offer is held: a key.peer.rotate.held row (severity warning, naming the kept and the offered fingerprints), the same notice saying the new key was not accepted, and the autocrypt-rotation-held count. Deadness is read from what is stored, and every learnt key is stored with the expiry its own material states — for OpenPGP the date in the primary key’s current self-signature, for S/MIME the certificate’s notAfter — so a key that has lapsed is dead from that moment. A key that states no expiry is dead only once it is marked revoked or expired, or an operator removes it (pepsi-keys peer remove) or imports the new one by hand (pepsi-keys peer import, which outranks the harvested rung). The policy is read through the stage section, so a pepsi.settings override for a recipient applies to a stage listed in EDITABLE_STAGES.

85.1.26.1.8. Proof that a correspondent holds our key

With [stage-encrypt] ATTACH_KEYS_AS_FILES on, pepsi-stage-encrypt(1) attaches the sender’s public key as a file until the correspondent provably holds it. This stage writes that proof, one row per (fingerprint, peer_address) in pepsi.peer_has_own_key, when an inbound message shows it:

• it was decrypted with one of our keys (state.crypto.in.decrypted and decrypted_with), and was not merely passed through for a user’s own mail client;

• it carries a signature whose verdict is valid or valid-untrusted and that covers the plaintext (signature.covers_plaintext), so a stranger who wraps somebody else’s ciphertext in their own signature proves nothing;

• the signer is the From: address itself (never the envelope fallback used for key learning), so a forged From: cannot suppress the attachment.

The store then re-checks that the fingerprint names an active MTA key (custody = local) of the address the message was decrypted for, and prunes, in the same statement, rows whose fingerprint no longer names an active identity. The record is keyed on the fingerprint, so a new key starts being attached again. A peer who encrypts without signing keeps receiving the file, which does no harm.

This runs before every key-learning gate: it records something about our own key, not about theirs, so neither LEARN_KEYS nor the spam belief has anything to say about it. Like everything else here it never fails the message.

85.1.26.1.9. Configuration

Options live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-autocrypt-learn): LEARN_KEYS, LEARN_GOSSIP, LEARN_FROM_SPAM, ACCEPT_ROTATION (newest, the default, or expired), the optional RESPONSE_STAGE for key-change notices, the shared locality options LOCAL_DOMAINS, TARGETS and RECIPIENT_DELIMITER (parsed, but only LOCAL_DOMAINS is consulted), and a mandatory NEXT_STAGE. The stage also parses [pepsi-keydiscovery], whose NEGATIVE_TTL it hands to the shared store-and-release path. They are documented in pepsi.conf(5).

The stage is unprivileged: it writes only public key material (and the fact that a correspondent holds our public key), so it runs as the ordinary pepsi service account and is folded into the multi-call pepsi binary. It carries no setuid or setgid bit.

85.1.26.1.10. State

Inputs: state.spam (the belief upstream stages record); state.crypto.in.encrypted, state.crypto.in.decrypted and state.crypto.in.outer_gossip, written by pepsi-stage-decrypt(1), which together decide whether this message’s gossip may be believed. The sender is taken from the from_header column, falling back to the envelope sender.

Outputs: none on the message — its state is left untouched. The side effects are pepsi.peer_key rows, pepsi.peer_has_own_key rows, key.peer.rotate/key.peer.rotate.held/key.peer.promote rows in pepsi.event_log, and the key-change notices injected at RESPONSE_STAGE (token <token>-key-rotation-<n>). The proof additionally reads state.crypto.in.decrypted_with, state.crypto.in.for_client and state.crypto.in.signature. The state layout is described in pepsi.state(7).

Transitions: always advances to NEXT_STAGE. There is no branch; the stage never pauses, fails, reroutes or finishes. Every error in learning is logged and swallowed: a message is being delivered, and nothing about a stranger’s key material may stand in the way of it. Key material that does not parse is the sender’s business and logged at debug; material that parsed and could not be stored (a database error) is this host’s, and logged at warn, so an operator notices a store that has stopped learning. The configuration is the exception: the stage’s section and [pepsi-keydiscovery] are checked when the worker starts (a broken one makes it refuse to start, exit 78, and the dispatcher holds the queue), and a section that stops parsing under a running worker is retried rather than silently skipped.

85.1.26.1.11. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

85.1.26.1.12. Global Options

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

85.1.26.1.13. Exit Status

0

The message was processed (whatever it taught, or nothing) and advanced.

1

An error occurred (message not found or not running, a misconfigured stage — e.g. a missing NEXT_STAGE — or a database error). The reason is written to the log.

78

The worker refused to start: its section or [pepsi-keydiscovery] does not parse. The dispatcher requeues what it handed over and retries the stage later.

85.1.26.1.14. Examples

Learn from message 42 (it must be running):

echo 42 | pepsi-stage-autocrypt-learn -c /etc/pepsi/pepsi.conf worker

An inbound pipeline placing it after the spam checks and before delivery:

[stage-check-whitelist]
PROGRAM = pepsi-stage-check-whitelist
NEXT_STAGE = autocrypt-learn
WHITELIST_NAME = correspondents

[stage-autocrypt-learn]
PROGRAM = pepsi-stage-autocrypt-learn
NEXT_STAGE = local

Tell recipients when a correspondent’s key changes, and accept a change only once the old key is dead:

[stage-autocrypt-learn]
PROGRAM = pepsi-stage-autocrypt-learn
NEXT_STAGE = local
RESPONSE_STAGE = dkim-sign
ACCEPT_ROTATION = expired

Keep opportunistic encryption but refuse third-party introductions:

[stage-autocrypt-learn]
PROGRAM = pepsi-stage-autocrypt-learn
NEXT_STAGE = local
LEARN_GOSSIP = no

85.1.26.1.15. See Also

pepsi-stage-decrypt(1), pepsi-stage-encrypt(1), pepsi-keydisc(1), pepsi-keys(1), pepsi-stage-check-whitelist(1), pepsi-dispatch(1), pepsi-status(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

85.1.26.1.16. Bugs

Report bugs to the Pepsi issue tracker.