70.1.7. pepsi-stage-decrypt

decrypt mail addressed to our users and verify what signed it

Manual section:

1

70.1.7.1.1. Name

pepsi-stage-decrypt - inbound end-to-end decryption and signature verification stage of the Pepsi pipeline.

70.1.7.1.2. Synopsis

pepsi-stage-decrypt [GLOBAL-OPTIONS] worker

70.1.7.1.3. Description

pepsi-stage-decrypt is a stage program run by pepsi-dispatch(1) as a persistent worker reading message ids on standard input. For a message addressed to a recipient this host serves it opens whatever ciphertext the message carries, verifies whatever signature it carries against a real trust chain, records the verdict, harvests the sender’s key, and removes any security indicator the sender forged.

Server-side decryption is what makes end-to-end mail cryptography usable without client plugins: the user reads ordinary mail in their usual client and the gateway did the work. That trade is only sound because the message is about to stop being an SMTP object and become a file in a mailbox — see Locality.

The verification half is deliberately strict about one distinction. A signature that is cryptographically sound tells you the message was signed by some key; whether that key is the sender’s is a separate question with a separate answer. The two are never reported as the same thing (Verdicts).

70.1.7.1.4. Locality

The stage runs only for recipients this host serves, and for a message it does not serve it does not even try — a failed decryption attempt should not colour the verdict, and there is no key for such a message anyway.

The reason is that decryption rewrites the body. The sender’s DKIM signature then no longer verifies, and an ARC seal made before decryption (see Placement) covers the encrypted form. For a message about to be delivered locally that is harmless. For a message being forwarded onward it is not: it would arrive at the next hop with a broken signature, which is a stronger negative signal than no signature at all.

Locality is the same test the delivery stages use — LOCAL_DOMAINS, TARGETS and RECIPIENT_DELIMITER — with one difference. By default only the domain has to match, because this stage does not have to know whose mailbox to write; a hosted domain whose users hold crypto identities but no shell accounts is an ordinary deployment. Set REQUIRE_ACCOUNT = yes to demand a passwd entry in TARGETS as well.

A message with both local and foreign recipients is split: the foreign recipients are peeled onto a sibling row at NEXT_STAGE carrying the message exactly as it arrived, and this row keeps the local ones. The per-recipient DSN state is sliced in lockstep by the same code every other recipient split uses.

70.1.7.1.5. Placement

The recommended inbound chain is:

init = arc -> decrypt -> aliases -> (anti-spam / language / …) -> local delivery

ARC runs first, and this is not a preference. An ARC set seals the message as it arrived; decrypting first would make the sealed AMS describe bytes nobody else ever saw, so the seal would be a claim about a message that never existed. pepsi-setup(1) warns when it finds an ARC stage reachable from a decrypt stage.

Content inspection runs after. Language detection, the pay-to-send gate and the From:-header whitelist check all read the body, and all of them want cleartext.

Placement is also a performance matter. The stage declares Load::Full, because whether a message is protected cannot be told from the envelope columns and Load is fixed per stage. Every message that passes through it therefore loads its whole body from the database, even one that turns out to be ordinary unencrypted mail. That is affordable on an inbound path terminating in local delivery; it is not a stage to put in front of bulk relay traffic.

70.1.7.1.6. Verdicts

state.crypto.in.signature.status is one of:

none

The message carries no signature.

valid

Cryptographically sound and the key was anchored: an X.509 chain to a configured ca_trust anchor, or an OpenPGP key from a ranked discovery source (DANE, WKD, a key server, or one the operator pinned).

valid-untrusted

Cryptographically sound, but the key could not be tied to anything: a key seen for the first time, learnt from the message itself or from an Autocrypt header, or a certificate from a CA this deployment holds no anchor for. Most of the world’s signed mail is in this state.

This is never reported as ``valid``. It is the distinction the whole stage exists to make, and a security indicator that conflates the two is worse than none.

invalid

The signature does not verify over the bytes it covers, or the signer’s certificate is expired, revoked, or bound to an address other than the message’s From:.

unverifiable

No key or certificate was available for the claimed signer, so nothing could be established either way. This is the verdict that makes the stage wait for a key (Key discovery).

A message carrying several signature layers takes the worst of their verdicts. One broken signature makes the message invalid however good the others are.

Only valid sets state.signature_verified, which pepsi-stage-check-whitelist(1) gates a whitelist entry on.

70.1.7.1.6.1. A signature that covers ciphertext

In signed(encrypted(body)) the signature is computed over the ciphertext, not over anything a reader will ever see. That proves the signer transmitted the blob; it does not prove they wrote it, and it does not prove they can read it — anybody who obtains an encrypted message can wrap it in their own signature and forward it. Such a layer is recorded with "covers_ciphertext": true in state.crypto.in.layers and, whatever its own verdict, it is never allowed to make the message ``valid``: a good one is reported as valid-untrusted, so it earns no [verified] tag and does not set state.signature_verified. A bad one is still reported as invalid and still routes under ON_BAD_SIGNATURE — only the positive verdict is withheld, because only the positive verdict grants anything.

The consequence is worth stating plainly: in signed(encrypted(signed(body))) — the triple wrap RFC 8551 §3.7 recommends — the inner signature is the one that speaks for the content, and it is the one this stage reports. That shape is therefore reported as valid where a single outer signature is not.

70.1.7.1.7. Failure handling

Both failure paths deliver by default, matching Pepsi’s existing and deliberate fail-open posture on inbound SPF, DKIM and DMARC:

ON_DECRYPT_FAILURE = passthrough

The still-encrypted message is delivered. The user may well hold the key in their own client, and bouncing mail we merely could not open helps nobody. quarantine and bounce are available.

ON_BAD_SIGNATURE = record

A bad signature is a recorded verdict, never a delivery failure. Mailing lists that rewrite bodies, forwarders and clients with broken canonicalisation all produce bad signatures on entirely legitimate mail; quarantining them would cost far more than it catches. quarantine and bounce are available.

Note that only invalid is a “bad signature” for this purpose. valid-untrusted and unverifiable are the normal state of mail from a correspondent nobody here has heard of, and routing those anywhere would route most of the internet.

The decryption policy is consulted first: a message that could not be opened has no signature verdict worth acting on.

70.1.7.1.8. Anti-forgery

Every X-Pepsi-* header field is removed from every message this stage sees — including a message it is not going to touch, and including one for which the stage is disabled. Only then is our own added.

This is not hygiene. X-Pepsi-Crypto is a private header, so it is forgeable by definition; the removal is the entire basis on which a client may believe it. The Pepsi-Origin proof-of-origin header is deliberately not in the namespace and survives.

The same argument applies to the Subject. With STRIP_SUBJECT_TAGS = yes (the default) this stage’s own tag vocabulary — TAG_DECRYPTED, TAG_VERIFIED, TAG_BAD_SIGNATURE, the built-in [decrypted] and [verified] whatever those were renamed to, and anything in STRIP_SUBJECT_KEYWORDS — is removed from the front of an inbound subject before ours is prepended. Otherwise an external sender simply writes [verified] themselves. Tags are stripped from the front only, and from behind a Re:/Fwd: prefix, so a subject that mentions one mid-sentence keeps it.

70.1.7.1.9. Subject tags

With SUBJECT_TAGS = yes (the default) the earned tags are prepended to the Subject in nesting order:

encrypted(signed(body))   ->  [decrypted][verified] quarterly figures
signed(encrypted(body))   ->  [verified][decrypted] quarterly figures

Both shapes open. The second line states the ordering rule; note that the second example earns its [verified] only from an inner signature, because an outer signature over ciphertext never reaches the valid verdict the tag is written for (A signature that covers ciphertext). A signed(encrypted(body)) message with no inner signature is therefore tagged [decrypted] alone.

For most users this is the only signal they will ever see — nobody reading mail in a webmail client inspects headers — so doing the work invisibly would waste it. The cost is that a user-visible field is mutated: it affects search, threading and the quoted subject of replies. Set SUBJECT_TAGS = no to rely on the header alone.

[verified] is written only for the valid verdict. TAG_BAD_SIGNATURE is empty by default: a bad signature is a routine property of legitimate mail, so a default-on tag would decorate a large part of a user’s inbox with an accusation.

70.1.7.1.10. The result header

With ADD_RESULT_HEADER = yes (the default) a message that carried any protection gains one header field, prepended at the top so it disturbs no existing signature:

X-Pepsi-Crypto: encrypted=yes; decrypted=yes; protocol=openpgp;
  signature=valid; signer=alice@example.net;
  fingerprint=0123456789ABCDEF; key-source=wkd; trust=wkd-advanced;
  layers="encrypted,signed"

The grammar is semicolon-separated key=value pairs in a fixed order. A value that is not a bare token (A–Z, a–z, 0–9 and . _ @ : + / -) is double-quoted; an embedded " becomes ', and control characters are replaced by spaces, so no value from the message can end the field or start another one. Long values are folded at whitespace.

The keys are:

encrypted

yes/no — the message arrived carrying ciphertext.

decrypted

yes/no — present only when encrypted=yes.

protocol

openpgp or smime.

signature

One of the verdicts above.

signer, fingerprint, key-source, trust, algorithm

The signing key, where it came from and what it was worth. Present only when known.

failure

The machine-readable reason a signature verdict is not good.

decrypt-failure

The machine-readable reason decryption failed.

layers

The protection layers, outermost first, comma separated.

RFC 8601 registers no smime or pgp method, so an Authentication-Results extension would mean inventing a method name in a registry we do not own — which other implementations are entitled to ignore or mistrust. A private header with a documented grammar is the honest choice, and the price of it is the unconditional stripping described above.

70.1.7.1.11. Key discovery

The stage performs no network I/O of any kind. When the signer’s key is neither in the message nor in the peer_key cache, it enqueues a discovery request and parks the message; a pepsi-keydisc(1) service releases it when the request settles, found or not, and the stage runs again. A settled request with no key releases the message just the same, and it then verifies as unverifiable. A fresh negative-cache entry short-circuits the wait entirely, which is what stops a spammer signing with an unknown key from making every message wait: the first one parks, the rest do not.

[pepsi-keydiscovery] INBOUND_TIMEOUT bounds the wait and is separately tunable from the outbound TIMEOUT; DISCOVERY_TIMEOUT in this stage’s own section overrides it. DISCOVERY = no suppresses the request altogether and leaves the stage on the cache-only path.

Whether a parked message is stored decrypted depends on the shape of the message, and the difference matters:

• An S/MIME signature nested inside an envelope survives decryption as an ordinary MIME layer, so the message is parked with the plaintext committed and is decrypted exactly once.

• An OpenPGP signature lives only inside the packet stream, so it is gone the moment the plaintext is out. Committing the plaintext would destroy the evidence the key is being fetched to check, so such a message is parked unchanged and decrypted again on resume.

A message parked with its plaintext committed sits in pepsi.ingress decrypted for up to the discovery deadline. That is the same exposure delivery would create a moment later anyway; it is a longer window, not a new one.

70.1.7.1.12. Certificates the message carries

Most signed mail carries the certificate that signed it, so before deciding anything the stage reads the S/MIME signer certificates and application/pgp-keys parts out of the message and offers them to the verifier. It does this again on the plaintext once the message is open, since a certificate attached inside the ciphertext is invisible until then, and re-evaluates once if that yielded something the first pass did not have.

These certificates are used and dropped. This stage stores nothing: learning from inbound mail — the Autocrypt: header, attached keys, and gossip — is pepsi-stage-autocrypt-learn(1), which runs later in the pipeline, after the stages that decide whether a message is spam. That split is what lets Autocrypt Level 1 §5.3’s spam rule be honoured at all: this stage runs first on the inbound path by design, before any such verdict exists. It also means nothing is written to the key store by the process that holds the private keys.

Using a certificate the message carried can only ever lower a verdict’s standing, never raise it: such material is untrusted provenance, so a signature checked against it comes back valid-untrusted and does not set state.signature_verified — the key pepsi-stage-check-whitelist(1) gates on. That is the same rung a stored harvested key would have had, which is why routing it through the database bought nothing.

70.1.7.1.13. Two facts recorded for the learn stage

Decryption commits the plaintext over the bytes that arrived, so two things only this stage can see are written to state.crypto.in for pepsi-stage-autocrypt-learn(1) to read: encrypted, that the message carried ciphertext at all, and outer_gossip, that the arriving header block already had an Autocrypt-Gossip: field. Both are preconditions for believing a message’s gossip, and neither is answerable once the plaintext is in the row.

70.1.7.1.14. Our own users’ keys are never overridden

Harvesting keys on the From: header, which the sender writes, and inbound authentication is deliberately fail-open. A message claiming to come from one of this deployment’s own users will therefore seed a peer_key row for that user with whatever key it carried — and a signature checked against it would otherwise verify, setting state.signature_verified, which pepsi-stage-check-whitelist(1) gates on.

So before the store is consulted, the stage asks whether the claimed sender is somebody it holds an identity for. If it is, that identity’s public key is the only candidate: no peer_key row for the address is offered to the verifier, whatever its rank and however it arrived. Anything but a revoked identity counts, expired ones included — a signature made last year was made with last year’s key, and refusing to look at it would report our own users’ mail as unverifiable every time a key rolled over.

The rule is conditional on holding a key, not on serving the address: a user who runs their own OpenPGP client and never uploaded a key here has no identity, so their harvested or gossiped key is used exactly as any correspondent’s would be. The cost is the mirror image — once an identity is held for an address, a signature made with that user’s separate personal key no longer verifies. See the manual’s Our own users’ keys are not discovered.

70.1.7.1.15. Trust anchors

X.509 anchors come from the pepsi.ca_trust table (pepsi-keys(1) ca) and are cached per worker process for TRUST_ANCHOR_TTL (default five minutes). An anchor added or disabled therefore becomes effective within that window; restart the workers to apply it at once. They are loaded lazily and only for a message that actually carries an S/MIME layer, so ordinary traffic costs no query.

Revocation data is not fetched: a CRL retrieval would be network I/O in a stage, which the discovery design exists to avoid. A chain is consequently reported as revocation not-checked, which is stated rather than being allowed to imply “not revoked”.

70.1.7.1.16. Configuration

The stage reads its own [stage-<name>] section (PROGRAM = pepsi-stage-decrypt, NEXT_STAGE (required), BOUNCE_STAGE, plus ENABLED, LOCAL_DOMAINS, TARGETS, RECIPIENT_DELIMITER, REQUIRE_ACCOUNT, ON_DECRYPT_FAILURE, ON_BAD_SIGNATURE, QUARANTINE_STAGE, SUBJECT_TAGS, TAG_DECRYPTED, TAG_VERIFIED, TAG_BAD_SIGNATURE, STRIP_SUBJECT_TAGS, STRIP_SUBJECT_KEYWORDS, ADD_RESULT_HEADER, MAX_LAYERS, TRUST_ANCHOR_TTL, DISCOVERY, DISCOVERY_TIMEOUT), all documented in pepsi.conf(5). Those are behavioural options and are per-address overridable through the pepsi.settings table.

The cryptographic policy — algorithm choice, weak-digest acceptance, minimum key sizes — lives in [pepsi]’s CRYPTO_* options and [pepsi-crypto]. Those are system-administrator-only and are deliberately not reachable from pepsi.settings or pepsi-stage-edit-settings(1).

One of them does not apply here. CRYPTO_ALLOW_DOWNGRADE governs what may be emitted: OpenPGP SEIPDv1+MDC and CMS EnvelopedData are always readable whatever it says, because refusing them inbound would make this host unable to receive mail from most of the world while protecting nobody. Only 3DES and the weak signature digests are gated inbound, and each refusal produces its own verdict rather than a generic failure. The container that was actually used is recorded per layer, so a deployment can see how much of its inbound mail is still pre-AEAD.

70.1.7.1.17. Privilege

The binary is installed setuid pepsi-crypto, mode 4750 pepsi-crypto:pepsi, and is therefore not folded into the unified pepsi multi-call binary.

Both halves of the key-custody boundary key on that identity. The database role pepsi-crypto is the only one granted crypto_identity.private_wrapped, and PostgreSQL peer authentication keys off the effective uid; the key-encryption key lives in secrets.d/pepsi-crypto.secret, mode 0640 owned by the same account. A setgid binary would gain the group — enough to read the fragment — and would still authenticate to the database as pepsi, precisely the role that is denied the column.

The group in the mode restricts execution to the dispatcher’s account and root; the program also refuses at run time unless its real uid is one of those. -c is accepted from those callers (the dispatcher’s documented spawn shape is PROGRAM -c cfg worker) and the environment is sanitised before the configuration is loaded, because the configuration layer is steerable through XDG_CONFIG_HOME, HOME, PATH and the PG* family.

Decryption keys are selected by capability: every non-revoked identity of every local recipient whose purpose is encrypt or both, including expired ones, because old mail must stay readable.

70.1.7.1.18. Resource limits

MAX_LAYERS (default 4) caps how many nested encrypted layers are unwrapped; a message with more is delivered with the remainder unopened and decrypt-failure=unsupported recorded.

An OpenPGP message is routinely compressed and the expansion ratio is the sender’s choice, so pepsi-crypto refuses a plaintext over 64 MiB outright rather than truncating it — a partial plaintext must never be reported as verified. That case is reported as its own verdict, too-large, and not as a generic decryption failure: “this message is a bomb” and “we could not open this” are different facts for an operator.

70.1.7.1.19. State

Inputs: state.dsn (honoured and sliced in lockstep with the locality split), state.crypto.in (on a resumed row, what the pass that decrypted it recorded).

Outputs: state.crypto.in — whether the message was encrypted, what became of the ciphertext, which identity opened it, the signature verdict and its detail, and the ordered layer list. Also state.encrypted = true when the message arrived encrypted, and state.signature_verified = true for the valid verdict only; pepsi-stage-check-whitelist(1) and pepsi-stage-auto-whitelist(1) read those. On a bounce, state.bounce. The state layout is described in pepsi.state(7).

Transitions: NEXT_STAGE for a delivered message; QUARANTINE_STAGE or BOUNCE_STAGE per the failure policies; paused while waiting for key discovery; pending at this same stage after a locality split.

70.1.7.1.20. Commands

worker

Run as a persistent dispatcher worker, reading message ids on standard input and writing one status line per message. This is the only mode; there is no one-shot form. To run a single message by hand, pipe its id:

echo 1234 | pepsi-stage-decrypt -c /etc/pepsi/pepsi.conf worker

70.1.7.1.21. Exit Status

0

The worker exited cleanly (standard input closed).

1

An error occurred (misconfiguration, an unreadable key store, or a database error). The reason is written to the log.

70.1.7.1.22. See Also

pepsi-stage-encrypt(1), pepsi-stage-arc(1), pepsi-keys(1), pepsi-keydisc(1), pepsi-stage-check-whitelist(1), pepsi-stage-relay-to-maildir(1), pepsi-stage-relay-to-lmtp(1), pepsi-stage-bounce(1), pepsi-dispatch(1), pepsi-setup(1), pepsi.conf(5), pepsi.state(7)

70.1.7.1.23. Bugs

Report bugs to the Pepsi issue tracker.