38. pepsi-stage-decrypt

Decrypt mail addressed to our users, and say what its signature is worth.

38.1. Role

pepsi-stage-decrypt is the inbound half of Pepsi’s end-to-end cryptography. For a message addressed to a recipient that this host serves, it opens whatever ciphertext the message carries, verifies whatever signature it carries against a real trust chain, records the verdict, and removes any security indicator that the sender forged. It uses the certificates that the message carries but stores none of them. It performs no network I/O: when the signer’s key is unknown the message is paused and a pepsi-keydisc service releases it. What it opens is filed as plaintext unless pepsi-stage-reencrypt seals it again to the recipient’s own mail-client key just before delivery. Reference: pepsi-stage-decrypt(1).

Both protocols it reads are standards:

  • OpenPGP — RFC 9580 (the crypto refresh, superseding RFC 4880) for the packet grammar and algorithms, RFC 3156 for the PGP/MIME containers (multipart/encrypted, multipart/signed and the micalg parameter). Inline (non-MIME) OpenPGP is read as well, though never written.

  • S/MIME — RFC 5652 for the Cryptographic Message Syntax, RFC 5083 with RFC 5084 for AuthEnvelopedData (AES-GCM), RFC 3565 for AES-CBC EnvelopedData, RFC 5753 for ECDH key agreement, RFC 9709 for content keys derived with HKDF over the algorithm identifier, and RFC 8551 for the message profile with RFC 8550 for certificate handling. For the certificates themselves, RFC 5280 covers path building, the emailProtection extended key usage and the rfc822Name binding that ties a certificate to an address, and RFC 9598 the SmtpUTF8Mailbox form that an internationalised address takes.

Two limits are stated here so that they are not mistaken for bugs. Certificate revocation soft-fails and has no OCSP: the stage itself fetches nothing, but pepsi-keydisc fetches into pepsi.crl_cache the CRLs (RFC 5280 §5) that cached S/MIME peer certificates name, and the stage checks signer chains against the current ones. A listed certificate makes the signature invalid (revoked). A chain with no usable list (none obtainable, stale, not signed by the issuer’s key, out of scope) is reported as revocation “not checked”, never as “not revoked”. OCSP (RFC 6960) is never used. A signer who sent no intermediate certificates is validated with the ones that pepsi-keydisc fetched from the caIssuers pointers of cached certificates, when [pepsi-keydiscovery] CA_ISSUERS_FETCH is on; a fetched certificate is never an anchor. And an unauthenticated container is not reported as good: AES-CBC EnvelopedData carries no MAC, so it opens with a verdict of untrusted rather than good, while AES-GCM AuthEnvelopedData, which Pepsi emits, is authenticated. Both are listed in RFC Index.

Warning

Put this stage after the ARC stage and before the content stages. The recommended inbound chain is init = arc → decrypt → aliases → (anti-spam / language / …) → local delivery. An ARC set seals the message as it arrived; decrypting first would make the sealed AMS describe bytes that nobody else ever saw. pepsi-setup warns when it finds an ARC stage reachable from a decrypt stage, and again when no local-delivery stage is reachable from one.

38.2. Features

  • It only runs for recipients this host serves, and for anything else it does not even try. Decryption rewrites the body, which invalidates the sender’s DKIM signature. That is harmless for a message about to be filed in a mailbox here, but not for one being relayed onward. A message with both kinds of recipient is split, so the forwarded copy is the one that arrived.

  • Each copy is opened with its own recipient’s keys. A ciphertext addressed to several local recipients is split one per row first, so a message to Bob and to someone else cannot have Bob’s key open it for the other recipient.

  • A verdict that distinguishes what matters. valid means the signature is sound and the key was anchored; valid-untrusted means that the signature is sound but the key could not be tied to anything (a first-seen key, a CA with no anchor here). Signed mail from a new correspondent often falls into this class, and it is never reported as valid. invalid, unverifiable and none complete the set, and a message with several signatures takes the worst of those that cover the plaintext; a signature wrapped around ciphertext can never make a message valid. Nor can an OpenPGP signature inside the ciphertext that names its intended recipients without naming the key that opened it: it was signed for somebody else and re-encrypted to us (wrong-recipient, see Who a signed and encrypted message was for). What the verdicts and the subject tags promise a user is set out in What a security indicator means.

  • Protected headers are compared with the outer ones. When a signer placed From:, To:, a reply-steering field (Reply-To:, Mail-Reply-To:, Mail-Followup-To:) or Subject: inside the signed entity (RFC 9788 injected headers, or a signed message/rfc822), a protected From: that names another address than the outer From: makes an otherwise valid signature valid-untrusted, with no [verified] tag and no state.signature_verified. This catches a signed message re-sent under someone else’s From:. A reply-steering field that was changed, added or removed in transit does the same. The exception is a field that a mailing list added when the signer protected none of them: the signature then stays valid under [verified (except reply-to)]. A Subject: that only gained bracketed segments in front (a list tag, an [external] marker) does not count as a difference. A subject changed beyond that is reported and, on a valid signature, the signed subject is shown instead, so the message stays [verified] under the subject that its signer wrote. A protected Subject: carrying a forged indicator ([verified] … or a look-alike of it, which the subject-tag sanitiser would remove but cannot remove inside a signed part) makes the signature valid-untrusted as well (withheld=protected-subject). A differing To: is only reported (state.crypto.in.protected_mismatch, protected-mismatch in X-Pepsi-Crypto). See Protected headers against the outer ones.

  • ``[verified]`` says who, and is granted for the ``From:`` line as the reader sees it. The tag names the address that the signer is bound to ([verified: alice@example.org]). When the tag is withheld, the verdict is valid-untrusted and X-Pepsi-Crypto gives the reason. It is withheld when the display name shows another address, when the From: names several mailboxes, when an anchored certificate names no mailbox at all (unless the operator opted that anchor in, pepsi-keys ca accept-nameless), and when the signer’s domain looks like a served domain or a known correspondent’s. That comparison uses the same look-alike skeleton as the subject-tag stripper, built from the UTS #39 confusables data, and the A-label then goes into X-Pepsi-Crypto. RFC 9598 SmtpUTF8Mailbox names are read and bound as that RFC says. It is withheld too when the signature was made more than MAX_SIGNATURE_AGE_DAYS (7) before the message arrived (withheld=old-signature) or more than a day after (future-signature): a captured signed message replayed under fresh outer headers keeps a genuine signature but is not its signer’s message of today. The arrival time is this host’s, never the Date: field; the signing time is shown as signed-at= in X-Pepsi-Crypto. See What [verified] binds the signer to.

  • A correspondent who usually signs is remembered. Every valid signature records its From: address in pepsi.signed_sender; a later message from that address without a valid signature, within MISSING_SIGNATURE_WINDOW_DAYS (90), is handled under ON_MISSING_SIGNATURE: record (default; usually-signed=<date> in X-Pepsi-Crypto and state.crypto.in.usually_signed), tag ([unsigned, usually signed]) or quarantine. The default only records, since a correspondent writing from a phone or a new client looks exactly like this. See Correspondents who usually sign.

  • A signature inside a ciphertext is always tagged. Anyone can re-send a captured ciphertext, or encrypt to a published key, from their own address. So beside [decrypted], a sound signature whose key is untrusted or unknown is tagged [unverified signer] and a bad one [bad signature]. Both tags are on by default (TAG_UNVERIFIED_SIGNER, TAG_BAD_SIGNATURE_ENCRYPTED), while outside a ciphertext TAG_BAD_SIGNATURE stays empty.

  • A verified clearsigned block that shows unsigned text is ``partial``. An inline clearsigned block is delivered as it arrived; when it carries an armor header line other than Hash: (which the armor reader accepts without hashing) or sits in a text/html part, the reader sees text that the signature does not cover. The block then gets no tag and no state.signature_verified; the reader’s client can still re-verify it.

  • S/MIME revocation from cached CRLs. A signer chain is checked against the revocation lists that pepsi-keydisc fetched; a revoked certificate makes the signature invalid, and a chain without a usable list reads “not checked”.

  • Both failure paths deliver by default. The default is ON_DECRYPT_FAILURE = passthrough, because the user may hold the key in their own client, and bouncing mail that we merely could not open helps nobody. It is ON_BAD_SIGNATURE = record, because mailing lists that rewrite bodies, forwarders and clients with broken canonicalisation all produce bad signatures on legitimate mail. quarantine is available for both, bounce for a bad signature and bounce-if-unproblematic for a decryption failure. Neither bounce ever answers the padding question: a failed ciphertext is bounced only when it is integrity-protected (OpenPGP, S/MIME AuthEnvelopedData), a bad signature only when it did not come out of an AES-CBC EnvelopedData, and any other such message is dropped (telemetry decrypt-dropped). ON_DECRYPT_FAILURE = bounce is an alias for the longer but unambiguous bounce-if-unproblematic; the worker and pepsi-setup warn about the short spelling and ask for the long one. A decryption-failure DSN never names the failure class, and no key lookup is started for a signature found inside an unauthenticated (CBC) container, since whether the lookup reaches the sender’s domain would be the same oracle.

  • AES-CBC can be left to the client. OPEN_ENVELOPED_DATA = no (default yes) passes every S/MIME EnvelopedData on unopened, as UNAUTHENTICATED_PLAINTEXT = require-signature does with an unsigned one, so the content key of a captured AES-GCM message re-submitted under a synthetic CBC container is never applied to it. A container whose key is derived the RFC 9709 way (id-alg-cek-hkdf-sha256) is opened in either form; Pepsi advertises the derivation in its own signatures. The option can be set for one recipient address or domain through a pepsi-config overlay; there is no per-correspondent switch, because the container’s sender is known only from an outer From: that the attacker writes.

  • Decrypted HTML is delivered as HTML unless asked otherwise. DECRYPTED_MARKUP = keep (default) leaves remote content in an authenticated decrypted message to the client’s own settings (Remote content); text reduces every decrypted plaintext to plain text, as is always done for an unauthenticated one.

  • A forwarded signature is not addressed to us. An OpenPGP signature whose Intended Recipient Fingerprints, or an S/MIME signature whose Pepsi recipient attribute, leaves out the certificate that opened the message is invalid (wrong-recipient).

  • Protected content is isolated first. Before any key is looked up, a message for a local recipient that carries ciphertext beside other parts keeps its first encrypted unit as the message and has the other parts moved into inert application/octet-stream attachments (ISOLATE_ENCRYPTED). This stops EFAIL direct exfiltration here and in a client that decrypts the message itself. A message without ciphertext keeps only its signed content beside unsigned content (ISOLATE_SIGNED). An attached message carrying ciphertext beside other parts is reduced the same way inside the attachment, through three levels of attached messages (nested), since a client opens and decrypts it on its own. Both options are on by default, and the numbers of moved parts that were delivered or dropped are recorded as demoted/dropped.

  • Forged indicators are removed, unconditionally. Every X-Pepsi-* header is removed from every inbound message, and this stage’s own subject tags are removed from the front of the Subject, before ours are added. X-Pepsi-Crypto is a private header and therefore forgeable by definition; the removal is the entire basis on which a client may believe it. (Pepsi-Origin is deliberately not in the namespace and survives; ingress’s X-Pepsi-Warning on the outer header block is kept, since ingress removes a sender’s copies before adding it.)

  • A signal the user actually sees. The stage prepends [decrypted][verified] to the Subject in nesting order. For most users this is the only signal that they will see, because few people look at the header fields, and webmail users hardly ever do. It can be switched off, and the wording is configurable.

  • Certificates the message carries. S/MIME signer certificates and application/pgp-keys parts are read out of the message and offered to the verifier before the stage decides whether to wait for a key, because most signed mail carries the certificate that signed it. They are used and then dropped: storing keys from inbound mail is the job of pepsi-stage-autocrypt-learn, which runs later, after the stages that decide whether the message is spam. Such a certificate can only lower a verdict’s standing, never raise it: a signature checked against one is valid-untrusted and never sets state.signature_verified.

  • Two facts recorded for the learn stage. Decryption commits the plaintext over the bytes that arrived, so state.crypto.in carries encrypted (there was ciphertext) and outer_gossip (the arriving header block already had an Autocrypt-Gossip: field, which anything on the path could have written). Both are preconditions for believing a message’s gossip, and neither is answerable once the plaintext is in the row.

  • Mail for the user’s own key is left for their client. A message encrypted to a recipient’s registered MUA key (custody = client, a key whose private half lives only in the mail client) is recognised from the ciphertext’s recipient identifiers without opening it, and passed on unopened, even when one of our own keys could open it too. It is recorded as decryption = for-client, is not a decryption failure, and so never triggers ON_DECRYPT_FAILURE.

  • Every non-revoked key is tried, including expired ones, selected by capability (purpose encrypt or both) rather than by shape. Old mail must stay readable, which is why a decryption key survives its own expiry.

38.3. Where it sits

ingress → arc → decrypt → aliases → check-whitelist → anti-spam →
detect-language → autocrypt-learn → local delivery

Everything that reads the body wants cleartext, and gets it because decryption is early. Everything that authenticates the message as it arrived, such as ARC, must come first.

Placement is a performance matter too. The stage declares Load::Full, so every message that passes through it loads its whole body from the database, even one that turns out to be ordinary mail: whether a message is protected cannot be told from the envelope columns, and Load is fixed per stage. That cost is affordable on an inbound path that ends in local delivery, but this is not a stage to put in front of bulk relay traffic.

38.4. Waiting for a key

When the signer’s key is in neither the message nor the peer_key cache, the message is parked: a discovery request is enqueued and a pepsi-keydisc service releases the message when it settles, found or not. So the first message from a new correspondent gets a real verdict rather than unverifiable. A fresh negative-cache entry short-circuits the wait entirely, so only that first message ever waits.

Whether a parked message is stored decrypted depends on the shape of the message, and the difference is not cosmetic. An S/MIME signature nested inside an envelope survives decryption as an ordinary MIME layer, so the message is parked with the plaintext committed and decrypted exactly once. An OpenPGP signature lives only inside the packet stream and is gone the moment the plaintext is out, so committing the plaintext would destroy the very evidence that the key is being fetched to check; such a message is parked unchanged and decrypted again on resume.

38.5. Configuration

[stage-<name>]: PROGRAM = pepsi-stage-decrypt, NEXT_STAGE (required) and BOUNCE_STAGE, plus ENABLED, LOCAL_DOMAINS / TARGETS / RECIPIENT_DELIMITER / REQUIRE_ACCOUNT, ON_DECRYPT_FAILURE, ON_BAD_SIGNATURE, ON_MISSING_SIGNATURE, MISSING_SIGNATURE_WINDOW_DAYS, TAG_MISSING_SIGNATURE, MAX_SIGNATURE_AGE_DAYS, UNAUTHENTICATED_PLAINTEXT, DECRYPTED_MARKUP, OPEN_ENVELOPED_DATA, QUARANTINE_STAGE, SUBJECT_TAGS, TAG_DECRYPTED, TAG_VERIFIED, TAG_VERIFIED_EXCEPT_REPLY_TO, TAG_UNAUTHENTICATED, TAG_BAD_SIGNATURE, TAG_BAD_SIGNATURE_ENCRYPTED, TAG_UNVERIFIED_SIGNER, STRIP_SUBJECT_TAGS, STRIP_SUBJECT_KEYWORDS, ADD_RESULT_HEADER, ISOLATE_ENCRYPTED, ISOLATE_SIGNED, MAX_LAYERS, TRUST_ANCHOR_TTL, DISCOVERY and DISCOVERY_TIMEOUT. All are documented in pepsi.conf(5).

Those are behavioural options and are per-address overridable through the pepsi.settings table (keyed, for inbound mail, on an envelope recipient). The cryptographic policy lives in [pepsi]’s CRYPTO_* options and [pepsi-crypto], is system-administrator-only, and is deliberately not reachable from pepsi.settings.

One of those does not apply in this direction: CRYPTO_ALLOW_DOWNGRADE governs what may be emitted. SEIPDv1+MDC and AES-CBC EnvelopedData are readable (unless OPEN_ENVELOPED_DATA = no), because refusing them inbound would make this host unable to receive mail from most of the world while protecting nobody.

38.6. Privilege

The stage is installed setuid pepsi-crypto, mode 4750 pepsi-crypto:pepsi, and is therefore standalone rather than folded into the unified pepsi binary. This is the same arrangement as pepsi-stage-encrypt, for the same two reasons: the database role pepsi-crypto is the only one granted crypto_identity.private_wrapped and PostgreSQL peer authentication keys off the effective uid, and the key-encryption key is a 0640 fragment owned by the same account. See Key management for the custody model and The private-key custody boundary and Gateway custody (custody = local, the MTA key) for what that boundary does and does not protect.

38.7. State

The stage writes state.crypto.in: whether the message arrived encrypted, what became of the ciphertext (not-encrypted, decrypted, failed or for-client), which identity opened it, the signature verdict with its signer, fingerprint, key source, covers_plaintext, binding and bound_to, withheld or unsigned_surface and its signed_at when valid was withheld or the verdict is partial, the ordered layer list, isolated, demoted, dropped and nested when isolation changed the message, protected_mismatch when signed protected header fields differ from the outer ones (subject_promoted when the signed subject was shown instead), signer_domain for an internationalised or look-alike signer domain, and encrypted and outer_gossip for pepsi-stage-autocrypt-learn. Sets state.encrypted = true for a message that arrived encrypted and state.signature_verified = true for the valid verdict only; these are the two keys that pepsi-stage-check-whitelist and pepsi-stage-auto-whitelist read. On a policy bounce it writes state.bounce with a 5.7.0 enhanced status.

38.8. See also

pepsi-stage-encrypt (the outbound half), pepsi-stage-arc, pepsi-keys, pepsi-keydisc (the service that settles the parks this stage creates), pepsi-stage-check-whitelist (which gates a whitelist row on the state.signature_verified this stage sets), Key management, Client interoperability, RFC Index, Security model (Protected content stands alone, Header forgery, Bounces and replies to encrypted mail), What a security indicator means, pepsi.conf(5), pepsi.state(7)