85.1.7. pepsi-stage-decrypt

decrypt mail addressed to our users and verify what signed it

Manual section:

1

85.1.7.1.1. Name

pepsi-stage-decrypt - inbound end-to-end decryption and signature verification stage of the Pepsi pipeline.

85.1.7.1.2. Synopsis

pepsi-stage-decrypt [GLOBAL-OPTIONS] worker

85.1.7.1.3. Description

pepsi-stage-decrypt is a stage program run by pepsi-dispatch(1) as a persistent worker reading message ids on standard input. For a message addressed to a recipient this host serves it opens whatever ciphertext the message carries, verifies whatever signature it carries against a real trust chain, records the verdict, and removes any security indicator the sender forged. It reads the certificates the message carries in order to check that signature, but stores none of them: learning correspondents’ keys is pepsi-stage-autocrypt-learn(1)’s job, and it is a separate stage because it must run after the spam verdict, which does not exist yet here.

Doing this on the server is what removes the need for a client plugin: the user reads ordinary mail in their usual client. The trade is only sound because the message is about to stop being an SMTP object and become a file in a mailbox — see Locality. That file is plaintext unless the recipient registered their own mail-client key and pepsi-stage-reencrypt(1) sits in front of local delivery, sealing what this stage opened to that key once the later stages have read it.

The verification half is deliberately strict about one distinction. A cryptographically sound signature tells you the message was signed by some key; whether that key is the sender’s is a separate question. The two are never reported as the same thing (Verdicts).

85.1.7.1.4. Locality

The stage runs only for recipients this host serves, and for a message it does not serve it does not even try — a failed decryption attempt should not colour the verdict, and there is no key for such a message anyway.

The reason is that decryption rewrites the body. The sender’s DKIM signature then no longer verifies, and an ARC seal made before decryption (see Placement) covers the encrypted form. For a message about to be delivered locally that is harmless. For a message being forwarded onward it is not: it would arrive at the next hop with a broken signature, which is a stronger negative signal than no signature at all.

Locality is the same test the delivery stages use — LOCAL_DOMAINS, TARGETS and RECIPIENT_DELIMITER — with one difference. By default only the domain has to match, because this stage does not have to know whose mailbox to write; a hosted domain whose users hold crypto identities but no shell accounts is an ordinary deployment. Set REQUIRE_ACCOUNT = yes to demand a passwd entry in TARGETS as well.

A message with both local and foreign recipients is split: the foreign recipients are peeled onto a sibling row at NEXT_STAGE carrying the message exactly as it arrived, and this row keeps the local ones. The per-recipient DSN state is sliced in lockstep by the same code every other recipient split uses.

85.1.7.1.5. Placement

The recommended inbound chain is:

init = arc -> decrypt -> aliases -> (anti-spam / language / …) -> local delivery

ARC runs first. An ARC set seals the message as it arrived; decrypting first would make the sealed AMS describe bytes nobody else ever saw, so the seal would be a claim about a message that never existed. pepsi-setup(1) warns when it finds an ARC stage reachable from a decrypt stage.

Content inspection runs after. Language detection, the pay-to-send gate and the From:-header whitelist check all read the body, and all of them want cleartext.

Placement is also a performance matter. The stage declares Load::Full, because whether a message is protected cannot be told from the envelope columns and Load is fixed per stage. Every message that passes through it therefore loads its whole body from the database, even one that turns out to be ordinary unencrypted mail. That is affordable on an inbound path terminating in local delivery; it is not a stage to put in front of bulk relay traffic.

85.1.7.1.6. Verdicts

state.crypto.in.signature.status is one of:

none

The message carries no signature.

valid

Cryptographically sound and the key was anchored: an X.509 chain to a configured ca_trust anchor, or an OpenPGP key from a ranked discovery source (DANE, WKD, a key server, or one the operator pinned).

valid-untrusted

Cryptographically sound, but the key could not be tied to anything: a key seen for the first time, learnt from the message itself or from an Autocrypt header, or a certificate from a CA this deployment holds no anchor for. Most of the world’s signed mail is in this state.

This is never reported as ``valid``. It is the distinction the stage exists to make, and a security indicator that conflates the two is worse than none.

invalid

The signature does not verify over the bytes it covers, or the signer’s certificate is expired, revoked, or bound to an address other than the message’s From:.

unverifiable

No key or certificate was available for the claimed signer, so nothing could be established either way. This is the verdict that makes the stage wait for a key (Key discovery).

A message carrying several signature layers takes the worst of the verdicts of the layers that speak for it. One broken signature makes the message invalid however good the others are. A layer that only covers ciphertext is one of those only when nothing covers the plaintext (see below): such a layer can never make the message valid, and once some layer does cover the plaintext it is disregarded entirely, however bad it is.

Only valid sets state.signature_verified, which pepsi-stage-check-whitelist(1) gates a whitelist entry on.

85.1.7.1.6.1. A signature that covers ciphertext

In signed(encrypted(body)) the signature is computed over the ciphertext, not over anything a reader will ever see. That proves the signer transmitted the blob; it does not prove they wrote it, and it does not prove they can read it — anybody who obtains an encrypted message can wrap it in their own signature and forward it. Such a layer is recorded with "covers_ciphertext": true in state.crypto.in.layers and, whatever its own verdict, it is never allowed to make the message ``valid``: a good one is reported as valid-untrusted, so it earns no [verified] tag and does not set state.signature_verified. A bad one is reported as invalid and routes under ON_BAD_SIGNATURE only when it is all there is — that is, when no layer covers the plaintext.

Once some layer does cover the plaintext, the ciphertext-covering ones are set aside entirely, whatever their own verdict. A layer that cannot grant a verdict must not be able to destroy one: anybody who obtains the encrypted blob can re-wrap it, so if a corrupt wrapper forced invalid then corrupting the wrapper would be strictly more useful to an attacker than removing it (which leaves an ordinary encrypted(signed(body)) that still verifies). The outer signature of a triple wrap is in any case a hop-by-hop artefact — §3.7 has a mailing-list agent applying it — so its failure says nothing about the author. The tampering is not lost: it stays visible in state.crypto.in.layers and in X-Pepsi-Crypto, as an observation rather than a routing input. The same reasoning covers the commoner case of an outer signature that is merely unverifiable (no key for the forwarder), so the triple wrap below is not punished for a signer we happen not to know.

So in signed(encrypted(signed(body))) — the triple wrap RFC 8551 §3.7 recommends — the inner signature is the one that speaks for the content, and it is the one this stage reports. That shape is therefore reported as valid where a single outer signature is not.

Whenever there is a signature verdict, state.crypto.in.signature also records covers_plaintext: true when the verdict was granted by a signature over the plaintext, false when it came from a layer wrapped around ciphertext. pepsi-stage-autocrypt-learn(1) accepts only the first kind as proof that a correspondent holds one of our keys.

85.1.7.1.7. Failure handling

Both failure paths deliver by default, matching Pepsi’s deliberate fail-open posture on inbound SPF, DKIM and DMARC:

ON_DECRYPT_FAILURE = passthrough

The still-encrypted message is delivered. The user may well hold the key in their own client, and bouncing mail we merely could not open helps nobody. quarantine and bounce are available.

ON_BAD_SIGNATURE = record

A bad signature is a recorded verdict, never a delivery failure. Mailing lists that rewrite bodies, forwarders and clients with broken canonicalisation all produce bad signatures on entirely legitimate mail; quarantining them would cost far more than it catches. quarantine and bounce are available.

Note that only invalid is a “bad signature” for this purpose. valid-untrusted and unverifiable are the normal state of mail from a correspondent nobody here has heard of, and routing those anywhere would route most of the internet.

The decryption policy is consulted first: a message that could not be opened has no signature verdict worth acting on.

A recipient key this host holds but cannot open — a key-encryption key that does not match the stored blob, a secrets.d fragment gone missing, a revoked database grant — also ends as decrypt-failure=no-decryption-key and takes the ON_DECRYPT_FAILURE route, exactly like a message addressed to a key we never had. That is deliberate (one broken identity must not stop the others from being tried), but it is a broken deployment and not the sender’s doing, so each such identity is logged at error with the address and the reason. The same goes for recipients who hold decryption identities on a host with no [pepsi-crypto] KEY_WRAP_SECRET configured at all.

85.1.7.1.8. Errors of this host

Everything else that goes wrong on this host is retried rather than failed (see pepsi-dispatch(1)): a database that cannot answer — including the lookup of the recipients’ own MUA keys, which is never read as “no such key”, since that would open mail meant for the user’s own client — is paused and tried again until the stage’s MAX_LIFETIME. The stage’s section, the [pepsi] crypto policy, [pepsi-keydiscovery] and, when a key-encryption key is configured, the key ring built from it are checked when the worker starts: one that does not work makes the worker refuse to start (exit 78), so the dispatcher holds the queue instead of each message failing on the same fault. A message that makes the cryptography crash (a panic while parsing it) is failed at once: it is the message’s doing, and it would only crash again.

85.1.7.1.9. Mail for the user’s own key

A user may register their own key, whose private half lives only in their mail client (an MUA key, crypto_identity.custody = client; see pepsi-stage-encrypt(1) and pepsi-keys(1)). Mail encrypted to such a key is for the mail client, and the stage recognises it before it tries to open anything: it reads the recipient identifiers of the ciphertext (OpenPGP PKESK key IDs, S/MIME RecipientIdentifiers) without decrypting and matches them against the recipients’ client keys that are not revoked. A wildcard (“hidden recipient”) key ID matches nothing and takes the ordinary path.

On a match the message is passed on unopened, even when one of our own MTA keys is among its recipients too: the user registered their own key to get end-to-end protection, and opening a copy on the server and storing the plaintext in the mailbox would silently undo that. The stage records state.crypto.in.decryption = for-client, for_client = true and client_fingerprint, sets state.encrypted, and adds for-client=yes to X-Pepsi-Crypto. This is not a decryption failure, so ON_DECRYPT_FAILURE is not consulted, and the message advances to NEXT_STAGE like any other: the spam checks and the rest of the pipeline still decide its fate. A message with several local recipients is first split one per row, since one recipient’s client key says nothing about another’s copy.

The consequence is that server-side features go blind for such mail. Content-reading stages (language detection, for example) see ciphertext and fall back as they do for any undecryptable message, and nothing was verified, so state.signature_verified is never set and a whitelist entry with signature_required does not match it.

85.1.7.1.10. Anti-forgery

Every X-Pepsi-* header field is removed from every message this stage sees — including a message it is not going to touch, and including one for which the stage is disabled. Only then is our own added.

X-Pepsi-Crypto is a private header, so it is forgeable by definition; the unconditional removal is the entire basis on which a client may believe it. The Pepsi-Origin proof-of-origin header is deliberately not in the namespace and survives.

The same argument applies to the Subject. With STRIP_SUBJECT_TAGS = yes (the default) this stage’s own tag vocabulary — TAG_DECRYPTED, TAG_VERIFIED, TAG_BAD_SIGNATURE, the built-in [decrypted] and [verified] whatever the tags are configured as, and anything in STRIP_SUBJECT_KEYWORDS — is removed from the front of an inbound subject before ours is prepended. Otherwise an external sender simply writes [verified] themselves. Tags are stripped from the front only, and from behind a Re:/Fwd: prefix, so a subject that mentions one mid-sentence keeps it.

85.1.7.1.11. Subject tags

With SUBJECT_TAGS = yes (the default) the earned tags are prepended to the Subject in nesting order:

encrypted(signed(body))   ->  [decrypted][verified] quarterly figures
signed(encrypted(body))   ->  [verified][decrypted] quarterly figures

Both shapes open. The second line states the ordering rule; note that the second example earns its [verified] only from an inner signature, because an outer signature over ciphertext never reaches the valid verdict the tag is written for (A signature that covers ciphertext). A signed(encrypted(body)) message with no inner signature is therefore tagged [decrypted] alone.

For most users this is the only signal they will ever see — nobody reading mail in a webmail client inspects headers — so doing the work invisibly would waste it. The cost is that a user-visible field is mutated: it affects search, threading and the quoted subject of replies. Set SUBJECT_TAGS = no to rely on the header alone.

[verified] is written only for the valid verdict. TAG_BAD_SIGNATURE is empty by default: a bad signature is a routine property of legitimate mail, so a default-on tag would decorate a large part of a user’s inbox with an accusation.

85.1.7.1.12. The result header

With ADD_RESULT_HEADER = yes (the default) a message that carried any protection gains one header field, prepended at the top so it disturbs no existing signature:

X-Pepsi-Crypto: encrypted=yes; decrypted=yes; protocol=openpgp;
  signature=valid; signer=alice@example.net;
  fingerprint=0123456789ABCDEF; key-source=wkd; trust=wkd-advanced;
  layers="encrypted,signed"

The grammar is semicolon-separated key=value pairs in a fixed order. A value that is not a bare token (A–Z, a–z, 0–9 and . _ @ : + / -) is double-quoted; an embedded " becomes ', and control characters are replaced by spaces, so no value from the message can end the field or start another one. Long values are folded at whitespace.

The keys are:

encrypted

yes/no — the message arrived carrying ciphertext.

decrypted

yes/no — present only when encrypted=yes.

for-client

yes when the message is encrypted to the recipient’s own MUA key and was passed through unopened (Mail for the user’s own key).

protocol

openpgp or smime.

signature

One of the verdicts above.

signer, fingerprint, key-source, trust, algorithm

The signing key, where it came from and what it was worth. Present only when known.

failure

The machine-readable reason a signature verdict is not good.

decrypt-failure

The machine-readable reason decryption failed.

layers

The protection layers, outermost first, comma separated.

RFC 8601 registers no smime or pgp method, so an Authentication-Results extension would mean inventing a method name in a registry we do not own — which other implementations are entitled to ignore or mistrust. A private header with a documented grammar is the honest choice, and the price of it is the unconditional stripping described above.

85.1.7.1.13. Key discovery

The stage performs no network I/O of any kind. When the signer’s key is neither in the message nor in the peer_key cache, it enqueues a discovery request and parks the message; a pepsi-keydisc(1) service releases it when the request settles, found or not, and the stage runs again. A settled request with no key releases the message just the same, and it then verifies as unverifiable. A fresh negative-cache entry short-circuits the wait entirely, which is what stops a spammer signing with an unknown key from making every message wait: the first one parks, the rest do not.

[pepsi-keydiscovery] INBOUND_TIMEOUT bounds the wait and is separately tunable from the outbound TIMEOUT; DISCOVERY_TIMEOUT in this stage’s own section overrides it. DISCOVERY = no suppresses the request altogether and leaves the stage on the cache-only path.

Whether a parked message is stored decrypted depends on the shape of the message, and the difference matters:

• An S/MIME signature nested inside an envelope survives decryption as an ordinary MIME layer, so the message is parked with the plaintext committed and is decrypted exactly once.

• An OpenPGP signature lives only inside the packet stream, so it is gone the moment the plaintext is out. Committing the plaintext would destroy the evidence the key is being fetched to check, so such a message is parked unchanged and decrypted again on resume.

A message parked with its plaintext committed sits in pepsi.workqueue decrypted for up to the discovery deadline. That is the same exposure delivery would create a moment later anyway; it is a longer window, not a new one.

85.1.7.1.14. Certificates the message carries

Most signed mail carries the certificate that signed it, so before deciding anything the stage reads the S/MIME signer certificates and application/pgp-keys parts out of the message and offers them to the verifier. It does this again on the plaintext once the message is open, since a certificate attached inside the ciphertext is invisible until then, and re-evaluates once if that yielded something the first pass did not have.

These certificates are used and dropped. This stage stores nothing: learning from inbound mail — the Autocrypt: header, attached keys, and gossip — is pepsi-stage-autocrypt-learn(1), which runs later in the pipeline, after the stages that decide whether a message is spam. That split is what lets Autocrypt Level 1 §5.3’s spam rule be honoured at all: this stage runs first on the inbound path by design, before any such verdict exists. It also means nothing is written to the key store by the process that holds the private keys.

Using a certificate the message carried can only ever lower a verdict’s standing, never raise it: such material is untrusted provenance, so a signature checked against it comes back valid-untrusted and does not set state.signature_verified — the key pepsi-stage-check-whitelist(1) gates on. That is the same rung a stored harvested key would have, so routing it through the database would gain nothing.

85.1.7.1.15. Two facts recorded for the learn stage

Decryption commits the plaintext over the bytes that arrived, so two things only this stage can see are written to state.crypto.in for pepsi-stage-autocrypt-learn(1) to read: encrypted, that the message carried ciphertext at all, and outer_gossip, that the arriving header block already had an Autocrypt-Gossip: field. Both are preconditions for believing a message’s gossip, and neither is answerable once the plaintext is in the row.

85.1.7.1.16. Our own users’ keys are never overridden

Harvesting keys on the From: header, which the sender writes, and inbound authentication is deliberately fail-open. A message claiming to come from one of this deployment’s own users will therefore seed a peer_key row for that user with whatever key it carried — and a signature checked against it would otherwise verify, setting state.signature_verified, which pepsi-stage-check-whitelist(1) gates on.

So before the store is consulted, the stage asks whether the claimed sender is somebody it holds an identity for. If it is, that identity’s public key is the only candidate: no peer_key row for the address is offered to the verifier, whatever its rank and however it arrived. Anything but a revoked identity counts, expired ones included — a signature made last year was made with last year’s key, and refusing to look at it would report our own users’ mail as unverifiable every time a key rolled over.

The rule is conditional on holding a key, not on serving the address: a user who runs their own OpenPGP client and never uploaded a key here has no identity, so their harvested or gossiped key is used exactly as any correspondent’s would be. The cost is the mirror image — once an identity is held for an address, a signature made with that user’s separate personal key no longer verifies. See the manual’s Our own users’ keys are not discovered.

85.1.7.1.17. Trust anchors

X.509 anchors come from the pepsi.ca_trust table (pepsi-keys(1) ca) and are cached per worker process for TRUST_ANCHOR_TTL (default five minutes). An anchor added or disabled therefore becomes effective within that window; restart the workers to apply it at once. They are loaded lazily and only for a message that actually carries an S/MIME layer, so ordinary traffic costs no query.

Revocation data is not fetched: a CRL retrieval would be network I/O in a stage, which the discovery design exists to avoid. A chain is consequently reported as revocation not-checked, which is stated rather than being allowed to imply “not revoked”.

85.1.7.1.18. Configuration

The stage reads its own [stage-<name>] section (PROGRAM = pepsi-stage-decrypt, NEXT_STAGE (required), BOUNCE_STAGE, plus ENABLED, LOCAL_DOMAINS, TARGETS, RECIPIENT_DELIMITER, REQUIRE_ACCOUNT, ON_DECRYPT_FAILURE, ON_BAD_SIGNATURE, QUARANTINE_STAGE, SUBJECT_TAGS, TAG_DECRYPTED, TAG_VERIFIED, TAG_BAD_SIGNATURE, STRIP_SUBJECT_TAGS, STRIP_SUBJECT_KEYWORDS, ADD_RESULT_HEADER, MAX_LAYERS, TRUST_ANCHOR_TTL, DISCOVERY, DISCOVERY_TIMEOUT), all documented in pepsi.conf(5). pepsi-setup(1) requires a BOUNCE_STAGE whenever either failure policy is bounce, and a QUARANTINE_STAGE whenever either is quarantine. Those are behavioural options and are per-address overridable through the pepsi.settings table.

The cryptographic policy — algorithm choice, weak-digest acceptance, minimum key sizes — lives in [pepsi]’s CRYPTO_* options and [pepsi-crypto]. Those are system-administrator-only and are deliberately not reachable from pepsi.settings or pepsi-stage-edit-settings(1).

One of them does not apply here. CRYPTO_ALLOW_DOWNGRADE governs what may be emitted: OpenPGP SEIPDv1+MDC and CMS EnvelopedData are always readable whatever it says, because refusing them inbound would make this host unable to receive mail from most of the world while protecting nobody. Only 3DES and the weak signature digests are gated inbound, and each refusal produces its own verdict rather than a generic failure. The 3DES gate is not the same on both sides: an inbound S/MIME 3DES message is accepted only under yes, an inbound OpenPGP one under negotiate as well, because an OpenPGP certificate states its own cipher capabilities and an X.509 one states none. The container that was actually used is recorded per layer, so a deployment can see how much of its inbound mail is still pre-AEAD.

85.1.7.1.19. Privilege

The binary is installed setuid pepsi-crypto, mode 4750 pepsi-crypto:pepsi, and is therefore not folded into the unified pepsi multi-call binary.

Both halves of the key-custody boundary key on that identity. The database role pepsi-crypto is the only one granted crypto_identity.private_wrapped, and PostgreSQL peer authentication keys off the effective uid; the key-encryption key lives in secrets.d/pepsi-crypto.secret, mode 0640 owned by the same account. A setgid binary would gain the group — enough to read the fragment — and would still authenticate to the database as pepsi, precisely the role that is denied the column.

The group in the mode restricts execution to the dispatcher’s account and root; the program also refuses at run time unless its real uid is root or one of the pepsi, pepsi-crypto and pepsi-owner accounts. That run-time check applies only when the binary actually carries the setuid bit — a build installed without it (a developer tree, cargo test) lets anyone run it, on the grounds that such a caller reaches the database as themselves and PostgreSQL’s own grants are then the authority. -c is accepted from those callers (the dispatcher’s documented spawn shape is PROGRAM -c cfg worker) and the environment is sanitised before the configuration is loaded, because the configuration layer is steerable through XDG_CONFIG_HOME, HOME, PATH and the PG* family.

Decryption keys are selected by capability: every non-revoked identity of every local recipient whose purpose is encrypt or both, including expired ones, because old mail must stay readable.

85.1.7.1.20. Resource limits

MAX_LAYERS (default 4) caps how many nested encrypted layers are unwrapped; a message with more is delivered with the remainder unopened and decrypt-failure=unsupported recorded.

An OpenPGP message is routinely compressed and the expansion ratio is the sender’s choice, so pepsi-crypto refuses a plaintext over 64 MiB outright rather than truncating it — a partial plaintext must never be reported as verified. That case is reported as its own failure reason, decrypt-failure=too-large, and not as a generic decryption failure: “this message is a bomb” and “we could not open this” are different facts for an operator.

85.1.7.1.21. State

Inputs: state.dsn (honoured and sliced in lockstep with the locality split), state.crypto.in (on a resumed row, what the pass that decrypted it recorded).

Outputs: state.crypto.in — whether the message was encrypted, what became of the ciphertext (decryption: not-encrypted, decrypted, failed or for-client), which identity opened it, the signature verdict and its detail (including covers_plaintext), the ordered layer list, and, for mail to a user’s own key, for_client and client_fingerprint. Also state.encrypted = true when the message arrived encrypted, and state.signature_verified = true for the valid verdict only; pepsi-stage-check-whitelist(1) and pepsi-stage-auto-whitelist(1) read those. On a bounce, state.bounce. The state layout is described in pepsi.state(7).

Transitions: NEXT_STAGE for a delivered message; QUARANTINE_STAGE or BOUNCE_STAGE per the failure policies; paused while waiting for key discovery; pending at this same stage after a locality split.

85.1.7.1.22. Commands

worker

Run as a persistent dispatcher worker, reading message ids on standard input and writing one status line per message. This is the only mode; there is no one-shot form. To run a single message by hand, pipe its id:

echo 1234 | pepsi-stage-decrypt -c /etc/pepsi/pepsi.conf worker

85.1.7.1.23. Exit Status

0

The worker exited cleanly (standard input closed).

1

An error occurred (misconfiguration, an unreadable key store, or a database error). The reason is written to the log.

78

The worker refused to start: its configuration or key ring does not work (see Errors of this host). The dispatcher requeues what it handed over and retries the stage later.

85.1.7.1.24. See Also

pepsi-stage-encrypt(1), pepsi-stage-reencrypt(1), pepsi-stage-arc(1), pepsi-keys(1), pepsi-keydisc(1), pepsi-stage-check-whitelist(1), pepsi-stage-relay-to-maildir(1), pepsi-stage-relay-to-lmtp(1), pepsi-stage-bounce(1), pepsi-dispatch(1), pepsi-setup(1), pepsi.conf(5), pepsi.state(7)

85.1.7.1.25. Bugs

Report bugs to the Pepsi issue tracker.