7. Supported Features

This chapter catalogues the protocol- and policy-level features Pepsi supports, with the governing RFC for each — the SMTP and authentication machinery of a forwarder, the end-to-end cryptography of a gateway, and the operational surface of both. The RFC Index cross-references every RFC to the code and configuration; the per-program chapters list which program realises each feature.

7.1. Inbound SMTP reception

pepsi-ingress is a standards SMTP server (RFC 5321) with these extensions:

  • Transport security. Cleartext, STARTTLS (RFC 3207) upgrade, and implicit TLS (RFC 8314, e.g. the submissions port 465) — selected per listener via MODE. Submission listeners (RFC 6409, SUBMISSION = yes) may be bound — see Message submission.

  • Message body transfer. Classic DATA and CHUNKING/BDAT (RFC 3030) for large messages. (BODY=BINARYMIME is not advertised.)

  • SIZE advertisement and enforcement (MAX_MESSAGE_SIZE; oversize → 552).

  • PIPELINING and ENHANCEDSTATUSCODES.

  • 8BITMIME (RFC 6152) and SMTPUTF8 (RFC 6531) — see Internationalised and 8-bit content.

  • DSN (RFC 3461) — see Delivery Status Notifications.

  • SMTP AUTH (RFC 4954) for submission — see Client authentication.

  • Acceptance policy: mail is accepted only for ACCEPTED_DOMAINS (others are rejected as relaying with 550); the domainless <Postmaster> mailbox is always accepted (RFC 5321 §4.5.1); recipients that are valid SRS tokens are reverse-decoded and relayed (see Sender Rewriting Scheme (SRS)).

  • Durability: the client is told 250 only after the row is committed; if the database is unavailable the client gets a transient 451 and retries.

7.2. Trace headers

Ingress prepends an RFC 5321 §4.4 Received: header to every message before authentication (so the ARC seal covers it). The with clause records the transmission type — SMTP, ESMTP or ESMTPS — per RFC 3848, and a for clause names the recipient for single-recipient transactions. The loop guard counts Received: headers against MAX_HOP_COUNT.

7.3. Client authentication

Each listener can authenticate the connecting client and mark its mail as locally originated (which lets it relay onward to any domain and stamps ESMTPSA in the Received: trace). A session is authenticated when any of three per-listener mechanisms succeeds:

  • the peer IP is in the listener’s MYNETWORKS (trusted up front);

  • an SMTP AUTH exchange succeeds (RFC 4954, PLAIN/LOGIN), verified against a configured SASL backend (SASL_TYPE/SASL_PATH; Dovecot’s auth-client socket today). AUTH is advertised and accepted only over TLS — a cleartext attempt is refused with 538;

  • a presented TLS client certificate matches a configured key/CA pin (TLS_AUTH_CLIENT).

This is distinct from Boundary authentication below, which verifies the message (SPF/DKIM/DMARC) regardless of how — or whether — the client itself authenticated. See pepsi-ingress.

7.4. Message submission

A listener marked SUBMISSION = yes behaves as a Message Submission Agent (RFC 6409) instead of an MX:

  • authentication is mandatory — an unauthenticated MAIL FROM is refused with 530 5.7.0 (so the flag requires TLS and one of the client-auth mechanisms above); and

  • submission fixups are applied to each accepted message: a missing Date: (§8.2) and Message-ID: (§8.3) are added before the message is stored; and

  • submission-identity enforcement (§6) via USERNAME_MAP: the envelope MAIL FROM and the From: header must be addresses the authenticated user is permitted to use (otherwise 550), and an allowed From: that differs from the user’s canonical identity gains a Sender: header naming it (§8.1).

The flag has no effect on an MX listener.

7.5. Boundary authentication

Before storing a message, ingress authenticates it so the eventual receiver can still see the verdict Pepsi reached at the boundary (forwarding will break the original SPF/DKIM signals):

  • SPF (RFC 7208) on the MAIL FROM identity using the client IP.

  • DKIM signature verification (RFC 6376 / RFC 8463).

  • DMARC evaluation (RFC 7489), with optional SMTP-time enforcement (DMARC_ENFORCE → 550 on a definite failure under a quarantine/reject policy). Authentication is otherwise fail-open: a DNS or parse error is recorded and the mail accepted.

  • iprev / FCrDNS (RFC 8601) — does the client’s PTR forward-confirm to its IP.

  • An Authentication-Results header (RFC 8601) recording all of the above, stamped with the ingress hostname as authserv-id.

7.6. Authenticated Received Chain (ARC)

The pepsi-stage-arc stage implements ARC (RFC 8617). It verifies any inbound ARC chain and seals the message as our ADMD by prepending an ARC-Authentication-Results / ARC-Message-Signature / ARC-Seal set, signed with the single ARC_ALGORITHM (ARC permits one signature per hop). This lets a downstream receiver trust the boundary verdict after forwarding breaks SPF/DKIM alignment. Sealing is fail-open. Because ARC preserves an upstream sender’s authentication across the forwarding hop, it applies only to mail Pepsi receives: locally-originated submissions (state.local_origin) are skipped — they are authenticated as the author domain by the DKIM-signing stage — so the generated pipeline places ARC on the inbound branch, after the state.local_origin split.

7.7. Sender Rewriting Scheme (SRS)

The pepsi-stage-srs stage rewrites the envelope sender into a local address of a Pepsi-controlled SRS_DOMAIN that HMAC-encodes the original sender (the truncated MAC is base32-encoded, RFC 4648), so SPF passes at the next hop. The null sender and an address already in the SRS domain are left unchanged; an already-SRS address is re-signed in the compact SRS1 form. Ingress performs the reverse direction: a bounce returned to a valid SRS address is verified and relayed to the original sender, while a forged or expired token is rejected (550).

7.8. End-to-end encryption and signing

pepsi-stage-encrypt signs a locally submitted message with the From: author’s own key and encrypts it to each recipient, in OpenPGP (PGP/MIME, RFC 3156) or S/MIME (CMS, RFC 8551 / 5083). The signature goes inside the ciphertext.

Every encrypted recipient gets their own ciphertext on their own queue row: there is no recipient-list disclosure, no content key shared between recipients, and Bcc leakage is structurally impossible rather than carefully avoided. Recipients that share an outcome share a row.

Recipient keys come from the Key management store, filled by the discovery layer (WKD, VKS, DANE, LDAP, inbound harvesting). The stage itself performs no network I/O: a recipient whose key is not cached pauses the message until a discovery service settles the request, and one with a fresh negative cache entry takes the no-key path at once.

What happens to a recipient with no key is policy: ordinary mail (opportunistic), a secure link, or a bounce. ENCRYPT = required never degrades to cleartext, and no cleartext outcome is ever silent — each is logged and recorded per recipient in state.crypto.out, together with the content container actually used and whether it was a downgrade.

The stage runs before DKIM signing, so DKIM covers the bytes actually transmitted; see Configuration for why that ordering is not optional.

7.9. Server-side decryption and verification

pepsi-stage-decrypt is the inbound half. For a message addressed to a recipient this host serves, it opens whatever ciphertext the message carries, verifies whatever signature it carries, records the verdict and removes any security indicator the sender forged. The user reads ordinary mail in their usual client and the gateway did the work — which is what makes end-to-end mail cryptography usable without client plugins.

It runs only for recipients this host serves, and for anything else does not even try. Decryption rewrites the body, which invalidates the sender’s DKIM signature; that is harmless for a message about to be filed in a mailbox here and is not for one being relayed onward. A message with both kinds of recipient is split, so the forwarded copy is the one that arrived.

Verification is where the prototype this replaces was weakest, and the verdict vocabulary is deliberately precise. valid means the signature is sound and the key was anchored — an X.509 chain to a configured CA, or an OpenPGP key from a ranked discovery source. valid-untrusted means sound with a key that could not be tied to anything, which is the state of most of the world’s signed mail and is never reported as valid. invalid, unverifiable and none complete the set, and a message with several signatures takes the worst of them. Only valid sets state.signature_verified, which pepsi-stage-check-whitelist gates a whitelist row on.

Both failure paths deliver by default, matching Pepsi’s deliberate fail-open posture on inbound SPF, DKIM and DMARC: a message we could not open is delivered still encrypted (the user may hold the key in their own client), and a bad signature is a recorded verdict rather than a delivery failure (mailing lists that rewrite bodies produce them on entirely legitimate mail). Quarantine and bounce routes exist for deployments that want them.

The result reaches the user two ways: an X-Pepsi-Crypto header with a documented grammar, and — by default — [decrypted][verified] prepended to the Subject in nesting order, which for most users is the only signal they will ever see. Both depend on the same discipline: every X-Pepsi-* field and every one of our own subject tags is removed from inbound mail before ours is added. A private header is forgeable by definition, and that removal is the entire basis on which it can be believed.

The sender’s key is harvested from the message itself — S/MIME signer certificates, application/pgp-keys parts, Autocrypt headers — at the lowest trust rank, before any decision to wait, because most signed mail carries the certificate that signed it. When it does not, the message pauses on a discovery request rather than blocking on a key server, so even the first message from a new correspondent gets a real verdict.

7.10. Key management, discovery and publication

Neither crypto stage performs network I/O of its own. Both draw on a key store in the shared schema, filled by pepsi-keydisc — one service instance per discovery method, so a slow key server delays nobody. A stage that needs a key it has not got commits what it may keep, pauses the message and enqueues a request; the service that answers releases every message parked on that address in the same round-trip.

  • Discovery sources, each a separate concurrently-run lookup with its own rank on a trust ladder: the Web Key Directory (advanced and direct forms), DANE OPENPGPKEY (RFC 7929) and SMIMEA (RFC 8162) — accepted only when the resolver’s AD bit says the answer was DNSSEC-validated — the VKS protocol of a verifying key server, LDAP, and material harvested from inbound mail (S/MIME signer certificates, application/pgp-keys parts and Autocrypt headers) at the bottom of the ladder.

  • Publication of our own users’ keys: the WKD endpoints served by pepsi-httpd, OPENPGPKEY/SMIMEA records printed by pepsi-keys and pepsi-setup, and — at the operator’s explicit choice — upload to a verifying key server, whose confirmation mail pepsi-stage-vks-confirm answers.

  • Custody. Private material is stored wrapped under a key-encryption key that lives outside the database, and is reachable only by the one database role the crypto stages run as.

  • A negative cache means only the first message to or from an unknown correspondent ever waits.

Key management is the chapter on all of this; the option reference is pepsi.conf(5).

7.12. Outbound DKIM signing

The pepsi-stage-dkim-sign stage prepends DKIM signatures (RFC 6376) — both an RSA-2048 and an Ed25519 (RFC 8463) signature — under the SIGNING_DOMAIN or the message’s From: domain. The optional l= body-length tag (COVER_BODY = no) tolerates a downstream footer. Signing is fail-open. Because header selection is bottom-up, the signature does not disturb existing signatures.

7.13. Delivery Status Notifications

Pepsi implements DSN end to end (RFC 3461 / RFC 3463 / RFC 3464):

  • Ingress advertises DSN and validates/stores RET/ENVID (on MAIL FROM) and NOTIFY/ORCPT (on RCPT TO) under state.dsn.

  • The relay stages propagate those parameters to a next hop that also advertises DSN, and omit them otherwise (RFC 3461 §6 — Pepsi does not become the DSN-responsible relay).

  • pepsi-stage-bounce emits the report as an RFC 3464 multipart/report with RFC 3463 enhanced status codes:

    • a failure report when NOTIFY requests FAILURE (the default when absent) — NOTIFY=NEVER drops silently;

    • a success report (Action: delivered) only when the global ORIGINATE_SUCCESS_DSN is set and NOTIFY=SUCCESS was requested;

    • a delay report (Action: delayed) when a relay stage’s DELAY_DSN_AFTER elapses on a still-queued message that asked for NOTIFY=DELAY (sent at most once).

A null-sender message (a bounce) is never itself bounced (RFC 5321 §6.1).

7.14. Internationalised and 8-bit content

Ingress advertises 8BITMIME (RFC 6152) and SMTPUTF8 (RFC 6531) and records the BODY= declaration. Because a next hop’s capabilities are unknown until after connection, the outbound SMTP client decides per hop:

  • If the hop supports the extension, Pepsi re-advertises BODY=8BITMIME / SMTPUTF8.

  • Otherwise it downgrades: 8-bit MIME leaf parts are re-encoded to a 7-bit transfer-encoding (RFC 2045 — quoted-printable for text/*, base64 otherwise); UTF-8 header fields are rewritten as RFC 2047 encoded-words.

  • A non-ASCII address (envelope, or inside a header address) that cannot be represented to a non-SMTPUTF8 hop is a permanent failure (bounce).

Body re-encoding necessarily breaks a body-covering DKIM/ARC signature; this is unavoidable (capabilities are late-bound) and rare.

7.15. Outbound relay

Two interchangeable relay stages send mail off-site:

  • pepsi-stage-relay-to-internet — direct-to-MX delivery (RFC 5321 §5): its own MX lookup with preference ordering and Happy-Eyeballs address selection (cached per address in pepsi.dns_address), implicit MX via address records (§5.1), and Null MX handling (RFC 7505). TLS is authenticated by MTA-STS (RFC 8461) with certificate identity checks (RFC 6125); enforce policies require STARTTLS to a listed MX.

  • pepsi-stage-relay-to-smarthost — relay through a configured smarthost, chosen by recipient domain (or a catch-all), with per-MTA transport (plain/tls/starttls), certificate verification and the full SMTP AUTH (RFC 4954) suite — PLAIN/LOGIN, CRAM-MD5, SCRAM-SHA-1/-SHA-256 (with optional -PLUS channel binding), OAUTHBEARER/XOAUTH2, SASL EXTERNAL (TLS client certificate), NTLM, GSSAPI/Kerberos and the deprecated DIGEST-MD5 — or auto negotiation. See Client authentication and the RFC index.

Both stages authenticate the next hop’s TLS with DANE (RFC 7672, DANE = off|warn|strict) — looking up the hop’s TLSA records and matching them against the presented chain, taking precedence over MTA-STS — and record every outbound TLS session for TLS Reporting (RFC 8460); pepsi-tlsrpt ships the daily aggregate reports. They share retry semantics: a transient failure pauses the message with exponential backoff (RETRY_INITIAL/ RETRY_MAX_INTERVAL/RETRY_FACTOR) until MAX_LIFETIME, after which it is bounced or failed; a permanent failure routes to BOUNCE_STAGE (or marks the row failed).

7.16. Local delivery

For recipients in a local domain, three stages file the mail on the host instead of relaying it (each forwards the recipients it cannot handle to its NEXT_STAGE, so they compose):

  • pepsi-stage-relay-to-maildir — writes the message directly into each local user’s Maildir/new/. Locality is decided by LOCAL_DOMAINS and TARGETS (passwd uid ranges); the privileged write is done by the setuid-root pepsi-helper-maildir-writer, reached through the stage’s own pepsi-maildir set-group-id bit.

  • pepsi-stage-relay-to-lmtp — hands the message to a local Mail Delivery Agent (typically Dovecot) over LMTP (RFC 2033), which runs each recipient’s Sieve (RFC 5228) filter as it files the mail. It delivers all recipients in one transaction and routes each one the MDA rejects onward by its RFC 3463 status. Needs no privileged helper (the MDA drops privilege).

  • pepsi-stage-dot-forward — processes each local user’s ~/.forward file (the classic sendmail/Postfix mechanism) via the setuid-root pepsi-helper-dot-forward, which drops to the user before reading it: forwarded addresses restart the pipeline, |pipe//file directives run as the user, and a recipient with no ~/.forward passes through unchanged.

pepsi-stage-aliases complements these by expanding envelope recipients through a Postfix virtual(5)-style map (full-address or @domain catch-all, expanded transitively) before local delivery.

pepsi-stage-vacation answers mail that arrives while a recipient is away, and tags the forwarded copy’s subject so they can see on their return which mail was answered for them. It is the one user-filtering action Pepsi implements itself rather than leaving to an MDA’s Sieve, because the reply is a new message that has to be signed and relayed — and because deciding not to send one is pipeline knowledge: RFC 3834 says never answer a bounce, mailing-list mail, anything already marked automatic, or a service address, and Pepsi adds its own “never answer spam” and a per-correspondent rate limit. Leave dates are ordinary per-address configuration, so the same option is a national holiday at global scope and one person’s holiday in their own pepsi.settings row.

pepsi-stage-route complements them differently: instead of delivering, it chooses a next hop per recipient from that recipient’s domain, splitting the message when its recipients disagree. It is what lets Pepsi front an existing mail system — the domains behind the gateway go to that system, the rest go to the internet — and it is where pepsi-setup proves no domain this host serves can be routed back into its own ingress. See Microsoft Exchange as a gateway.

7.17. Operational features

  • Single-table pipeline with crash recovery: orphaned running rows are reset on dispatcher start-up; in-flight children are reset on shutdown.

  • Elastic, pipelined worker pools and watchdog: each stage runs persistent worker processes started on demand up to its PARALLELISM and reaped after WORKER_IDLE_TIMEOUT, recycled after MAX_MESSAGES; each worker is fed up to QUEUE_LIMIT messages at once (in-flight capacity QUEUE_LIMIT × PARALLELISM, at no extra connection cost), and a worker exceeding MAX_RUNTIME on its head-of-line message is killed (timeout) and replaced.

  • Queue tooling: pepsi-queue lists, deletes, re-stages and bulk-unsticks messages.

  • Health monitoring: pepsi-status prints a read-only summary of the queue, stuck messages, cumulative delivery/failure counters and recent outbound TLS outcomes (also as --json), suitable for running over SSH.

  • Provisioning and DNS verification: pepsi-setup installs the schema, generates keys and prints/validates DNS (DKIM, SPF, MTA-STS).

  • HTTP server: pepsi-httpd serves, over one or more TLS (SNI-selected) or plaintext listeners, the MTA-STS policy file (/.well-known/mta-sts.txt), a Prometheus /metrics page, the Web Key Directory endpoints that publish this deployment’s own identities (/.well-known/openpgpkey/…, direct and advanced forms), the The secure-link fallback portal portal, and the mail autoconfiguration document below.

  • Mail client autoconfiguration (draft-ietf-mailmaint-autoconfig): a client given nothing but fred@example.org fetches https://autoconfig.example.org/mail/config-v1.1.xml and configures itself — submission host, port, transport security and authentication, and the same for the mailbox server. The submission half is derived from the running submission listener, so it cannot drift from the server it describes; the mailbox half names whatever MDA the deployment pairs with. Served publicly and without authentication, as the draft requires, because a client must read it before it can know how to authenticate.

  • Administrative API and console: on a listener explicitly flagged ADMIN = yes — and never on one that would carry it in cleartext off the host — the same server exposes the The administrative API under /api/v1 (queue, health, configuration, key store, audit and mail logs) and the browser The administration console under /ui, which is a client of exactly that API. Everywhere else those paths answer the same 404 an unknown path gets. Authentication is by UNIX socket peer credentials, a session cookie or a bearer token, and every principal carries scopes; all of it is audited.

  • Metrics: pepsi-dispatch records per-stage throughput, kills/timeouts, crashes and processing time and the global stage/message totals (flushed to the database in one transaction roughly once a minute), exported by pepsi-httpd alongside live active/paused gauges read from the queue.

  • Structured logging via tracing at configurable levels.

For features still on the roadmap (BINARYMIME), see Applicable RFCs not yet implemented.

7.18. Feature stability

Every feature described in this chapter is tracked in a single Feature stability table, which records — per feature — its governing RFC, whether it has an automated test (and of what kind: U unit, I integration, I/U both), whether it has been verified by hand, and how widely it is deployed and exercised according to anonymous, opt-in usage telemetry (pepsi-telemetry).

That table is generated: contrib/update-feature-stability.sh merges the hand-maintained registry contrib/feature-registry.tsv (name, RFC, test and manual-test columns) with a pepsi-telemetry GET /telemetry/report (the deployment and usage counts). See Extending the Pipeline for how to keep it current when you add a feature, write a test, or refresh the telemetry counts.