16. Microsoft Exchange as a gateway

Pepsi can sit in front of and behind a Microsoft Exchange or Exchange Online deployment as a transparent crypto gateway: inbound mail is decrypted and verified before Exchange stores it, outbound mail is signed and encrypted after Exchange has sent it. Users get S/MIME and OpenPGP without touching their clients.

Warning

Nothing in this chapter has been tested against a real Exchange or Exchange Online tenant. Pepsi’s own test suite uses a plain SMTP stand-in and .invalid domains. The connector settings below follow Microsoft’s documented behaviour, but expect to adjust them, and read Troubleshooting before concluding that Pepsi is at fault.

16.1. Topologies

Three arrangements are supported. Which one applies decides what the rest of this chapter asks you to configure.

Both directions (the full gateway). One Pepsi instance is the organisation’s MX and the smarthost Exchange sends through. This is the arrangement that gives end-to-end crypto in both directions.

Internet ──▶ pepsi-ingress :25 ──▶ arc ──▶ decrypt ──▶ autocrypt-learn
             ──▶ aliases ──▶ route ──┬─▶ exchange (smarthost)
                                     └─▶ internet (direct MX)

Exchange ──▶ pepsi-ingress :587 ──▶ encrypt ──▶ srs ──▶ dkim-sign ──▶ internet
             (authenticated)

pepsi-stage-autocrypt-learn follows decrypt because it needs the plaintext: it is where correspondents’ keys are learnt from inbound mail (their Autocrypt: header, attached keys, S/MIME certificates, and the Autocrypt-Gossip: of a message that arrived encrypted), which is what gives the outbound encrypt stage keys to encrypt to without asking the network. It also records the proof that a correspondent holds one of our keys (they sent mail encrypted to it and signed by them); that record is what stops ATTACH_KEYS_AS_FILES attaching our key file to mail for them. Leave it out and both halves are lost.

Front-only. Pepsi is the MX and hands everything to Exchange; Exchange’s own outbound path is left alone. Inbound mail is decrypted and verified; outbound mail is untouched. A real deployment for an organisation that cannot change its outbound flow.

Behind-only. Exchange keeps the MX; Pepsi is only the smarthost Exchange relays through. Outbound mail is signed and encrypted; inbound is untouched. Suits an organisation whose inbound already passes through a compliance or filtering gateway.

The two partial cases are not second-class: the routing stage, the loop prevention and the ADDRESS_FAMILY control all behave the same in each. What differs is only which half of the pipeline you configure.

16.2. The routing decision

A gateway in front of another mail system has one routing decision on every inbound message: does this recipient belong to the system behind me, or to the internet?

Getting it wrong is not a delivery failure. The domains behind the gateway are exactly the domains whose public MX is the gateway, so handing one to a direct-to-MX relay posts the message back into Pepsi’s own ingress, which accepts it, relays it out again, and repeats.

pepsi-stage-route exists for this decision:

[stage-route]
PROGRAM = pepsi-stage-route
# Recipients at a domain we serve go to Exchange...
MANAGED_STAGE = exchange
# ...everything else out to the public internet.
NEXT_STAGE = internet

MANAGED_DOMAINS defaults to [pepsi-ingress] ACCEPTED_DOMAINS, so the set of domains treated as “behind the gateway” cannot drift from the set ingress accepts. Add ROUTES only for exceptions — a legacy subdomain delivered locally, a partner domain with its own next hop.

Tell Pepsi how to find out whether an address exists in Exchange. Then pepsi-ingress can refuse an unknown one while the sender is still connected, instead of accepting it and having Exchange bounce it later — to an envelope sender that spam forges (see A remote sender). RECIPIENT_CHECK = probe asks Exchange with RCPT TO and no message. That only helps where Exchange itself refuses unknown recipients at RCPT: on-premises with recipient filtering, or Exchange Online with Directory-Based Edge Blocking. Otherwise export the valid addresses to a file and use RECIPIENT_CHECK = map:

[stage-route]
PROGRAM = pepsi-stage-route
MANAGED_STAGE = exchange
NEXT_STAGE = internet
RECIPIENT_CHECK = probe
PROBE_HOST = example-com.mail.protection.outlook.com
# or: RECIPIENT_CHECK = map
#     RECIPIENT_MAP = /etc/pepsi/exchange-recipients

Without either, every address at the managed domains is accepted, as before, and pepsi-setup run says so.

In a behind-only deployment there is no routing decision to make: Pepsi only ever relays outward, and the routing stage is not needed.

16.3. Exchange Online: the tenant endpoint

Important

Point the smarthost at the tenant-specific endpoint, never at your own domain’s public MX.

Your domain’s MX record points at Pepsi (that is the point of the gateway). Configuring Exchange Online as example-com.mail.protection.outlook.com reaches Microsoft; configuring it as mail.example.com reaches Pepsi, and every message loops.

The tenant endpoint is shown in the Microsoft 365 admin centre under the domain’s DNS records, and has the form <tenant>.mail.protection.outlook.com — where <tenant> is normally your domain with dots replaced by hyphens (example.com → example-com). Regional tenants may differ (for instance <tenant>-ch.mail.protection.outlook.com); use whatever the admin centre shows, not a constructed guess.

[pepsi-stage-relay-to-smarthost-mta-exchange]
HOST = example-com.mail.protection.outlook.com
PORT = 25
MODE = starttls
TLS_VERIFY = yes
AUTH = none
DOMAINS = example.com example.net
# Exchange Online rejects a sending IPv6 address with no PTR; see below.
ADDRESS_FAMILY = ipv4

AUTH = none is correct here: Exchange Online authenticates an inbound connector by the sending IP address or by the TLS certificate, not by SASL. Pepsi still uses STARTTLS, which Exchange Online requires.

pepsi-setup refuses a smarthost whose host is this machine and whose port is one an ingress listener is bound to, so the most direct form of this mistake is caught before it can loop.

16.4. Address family selection

Exchange Online rejects mail from a sending IPv6 address whose reverse DNS (PTR) is missing, with a 550 5.7.1 ... S820 error. A dual-stacked host prefers IPv6, so a machine whose IPv4 reverse DNS is correct and whose IPv6 reverse DNS was never delegated delivers fine to everyone except Exchange Online.

The fix is to reach that one destination over IPv4. ADDRESS_FAMILY takes any (the default), ipv4 (also spelled v4/v4only) or ipv6 (v6/v6only).

Precedence is: the ``[…-mta-*]`` entry, then the owning ``[stage-*]`` section, then ``any``. The per-entry level is the one that matters: pinning the whole stage to IPv4 would also force every other destination onto IPv4, including one that publishes only AAAA.

When a delivery is using a family you did not expect, check the MTA entry first, then the stage section, and remember that the configured family is intersected with what the local host can actually route — pinning ipv6 on a host with no global IPv6 route leaves nothing to connect to.

The better fix, where you control the reverse zone, is to publish the PTR record. ADDRESS_FAMILY is for when you do not.

16.5. Loop prevention

A gateway on both sides of another MTA is the classic loop topology, and Pepsi has three independent defences. None of them is optional in this deployment.

1. Configuration refusal (``pepsi-setup``). Two checks, both errors rather than warnings:

• A routing stage from which a served domain can reach pepsi-stage-relay-to-internet. pepsi-setup walks forward from every stage a managed recipient can reach and refuses if a direct-to-MX relay is among them. (BOUNCE_STAGE is not followed — a bounce is a new null-sender message about a delivery that already failed, and it legitimately reaches a relay.)

• A smarthost whose HOST is this host — its [pepsi-ingress] HOSTNAME, localhost, or a loopback address — and whose PORT is one an ingress listener is bound to. Both halves are required: a site may legitimately run an unrelated relay on the same host, or reach a different machine on port 25.

2. The hop count (``MAX_HOP_COUNT``). Both relay stages count Received: header fields and refuse to relay a message with more than MAX_HOP_COUNT (default 30, RFC 5321 §6.3). This is the backstop, not the defence: by the time it fires, thirty copies of the message have crossed the gateway, and the bounce it produces is addressed back into the loop.

3. Exchange’s own connector scoping. Restrict the inbound connector to Pepsi’s IP addresses (below), so a message arriving from anywhere else is not treated as gateway traffic.

16.6. Exchange connector configuration

The exact settings depend on whether Exchange sends to Pepsi, receives from Pepsi, or both.

16.6.1. Inbound to Exchange (Pepsi → Exchange)

In Exchange Online this is an inbound connector (Exchange admin centre → Mail flow → Connectors → From: Your organisation’s email server, To: Office 365):

Setting	Value
Connector type	From your organisation’s email server to Office 365
How to identify the sender	By verifying that the sending server’s IP address matches — list every address Pepsi sends from (see PUBLIC_IP in the relay stage’s own [stage-<name>] section; it is a per-stage option, found by PROGRAM, never by a program-named section), or, preferably, by the certificate subject name
Reject messages if not sent over TLS	Yes

On on-premises Exchange the equivalent is a Receive Connector on the Hub Transport / Frontend role, with RemoteIPRanges set to Pepsi’s addresses and PermissionGroups including AnonymousUsers (with Ms-Exch-SMTP-Accept-Any-Recipient granted if Exchange must accept for domains it is not authoritative for).

Note

If the connector identifies Pepsi by IP, and Pepsi is pinned to IPv4 for this destination (Address family selection), list the IPv4 address. Listing only the IPv6 address of a host that will never use it is a silent authentication failure.

16.6.2. Outbound from Exchange (Exchange → Pepsi)

In Exchange Online this is an outbound connector (From: Office 365, To: Partner organisation), with routing set to Route email through these smart hosts naming Pepsi’s hostname, and Always use Transport Layer Security on with certificate validation against Pepsi’s name.

On the Pepsi side, Exchange arrives at a submission listener:

[pepsi-ingress-listener-submission]
SERVE = tcp
BIND_TO = 0.0.0.0
PORT = 587
MODE = starttls
SUBMISSION = yes
SASL_TYPE = dovecot
SASL_PATH = /run/dovecot/auth-client

16.6.3. Authenticating the connector

There are two ways to let Exchange relay through Pepsi, and they are not equivalent.

SASL with a dedicated account (recommended). Exchange authenticates with a username and password. Because a submission listener enforces RFC 6409 §6 — a sender may only use an address the authenticated identity is entitled to — the connector’s account needs a USERNAME_MAP entry permitting the served domains, or every message Exchange relays is refused:

# /etc/pepsi/username.map — the connector account may send as any of our users.
exchange-connector:   *@example.com *@example.net

Without that entry the account may send only as itself, and the first message Exchange relays gets a 550.

Network trust (``MYNETWORKS``). Simpler — list Exchange’s addresses and skip authentication — but it carries the self-relay risk this chapter is about: any host that can reach the listener from a trusted address can relay through it, and a mistake in the range turns Pepsi into an open relay. Prefer SASL; if you use MYNETWORKS, list individual addresses rather than ranges, and never include loopback.

16.7. The Outlook add-in

Pepsi’s encryption stage acts on two headers a message can carry: X-Pepsi-Sign and X-Pepsi-Encrypt. The add-in is what lets a user set them from Outlook.

Enable the routes and reload pepsi-httpd:

[pepsi-httpd]
ADDIN = yes
# Only needed when the request's own Host header does not name this gateway
# (for example behind a proxy that rewrites it).
# ADDIN_URL = https://mail.example.com

The manifest is then served from https://<gateway>/addin/manifest.xml, with the gateway URL and a per-deployment identifier substituted for you.

Deployment is by sideloading. In the Microsoft 365 admin centre, go to Settings → Integrated apps → Upload custom apps, choose Provide link to manifest file, and give it that URL. There is no AppSource listing: the manifest’s content depends on your gateway’s URL, so there is no single artifact to publish, and a store review sitting between you and a fix is a poor trade for one page of JavaScript.

Users without the add-in get the same effect from a subject keyword — [sign], [encrypt] or [secure] anywhere in the Subject. The encryption stage acts on it and removes it before the message is sent, so the recipient never sees it. The keywords are configurable (SUBJECT_KEYWORDS_SIGN, SUBJECT_KEYWORDS_ENCRYPT, SUBJECT_KEYWORDS_BOTH).

Note

A subject keyword means required, not try: a message a user deliberately marked [encrypt] is not sent in the clear if a recipient has no key. What happens instead is ON_NO_KEY’s business.

16.8. The X-Pepsi-* header contract

Three fields in one namespace, plus the rule that makes the result field believable.

Field	Meaning
X-Pepsi-Sign	Request: sign this message. no/yes/required.
X-Pepsi-Encrypt	Request: encrypt this message. no/yes/required.
X-Pepsi-Crypto	Result: what the inbound stage found. Semicolon-separated key=value.

Requests are honoured only for locally submitted mail (state.local_origin) and are stripped from outbound mail unconditionally, whatever STRIP_REQUEST_HEADERS says: a control channel that survived onto the wire would be one the next hop could be told to obey.

The result header can be believed because the inbound stage removes the entire ``X-Pepsi-*`` namespace from every message it sees — including one it is disabled for, one addressed to a domain it does not serve, and one it is about to split — before adding its own. It follows that a message that did not traverse a decrypt stage carries no trustworthy result header, and nothing should present one as authoritative.

Pepsi-Origin is deliberately not in this namespace and is never stripped: the payment-correlation path keys on it.

The result header’s signature value is one of none, valid, valid-untrusted, invalid or unverifiable.

Danger

valid-untrusted means the signature verified against a key that is not anchored in any trust path — anyone can produce one. It must never be rendered as verified in a user interface, a task pane or a mail client rule.

16.9. Identity auto-provisioning

[pepsi-crypto] AUTO_CREATE_IDENTITY (on by default) lets Pepsi generate a key for a local sender who has none. When is the encrypt stage’s ENABLE_PEP:

• ENABLE_PEP = yes (the default, the pEp-style preset): on the sender’s first submitted message, whatever it asked for. Behind Exchange this covers everybody who sends through the connector, because mail the connector relays is locally submitted however the connector authenticates (SASL, MYNETWORKS or a client certificate) and the preset applies on every submission route – for every sender whose domain Pepsi also receives mail for.

• ENABLE_PEP = no: the first time that sender explicitly asks for protection – with the add-in, or with a subject keyword. It does not fire on ordinary mail.

Either way an address that already has any key – including a user’s own key registered from their mail client, or a revoked one – is never given another automatically.

This is how an organisation gets end-to-end coverage without per-user work: a key appears, and it is published so correspondents can use it.

It is also how an organisation publishes the address of every person who sends mail (or, without the preset, who ever asked for a signed message). An auto-created identity is marked published, and a published identity is served from this host’s own Web Key Directory — which is exactly what makes it usable, and exactly what makes the address discoverable by anyone who guesses it. It is not uploaded to a public key server unless its owner asks (vks_wanted); [pepsi-keys] VKS_PUBLISH only decides the default for keys made by hand.

Both halves are wanted for a general-purpose deployment and neither is obviously right for an organisation whose address list is itself sensitive, or one bound by a policy on disclosing staff addresses. The dials, from most to least automatic:

• leave everything at its default – every sender gets a key on their first message, discoverable via this host’s WKD, so withdrawal means unpublishing one identity rather than asking a key server to forget;

• set [stage-encrypt] ENABLE_PEP = no – keys appear only for senders who asked for protection;

• turn AUTO_CREATE_IDENTITY off – key provisioning becomes an explicit act (an operator with pepsi-keys, or the user through the web console or the pepsi-keys@ control address), and a user who asks for encryption without a key gets whatever ON_NO_KEY says instead.

16.10. Key distribution

A correspondent can only encrypt to a key they have, so the sender’s public key travels with their mail. How:

• Autocrypt (the default). Every locally submitted message carries an Autocrypt: header with the sender’s minimal OpenPGP key ([stage-encrypt] AUTOCRYPT, default yes). Thunderbird, K-9/FairEmail, Delta Chat and other Autocrypt-aware clients learn it silently. It rides on cleartext mail too, which is the point: until a correspondent has the key there is nothing to encrypt with.

• An attached key file (optional). With ATTACH_KEYS_AS_FILES = yes the key is also attached as OpenPGP_0x<long key ID>.asc (application/pgp-keys), the classic OpenPGP way, for correspondents whose client does not read Autocrypt and imports keys from attachments. On encrypted mail the file is inside the ciphertext; on cleartext the message becomes multipart/mixed. It is attached only until the correspondent has shown they hold the key – they sent back a message encrypted to it and signed by them, which pepsi-stage-autocrypt-learn records on the inbound path – so regular correspondents stop receiving it, and a new key starts it again. Turn AUTOCRYPT off as well if you want the file only.

• The Web Key Directory, served by pepsi-httpd for every published identity, for clients that look keys up rather than learn them from mail.

When a user has registered their own key from their mail client, that key is their public face: Pepsi then adds no Autocrypt: header and no key file (the client advertises its own), and the Web Key Directory serves only that key. See Key management.

16.11. Troubleshooting

Messages loop between Pepsi and Exchange.

The smarthost is pointed at your public MX rather than the tenant endpoint, or a served domain is routed to the direct-to-MX stage. Re-run pepsi-setup run, which is where configuration validation lives: both are refused there, so a configuration that still loops means the running configuration is not the one you validated. (pepsi-setup check is a different command — it compares the live DNS against the records run would publish, and validates no pipeline.) Look for the loop in pepsi-queue — a message with a large Received: count is the tell.

Exchange Online rejects with 550 5.7.1 … S820.

The sending IPv6 address has no PTR. Publish one, or set ADDRESS_FAMILY = ipv4 on that MTA entry (Address family selection).

Exchange Online rejects with 550 5.7.64 TenantAttribution.

The inbound connector does not recognise the sender. Check that the connector lists the address Pepsi actually sends from — the IPv4 one if ADDRESS_FAMILY pins it.

Everything Exchange relays gets 550 5.7.1 Envelope sender not permitted.

RFC 6409 §6: the connector’s SASL account may only send as itself. Give it a USERNAME_MAP entry covering the served domains.

A delivery is using the wrong address family.

Precedence is the […-mta-*] entry, then the [stage-*] section, then any — and the result is intersected with what the host can route.

X-MS-* headers.

Exchange adds and consumes a number of X-MS-* headers. Pepsi does not read, rewrite or remove them; only the X-Pepsi-* namespace is touched. Be aware, though, that re-encoding a message for a next hop that lacks 8BITMIME rewrites the body and so invalidates any signature over it — see SMTP Protocol Extensions.

16.12. What Pepsi does not do

Journaling and archiving are out of scope. Exchange deployments frequently have a retention requirement, and Pepsi does not answer it: there is no archive store, no retention engine, no search and no legal-hold facility. See docs/ROADMAP.md.

Warning

[pepsi] MAIL_LOG is not an archive, and must not be presented to an auditor as one. It records no message content at any setting — at most the envelope, the outcome and (at full) the Subject: line — and its rows are pruned after [pepsi-admin] MAIL_LOG_RETENTION_DAYS, thirty days by default. It is an operational log with a retention limit, which is the opposite of a retention mechanism.

If you have a retention obligation, satisfy it with Exchange’s own journaling or a dedicated archiving product, and treat MAIL_LOG as telling you what happened last month, not what was said last year.

The narrower question a crypto gateway is actually expected to answer — was this message encrypted, and what did we decide about it — is what MAIL_LOG does answer, when it is enabled. Note that turning it on means the deployment keeps a record of who corresponds with whom; pepsi.conf(5) discusses that trade-off.