95.1.12. pepsi-stage-relay-to-smarthost

relay a message through an upstream smarthost

Manual section:

1

95.1.12.1.1. Name

pepsi-stage-relay-to-smarthost - the smarthost delivery stage of the Pepsi pipeline.

95.1.12.1.2. Synopsis

pepsi-stage-relay-to-smarthost [GLOBAL-OPTIONS] worker

95.1.12.1.3. Description

pepsi-stage-relay-to-smarthost is a stage program run by pepsi-dispatch(1) as a persistent worker reading message ids on standard input. It loads that pepsi.workqueue row (refusing to act unless its status is running), reads its [stage-<stage>] section, and relays the message to a configured upstream MTA (smarthost). As in pepsi-stage-relay-to-internet(1), a row holding several recipients is first split into one row per recipient, and a message carrying more than MAX_HOP_COUNT Received: header fields fails permanently as a mail loop.

The recipient’s domain selects the smarthost: the [pepsi-stage-relay-to-smarthost-mta-<name>] section whose DOMAINS lists it, or the catch-all MTA (CATCH_ALL = yes). A domain may be listed by only one MTA, and at most one MTA may be the catch-all. These MTA sections are shared across all egress stages (see pepsi.conf(5)). The message is sent with its own envelope sender, using the MTA’s configured transport (MODE = plain/tls/starttls, default starttls), certificate verification and authentication.

On success the row is removed (or, if the stage sets NEXT_STAGE, advanced). A transient failure pauses the message with an exponential backoff (re-queued by pepsi-dispatch(1) when its timeout elapses); past MAX_LIFETIME the failure is treated as permanent. A permanent failure — including a recipient with no matching MTA — is routed to the stage’s BOUNCE_STAGE (or, with none, the row is marked failed). A permanent failure of a bounce (null sender) is never bounced again. With POSTMASTER set, the row is the last of the report, so it is removed only once a copy has been delivered there, along the route of that address’s domain. While the copy is deferred the row keeps retrying like any other, and when the copy is refused, or still deferred past MAX_LIFETIME, the row is marked failed for an operator to find. pepsi-failure-bouncer(1) removes such a row after its MIN_AGE, since a bounce is never bounced. Without POSTMASTER the bounce is discarded, and so is a bounce addressed to POSTMASTER itself.

pepsi-setup(1) refuses a stage with no [pepsi-stage-relay-to-smarthost-mta-*] section at all (it would bounce every message), and warns when no MTA sets CATCH_ALL, naming the domains that are routed.

Unlike pepsi-stage-relay-to-internet(1), pepsi-stage-relay-to-smarthost does no MX discovery and never uses DNS-learned MTA-STS: it speaks only to the smarthosts named in the configuration.

DANE (RFC 7672): unless an MTA sets DANE = off (the default is strict) or MODE = plain, the TLSA records at _<PORT>._tcp.<HOST> are looked up through the stage’s DNS_SERVERS (else [pepsi] DNS_SERVERS, else resolv.conf). That resolver must validate DNSSEC, because Pepsi trusts the AD bit of its response. A worker started with DANE on for some MTA therefore logs a warning when the resolver is neither a loopback address nor reached over DNS over TLS or HTTPS (tls://, https://). It logs another when a probe for the signed ietf.org SOA gets no answer or comes back without the AD bit; in the latter case the resolver does not validate, so DANE is never applied. When usable, DNSSEC-validated records exist, a certificate that matches one is accepted without the TLS_VERIFY (PKIX) check. A certificate that matches none is refused under strict, as RFC 7672 section 2.2 requires. Under warn (an explicit opt-out), the mismatch is logged and the certificate is then checked exactly as without DANE, by TLS_VERIFY. So with the default TLS_VERIFY = yes, an impostor still cannot collect the relayed mail or the AUTH credentials; with [pepsi-tlsrpt] SEND_REPORTS on, such a session is recorded as tlsa-invalid. (TLS itself is never optional here: the smarthost MODE is starttls or tls whenever DANE applies.) If the TLSA lookup fails, or its nameservers cannot be determined, the message is deferred under strict; under warn the failure is logged and the hop proceeds with TLS_VERIFY. A smarthost without usable records is reached with the configured PKIX policy in either mode. Where one usage and selector are published under both SHA-256 and SHA-512, only the SHA-512 records are matched (RFC 7671 section 9).

Trust store: with TLS_VERIFY on, the certificate is validated against the system roots, plus the CA certificates of TLS_CA if set; an entry with TLS_CA_BUNDLE is validated against that bundle alone, so no other root of the host can authenticate the smarthost. A bundle that cannot be read or holds no certificate refuses the worker at start.

Revocation (RFC 6960): with TLS_VERIFY on (or TLS_CA/TLS_CA_BUNDLE), an OCSP response that the smarthost staples in the handshake is verified against its certificate’s issuer or an authorised responder. A current revoked answer refuses the session like any other certificate failure; with TLS_VERIFY = no it is only logged. No responder is queried, and a smarthost that staples nothing is not checked. For a certificate that a TLSA record matched, the stapled response is not consulted.

Require TLS (RFC 8689): a message tagged state.tls_required = "required" (the REQUIRETLS MAIL FROM parameter at ingress) is relayed only over a TLS session whose certificate was validated — by DANE, or by PKIX against the system roots: an entry with TLS_VERIFY = no is relayed as if it said yes for such a message, and an entry with MODE = plain cannot take it at all (a permanent failure, 5.7.10, routed to BOUNCE_STAGE). After the upgrade the smarthost must advertise REQUIRETLS, or the message bounces with 5.7.30; the parameter is then passed on, and the smarthost takes over the obligation for the hops beyond it. A bounce of such a message is tagged itself and returns headers only, but is relayed to a smarthost that lacks the extension rather than lost (RFC 8689 §5). A message tagged "no" (TLS-Required: No) is relayed exactly as an untagged one: the entry’s TLS settings are this site’s policy toward its own smarthost, not the recipient domain’s, which is what the header waives.

DSN (RFC 3461): when the smarthost advertises the DSN extension, the parameters that the origin requested are propagated to it: RET/ENVID on MAIL FROM and the recipient’s NOTIFY/ORCPT on RCPT TO, read from the row’s state.dsn. When it does not advertise DSN, they are omitted. On a permanent failure the recipient’s NOTIFY/ORCPT and the ENVID are handed to the bounce stage so a failure DSN is generated only when the sender asked for one. This stage preserves state.dsn.

RRVS (RFC 7293): a recipient that arrived with an RRVS= parameter (kept in state.dsn.rcpt) has it passed on verbatim, no-support action included, when the smarthost advertises RRVS. When it does not, the default action R fails the recipient permanently with 5.7.19 (a DSN, so the sender learns the request could not be enforced), and C continues with the request written as a Require-Recipient-Valid-Since: header field naming that one recipient – each relay transaction carries a single recipient, so the field never discloses one recipient’s date to another.

When the global [pepsi] ORIGINATE_SUCCESS_DSN is enabled and a successful delivery’s recipient requested NOTIFY=SUCCESS, the message is routed (after delivery) to BOUNCE_STAGE, which emits a positive (Action: delivered) report to the sender. With the flag off, or without a SUCCESS request, a successful delivery is simply finished.

When DELAY_DSN_AFTER is set and a still-undelivered message has been queued at least that long, a one-shot “delayed” (Action: delayed) DSN is sent to the sender — but only if the sender requested NOTIFY=DELAY (delay has no implicit default). The original message keeps being retried; the warning is a separate message routed through BOUNCE_STAGE, and is sent at most once.

Content downgrading (RFC 6152 / RFC 6531): the message is matched to the extensions the smarthost actually advertises. When the body is 8-bit and the smarthost supports 8BITMIME the envelope carries BODY=8BITMIME; when it does not, each 8-bit MIME leaf part is re-encoded with a 7-bit content-transfer-encoding (quoted-printable for text/*, base64 otherwise). Which parts are 8-bit is judged from their octets, not from the declared encoding. The result is re-tested: a body that still carries 8-bit octets is a permanent failure (554) rather than a message sent in violation of RFC 6152 §3. When the message needs SMTPUTF8 and the smarthost supports it the envelope carries SMTPUTF8; when it does not, UTF-8 header fields are rewritten as RFC 2047 encoded-words wherever §5 permits one, non-ASCII Content-Type and Content-Disposition parameters of any MIME part take the RFC 2231 form (filename*=UTF-8''…), a non-ASCII boundary= is replaced by a fresh ASCII one throughout its multipart, and a message with a non-ASCII address is bounced. Re-encoding a body breaks any body-covering signature already on the message (the originator’s DKIM body hash and the ARC-Message-Signature; not, in the common case, Pepsi’s own DKIM signature, which pepsi-stage-dkim-sign(1) makes over a body already converted to 7-bit); see pepsi-stage-relay-to-internet(1).

Proof of origin: when the shared [pepsi-origin] section is configured (on by default), this stage stamps every outbound message with a Pepsi-Origin header and records its nonce, exactly as pepsi-stage-relay-to-internet(1) does (the smarthost passes the header through to the eventual MX), so a bounce returning for smarthost-relayed mail can be verified by pepsi-stage-anti-spam(1). See that stage and pepsi.conf(5) for details. Without the section no header is added. As there, a nonce that cannot be recorded holds the message back for a retry.

The configuration that a delivery reads is checked when a worker starts: this section, [pepsi-tlsrpt], [pepsi-origin] and [pepsi]. A section that does not parse keeps the workers from starting (the queue is held, and the journal says why) instead of failing each message. Nothing is read from the configuration after the smarthost has accepted a message, so an accepted message is never retried over a configuration error.

SMTP TLS Reporting (RFC 8460): when [pepsi-tlsrpt] SEND_REPORTS is set, each TLS session to the smarthost is recorded (best-effort) as an aggregate counter in pepsi.tls_session, either as a success or as a classified TLS failure. The counter is keyed by the applied policy (tlsa when DANE authenticated the smarthost, else no-policy-found) and the smarthost host. pepsi-tlsrpt(1) compiles these into daily reports; recording is skipped for a report message itself (state.tlsrpt).

A smarthost that answers EHLO with 500 or 502 does not know the command, so the stage greets it with HELO instead (RFC 5321 section 3.2). Such a session uses no extension and offers no STARTTLS, so an entry with MODE = starttls fails transiently.

95.1.12.1.4. Configuration

Pipeline wiring and the operational options (SERVER_NAME, EHLO_FROM_PTR, the connection timeouts, the retry-backoff schedule and MAX_LIFETIME, DELAY_DSN_AFTER, MAX_HOP_COUNT, POSTMASTER, ADDRESS_FAMILY and the DANE DNS settings) live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-relay-to-smarthost). The upstream smarthosts are defined once in the shared [pepsi-stage-relay-to-smarthost-mta-<name>] sections: HOST and PORT (both required), MODE, TLS_VERIFY/TLS_CA/TLS_CA_BUNDLE/DANE, the AUTH credentials, HELO_NAME, an ADDRESS_FAMILY of their own (which defaults to the stage’s), and DOMAINS/CATCH_ALL, of which at least one is required. All are documented in pepsi.conf(5).

95.1.12.1.5. Authentication

When relaying through a smarthost, Pepsi proves its own identity to that upstream MTA via SMTP AUTH (RFC 4954), configured per smarthost with the [pepsi-stage-relay-to-smarthost-mta-<name>] AUTH option. The supported mechanisms are:

  • none (default) — no client authentication.

  • plain — SASL PLAIN (RFC 4616): the USERNAME/PASSWORD are sent base64-encoded.

  • login — SASL LOGIN: the legacy two-step user/password exchange used by some older smarthosts.

  • cram-md5 — SASL CRAM-MD5 (RFC 2195): a challenge/response in which Pepsi proves knowledge of the PASSWORD via HMAC-MD5 over the server’s challenge, so the password itself is never sent. Weaker than SCRAM (no salting, no mutual authentication, MD5) — prefer scram-* where available.

  • digest-md5 — SASL DIGEST-MD5 (RFC 2831): a salted challenge/response that also authenticates the server to Pepsi via rspauth (a mismatch aborts the exchange). Deprecated — RFC 6331 moved DIGEST-MD5 to Historic status; it is provided only for legacy smarthosts that offer nothing better, and Pepsi emits a warning (both from pepsi-setup(1) at configuration time and in the relay logs the first time it is used). It is never selected by auto. Prefer scram-* or cram-md5.

  • scram-sha-1 / scram-sha-256 — SASL SCRAM-SHA-1 / SCRAM-SHA-256 (RFC 5802 / RFC 7677): a salted challenge/response that, unlike cram-md5, also authenticates the server to Pepsi — the exchange aborts if the server’s signature does not verify against the PASSWORD, or if the server accepts the login without returning its server-final signature at all (so a peer cannot dodge mutual authentication by replying 235 early). SCRAM-SHA-256 is preferred. USERNAME/PASSWORD are normalised with SASLprep (RFC 4013).

  • scram-sha-1-plus / scram-sha-256-plus — the channel-binding -PLUS variants of the above, binding the SASL exchange to the underlying TLS channel with tls-server-end-point (RFC 5929; a hash of the server certificate). This detects a man-in-the-middle that terminates TLS and re-originates it. Requires an encrypting MODE (there is no channel to bind otherwise), and the smarthost must advertise the -PLUS mechanism.

  • auto — negotiate the strongest password mechanism the smarthost advertises in its EHLO, from the same USERNAME/PASSWORD. Preference order: SCRAM-SHA-256-PLUS > SCRAM-SHA-256 > SCRAM-SHA-1-PLUS > SCRAM-SHA-1 > CRAM-MD5 > LOGIN > PLAIN (the -PLUS variants are only chosen on a TLS session). The recommended setting when you do not need to pin a specific mechanism.

  • external — SASL EXTERNAL (RFC 4422 App. A): the smarthost authenticates Pepsi by the TLS client certificate it presents during the (mutual-TLS) handshake, so no password is sent. Requires TLS_CLIENT_CERT / TLS_CLIENT_KEY to be configured (see Mutual TLS below). An optional USERNAME is sent as the SASL authorization identity; when omitted the smarthost derives the identity from the certificate.

  • oauth — SASL OAuth 2.0 bearer token, required by large providers such as Gmail and Microsoft 365. The concrete mechanism is chosen automatically from the smarthost’s EHLO, preferring the standardised OAUTHBEARER (RFC 7628) and falling back to the de-facto XOAUTH2. USERNAME is the account address sent in the SASL exchange and TOKEN_FILE is the path to a file holding the current OAuth 2.0 access token (leading/trailing whitespace is trimmed). The file is re-read on every delivery, so an external refresher can replace an expired token without restarting Pepsi.

    Access tokens are short-lived (typically about an hour). The relay stage only reads the token; keeping TOKEN_FILE current is an external job. For this, Pepsi ships pepsi-helper-token-refresh(1), a service that obtains fresh tokens from the provider’s token endpoint and rewrites the file. It is optional and not part of pepsi.target; a site may use any equivalent (a cron job, a cloud agent). A delivery attempted while the token is missing, empty, expired or otherwise rejected is treated as a transient failure, so the message stays queued and is retried once a fresh token is in place rather than being bounced.

    For the relay worker (run as pepsi) to read a token written by the refresher, the stage binary is installed SGID the pepsi-token group (mode 2550, owner pepsi:pepsi-token, so only the dispatcher’s account and root can run it) and the token files are group-readable to it (mode 0640 in the SGID /var/pepsi/tokens directory). See the Extending the Pipeline manual chapter for the full refresh contract and how to substitute your own refresher.

  • ntlm — the de-facto Microsoft NTLM mechanism (no RFC), for older Exchange / Office 365 hybrid deployments. Only NTLMv2 is implemented (NTLMv1 is cryptographically broken and never sent). Because Pepsi only ever runs NTLM over TLS, the response always carries Extended Protection for Authentication (EPA): the MsvAvChannelBindings (tls-server-end-point, RFC 5929) and MsvAvTargetName (the smtp/<host> SPN) AV pairs plus the message-integrity code, so it authenticates against EPA-enforcing servers. The required NT_DOMAIN is the Windows domain and the optional NT_WORKSTATION is a cosmetic client name; USERNAME/PASSWORD are the account credentials. NTLM is weak (it uses only the NT hash) and does not authenticate the server to Pepsi — prefer gssapi, scram-* or oauth where the smarthost supports them. pepsi-setup warns when it is configured.

  • gssapi — SASL GSSAPI / Kerberos (RFC 4752), for Kerberized smarthosts (typically an on-premise Exchange in an Active Directory realm). Authentication uses a Kerberos service ticket for SERVICE_NAME (a host-based service name, default smtp@<HOST>) obtained from a Kerberos credential cache; after the GSS token exchange the SASL security layer is negotiated to no security layer (the session is already protected by TLS). No password is configured.

    Pepsi only reads the credential cache and never holds the long-term Kerberos key: an external process, such as a keytab plus k5start/cron, must keep a ticket-granting ticket current. The cache is the ambient KRB5CCNAME (set on the dispatcher service environment) unless the per-MTA KRB5CCNAME option overrides it. For the relay worker (run as pepsi) to read a cache written by the refresher, place a FILE: cache under the SGID /var/pepsi/krb5 directory (group pepsi-token, which the stage binary already runs SGID for OAuth/mTLS). A delivery attempted while the cache is missing or the ticket has expired is a transient failure (the message stays queued, like oauth). See the Extending the Pipeline manual chapter for the refresh contract.

The password-based mechanisms (plain, login, cram-md5, digest-md5, the scram-* family, auto and ntlm), oauth and gssapi all send or derive a credential (or a security context). Pepsi therefore offers them only once the session is encrypted (use MODE = tls or starttls), and each is refused if the smarthost does not advertise the corresponding AUTH mechanism. Encryption protects the credential only from a party that is verified to be the smarthost: with TLS_VERIFY = no any certificate is accepted, so whoever can intercept the connection (a man in the middle) completes the handshake with a certificate of their own and receives the plain/login password, the oauth token and every relayed message. TLS_VERIFY = no is therefore not a troubleshooting step for a certificate error; a smarthost with a private CA is configured with TLS_CA_BUNDLE (or TLS_CA) instead. The challenge-response mechanisms (cram-md5, digest-md5, scram-*, ntlm) never transmit the password itself, but Pepsi still requires TLS for them. external and gssapi send no password over SMTP, since the credential is the TLS client certificate or a Kerberos ticket, but they likewise require an encrypted session. For ntlm and gssapi a MODE = plain smarthost is rejected at configuration time.

A smarthost that rejects the credential (535/534/504), like a failed server verification in digest-md5 or scram-*, is a transient failure: the message stays queued and is retried, exactly as for a missing Kerberos ticket. What is wrong in that case is the local configuration: a rotated password, a locked account, or a mechanism that the smarthost does not implement. Bouncing would return the whole outbound queue to the site’s own users while the operator is still in a position to fix it. Watch the log (or pepsi-status(1)) for repeated AUTH failures; pepsi-setup --wizard checks a credential against the smarthost while it is being entered.

95.1.12.1.6. Mutual TLS

Independently of (or together with) AUTH, a smarthost can be configured to present a client certificate during the TLS handshake (mutual TLS):

  • TLS_CLIENT_CERT — path to the PEM certificate chain to present.

  • TLS_CLIENT_KEY — path to the matching PEM private key.

Both must be set together, and only with an encrypting MODE (tls or starttls). The certificate is presented on every connection and is loaded fresh each time, so a renewed certificate is picked up without restarting Pepsi. It applies to PKIX, TLS_VERIFY = no and DANE sessions alike (the client certificate is orthogonal to how the server certificate is validated).

The certificate alone satisfies many smarthosts (transport-layer mutual TLS, with AUTH = none). Setting AUTH = external additionally issues SMTP AUTH EXTERNAL so the smarthost ties the SMTP session to the presented certificate; it can also be combined with AUTH = plain/oauth if a smarthost wants both a client certificate and a password/token.

The private key is secret material. As with the OAuth token files, the relay worker (run as pepsi) reads it through the SGID pepsi-token group: on a Debian install drop the cert and key into /var/pepsi/tls (created SGID pepsi-token), so the key inherits group pepsi-token and is readable by the stage. Keep the key mode 0640 (or tighter) and never world-readable.

95.1.12.1.7. State

Inputs (read from the row’s state; all optional):

  • state.dsn — the RFC 3461 parameters; propagated to the smarthost when it advertises DSN and consulted to gate failure/success/delay reports.

  • state.origin — the 8BITMIME/SMTPUTF8 hints used by content downgrading.

  • state.tls_required — the RFC 8689 tag, see Require TLS above.

Outputs (merged into the row’s state):

  • On a transient failure (pause): attempts, last_error and, once a delay DSN has been emitted, delay_sent.

  • On a permanent failure with a BOUNCE_STAGE (reroute), or on a delay warning (an enqueued clone): a state.bounce object (kind = permanent or delay). For an SMTP-level failure it also records the structured next-hop detail — remote_mta (the smarthost), smtp_code, enhanced_status, phase and reply_text — so the bounce can state precisely why the smarthost refused the message.

  • On a successful relay with ORIGINATE_SUCCESS_DSN and NOTIFY=SUCCESS: a state.bounce object with kind = success.

  • On a permanent failure with no BOUNCE_STAGE (fail): last_error.

state.dsn, state.origin and state.tls_required are always preserved. The state layout is described in full in pepsi.state(7).

Transitions (the retry schedule is RETRY_INITIAL / RETRY_FACTOR / RETRY_MAX_INTERVAL, bounded by MAX_LIFETIME):

  • successful relay → finish (the row is deleted), or advance to NEXT_STAGE if one is set; with [pepsi] ORIGINATE_SUCCESS_DSN and a recipient that asked for NOTIFY=SUCCESS it instead reroutes to BOUNCE_STAGE to emit a positive (Action: delivered) DSN;

  • transient failure → pause for the next retry interval;

  • still queued past DELAY_DSN_AFTER with a NOTIFY=DELAY recipient → enqueue a one-shot delay-DSN clone at BOUNCE_STAGE (the original stays queued);

  • permanent failure (including a recipient with no matching MTA), or MAX_LIFETIME exhausted → reroute to BOUNCE_STAGE, or fail (terminal failed) if no BOUNCE_STAGE is configured;

  • an undeliverable bounce (null sender) is never re-bounced: with POSTMASTER set, the row is finished once a copy is delivered there, paused while the copy is deferred, and failed when the copy is refused or MAX_LIFETIME is exhausted; without it the row is finished.

95.1.12.1.8. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

95.1.12.1.9. Global Options

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default [pepsi] LOG, else info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

95.1.12.1.10. Exit Status

The outcome of each message is reported to pepsi-dispatch(1) on the worker’s status line, not as the exit status.

0

The worker ran until its standard input was closed.

1

A fatal error occurred (the configuration file could not be read, the database could not be opened, or standard input/output failed). The reason is written to the log.

78

The worker refused to start: its [stage-<name>] section, an MTA section, [pepsi], [pepsi-tlsrpt] or [pepsi-origin] does not parse, or the database schema does not match this build. The dispatcher requeues what it handed over and retries the stage later.

95.1.12.1.11. Examples

Run the egress stage on message 42 (it must be running):

echo 42 | pepsi-stage-relay-to-smarthost -c /etc/pepsi/pepsi.conf worker

95.1.12.1.12. See Also

pepsi-config(1), pepsi-stage-bounce(1), pepsi-stage-dkim-sign(1), pepsi-stage-relay-to-internet(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

The transport-security guarantees and their limits (what an active network attacker can still do with and without MTA-STS or DANE) are stated in the Threat model chapter of the Pepsi manual; the privileged-program table and the DNS resolver assumption are in its Security chapter.

95.1.12.1.13. Bugs

Report bugs to the Pepsi issue tracker.