85.1.11. pepsi-stage-relay-to-smarthost

relay a message through an upstream smarthost

Manual section:

1

85.1.11.1.1. Name

pepsi-stage-relay-to-smarthost - the smarthost delivery stage of the Pepsi pipeline.

85.1.11.1.2. Synopsis

pepsi-stage-relay-to-smarthost [GLOBAL-OPTIONS] worker

85.1.11.1.3. Description

pepsi-stage-relay-to-smarthost is a stage program run by pepsi-dispatch(1) as a persistent worker reading message ids on standard input. It loads that pepsi.workqueue row (refusing to act unless its status is running), reads its [stage-<stage>] section, and relays the message to a configured upstream MTA (smarthost). As in pepsi-stage-relay-to-internet(1), a row holding several recipients is first split into one row per recipient, and a message carrying more than MAX_HOP_COUNT Received: header fields fails permanently as a mail loop.

The recipient’s domain selects the smarthost: the [pepsi-stage-relay-to-smarthost-mta-<name>] section whose DOMAINS lists it, or the catch-all MTA (CATCH_ALL = yes). A domain may be listed by only one MTA, and at most one MTA may be the catch-all. These MTA sections are shared across all egress stages (see pepsi.conf(5)). The message is sent with its own envelope sender, using the MTA’s configured transport (MODE = plain/tls/starttls, default starttls), certificate verification and authentication.

On success the row is removed (or, if the stage sets NEXT_STAGE, advanced). A transient failure pauses the message with an exponential backoff (re-queued by pepsi-dispatch(1) when its timeout elapses); past MAX_LIFETIME the failure is treated as permanent. A permanent failure — including a recipient with no matching MTA — is routed to the stage’s BOUNCE_STAGE (or, with none, the row is marked failed). A permanent failure of a bounce (null sender) is discarded rather than bounced again. pepsi-setup(1) refuses a stage with no [pepsi-stage-relay-to-smarthost-mta-*] section at all (it would bounce every message), and warns when no MTA sets CATCH_ALL, naming the domains that are routed.

Unlike pepsi-stage-relay-to-internet(1), pepsi-stage-relay-to-smarthost does no MX discovery and never uses DNS-learned MTA-STS: it speaks only to the smarthosts named in the configuration.

DSN (RFC 3461): when the smarthost advertises the DSN extension, the parameters the origin requested (RET/ENVID on MAIL FROM and the recipient’s NOTIFY/ORCPT on RCPT TO, read from the row’s state.dsn) are propagated to it; when it does not advertise DSN they are omitted. On a permanent failure the recipient’s NOTIFY/ORCPT and the ENVID are handed to the bounce stage so a failure DSN is generated only when the sender asked for one. This stage preserves state.dsn.

When the global [pepsi] ORIGINATE_SUCCESS_DSN is enabled and a successful delivery’s recipient requested NOTIFY=SUCCESS, the message is routed (after delivery) to BOUNCE_STAGE, which emits a positive (Action: delivered) report to the sender. With the flag off, or without a SUCCESS request, a successful delivery is simply finished.

When DELAY_DSN_AFTER is set and a still-undelivered message has been queued at least that long, a one-shot “delayed” (Action: delayed) DSN is sent to the sender — but only if the sender requested NOTIFY=DELAY (delay has no implicit default). The original message keeps being retried; the warning is a separate message routed through BOUNCE_STAGE, and is sent at most once.

Content downgrading (RFC 6152 / RFC 6531): the message is matched to the extensions the smarthost actually advertises. When the body is 8-bit and the smarthost supports 8BITMIME the envelope carries BODY=8BITMIME; when it does not, each 8-bit MIME leaf part is re-encoded with a 7-bit content-transfer-encoding (quoted-printable for text/*, base64 otherwise) — on the evidence of the octets, not of the declared encoding — and the result is re-tested, a body still carrying 8-bit octets being a permanent failure (554) rather than a message sent in violation of RFC 6152 §3. When the message needs SMTPUTF8 and the smarthost supports it the envelope carries SMTPUTF8; when it does not, UTF-8 header fields are rewritten as RFC 2047 encoded-words wherever §5 permits one, non-ASCII Content-Type and Content-Disposition parameters of any MIME part take the RFC 2231 form (filename*=UTF-8''…), a non-ASCII boundary= is replaced by a fresh ASCII one throughout its multipart, and a message with a non-ASCII address is bounced. Re-encoding a body breaks any body-covering signature already on the message (the originator’s DKIM body hash and the ARC-Message-Signature); see pepsi-stage-relay-to-internet(1).

Proof of origin: when the shared [pepsi-origin] section is configured (on by default), this stage stamps every outbound message with a Pepsi-Origin header and records its nonce, exactly as pepsi-stage-relay-to-internet(1) does (the smarthost passes the header through to the eventual MX), so a bounce returning for smarthost-relayed mail can be verified by pepsi-stage-anti-spam(1). See that stage and pepsi.conf(5) for details. Without the section no header is added. As there, a nonce that cannot be recorded holds the message back for a retry.

Configuration read by a delivery — this section, [pepsi-tlsrpt], [pepsi-origin] and [pepsi] — is checked when a worker starts; a section that does not parse keeps the workers from starting (the queue is held, and the journal says why) instead of failing each message. Nothing is read from the configuration after the smarthost has accepted a message, so an accepted message is never retried over a configuration error.

SMTP TLS Reporting (RFC 8460): when [pepsi-tlsrpt] SEND_REPORTS is set, each TLS session to the smarthost is recorded (best-effort) as an aggregate counter in pepsi.tls_session — a success, or a classified TLS failure — keyed by the applied policy (tlsa when DANE authenticated the smarthost, else no-policy-found) and the smarthost host. pepsi-tlsrpt(1) compiles these into daily reports; recording is skipped for a report message itself (state.tlsrpt).

85.1.11.1.4. Configuration

Pipeline wiring and the operational options (SERVER_NAME, the connection timeouts, the retry-backoff schedule and MAX_LIFETIME, DELAY_DSN_AFTER, MAX_HOP_COUNT, ADDRESS_FAMILY and the DANE DNS settings) live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-relay-to-smarthost); the upstream smarthosts are defined once in the shared [pepsi-stage-relay-to-smarthost-mta-<name>] sections (HOST and PORT, both required, MODE, TLS_VERIFY/TLS_CA/DANE, the AUTH credentials, HELO_NAME, its own ADDRESS_FAMILY — which defaults to the stage’s — and DOMAINS/CATCH_ALL, of which at least one is required). All are documented in pepsi.conf(5).

85.1.11.1.5. Authentication

When relaying through a smarthost, Pepsi proves its own identity to that upstream MTA via SMTP AUTH (RFC 4954), configured per smarthost with the [pepsi-stage-relay-to-smarthost-mta-<name>] AUTH option. The supported mechanisms are:

  • none (default) — no client authentication.

  • plain — SASL PLAIN (RFC 4616): the USERNAME/PASSWORD are sent base64-encoded.

  • login — SASL LOGIN: the legacy two-step user/password exchange used by some older smarthosts.

  • cram-md5 — SASL CRAM-MD5 (RFC 2195): a challenge/response in which Pepsi proves knowledge of the PASSWORD via HMAC-MD5 over the server’s challenge, so the password itself is never sent. Weaker than SCRAM (no salting, no mutual authentication, MD5) — prefer scram-* where available.

  • digest-md5 — SASL DIGEST-MD5 (RFC 2831): a salted challenge/response that also authenticates the server to Pepsi via rspauth (a mismatch aborts the exchange). Deprecated — RFC 6331 moved DIGEST-MD5 to Historic status; it is provided only for legacy smarthosts that offer nothing better, and Pepsi emits a warning (both from pepsi-setup(1) at configuration time and in the relay logs the first time it is used). It is never selected by auto. Prefer scram-* or cram-md5.

  • scram-sha-1 / scram-sha-256 — SASL SCRAM-SHA-1 / SCRAM-SHA-256 (RFC 5802 / RFC 7677): a salted challenge/response that, unlike cram-md5, also authenticates the server to Pepsi — the exchange aborts if the server’s signature does not verify against the PASSWORD, or if the server accepts the login without returning its server-final signature at all (so a peer cannot dodge mutual authentication by replying 235 early). SCRAM-SHA-256 is preferred. USERNAME/PASSWORD are normalised with SASLprep (RFC 4013).

  • scram-sha-1-plus / scram-sha-256-plus — the channel-binding -PLUS variants of the above, binding the SASL exchange to the underlying TLS channel with tls-server-end-point (RFC 5929; a hash of the server certificate). This detects a man-in-the-middle that terminates TLS and re-originates it. Requires an encrypting MODE (there is no channel to bind otherwise), and the smarthost must advertise the -PLUS mechanism.

  • auto — negotiate the strongest password mechanism the smarthost advertises in its EHLO, from the same USERNAME/PASSWORD. Preference order: SCRAM-SHA-256-PLUS > SCRAM-SHA-256 > SCRAM-SHA-1-PLUS > SCRAM-SHA-1 > CRAM-MD5 > LOGIN > PLAIN (the -PLUS variants are only chosen on a TLS session). The recommended setting when you do not need to pin a specific mechanism.

  • external — SASL EXTERNAL (RFC 4422 App. A): the smarthost authenticates Pepsi by the TLS client certificate it presents during the (mutual-TLS) handshake, so no password is sent. Requires TLS_CLIENT_CERT / TLS_CLIENT_KEY to be configured (see Mutual TLS below). An optional USERNAME is sent as the SASL authorization identity; when omitted the smarthost derives the identity from the certificate.

  • oauth — SASL OAuth 2.0 bearer token, required by large providers such as Gmail and Microsoft 365. The concrete mechanism is chosen automatically from the smarthost’s EHLO, preferring the standardised OAUTHBEARER (RFC 7628) and falling back to the de-facto XOAUTH2. USERNAME is the account address sent in the SASL exchange and TOKEN_FILE is the path to a file holding the current OAuth 2.0 access token (leading/trailing whitespace is trimmed). The file is re-read on every delivery, so an external refresher can replace an expired token without restarting Pepsi.

    Access tokens are short-lived (typically about an hour). The relay stage only reads the token; keeping TOKEN_FILE current is an external job. Pepsi ships pepsi-helper-token-refresh(1) for this — a service that obtains fresh tokens from the provider’s token endpoint and rewrites the file — but it is optional and not part of pepsi.target; a site may use any equivalent (a cron job, a cloud agent). A delivery attempted while the token is missing, empty, expired or otherwise rejected is treated as a transient failure, so the message stays queued and is retried once a fresh token is in place rather than being bounced.

    For the relay worker (run as pepsi) to read a token written by the refresher, the stage binary is installed SGID the pepsi-token group (mode 2550, owner pepsi:pepsi-token, so only the dispatcher’s account and root can run it) and the token files are group-readable to it (mode 0640 in the SGID /var/pepsi/tokens directory). See the Extending the Pipeline manual chapter for the full refresh contract and how to substitute your own refresher.

  • ntlm — the de-facto Microsoft NTLM mechanism (no RFC), for older Exchange / Office 365 hybrid deployments. Only NTLMv2 is implemented (NTLMv1 is cryptographically broken and never sent). Because Pepsi only ever runs NTLM over TLS, the response always carries Extended Protection for Authentication (EPA): the MsvAvChannelBindings (tls-server-end-point, RFC 5929) and MsvAvTargetName (the smtp/<host> SPN) AV pairs plus the message-integrity code, so it authenticates against EPA-enforcing servers. The required NT_DOMAIN is the Windows domain and the optional NT_WORKSTATION is a cosmetic client name; USERNAME/PASSWORD are the account credentials. NTLM is weak (it uses only the NT hash) and does not authenticate the server to Pepsi — prefer gssapi, scram-* or oauth where the smarthost supports them. pepsi-setup warns when it is configured.

  • gssapi — SASL GSSAPI / Kerberos (RFC 4752), for Kerberized smarthosts (typically an on-premise Exchange in an Active Directory realm). Authentication uses a Kerberos service ticket for SERVICE_NAME (a host-based service name, default smtp@<HOST>) obtained from a Kerberos credential cache; after the GSS token exchange the SASL security layer is negotiated to no security layer (the session is already protected by TLS). No password is configured.

    Pepsi only reads the credential cache and never holds the long-term Kerberos key: an external process — a keytab plus k5start/cron — must keep a ticket-granting ticket current. The cache is the ambient KRB5CCNAME (set on the dispatcher service environment) unless the per-MTA KRB5CCNAME option overrides it. For the relay worker (run as pepsi) to read a cache written by the refresher, place a FILE: cache under the SGID /var/pepsi/krb5 directory (group pepsi-token, which the stage binary already runs SGID for OAuth/mTLS). A delivery attempted while the cache is missing or the ticket has expired is a transient failure (the message stays queued, like oauth). See the Extending the Pipeline manual chapter for the refresh contract.

All the password-based mechanisms (plain, login, cram-md5, digest-md5, the scram-* family, auto and ntlm) as well as oauth and gssapi send or derive a credential (or a security context), so Pepsi only offers them once the session is encrypted (use MODE = tls or starttls); each is refused if the smarthost does not advertise the corresponding AUTH mechanism. The challenge-response mechanisms (cram-md5, digest-md5, scram-*, ntlm) never transmit the password itself, but Pepsi still requires TLS for them. external and gssapi send no password over SMTP — the credential is the TLS client certificate or a Kerberos ticket — but likewise require an encrypted session. For ntlm and gssapi a MODE = plain smarthost is rejected at configuration time.

A smarthost that rejects the credential (535/534/504), like a failed server verification in digest-md5 or scram-*, is a transient failure: the message stays queued and is retried, exactly as for a missing Kerberos ticket. What is wrong in that case is the local configuration — a rotated password, a locked account, a mechanism the smarthost does not implement — so bouncing would return the whole outbound queue to the site’s own users while the operator is still in a position to fix it. Watch the log (or pepsi-status(1)) for repeated AUTH failures; pepsi-setup --wizard checks a credential against the smarthost while it is being entered.

85.1.11.1.6. Mutual TLS

Independently of (or together with) AUTH, a smarthost can be configured to present a client certificate during the TLS handshake (mutual TLS):

  • TLS_CLIENT_CERT — path to the PEM certificate chain to present.

  • TLS_CLIENT_KEY — path to the matching PEM private key.

Both must be set together, and only with an encrypting MODE (tls or starttls). The certificate is presented on every connection and is loaded fresh each time, so a renewed certificate is picked up without restarting Pepsi. It applies to PKIX, TLS_VERIFY = no and DANE sessions alike (the client certificate is orthogonal to how the server certificate is validated).

The certificate alone satisfies many smarthosts (transport-layer mutual TLS, with AUTH = none). Setting AUTH = external additionally issues SMTP AUTH EXTERNAL so the smarthost ties the SMTP session to the presented certificate; it can also be combined with AUTH = plain/oauth if a smarthost wants both a client certificate and a password/token.

The private key is secret material. As with the OAuth token files, the relay worker (run as pepsi) reads it through the SGID pepsi-token group: on a Debian install drop the cert and key into /var/pepsi/tls (created SGID pepsi-token), so the key inherits group pepsi-token and is readable by the stage. Keep the key mode 0640 (or tighter) and never world-readable.

85.1.11.1.7. State

Inputs (read from the row’s state; all optional):

  • state.dsn — the RFC 3461 parameters; propagated to the smarthost when it advertises DSN and consulted to gate failure/success/delay reports.

  • state.origin — the 8BITMIME/SMTPUTF8 hints used by content downgrading.

Outputs (merged into the row’s state):

  • On a transient failure (pause): attempts, last_error and, once a delay DSN has been emitted, delay_sent.

  • On a permanent failure with a BOUNCE_STAGE (reroute), or on a delay warning (an enqueued clone): a state.bounce object (kind = permanent or delay). For an SMTP-level failure it also records the structured next-hop detail — remote_mta (the smarthost), smtp_code, enhanced_status, phase and reply_text — so the bounce can state precisely why the smarthost refused the message.

  • On a successful relay with ORIGINATE_SUCCESS_DSN and NOTIFY=SUCCESS: a state.bounce object with kind = success.

  • On a permanent failure with no BOUNCE_STAGE (fail): last_error.

state.dsn and state.origin are always preserved. The state layout is described in full in pepsi.state(7).

Transitions (the retry schedule is RETRY_INITIAL / RETRY_FACTOR / RETRY_MAX_INTERVAL, bounded by MAX_LIFETIME):

  • successful relay → finish (the row is deleted), or advance to NEXT_STAGE if one is set; with [pepsi] ORIGINATE_SUCCESS_DSN and a recipient that asked for NOTIFY=SUCCESS it instead reroutes to BOUNCE_STAGE to emit a positive (Action: delivered) DSN;

  • transient failure → pause for the next retry interval;

  • still queued past DELAY_DSN_AFTER with a NOTIFY=DELAY recipient → enqueue a one-shot delay-DSN clone at BOUNCE_STAGE (the original stays queued);

  • permanent failure (including a recipient with no matching MTA), or MAX_LIFETIME exhausted → reroute to BOUNCE_STAGE, or fail (terminal failed) if no BOUNCE_STAGE is configured;

  • an undeliverable bounce (null sender) is never re-bounced — it is simply discarded (this stage has no postmaster double-bounce copy).

85.1.11.1.8. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

85.1.11.1.9. Global Options

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

85.1.11.1.10. Exit Status

The outcome of each message is reported to pepsi-dispatch(1) on the worker’s status line, not as the exit status.

0

The worker ran until its standard input was closed.

1

A fatal error occurred (unreadable configuration, the database could not be opened, or standard input/output failed). The reason is written to the log.

85.1.11.1.11. Examples

Run the egress stage on message 42 (it must be running):

echo 42 | pepsi-stage-relay-to-smarthost -c /etc/pepsi/pepsi.conf worker

85.1.11.1.12. See Also

pepsi-config(1), pepsi-stage-bounce(1), pepsi-stage-dkim-sign(1), pepsi-stage-relay-to-internet(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

85.1.11.1.13. Bugs

Report bugs to the Pepsi issue tracker.