95.1.22. pepsi-stage-list-deliver

finalise one member’s copy of a mailing-list post

Manual section:

1

95.1.22.1.1. Name

pepsi-stage-list-deliver - the per-member delivery stage of the Pepsi mailing-list pipeline.

95.1.22.1.2. Synopsis

pepsi-stage-list-deliver [GLOBAL-OPTIONS] worker

95.1.22.1.3. Description

pepsi-stage-list-deliver is a stage program run by pepsi-dispatch(1). pepsi-stage-list-post(1) has already made one row per recipient, and this stage makes each of them that member’s copy rather than another copy of the same message. One row in, one row out.

Three things happen:

  • ``List-Unsubscribe`` is finalised with that member’s one-click https: URI, alongside the mailto: form, and List-Unsubscribe-Post: List-Unsubscribe=One-Click is added (RFC 8058).

  • The message is decorated with the list’s header and footer, in the member’s language, when the list’s personalize is individual or full. A digest issue is never decorated: it carries its own header and footer.

  • ``To:`` is munged to name the member, for full only. Upstream’s personalize_to is a no-op for individual, and so is this.

A copy that avoid-duplicates flagged is stamped X-Pepsi-List-Copy: yes, so that the reader can see why they have two. Such a copy goes to a member who was already named in the message’s own To or Cc and kept their list copy anyway.

Warning

This stage must run before signing. Every header it touches belongs in the DKIM h= list, and List-Unsubscribe is exactly the header that a mailbox provider inspects when it decides whether to show an unsubscribe button. A pipeline that signed first would produce mail that is signed and then modified, which fails DKIM everywhere. So NEXT_STAGE points at pepsi-stage-dkim-sign(1), not at a relay, and pepsi-setup refuses the reverse. Upstream says the same thing in the comment that removes arc-sign from their own pipeline.

95.1.22.1.4. One-click unsubscribe

The token in the https: URI is a keyed hash over the list, the member’s address and a serial that the member can rotate. It is not stored: a List-Unsubscribe URI is emitted once per member per post, so a table of them would grow with deliveries rather than with members. Three things follow:

  • the token never travels between stages, because this stage has the secret and computes it per copy;

  • revoking every URI ever emitted for one subscription is a bump of that member’s unsubscribe_serial, a counter update rather than a table sweep;

  • and there is nothing to expire, because a subscription that no longer exists fails the membership lookup whatever the token says.

The secret is [pepsi-list] UNSUBSCRIBE_SECRET. With no secret, or no BASE_URL to build a URI from, the mailto: form stands alone, as upstream’s does: an unverifiable URI is worse than none. Rotating the secret invalidates every URI already sitting in every subscriber’s mailbox.

95.1.22.1.5. Decoration

The header and footer are upstream’s list:member:regular:header and list:member:regular:footer, resolved like every other list notice: a list_template override for the list, then its domain, then the site, and otherwise the shipped list-member-regular-header.<lang>.body and list-member-regular-footer.<lang>.body from the [pepsi] TEMPLATE_DIR, with the member’s language chosen first and en as the fallback. The shipped header is empty and the shipped footer is upstream’s “To unsubscribe send an email to …” trailer. An override with an empty URI (or one that answers 404) means no header or footer, and a text that renders to nothing but whitespace adds nothing. A worker reuses a fetched override for a minute, so that one post does not fetch it once per member. A shipped template that exists but cannot be read or rendered is an error, and the copy is retried rather than sent without the footer the owner configured.

The footer is appended to the first ``text/plain`` part outside any multipart/signed or multipart/encrypted container, and to nothing else. Appending inside a signed part breaks the signature and appending to a PDF corrupts it, so a message with no such part is delivered undecorated rather than mangled.

Only a copy whose body changed gets a body of its own. The unsubscribe pair, the copy stamp and the To: rewrite are header changes, and a copy that received only those keeps referencing the one body that the fan-out stored for every member.

95.1.22.1.6. Two invariants

Sibling rows start here, never at the pipeline head. Re-entering at init would re-run ARC verification and re-enter the router, which explodes combinatorially on an umbrella list.

A member’s personal settings do not reconfigure list delivery. Pepsi layers per-address pepsi.settings overrides automatically, and a sibling row’s single recipient is a member, whose vacation or language override has no business changing this stage’s targets: a NEXT_STAGE taken from it could route the copy past signing. So NEXT_STAGE, the [pepsi-list] options and the [pepsi] TEMPLATE_DIR are read from the site-wide configuration, which no domain: or address: configuration override and no pepsi.settings row reaches, and the member’s language comes from the descriptor that the fan-out wrote. Nothing is taken from the overlay.

95.1.22.1.7. Configuration

NEXT_STAGE (required)

The signing tail. pepsi-setup refuses a relay here.

Everything else that this stage needs is the member descriptor that the fan-out wrote and the list’s own attributes; [pepsi-list] BASE_URL and UNSUBSCRIBE_SECRET are read from the site section.

95.1.22.1.8. State

Inputs: state.list (the list id, the member’s language, their unsubscribe serial and the duplicate flag) and the row’s single rcpt_to, which is the authority on the address.

Outputs: a rewritten header block and body. No state is changed.

Transitions: advances to NEXT_STAGE, or finishes (deletes) a copy whose list no longer exists.

95.1.22.1.9. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

95.1.22.1.10. Global Options

The usual set: -c/–config, -L/–log, -v/–verbose, -h/–help and -V/–version, which behave as they do for every Pepsi program; see pepsi-config(1).

95.1.22.1.11. Errors and retries

  • A copy whose list was deleted after the fan-out is dropped, with a warning in the log: the owner deleted the list, a copy delivered now would carry an unsubscribe link to nothing, and there is nobody left to report to.

  • A copy that cannot be parsed for decoration is delivered undecorated, with a warning. The fan-out built those bytes from a message that it had parsed, so this is not the sender’s doing and a retry cannot change it; the copy still carries the list’s own List-* headers (the mailto: unsubscribe included), and dropping it would lose the member’s mail.

  • A row with no member descriptor is failed at once.

  • Any other error (the database, a template) is retried with back-off until MAX_LIFETIME (see pepsi-dispatch(1)). A copy that is finally given up on is deleted by pepsi-failure-bouncer(1), not bounced, and so never counts against the member.

95.1.22.1.12. Exit Status

The outcome of each message is reported to pepsi-dispatch(1) on the worker’s status line, not as the exit status.

0

The worker ran until its standard input was closed.

1

A fatal error occurred (unreadable configuration, the database could not be opened, or standard input/output failed). The reason is written to the log.

78

The worker refused to start: a [stage-*] section it serves, [pepsi-list] or [pepsi] does not parse (e.g. a missing NEXT_STAGE). The dispatcher requeues what it handed over and retries the stage later.

95.1.22.1.13. Examples

[stage-list-deliver]
PROGRAM = pepsi-stage-list-deliver
NEXT_STAGE = dkim-sign

95.1.22.1.14. See Also

pepsi-list(1), pepsi-stage-list-post(1), pepsi-stage-list(1), pepsi-stage-dkim-sign(1), pepsi-stage-arc(1), pepsi-dispatch(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

95.1.22.1.15. Bugs

Report bugs to the Pepsi issue tracker.