95.1.17. pepsi-stage-anti-spam

gate a message behind a GNU Taler payment

Manual section:

1

95.1.17.1.1. Name

pepsi-stage-anti-spam - the pay-to-send anti-spam stage of the Pepsi pipeline.

95.1.17.1.2. Synopsis

pepsi-stage-anti-spam [GLOBAL-OPTIONS] worker

95.1.17.1.3. Description

pepsi-stage-anti-spam is a stage program run by pepsi-dispatch(1). It loads a pepsi.workqueue row (refusing to act unless its status is running), reads its [stage-<stage>] section, and decides what to do from the message’s state JSON, requiring a GNU Taler payment for an otherwise-unclassified message before it may continue down the pipeline.

The decision is:

  • the message has the null envelope sender (a bounce or RFC 3834 automatic reply) — it is never held for payment and is never itself bounced. When proof-of-origin is enabled (a [pepsi-origin] section, see PROOF OF ORIGIN below; it is on by default) the bounce is verified to be one we provoked and a verified bounce is forwarded to NEXT_STAGE. A bounce that we cannot verify is routed to BOUNCE_TARGET_STAGE when one is wired, otherwise forwarded to NEXT_STAGE unchanged, past the gate. That is a bounce with no valid Pepsi-Origin proof, or every bounce when proof-of-origin is not configured. pepsi-setup(1) --wizard wires BOUNCE_TARGET_STAGE = discard-rejected, a pepsi-stage-discard(1) sink, so a forged bounce is dropped rather than delivered; without the option a forged bounce reaches the mailbox (the stage itself never drops one).

  • state.spam = true — the message is dropped (the row is deleted).

  • the message is one this deployment injected itself — it is forwarded to NEXT_STAGE without an order. Such a message is either a reply composed in the secure-link portal (state.secure_link.reply, set by pepsi-httpd(1), which injects it straight into the inbound path), or a message whose own header block carries a Pepsi-Origin stamp minted for injected mail (see PROOF OF ORIGIN below). The stamp must verify, must name every envelope recipient of the row, and its nonce must still be tracked. The link and PIN notifications of pepsi-stage-secure-link(1) carry such a stamp when they come back in through this deployment’s MX.

  • state.spam = false or state.paid = true — the message is forwarded to NEXT_STAGE. An explicit classification (set by some other component) overrides the payment flow.

  • state.paid = false is present — an order was created earlier and the message has been woken again (by the pepsi-resume payment webhook calling pepsi-httpd’s /resume, or by pepsi-dispatch(1) when the deadline lapsed). The stage asks the merchant backend whether the order is now paid:

    • paid — forward to NEXT_STAGE;

    • unpaid and past the stored deadline — reject the message: reroute to the stage’s BOUNCE_STAGE (wire it to pepsi-stage-bounce(1) for a rejection DSN, or pepsi-stage-discard(1) to drop it silently). Each recipient goes there on a row of its own, so that a DSN reports on every one of them and honours each recipient’s NOTIFY. When no BOUNCE_STAGE is configured the row is simply deleted;

    • unpaid but still before the deadline — the message is re-paused until the deadline and a warning is logged. This should not happen in normal operation (a paused message only wakes on payment or at the deadline). It stays paused, not pending: a row that the dispatcher could claim again at once would busy-loop against the early resume that woke it.

  • no paid field (first encounter) — the stage creates a v1 merchant order priced by ORDER_CHOICES (order_id is the message’s external token, so the pepsi-resume webhook’s {{order_id}} resolves back to this message). It then pauses the message, recording paid: false and the absolute deadline in state. The pause lasts until the DELAY_DSN_AFTER checkpoint when one is configured and falls before the deadline, otherwise until PAYMENT_DEADLINE. A message whose sender failed SPF or DMARC outright (state.auth.spf or state.auth.dmarc is fail) is the exception: it is never sent the payment request (see below), so nobody could pay for it, and holding it would only fill the merchant’s order table and the queue. It gets no order and is rejected as unpaid at once, exactly as at the deadline (BOUNCE_STAGE, else deleted).

The deadline is stored in state.pay_deadline (epoch seconds) because pepsi-dispatch(1) clears the row’s timeout column when it requeues a paused message, so it could not otherwise be recovered on the re-run.

After pausing the message on that first encounter, the stage also originates a payment-request auto-reply to the sender (when BLOCK_RESPONSE_STAGE is configured). The reply explains that the recipient requires unknown senders to pay before their mail is accepted. It carries the taler://pay/… URI, both as a link and as a QR code (an inline image/png part referenced by cid:), together with the acceptable payment options and a pointer to https://wallet.taler.net/ for senders without a wallet. The options are the ORDER_CHOICES, omitting any choice that involves a token family. A sender that passed SPF or DMARC is addressed by the display name from the original From: header when one is available, and its subject is quoted; any other sender gets the request without either, since its envelope sender may be forged and quoting would let a forger have this host deliver text of their choosing to a third party. The human-readable part is a text/html body rendered from the payment-request.<lang>.body Mustache template under [pepsi] TEMPLATE_DIR. The language(s) are taken from state.language as set by pepsi-stage-detect-language (run earlier in the pipeline): the reply renders one chunk per detected language that has a template, concatenated in the detected order, each with its payment options localized to that language. When the message carries no detected language, or none of the detected languages has a template, the PAYMENT_MESSAGE_DEFAULT_LANGUAGE template is used instead. The reply uses the null sender <> (so it is never itself bounced or gated) and is injected at BLOCK_RESPONSE_STAGE (normally a signing/relay chain), so it is DKIM-signed and delivered like any other outbound mail.

A message whose content came out of, or failed to open from, an S/MIME EnvelopedData (AES-CBC, no integrity protection) counts as having no detected language: the language depends on whether a modified CBC ciphertext opened, and a request written in it would tell the sender whether it did.

The reply goes to an envelope sender that nobody has verified, so it is withheld in two cases; the message is held for payment all the same. First, when RFC 3834 says the message must not be answered automatically (Auto-Submitted other than no, Precedence: bulk|list|junk, X-Auto-Response-Suppress, any List-* field, a service sender, a multipart/report), or when REQUIRE_AUTHENTICATED is on and the sender did not pass SPF or DMARC. (A sender that failed authentication outright, which makes the envelope sender most likely forged, gets no request and no order at all, as above; softfail, neutral, none and temporary errors are answered.) Second, when the same sender was already sent a payment request for the same protected mailbox (the first envelope recipient) within BLOCK_RESPONSE_SUPPRESS (default one hour): otherwise a flood of messages bearing one forged sender would become the same flood of requests at that address. The window is claimed and recorded in one statement (the payment_request_should_send function over pepsi.payment_request_reply), after the message is paused and the reply rendered, so two workers holding two messages from one sender send one request, and a failure before that point does not use the window up. A sender whose second message falls inside the window gets no prompt for it; that message can still be paid (its order exists) and otherwise bounces at the deadline like any unpaid one. 0 s answers every message.

The auto-reply also carries two machine-readable headers so that a sending Pepsi can settle the demand automatically (see pepsi-stage-auto-pay(1)): a Taler: header holding the taler://pay/… URI, and the original message’s Pepsi-Origin: header copied verbatim (when the incoming message carried one). Echoing the proof-of-origin lets the originating site verify that the returning demand is for mail it actually sent, and meter what it spends per original message: all demands for one original (for example, a mailing-list fan-out) share one Pepsi-Origin nonce. The header is copied as-is, not re-signed: only the originator can verify it (its HMAC is keyed with the originator’s secret).

The reply is rendered before the message is paused. If no reply can be produced, notably when the PAYMENT_MESSAGE_DEFAULT_LANGUAGE template is missing, the message is held for payment without one, with a warning in the log; nothing is claimed in the window, so a later message from the same sender is answered once the template is back. pepsi-setup validates that this template exists up front whenever BLOCK_RESPONSE_STAGE is set. Injecting the already-rendered reply after the pause is best-effort: a database error there is logged, the window claimed for it is given back, and the message stays paused for payment.

95.1.17.1.3.1. Merchant backend unavailable

A merchant backend that cannot be reached, or answers an order or status request with an error, is the host’s problem and not the sender’s. The message stays held (paused, the reason in state.last_error) and the merchant is asked again every five minutes, never past PAYMENT_DEADLINE; the deadline is fixed at the first attempt, so an outage does not lengthen it. If the merchant still cannot be asked once the deadline has passed, the stage fails open: the message is delivered to NEXT_STAGE unverified, with a warning in the log. A payment gate that cannot tell whether it was paid must not turn a merchant outage into lost mail. A status request for an order the merchant does not know (404) is an answer, not an outage: the order is treated as unpaid.

The stage requires the shared [pepsi-payments] section (the merchant backend and access token; see pepsi.conf(5)) and, for the payment webhook to release paused messages, the [pepsi-httpd] RESUME_AUTHORIZATION_TOKEN (see pepsi-httpd(1)).

95.1.17.1.3.2. Mail the gate holds by mistake

Caution

Only the null envelope sender and mail that this deployment injected itself bypass the gate. Everything else is held unless state.spam = false clears it, including mail that nobody can or will pay for. That flag is normally set by pepsi-stage-check-whitelist(1), placed before this stage.

  • Mailing-list postings. A posting carries its author’s From:, so a correspondent whitelist does not cover it, and its List-* fields suppress the payment request, so nobody is even asked: every posting to a gated subscriber is held and then rejected at the deadline.

Vacation notices (pepsi-stage-vacation(1)) and this stage’s own payment requests use the null sender and are never held for payment; with BOUNCE_TARGET_STAGE wired they are subject to the proof-of-origin check below like any other null-sender message. The secure-link portal’s link and PIN notifications, which carry the original sender as envelope sender, are exempt as mail this deployment injected, provided proof of origin is on ([pepsi-origin] ENABLED, the default); with it switched off they are held like any other mail and have to be whitelisted. Read receipts use the null sender, and portal replies are recognised by their state.

The workaround for a list is to whitelist its identifier in a group that the check stage’s WHITELIST_NAME names, naming the ARC sealer that must vouch for it:

pepsi-whitelist add-list --sealer lists.example.org \
    correspondents users.lists.example.org

See pepsi-whitelist(1).

95.1.17.1.4. Configuration

Options live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-anti-spam): the required NEXT_STAGE, BOUNCE_STAGE, BOUNCE_TARGET_STAGE (see PROOF OF ORIGIN below), the order parameters (the required ORDER_CHOICES, plus PAYMENT_DEADLINE, DELAY_DSN_AFTER, SUMMARY, FULFILLMENT_MESSAGE) and the auto-reply parameters (BLOCK_RESPONSE_STAGE, BLOCK_RESPONSE_FROM, BLOCK_RESPONSE_SUBJECT, BLOCK_RESPONSE_SUPPRESS, REQUIRE_AUTHENTICATED, PAYMENT_MESSAGE_DEFAULT_LANGUAGE). The stage also requires the shared [pepsi-payments] section; the payment webhook that releases a paid message before its deadline needs the [pepsi-httpd] RESUME_AUTHORIZATION_TOKEN. All are documented in pepsi.conf(5). The payment request is marked as Pepsi’s own (state.notice, kind payment-request), which a NOTICE_SIGNER stage at BLOCK_RESPONSE_STAGE signs (see pepsi-stage-encrypt(1)), except while a per-address override applies to the gated message.

95.1.17.1.5. Proof of origin

Forwarding every null-sender bounce unconditionally is safe only because a bounce is never itself bounced. But it lets an attacker inject backscatter (forged bounces) that Pepsi then relays back to a forged “original sender”. To reject such forgeries, the shared [pepsi-origin] section (pepsi.conf(5)) is enabled by default; pepsi-setup(1) generates a random secret on first run if none is configured. Both relay stages (pepsi-stage-relay-to-internet(1) and pepsi-stage-relay-to-smarthost(1)) then stamp every outbound message with a Pepsi-Origin header and record the nonce in pepsi.origin_nonce for two weeks. The header holds an HMAC over the originating sender account, a fresh 128-bit nonce, a timestamp and our HOSTNAME.

A genuine bounce embeds the original message (at least its headers, and so the Pepsi-Origin header) in its DSN body. This stage scans the whole returning bounce for that header and trusts the bounce only if both checks pass, in this order:

  1. the header’s HMAC verifies against our secret and its host matches our HOSTNAME (a fast, purely cryptographic check); and

  2. only then — the nonce is still present (and unexpired) in pepsi.origin_nonce.

A bounce that we cannot verify is routed to BOUNCE_TARGET_STAGE when one is wired, otherwise forwarded to NEXT_STAGE unchanged. A bounce cannot be verified when it carries no such header, fails the HMAC, or has a nonce that is no longer tracked. When proof of origin is switched off ([pepsi-origin] ENABLED = no), every bounce is unverifiable, since none can then be proven ours. The stage itself never drops a bounce; what happens to an unverifiable one is the target stage’s choice.

Without BOUNCE_TARGET_STAGE that choice is delivery: a null-sender message is never held for payment, so a spammer who sends with MAIL FROM:<> and no proof is past the gate. pepsi-setup(1) --wizard, which always writes [pepsi-origin], therefore also writes BOUNCE_TARGET_STAGE = discard-rejected into the section it generates for this stage and emits that [stage-discard-rejected] (PROGRAM = pepsi-stage-discard, the sink that milter rejects share): an unverifiable bounce is dropped, and the verdict stays in the log. For a section written by hand, pepsi-setup(1) warns when the option is missing, and when it is set while proof of origin is switched off, which sends every bounce there. The cost is accepted knowingly. Standard non-delivery reports from major MTAs (Postfix, Exim, Gmail, Exchange) return the original headers and so verify. A real tail of legitimate bounces, however, cannot be verified and is lost silently: spam-rejection notices that quote little of the original, header-stripping gateways, minimal/legacy bouncers, bounces arriving after the two-week nonce window, and bounces for mail sent before the origin secret existed. A site that wants to see them points BOUNCE_TARGET_STAGE at a quarantine mailbox or a review stage instead; that is a one-word change in the generated section.

Some mail that this deployment composes on its own behalf has a human envelope sender: the secure-link link and PIN notifications. When such mail is injected, it is stamped with a second form of the header, one that also binds the message’s envelope recipients (for=) under the HMAC. Only that form, found in the message’s own header block (not in a body or an attached message) with every envelope recipient of the row named and its nonce tracked, exempts a message from the payment gate. An ordinary outbound stamp never does: every recipient of our mail holds one, and could otherwise paste it into a message to another of our users. The recipient binding limits what a recipient of an injected message can do with its stamp to mail addressed to themselves.

95.1.17.1.6. Files

<TEMPLATE_DIR>/payment-request.<lang>.body

The Mustache template for the payment-request reply body (TEMPLATE_DIR is the [pepsi] option, default ${DATADIR}/templates, i.e. /usr/share/pepsi/templates for a --prefix=/usr installation). The template for PAYMENT_MESSAGE_DEFAULT_LANGUAGE (default payment-request.en.body) is required; add the languages that pepsi-stage-detect-language may detect alongside it. The template is given the variables recipient_name / has_name, protected_mailbox, original_subject (empty for a message that arrived encrypted, for the reason given in pepsi-stage-bounce(1), and, with recipient_name, for a sender that did not pass SPF or DMARC), pay_uri / pay_link, qr_cid, wallet_url, has_options and the payment_options list (each amount plus has_description and description, localized to that template’s language).

95.1.17.1.7. State

Inputs: state.spam / state.paid (booleans, optional) classify the message; state.secure_link.reply (boolean) marks a reply composed in the secure-link portal, which passes the gate; state.pay_deadline (epoch seconds) is the deadline this stage wrote; state.dsn carries the recipient’s notify/orcpt (echoed into a rejection).

Outputs: on the first encounter the stage merges {"paid": false, "pay_deadline": <epoch>} into state and pauses. At the DELAY_DSN_AFTER checkpoint it adds state.delay_sent = true once a delay DSN has been sent (so it fires at most once) and re-pauses. While the merchant backend is unavailable it records the reason in state.last_error. On rejection it reroutes to BOUNCE_STAGE with a state.bounce object (kind = permanent) for pepsi-stage-bounce(1). The state layout is described in pepsi.state(7).

Transitions (driven by state.spam/state.paid, PAYMENT_DEADLINE and the merchant order status):

  • whitelisted (state.spam = false) or already paid (state.paid = true) → advance to NEXT_STAGE;

  • explicit state.spam = true → finish (the row is deleted);

  • first encounter, sender failed SPF or DMARC (fail) → no order; reroute to BOUNCE_STAGE (or finish, if no BOUNCE_STAGE is configured), as for an unpaid message at its deadline;

  • first encounter → create a Taler order, inject the payment request at BLOCK_RESPONSE_STAGE (unless suppressed, see above), and pause (until the DELAY_DSN_AFTER checkpoint when one is configured and falls before it, else until PAYMENT_DEADLINE);

  • woken (resume webhook or deadline) and now paid → advance to NEXT_STAGE;

  • woken, still unpaid, deadline passed → reroute to BOUNCE_STAGE (or finish, if no BOUNCE_STAGE is configured);

  • merchant unavailable (see above) → re-pause before the deadline, advance to NEXT_STAGE after it;

  • woken, still unpaid, deadline not yet reached → with DELAY_DSN_AFTER configured this is the delay checkpoint: send the sender a one-shot delayed DSN (RFC 3461, only if they requested NOTIFY=DELAY) without releasing the message, then re-pause until PAYMENT_DEADLINE; without DELAY_DSN_AFTER an early wake-up is anomalous and the message is re-paused (paused) until the deadline.

95.1.17.1.8. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

95.1.17.1.9. Global Options

These global options precede the subcommand.

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default [pepsi] LOG, else info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

95.1.17.1.10. Exit Status

The outcome of each message is reported to pepsi-dispatch(1) on the worker’s status line, not as the exit status. A message that is not running, or a per-address setting that breaks the stage’s section, fails that message; a fault of the host (the database, a template or helper that cannot be used) is not a failure: the message is paused and retried, as Stage errors in pepsi-dispatch(1) describes.

0

The worker ran until its standard input was closed.

1

A fatal error occurred (unreadable configuration, the database could not be opened, or standard input/output failed). The reason is written to the log.

78

The worker refused to start: a [stage-*] section it serves, or the [pepsi-payments] backend it needs, does not parse or is missing. The dispatcher requeues what it handed over and retries the stage later.

95.1.17.1.11. Examples

Process message 42 (it must be running):

echo 42 | pepsi-stage-anti-spam -c /etc/pepsi/pepsi.conf worker

A stage that charges one KUDOS and rejects unpaid mail after two days:

[stage-anti-spam]
PROGRAM = pepsi-stage-anti-spam
NEXT_STAGE = srs
BOUNCE_STAGE = bounce
BLOCK_RESPONSE_STAGE = dkim-sign
PAYMENT_DEADLINE = 48 h
ORDER_CHOICES = [{"amount":"KUDOS:1"}]

95.1.17.1.12. See Also

pepsi-config(1), pepsi-stage-bounce(1), pepsi-stage-discard(1), pepsi-httpd(1), pepsi-dispatch(1), pepsi-setup(1), pepsi-stage-auto-pay(1), pepsi.conf(5), pepsi.state(7)

The manual’s Threat model chapter (the remote-sender and availability sections) states what the gate and its payment request are meant to withstand; its Security model chapter (Unknown recipients and backscatter, Resource limits) describes the mechanisms.

95.1.17.1.13. Bugs

Report bugs to the Pepsi issue tracker.