85.1.16. pepsi-stage-anti-spam

gate a message behind a GNU Taler payment

Manual section:

1

85.1.16.1.1. Name

pepsi-stage-anti-spam - the pay-to-send anti-spam stage of the Pepsi pipeline.

85.1.16.1.2. Synopsis

pepsi-stage-anti-spam [GLOBAL-OPTIONS] worker

85.1.16.1.3. Description

pepsi-stage-anti-spam is a stage program run by pepsi-dispatch(1). It loads a pepsi.workqueue row (refusing to act unless its status is running), reads its [stage-<stage>] section, and decides what to do from the message’s state JSON, requiring a GNU Taler payment for an otherwise-unclassified message before it may continue down the pipeline.

The decision is:

  • the message has the null envelope sender (a bounce or RFC 3834 automatic reply) — it is never held for payment and is never itself bounced. When proof-of-origin is enabled (a [pepsi-origin] section, see PROOF OF ORIGIN below; it is on by default) the bounce is verified to be one we provoked and a verified bounce is forwarded to NEXT_STAGE. Any bounce we cannot verify — one with no valid Pepsi-Origin proof, or every bounce when proof-of-origin is not configured — is routed to BOUNCE_TARGET_STAGE when one is wired (e.g. a quarantine or discard stage), otherwise forwarded to NEXT_STAGE unchanged (a bounce is never silently dropped).

  • state.spam = true — the message is dropped (the row is deleted).

  • state.spam = false or state.paid = true — the message is forwarded to NEXT_STAGE. An explicit classification (set by some other component) overrides the payment flow.

  • state.paid = false is present — an order was created earlier and the message has been woken again (by the pepsi-resume payment webhook calling pepsi-httpd’s /resume, or by pepsi-dispatch(1) when the deadline lapsed). The stage asks the merchant backend whether the order is now paid:

    • paid — forward to NEXT_STAGE;

    • unpaid and past the stored deadline — reject the message: reroute to the stage’s BOUNCE_STAGE (wire it to pepsi-stage-bounce(1) for a rejection DSN, or pepsi-stage-discard(1) to drop it silently); when no BOUNCE_STAGE is configured the row is simply deleted;

    • unpaid but still before the deadline — the message is re-paused until the deadline and a warning is logged. This should not happen in normal operation (a paused message only wakes on payment or at the deadline). It stays paused, not pending: a row the dispatcher could claim again at once would busy-loop against the early resume that woke it.

  • no paid field (first encounter) — the stage creates a v1 merchant order priced by ORDER_CHOICES (order_id is the message’s external token, so the pepsi-resume webhook’s {{order_id}} resolves back to this message), then pauses the message — until the DELAY_DSN_AFTER checkpoint when one is configured and falls before the deadline, otherwise until PAYMENT_DEADLINE — recording paid: false and the absolute deadline in state.

The deadline is stored in state.pay_deadline (epoch seconds) because pepsi-dispatch(1) clears the row’s timeout column when it requeues a paused message, so it could not otherwise be recovered on the re-run.

After pausing the message on that first encounter, the stage also originates a payment-request auto-reply to the sender (when BLOCK_RESPONSE_STAGE is configured). The reply explains that the recipient requires unknown senders to pay before their mail is accepted, and carries the taler://pay/… URI — both as a link and as a QR code (an inline image/png part referenced by cid:) — together with the acceptable payment options (the ORDER_CHOICES, omitting any choice that involves a token family) and a pointer to https://wallet.taler.net/ for senders without a wallet. The sender is addressed by the display name from the original From: header when one is available. The human-readable part is a text/html body rendered from the payment-request.<lang>.body Mustache template under [pepsi] TEMPLATE_DIR. The language(s) are taken from state.language as set by pepsi-stage-detect-language (run earlier in the pipeline): the reply renders one chunk per detected language that has a template, concatenated in the detected order, each with its payment options localized to that language. When the message carries no detected language — or none of the detected languages has a template — the PAYMENT_MESSAGE_DEFAULT_LANGUAGE template is used instead. The reply uses the null sender <> (so it is never itself bounced or gated) and is injected at BLOCK_RESPONSE_STAGE (normally a signing/relay chain), so it is DKIM-signed and delivered like any other outbound mail.

The reply goes to an envelope sender nobody has verified, so it is withheld — the message is held for payment all the same — in two cases. First, when RFC 3834 says the message must not be answered automatically (Auto-Submitted other than no, Precedence: bulk|list|junk, X-Auto-Response-Suppress, any List-* field, a service sender, a multipart/report). Second, when the same sender was already sent a payment request for the same protected mailbox (the first envelope recipient) within BLOCK_RESPONSE_SUPPRESS (default one hour): otherwise a flood of messages bearing one forged sender would become the same flood of requests at that address. The window is claimed and recorded in one statement (the payment_request_should_send function over pepsi.payment_request_reply), after the message is paused and the reply rendered, so two workers holding two messages from one sender send one request, and a failure before that point does not use the window up. A sender whose second message falls inside the window gets no prompt for it; that message can still be paid (its order exists) and otherwise bounces at the deadline like any unpaid one. 0 s answers every message.

The auto-reply also carries two machine-readable headers so that a sending Pepsi can settle the demand automatically (see pepsi-stage-auto-pay(1)): a Taler: header holding the taler://pay/… URI, and the original message’s Pepsi-Origin: header copied verbatim (when the incoming message carried one). Echoing the proof-of-origin lets the originating site verify the returning demand is for mail it actually sent and meter what it spends per original message (all demands for one original — e.g. a mailing-list fan-out — share one Pepsi-Origin nonce). The header is copied as-is, not re-signed: only the originator can verify it (its HMAC is keyed with the originator’s secret).

The reply is rendered before the message is paused. If no reply can be produced — notably when the PAYMENT_MESSAGE_DEFAULT_LANGUAGE template is missing — the message is held for payment without one, with a warning in the log; nothing is claimed in the window, so a later message from the same sender is answered once the template is back. pepsi-setup validates that this template exists up front. Injecting the already-rendered reply after the pause is best-effort: a database error there is logged, the window claimed for it is given back, and the message stays paused for payment.

85.1.16.1.3.1. Merchant backend unavailable

A merchant backend that cannot be reached, or answers an order or status request with an error, is the host’s problem and not the sender’s. The message stays held (paused, the reason in state.last_error) and the merchant is asked again every five minutes, never past PAYMENT_DEADLINE; the deadline is fixed at the first attempt, so an outage does not lengthen it. If the merchant still cannot be asked once the deadline has passed, the stage fails open: the message is delivered to NEXT_STAGE unverified, with a warning in the log. A payment gate that cannot tell whether it was paid must not turn a merchant outage into lost mail. A status request for an order the merchant does not know (404) is an answer, not an outage: the order is treated as unpaid.

The stage requires the shared [pepsi-payments] section (the merchant backend and access token; see pepsi.conf(5)) and, for the payment webhook to release paused messages, the [pepsi-httpd] RESUME_AUTHORIZATION_TOKEN (see pepsi-httpd(1)).

85.1.16.1.3.2. Mail the gate holds by mistake

Caution

Only the null envelope sender bypasses the gate. Everything else that is not cleared by state.spam = false — normally set by pepsi-stage-check-whitelist(1) placed before this stage — is held, including mail nobody can or will pay for:

  • Pepsi’s own secure-link mail. The link notification and the PIN mail of pepsi-stage-secure-link(1) carry the original sender as envelope sender (deliberately, so a failure reaches them), not <>. When one of them re-enters this deployment as inbound mail to a gated mailbox — a PIN sent back to a local sender, a link to a local recipient — it is held until PAYMENT_DEADLINE and then rejected, and the feature fails silently. A reply composed in the portal is injected on the inbound path unsigned, with the external recipient as sender, and is held like any other mail from that address. Read receipts use the null sender and pass.

  • Mailing-list postings. A posting carries its author’s From:, so a correspondent whitelist does not cover it, and its List-* fields suppress the payment request, so nobody is even asked: every posting to a gated subscriber is held and then rejected at the deadline.

Vacation notices (pepsi-stage-vacation(1)) and this stage’s own payment requests use the null sender and are never held for payment; with BOUNCE_TARGET_STAGE wired they are subject to the proof-of-origin check below like any other null-sender message.

The workaround is to whitelist those senders in a group the check stage’s WHITELIST_NAME names. For mail your own domains originate, keep the default DKIM requirement: an aligned DKIM pass for example.com can only come from your own signing stage, so the pattern cannot be satisfied by a forged From::

pepsi-whitelist add correspondents '^[^@]+@example\.com$'

(repeat for every served domain, and for the domain of [pepsi-secure-link] NOTIFY_FROM if it is set elsewhere). For a mailing list, whitelist its identifier, naming the ARC sealer that must vouch for it:

pepsi-whitelist add-list --sealer lists.example.org \
    correspondents users.lists.example.org

See pepsi-whitelist(1).

85.1.16.1.4. Configuration

Options live in the stage’s own [stage-<name>] section (PROGRAM = pepsi-stage-anti-spam): the required NEXT_STAGE, BOUNCE_STAGE, BOUNCE_TARGET_STAGE (see PROOF OF ORIGIN below), the order parameters (the required ORDER_CHOICES, plus PAYMENT_DEADLINE, DELAY_DSN_AFTER, SUMMARY, FULFILLMENT_MESSAGE) and the auto-reply parameters (BLOCK_RESPONSE_STAGE, BLOCK_RESPONSE_FROM, BLOCK_RESPONSE_SUBJECT, BLOCK_RESPONSE_SUPPRESS, PAYMENT_MESSAGE_DEFAULT_LANGUAGE). The stage also requires the shared [pepsi-payments] section; the payment webhook that releases a paid message before its deadline needs the [pepsi-httpd] RESUME_AUTHORIZATION_TOKEN. All are documented in pepsi.conf(5).

85.1.16.1.5. Proof of origin

Forwarding every null-sender bounce unconditionally is safe only because a bounce is never itself bounced — but it lets an attacker inject backscatter (forged bounces) that Pepsi then relays back to a forged “original sender”. To reject such forgeries, the shared [pepsi-origin] section (pepsi.conf(5)) is enabled by default — pepsi-setup(1) generates a random secret on first run if none is configured. Both relay stages (pepsi-stage-relay-to-internet(1) and pepsi-stage-relay-to-smarthost(1)) then stamp every outbound message with a Pepsi-Origin header — an HMAC over the originating sender account, a fresh 128-bit nonce, a timestamp and our HOSTNAME — and record the nonce in pepsi.origin_nonce for two weeks.

A genuine bounce embeds the original message (at least its headers, and so the Pepsi-Origin header) in its DSN body. This stage scans the whole returning bounce for that header and trusts the bounce only if both checks pass, in this order:

  1. the header’s HMAC verifies against our secret and its host matches our HOSTNAME (a fast, purely cryptographic check); and

  2. only then — the nonce is still present (and unexpired) in pepsi.origin_nonce.

A bounce we cannot verify — one that carries no such header, fails the HMAC, or whose nonce is no longer tracked, and every bounce at all when proof of origin is switched off ([pepsi-origin] ENABLED = no; none can then be proven ours) — is routed to BOUNCE_TARGET_STAGE when one is wired, otherwise forwarded to NEXT_STAGE unchanged (never silently dropped).

BOUNCE_TARGET_STAGE should normally point at a quarantine mailbox or a review stage rather than a hard pepsi-stage-discard(1) sink: standard non-delivery reports from major MTAs (Postfix, Exim, Gmail, Exchange) return the original headers and so verify, but a real tail of legitimate bounces — spam-rejection notices that quote little of the original, header-stripping gateways, minimal/legacy bouncers, and bounces arriving after the two-week nonce window — cannot be verified, and discarding them outright loses non-delivery reports your own users need.

85.1.16.1.6. Files

<TEMPLATE_DIR>/payment-request.<lang>.body

The Mustache template for the payment-request reply body (TEMPLATE_DIR is the [pepsi] option, default ${DATADIR}/templates, i.e. /usr/share/pepsi/templates for a --prefix=/usr installation). The template for PAYMENT_MESSAGE_DEFAULT_LANGUAGE (default payment-request.en.body) is required; add the languages pepsi-stage-detect-language may detect alongside it. The template is given the variables recipient_name / has_name, protected_mailbox, original_subject, pay_uri / pay_link, qr_cid, wallet_url, has_options and the payment_options list (each amount plus has_description and description, localized to that template’s language).

85.1.16.1.7. State

Inputs: state.spam / state.paid (booleans, optional) classify the message; state.pay_deadline (epoch seconds) is the deadline this stage wrote; state.dsn carries the recipient’s notify/orcpt (echoed into a rejection).

Outputs: on the first encounter the stage merges {"paid": false, "pay_deadline": <epoch>} into state and pauses. At the DELAY_DSN_AFTER checkpoint it adds state.delay_sent = true once a delay DSN has been sent (so it fires at most once) and re-pauses. On rejection it reroutes to BOUNCE_STAGE with a state.bounce object (kind = permanent) for pepsi-stage-bounce(1). The state layout is described in pepsi.state(7).

Transitions (driven by state.spam/state.paid, PAYMENT_DEADLINE and the merchant order status):

  • whitelisted (state.spam = false) or already paid (state.paid = true) → advance to NEXT_STAGE;

  • explicit state.spam = true → finish (the row is deleted);

  • first encounter → create a Taler order, inject the payment request at BLOCK_RESPONSE_STAGE (unless suppressed, see above), and pause (until the DELAY_DSN_AFTER checkpoint when one is configured and falls before it, else until PAYMENT_DEADLINE);

  • woken (resume webhook or deadline) and now paid → advance to NEXT_STAGE;

  • woken, still unpaid, deadline passed → reroute to BOUNCE_STAGE (or finish, if no BOUNCE_STAGE is configured);

  • merchant unavailable (see above) → re-pause before the deadline, advance to NEXT_STAGE after it;

  • woken, still unpaid, deadline not yet reached → with DELAY_DSN_AFTER configured this is the delay checkpoint: send the sender a one-shot delayed DSN (RFC 3461, only if they requested NOTIFY=DELAY) without releasing the message, then re-pause until PAYMENT_DEADLINE; without DELAY_DSN_AFTER an early wake-up is anomalous and the message is re-paused (paused) until the deadline.

85.1.16.1.8. Commands

worker

Run as a persistent pepsi-dispatch(1) worker, reading message ids on standard input.

85.1.16.1.9. Global Options

These global options precede the subcommand.

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity (default info).

-v, –verbose

Show log messages from all sources.

-h, –help; -V, –version

Print a usage summary / the version and exit.

85.1.16.1.10. Exit Status

0

The message was processed (dropped, forwarded, paused for payment, or rerouted for rejection).

1

An error occurred (message not found or not running, or a per-address setting that breaks the stage’s section). The reason is written to the log.

A fault of the host (the database, a template or helper that cannot be used) is not reported as a failure: the message is paused and retried, as Stage errors in pepsi-dispatch(1) describes. A section that does not parse makes the worker refuse to start (status 78) instead of failing each message in turn. A missing [pepsi-payments] backend is such a section.

85.1.16.1.11. Examples

Process message 42 (it must be running):

echo 42 | pepsi-stage-anti-spam -c /etc/pepsi/pepsi.conf worker

A stage that charges one KUDOS and rejects unpaid mail after two days:

[stage-anti-spam]
PROGRAM = pepsi-stage-anti-spam
NEXT_STAGE = srs
BOUNCE_STAGE = bounce
BLOCK_RESPONSE_STAGE = dkim-sign
PAYMENT_DEADLINE = 48 h
ORDER_CHOICES = [{"amount":"KUDOS:1"}]

85.1.16.1.12. See Also

pepsi-config(1), pepsi-stage-bounce(1), pepsi-stage-discard(1), pepsi-httpd(1), pepsi-dispatch(1), pepsi-setup(1), pepsi.conf(5), pepsi.state(7)

85.1.16.1.13. Bugs

Report bugs to the Pepsi issue tracker.