85.1.23. pepsi-stage-list-bounce

score an inbound bounce against the member it names

Manual section:

1

85.1.23.1.1. Name

pepsi-stage-list-bounce - the mailing-list bounce stage of the Pepsi pipeline.

85.1.23.1.2. Synopsis

pepsi-stage-list-bounce [GLOBAL-OPTIONS] worker

85.1.23.1.3. Description

pepsi-stage-list-bounce is a stage program run by pepsi-dispatch(1). It consumes bounces: a delivery failure that came back to a list’s -bounces address is attributed to the member it concerns, scored, and — once the score crosses the list’s threshold — that member’s delivery is disabled, they are warned, and eventually they are unsubscribed.

Three things in this tree are called “bounce” and they are not the same thing. This one reads an inbound failure report. pepsi-stage-bounce(1) generates a DSN for a message Pepsi itself could not deliver. pepsi-failure-bouncer(1) rescues messages the pipeline gave up on and hands them to that generator. A configuration that points this stage’s NEXT_STAGE at the generator answers a bounce with a bounce; pepsi-setup(1) refuses it.

The stage is reached from pepsi-stage-list(1), the router, which sends a message here when its envelope recipient is a list’s -bounces address. It is never reached any other way: a message arriving here without the router’s state.list descriptor fails.

85.1.23.1.4. Attribution

Four steps, in decreasing order of certainty. The first two are exact, because Pepsi wrote the address they read.

  1. The VERP envelope. Every copy a list sends carries an envelope sender of <list>-bounces+<local>=<domain>@<host>, so a bounce of it names the member unambiguously. The <list>-bounces part must match this list’s own bounces mailbox before the address is believed: without that check, anybody could score a bounce against any member of any list by sending mail to an address they made up.

    A VERP match still has to look like a failure. A temporary failure is ignored outright, and a VERP’d message with no recognised failure at all is forwarded rather than scored — it may be a vacation reply or a virus scanner’s report, which arrived at this address only because that is what was on the envelope.

  2. A probe token (<list>-bounces+<token>@<host>), when BOUNCE_PROBES is on. See Probes.

  3. The heuristic detectors, over the whole message. See Recognised bounce formats.

  4. Nothing resolved, in which case the bounce is wrapped up and forwarded to a human — see Unrecognised bounces.

85.1.23.1.5. Scoring

The machine is GNU Mailman 3’s, reproduced step for step, because every attribute it reads is visible through the REST API and a site’s documentation says what each one does.

  1. the list is gone — drop

  2. the address is not a member — drop

  3. the bounce is a probe — disable immediately, without scoring; a probe bounce is decisive

  4. the member is already disabled by bounces — a residual bounce; drop

  5. a bounce was already scored for this member today — update the timestamp only. At most one increment per calendar day.

  6. no previous bounce, or one older than bounce_info_stale_after — reset the score to 1; otherwise increment it by one

  7. if bounce_notify_owner_on_bounce_increment and (the score is below the threshold or probes are off) — tell the owners

  8. at or above bounce_score_threshold: send a probe if BOUNCE_PROBES is on (which resets the score to zero), otherwise zero the score and disable delivery

  9. mark the event processed

Step 5 is why a bounce storm cannot purge a list in an afternoon: a member with the default threshold of 5 needs failures on five different days.

85.1.23.1.6. Warning and removal

Disabling a member is not unsubscribing them. Two periodic passes, run by pepsi-list tasks --once (see pepsi-list(1)), finish the story:

  • warn — a member disabled by bounces who has had fewer than bounce_you_are_disabled_warnings warnings, and whose last warning is at least bounce_you_are_disabled_warnings_interval old, is sent one;

  • remove — once the warnings have run out and the interval has elapsed again, the member is unsubscribed and the owners are told if bounce_notify_owner_on_removal is set.

A member disabled by themselves or by a moderator is never touched by either pass.

Setting bounce_you_are_disabled_warnings to zero removes a disabled member on the next sweep with no warning mail at all. That is upstream’s documented behaviour and it is deliberate, not an edge case.

The notices these passes send are injected at [pepsi-list] NOTICE_STAGE, because the passes run from a timer rather than from a stage. pepsi-setup(1) refuses a configuration that runs this stage without it: without a stage to inject into, a member would be warned three times in the database and never once by mail.

85.1.23.1.7. Probes

With [pepsi-list] BOUNCE_PROBES = yes, crossing the threshold sends the member a probe instead of disabling them: a message addressed to them alone, from an envelope sender carrying a one-use token, with the triggering bounce attached. The member’s score is reset to zero while the probe is outstanding. If the probe bounces, the token identifies them exactly and delivery is disabled at once; if it does not, nothing happens and they keep receiving mail.

Probes are off by default, as they are upstream. The trade is one more message to an address that is probably dead, against not disabling a member whose provider had one bad afternoon. A probe token is good for ten days and carries no Precedence: bulk header — a probe exists in order to be bounced, and that header invites some systems not to bounce it.

85.1.23.1.8. Unrecognised bounces

A bounce nothing recognised is wrapped in an explanatory notice with the original attached verbatim as message/rfc822, and sent to whoever the list’s forward_unrecognized_bounces_to names: administrators (the default), site_owner, or discard.

This is a feature rather than a fallback. Every bounce that falls through is a format the corpus does not have, and the person who receives it can send the sample to the Pepsi maintainers so that the detector can be written. Under every disposition — discard included — the fall-through is counted in pepsi.event_log as list.bounce.unrecognized, because how often it happens is what says whether a site needs more detectors at all.

85.1.23.1.9. Recognised bounce formats

Seventeen detectors, tried in this order, first match winning. They are ported from flufl.bounce (Apache-2.0, copyright Barry Warsaw), and each is tested against that library’s own fixtures:

dsn, exim, sina, yahoo, yale, smtp32, postfix, qmail, groupwise, microsoft, caiwireless, exchange, netscape, aol, simplematch, simplewarning, llnl

Exactly one of them — dsn, RFC 3464 — is a standard; the rest name a vendor or a site. In practice they matter far less here than they do upstream: Pepsi VERPs every copy of every post, so a bounce of a message Pepsi sent is attributed by its envelope without any of this being consulted. The detectors are what reads a bounce that reached a list some other way.

A message that no detector recognises is not a bounce as far as this stage is concerned, and is forwarded rather than guessed at. That distinction is worth knowing when reading a log: “Pepsi did not recognise this” and “Pepsi decided this was not a failure” are different sentences.

85.1.23.1.10. Configuration

[stage-list-bounce]

PROGRAM

pepsi-stage-list-bounce.

NEXT_STAGE (required)

Where an unrecognised bounce goes once it has been wrapped for a human: ordinary delivery, not a list stage. pepsi-setup(1) refuses a value that names pepsi-stage-bounce(1) or the list router.

RESPONSE_STAGE (required)

Where the member’s probe and the owners’ notices are injected — the signing tail, not the list delivery stage. That stage only accepts a member’s copy from the fan-out and fails anything else, so a notice sent there never goes out; pepsi-setup(1) refuses the wrong value.

[pepsi-list]

NOTICE_STAGE

Where the periodic warn-and-remove passes inject their notices. Required as soon as this stage exists.

BOUNCE_PROBES

yes or no (default no). See Probes.

BOUNCE_EVENT_RETENTION

How long a processed bounce event row is kept, in days (default 1). The rows are diagnostic once scored. This is not score decay — that is the per-list bounce_info_stale_after attribute, and it is applied lazily rather than by a nightly pass.

The nine per-list attributes — process_bounces, bounce_score_threshold, bounce_info_stale_after, bounce_notify_owner_on_bounce_increment, bounce_notify_owner_on_disable, bounce_notify_owner_on_removal, bounce_you_are_disabled_warnings, bounce_you_are_disabled_warnings_interval and forward_unrecognized_bounces_to — live in the database, where the REST API can see them. Set them with pepsi-list list set.

85.1.23.1.11. Differences from GNU Mailman 3

  • Scoring is synchronous. Upstream registers a bounce event and scores it in a later pass; Pepsi scores it in the transaction that records it. The “at most one increment per calendar day” rule is what makes two simultaneous bounces for one member harmless.

  • A migrated site’s bounce history does not come across. Upstream’s own importer never reads it either, so a migrated member starts at score zero and a long-dead address gets one more delivery attempt.

85.1.23.1.12. Exit status

The worker exits non-zero only when it cannot run at all. A bounce it cannot make sense of is forwarded, not failed; one for a list that no longer exists, or one the MIME parser refuses, is dropped with a log line (it came to the list’s own -bounces address, and answering it is how loops start). Any other error is retried with back-off until MAX_LIFETIME (see pepsi-dispatch(1)).

When one bounce names several members, each member’s owner notices carry a tag derived from the member’s address as well as the owner’s position, so every member’s notice is sent rather than only the first.

85.1.23.1.13. See also

pepsi-stage-list(1), pepsi-stage-list-post(1), pepsi-stage-list-command(1), pepsi-stage-list-deliver(1), pepsi-list(1), pepsi-stage-bounce(1), pepsi-failure-bouncer(1), pepsi-dispatch(1), pepsi.conf(5).