70.1.40. pepsi-failure-bouncer

Funnel failed/timeout messages to the bounce stage

Manual section:

1

70.1.40.1.1. Name

pepsi-failure-bouncer - move stuck (failed/timeout) messages to the bounce stage.

70.1.40.1.2. Synopsis

pepsi-failure-bouncer [GLOBAL-OPTIONS] [–failed-only | –timeout-only]

pepsi-failure-bouncer [GLOBAL-OPTIONS] –once [–failed-only | –timeout-only]

70.1.40.1.3. Description

A message moving through the post-ingress stage pipeline (see pepsi-dispatch(1)) lands in one of two terminal states when its current stage gives up:

failed

A stage called fail() (a permanent error), or its worker process crashed.

timeout

A worker exceeded [pepsi-dispatch] MAX_RUNTIME and was killed.

The dispatcher never re-queues a failed or timeout row, so without operator action such messages accumulate in pepsi.ingress and the original sender is never told their mail was not delivered.

pepsi-failure-bouncer closes that gap: it moves each stuck message to the configured [pepsi-failure-bouncer] BOUNCE_STAGE and resets its status to pending. The dispatcher then runs the bounce stage (pepsi-stage-bounce(1)), which — honouring the sender’s RFC 3461 NOTIFY — turns the message into a delivery-status notification and relays it back to the sender.

A message that is already at the bounce stage is left untouched, so a bounce that itself fails at that stage does not loop. A bounce that fails downstream of the bounce stage (a null-sender DSN that could not be relayed) is still moved back to the bounce stage, which simply drops it — a bounce is never re-bounced.

Having moved rows to pending, the bouncer notifies the ingress channel itself, so the dispatcher processes the bounce immediately rather than at its next POLL_INTERVAL sweep. It has to: it runs outside the dispatcher’s loop, and pepsi.ingress carries no notifying trigger (see pepsi-dispatch(1)).

It connects to the shared database through the [pepsi-postgres] section and is configured by [pepsi-failure-bouncer] (see pepsi.conf(5)). When started as root — a root shell, a cron job, a unit without a User= — it continues as the unprivileged pepsi service account, the role that owns the queue, before connecting: root has no PostgreSQL role of its own, so without that switch the connection would be refused. The configuration is read first, and if the pepsi account does not exist the identity is left untouched and a warning is logged. It is not a stage.

70.1.40.1.4. Modes

With no --once flag, pepsi-failure-bouncer runs as a long-lived service. It LISTENs on the PostgreSQL ingress_failed channel — fired by the ingress_failed_notify trigger whenever any writer makes a row failed or timeout — and bounces each message as it arrives. On start-up, and again on every reconnect, it first sweeps all messages that are already stuck, so nothing committed before the listener attached (or during a dropped connection) is missed. It runs until it receives SIGINT or SIGTERM.

With –once, it performs that sweep a single time over all currently-stuck messages and exits — the form for a manual operator run or a periodic cron job when running the service is not desired.

70.1.40.1.5. Options

–once

Sweep the messages that are stuck now and exit, instead of running as a service that listens for new failures.

–failed-only

Act only on failed messages, ignoring timeout.

–timeout-only

Act only on timeout messages, ignoring failed. Mutually exclusive with –failed-only.

70.1.40.1.6. Global Options

These options may appear before or after the other flags.

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace (default: info).

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

70.1.40.1.7. Exit Status

0

Successful completion (a --once sweep finished, or the service shut down cleanly on a signal).

1

An error occurred: a malformed configuration, the missing required BOUNCE_STAGE option, or a failed database connection. In service mode a lost database connection is not fatal — it is logged and retried with back-off.

70.1.40.1.8. Examples

Run as a service (the typical deployment, under an init/supervisor):

pepsi-failure-bouncer -c /etc/pepsi/pepsi.conf

Bounce everything currently stuck, once, by hand:

pepsi-failure-bouncer -c /etc/pepsi/pepsi.conf --once

Bounce only the messages that timed out:

pepsi-failure-bouncer -c /etc/pepsi/pepsi.conf --once --timeout-only

70.1.40.1.9. See Also

pepsi-dispatch(1), pepsi-stage-bounce(1), pepsi-queue(1), pepsi-status(1), pepsi.conf(5)

70.1.40.1.10. Bugs

Report bugs to the Pepsi issue tracker.