69. pepsi-queue

Inspect and repair the message workqueue.

69.1. Role

pepsi-queue is the operator’s diagnostic and repair tool for the pepsi.workqueue table. It connects through the shared [pepsi-postgres] section; its one option of its own is [pepsi-queue] QUARANTINE_MAX_AGE_DAYS (read by gc). Started as root it continues as the pepsi service account before connecting, so no sudo -u pepsi is needed. Reference: pepsi-queue(1).

69.2. Features

  • list — list queued messages ordered by workqueue_id, optionally as --json (workqueue_id, stage, status, mail_from, subject, received_at, state), with --limit (100 by default), --stage and --status filters, an optional WORKQUEUE_ID positional that restricts the output to one message, and --state-only to print each matching message’s full, un-abbreviated JSON state instead of the row summary. --headers/--body write one message’s stored header block and/or body byte for byte. list is the default action, so its positional and options are also accepted bare (pepsi-queue 5, pepsi-queue --state-only 5), but not in front of another subcommand, where they would look like a filter that the subcommand does not apply.

  • delete WORKQUEUE_ID — permanently remove a message.

  • set-stage WORKQUEUE_ID STAGE — re-assign a message to a given stage and reset its status to pending (its state is left unchanged) so the new stage picks it up.

  • clear-all — bulk “unstick”: reset every running/paused message back to pending (failed/timeout are left untouched, and so is mail held at a pepsi-stage-quarantine stage) — the recovery used after a crash or a stalled stage.

  • quarantine list|show|release|delete — the messages a pepsi-stage-quarantine stage holds, by token: list (--json, --limit) shows when each arrived and since when it is held, its envelope, and who sent it to the quarantine and why (state.quarantine.by and reason); show shows one; release hands it back to its stage, which sends it on to its RELEASE_STAGE; delete removes it after a confirmation (--yes in a script), recorded in the opt-in mail log.

  • gc — garbage-collect expired data. It deletes the proof-of-origin nonces (pepsi.origin_nonce) whose expiration has passed and the rows of the MX address cache (pepsi.dns_address) whose DNS TTL has passed. In the MTA-STS policy cache (pepsi.mta_sts_policy), it refreshes every policy that is due, independently of mail flow (RFC 8461 section 10.2), and deletes the policies whose max_age has passed without a successful refresh. It also deletes the recorded mailing-list confirmation requests (pepsi.list_confirmation) past their window, the learnt list-id whitelist rows idle past their max_idle, the second factors that were set up by e-mail and not confirmed in time (pepsi.otp_pending), and the DMARC aggregate reports (pepsi.dmarc_report) older than [pepsi-dmarc-reports] RETAIN_DAYS. The same run re-checks every login in the RFC 7293 login assignment record against passwd, so a login that changed owner loses its previous owner’s state (A login that changes owner). Finally, it ends holds at a quarantine stage after [pepsi-queue] QUARANTINE_MAX_AGE_DAYS (default 14), at every stage that a QUARANTINE_STAGE names and every pepsi-stage-quarantine stage without a MAX_HOLD of its own. An expired message is bounced without its content, with two exceptions that are dropped instead: a member’s copy of a list post, and a message whose content came out of, or failed to open from, an S/MIME EnvelopedData (AES-CBC) ciphertext. A bounce for the latter would tell its sender whether a modified ciphertext’s padding was valid (Bounces and replies to encrypted mail). The command is meant for a timer; the Debian package wires an hourly one (pepsi-origin-gc.timer).

delete, set-stage, clear-all and quarantine release/delete are recorded in the audit log, as the same actions taken through the administrative API are.

The mutating subcommands act immediately and are not reversible. set-stage and clear-all also wake the dispatcher (pepsi.workqueue carries no notifying trigger, and these run outside the dispatcher’s own loop, so nothing else would). Run ``clear-all`` only while ``pepsi-dispatch`` is stopped: the running half names rows that a live worker may still be holding, and resetting them means the same message is claimed and handled twice.

69.3. Relationship to the dispatcher

pepsi-queue manipulates the same stage/status/state columns that the dispatcher acts on; clear-all complements the dispatcher’s own start-up reset of orphaned running rows. It does not start stage programs.

69.4. See also

pepsi-dispatch, Architecture, pepsi-queue(1), pepsi.conf(5).