85.1.40. pepsi-queue

inspect and repair the pepsi-ingress message queue

Manual section:

1

85.1.40.1.1. Name

pepsi-queue - list, delete, re-stage and unstick messages in the work queue.

85.1.40.1.2. Synopsis

pepsi-queue [GLOBAL-OPTIONS] [list] [–json] [–state-only] [–headers] [–body] [–limit N] [–stage STAGE] [–status STATUS] [WORKQUEUE-ID]

pepsi-queue [GLOBAL-OPTIONS] delete WORKQUEUE-ID

pepsi-queue [GLOBAL-OPTIONS] set-stage WORKQUEUE-ID STAGE

pepsi-queue [GLOBAL-OPTIONS] clear-all

pepsi-queue [GLOBAL-OPTIONS] gc

The list options are accepted without the word list because listing is the default action, but only then: given in front of another subcommand (pepsi-queue --stage relay delete 5) they are refused, not silently ignored, since they would read like a filter the command does not apply.

85.1.40.1.3. Description

pepsi-queue is a diagnostic and repair tool for the pepsi.workqueue table shared by the Pepsi pipeline. Every accepted message advances through a filter pipeline tracked by three columns on each row: a stage (the configuration section of the program currently responsible for the message; init for a freshly accepted message), a status (pending, running, paused, failed or timeout), and an optional JSON state (per-stage scratch data). See pepsi.state(7) for the keys state may carry.

The tool connects to the same database as the other components, through the shared [pepsi-postgres] section; it introduces no configuration of its own. When started as root it continues as the pepsi service account, which owns the queue; see Running as root.

Warning

The mutating subcommands act immediately and are not reversible. delete removes a message permanently, and set-stage / clear-all change processing state in place.

set-stage and clear-all do emit a database notification: when they moved at least one row they reset it to pending and then wake pepsi-dispatch(1), so the change takes effect at once rather than at the next poll. pepsi.workqueue carries no notifying trigger, and pepsi-queue runs outside the dispatcher’s own loop, so nothing else would. delete emits none — a row that is gone gives the dispatcher nothing to claim.

That makes clear-all unsafe on a running pipeline: it resets rows that are currently running — a worker may still be handling one — and then hands them straight back to the dispatcher, which claims them again. The message is then processed twice, which for a delivery stage means a duplicate delivery and for a bounce stage a duplicate bounce. Stop pepsi-dispatch first, or use it only after a crash, when no worker is left to collide with.

85.1.40.1.4. Commands

list [–json] [–state-only] [–headers] [–body] [–limit N] [–stage STAGE] [–status STATUS] [WORKQUEUE-ID]

List queued messages, ordered by workqueue_id. This is also the default action when no subcommand is given, so the list keyword may be omitted: pepsi-queue lists everything and pepsi-queue 5 shows only the message with workqueue_id 5.

WORKQUEUE-ID

An optional trailing message id. When given, at most that one message is shown. --limit is then moot, but --stage and --status are still applied on top of it, so a message that does not also match them is not shown. pepsi-queue 5 and pepsi-queue list 5 are equivalent.

–json

Emit a pretty-printed JSON array of objects (workqueue_id, stage, status, mail_from, subject, received_at, state) instead of the human-readable table. subject and state are null when the message carries neither.

–state-only

Print each matching message’s full, un-truncated JSON state (the table view abbreviates it), one pretty-printed block per message, each headed by a # workqueue_id comment line. With an explicit WORKQUEUE-ID that header is dropped and the output is just that message’s state, so pepsi-queue --state-only 5 prints the complete state of message 5 — convenient to pipe into a JSON tool. A message with no state prints null. This takes precedence over –json.

–headers, –body

Write the stored header block (–headers) or body (–body) of the message WORKQUEUE-ID to standard output, byte for byte and with nothing else printed; both together write the whole message — header block, the separating blank line, body — exactly as a stage that loads the full message reassembles it. Both need an WORKQUEUE-ID and cannot be combined with –json or –state-only. The output is raw mail (CRLF line endings, possibly 8-bit), meant for a file or a pipe, e.g. pepsi-queue --headers --body 5 | pepsi-detect-language.

–limit N

Show at most N messages. Defaults to 100. Has no effect when an WORKQUEUE-ID is given, since at most one row can match.

–stage STAGE

Only show messages whose stage equals STAGE.

–status STATUS

Only show messages with the given status.

delete WORKQUEUE-ID

Permanently delete the message with the given workqueue_id.

set-stage WORKQUEUE-ID STAGE

Re-assign the message to STAGE and implicitly reset its status to pending so the new stage picks it up. The message’s state is left unchanged.

clear-all

Reset every message whose status is running or paused back to pending — the bulk “unstick” used after a crash or a stalled stage. Messages in failed or timeout are left untouched.

gc

Garbage-collect the proof-of-origin nonce table (pepsi.origin_nonce), deleting every entry whose two-week expiration has passed. An expired nonce can no longer authenticate a returning bounce, so dropping it is safe. The same run deletes the MX address cache rows (pepsi.dns_address) whose DNS TTL has passed: the relay stage never reads an expired row and replaces a host’s rows when it resolves it again, but a host it never resolves again would otherwise keep them for ever. Intended to run periodically; the Debian package ships a pepsi-origin-gc.timer that invokes it hourly. See pepsi-stage-relay-to-internet(1) and pepsi-stage-relay-to-smarthost(1) (which record the nonces as they relay) and pepsi-stage-anti-spam(1) and pepsi-stage-auto-pay(1) (which verify them).

85.1.40.1.5. Audit log

The three mutating subcommands are recorded in the shared audit log, through the same helper the administrative API uses for the same actions: delete as queue.cancel, set-stage as queue.set-stage (with the new stage) and clear-all as queue.clear-all (with the number of rows reset). Moving a message by hand is an administrative act whichever surface performs it, and a log that saw only one of them would be misleading rather than merely incomplete. list and gc are not recorded: reading is not an action, and the garbage collector is a timer rather than an operator.

85.1.40.1.6. Running as root

Pepsi gives every component its own PostgreSQL role, authenticated over the local socket by the operating-system account it runs as, and root is deliberately not one of them. Rather than fail to connect and oblige you to remember sudo -u pepsi pepsi-queue …, the tool detects that it was started as root and becomes the unprivileged pepsi service account — the account the dispatcher runs as, which owns pepsi.workqueue — before it connects. The change is permanent for the life of the process; the tool needs no privilege of its own.

The configuration file (and any @inline-secret@ fragment it references) is read before the switch, so a root-only configuration is still loaded normally.

If the pepsi account does not exist — an uninstalled source tree, a test rig — the identity is left untouched, a warning is logged, and the connection is attempted as the invoking user, so a setup in which root can reach the database keeps working.

85.1.40.1.7. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace (default: info).

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

85.1.40.1.8. Exit Status

0

Successful completion. An WORKQUEUE-ID that matches no row is not an error: list, delete and set-stage say so on standard output and still exit 0.

1

An error occurred: a malformed configuration file or a failed database connection or query. The reason is written to the log.

85.1.40.1.9. Files

When –config is not given, the first existing file from the following list is used. Every Pepsi component shares the same configuration file, so this is the same list each one searches:

• $XDG_CONFIG_HOME/pepsi.conf

• $HOME/.config/pepsi.conf

• /etc/pepsi/pepsi.conf

• /etc/pepsi.conf

85.1.40.1.10. Examples

List the first 20 queued messages as a table:

pepsi-queue -c /etc/pepsi/pepsi.conf list --limit 20

Show, as JSON, every message currently held by the spamfilter stage:

pepsi-queue -c /etc/pepsi/pepsi.conf list --json --stage spamfilter

Inspect just message 5, then dump its complete state JSON:

pepsi-queue -c /etc/pepsi/pepsi.conf 5
pepsi-queue -c /etc/pepsi/pepsi.conf --state-only 5

Save message 5 as it is stored, then look at its header block alone:

pepsi-queue -c /etc/pepsi/pepsi.conf --headers --body 5 > msg5.eml
pepsi-queue -c /etc/pepsi/pepsi.conf --headers 5

Hand message 42 to the spamfilter stage again:

pepsi-queue -c /etc/pepsi/pepsi.conf set-stage 42 spamfilter

Recover after a crash by re-queuing everything that was in flight:

pepsi-queue -c /etc/pepsi/pepsi.conf clear-all

85.1.40.1.11. See Also

pepsi-status(1), pepsi-config(1), pepsi-ingress(1), pepsi-dispatch(1), pepsi-failure-bouncer(1), pepsi.conf(5), pepsi.state(7), pepsi-setup(1)

85.1.40.1.12. Bugs

Report bugs to the Pepsi issue tracker.