95.1.53. pepsi-list

manage Pepsi’s mailing lists

Manual section:

1

95.1.53.1.1. Name

pepsi-list - create and manage mailing lists, their members and their owners.

95.1.53.1.2. Synopsis

pepsi-list [GLOBAL-OPTIONS] domain add|list|remove [ARGS]

pepsi-list [GLOBAL-OPTIONS] list create|show|set|set-ext|remove|find|styles [ARGS]

pepsi-list [GLOBAL-OPTIONS] members list|add|remove [ARGS]

pepsi-list [GLOBAL-OPTIONS] owner add|remove|reset-password [ARGS]

pepsi-list [GLOBAL-OPTIONS] import mailman3|mailman21 [ARGS]

pepsi-list [GLOBAL-OPTIONS] invite send|status [ARGS]

pepsi-list [GLOBAL-OPTIONS] digest send|bump|periodic|status [ARGS]

pepsi-list [GLOBAL-OPTIONS] ban add|list|remove [ARGS]

pepsi-list [GLOBAL-OPTIONS] template list|set|clear [ARGS]

pepsi-list [GLOBAL-OPTIONS] unsub-token LIST EMAIL [–serial N] [–uri]

pepsi-list [GLOBAL-OPTIONS] tasks –once

pepsi-list [GLOBAL-OPTIONS] check [–strict]

95.1.53.1.3. Description

pepsi-list is the operator’s command-line tool for Pepsi’s mailing-list subsystem. It is not a stage: it reads and writes the pepsi schema’s list tables directly, in the same way as pepsi-whitelist(1) and pepsi-settings(1).

Pepsi’s mailing-list subsystem is a reimplementation of GNU Mailman 3. Its data model, its rule/chain and handler/pipeline architecture, its REST API, its e-mail command vocabulary and its notice-template names are the GNU Mailman project’s design, copyright the Free Software Foundation, and Pepsi exists to be compatible with them. Upstream’s documentation at <https://docs.mailman3.org> describes concepts that apply directly here and is worth reading alongside this page. Some files in Pepsi are taken from GNU Mailman, Postorius or HyperKitty and remain under the GNU General Public License; vendor/PEPSI-VENDORING.md lists them.

Per-list configuration lives in the database, not in pepsi.conf(5). Lists are created at runtime, potentially in their thousands, by people who are not the operator; the configuration file holds only what the operator owns, in the [pepsi-list] section.

95.1.53.1.4. Global options

The global flags come before the subcommand, as everywhere else in Pepsi:

-c, --config FILE

Configuration file to read.

-L, --log LEVEL

Log level.

-v, --verbose

Show log messages from all sources, including third-party libraries.

-V, --version

Print the version and exit.

95.1.53.1.5. Naming a list

Every verb that takes a list accepts either spelling:

name@domain

The posting address, which is what people type.

name.domain

Upstream’s list_id, with a dot. This is the identifier the REST API uses, and it is not interchangeable with the posting address: both appear in the API, and writing the wrong one is a compatibility bug.

95.1.53.1.6. Commands

domain add DOMAIN [–base-url URL] [–description TEXT]

Register a mail domain in which lists may be created. This is not the same as pepsi-ingress(1)’s ACCEPTED_DOMAINS: a domain can accept mail without hosting lists. If a list domain is not one that ingress accepts, mail to the list is refused at RCPT and nothing in this subsystem’s logs says so.

domain list

Show the list domains and how many lists each holds.

domain remove DOMAIN

Remove a list domain. Refused while it still holds lists: deleting it would cascade through every list’s roster, held messages and archive.

list create ADDRESS [–style STYLE] [–set NAME=VALUE]…

Create a mailing list. --style applies a named set of attribute defaults before any --set; see Styles below. An unknown style name is an error, not a silent no-op.

list show ADDRESS [–explain]

Print every one of the 99 REST attributes with its current value. --explain adds each attribute’s one-line explanation. (The flag is --explain rather than --help because the latter belongs to the argument parser.) The Pepsi-only settings follow, each with its stored value or the value that applies without one.

list set ADDRESS NAME VALUE

Set one attribute. The value goes through the same validator that the REST API and the owner console use, so the three cannot disagree about what a valid value is. Multi-valued attributes take one entry per line; a comma is not a separator, because a regular expression may contain one.

list set-ext ADDRESS KEY VALUE

Set one Pepsi-only per-list setting. These settings are deliberately absent from the Mailman REST API’s /config. Upstream’s PUT replaces the whole resource and requires every writable attribute to be present, so a client built against Mailman 3.3.10 would omit anything that Pepsi invented, and its request would be rejected. A REST client reaches these settings at /lists/<id>/pepsi instead. An empty value restores the default or the site setting. The keys are search_trigram, archive_show_addresses, hold_dmarc_failures (yes, the default: hold for moderation every post whose From: domain failed DMARC at intake) and archive_retention_days.

list remove ADDRESS –yes

Delete a list, its roster, its held messages and its archive. --yes is required.

list find [SUBSTRING]

List the mailing lists, optionally those matching a substring.

list styles

Show the styles and their aliases.

members list ADDRESS [–role ROLE]

Show a list’s roster. ROLE is member, owner, moderator or nonmember.

members add ADDRESS EMAIL [–display-name NAME] [–role ROLE]

Subscribe an address. This bypasses the subscription policy entirely, which is what a command-line tool is for and also what makes it dangerous: the whole point of the confirmation workflow is that an address proves it wants to be there. Use it to migrate a roster or to fix a mistake, not to add people who did not ask.

members remove ADDRESS EMAIL [–role ROLE]

Unsubscribe an address from one role.

owner add ADDRESS EMAIL

Make an address an owner. Unlike an ordinary subscriber, an owner gets a user account, because they will sign in to the owner console.

owner remove ADDRESS EMAIL

Remove an owner.

owner reset-password ADDRESS EMAIL [–password PASSWORD]

Set an owner’s account password and print it. The command prints the password instead of mailing it on purpose: this path has to work when mail is broken, which is exactly when an operator needs it.

The ordinary way to change a password is the web reset flow at /lists/reset, which mails a link and never a password, because a mailed password stays in a mailbox for ever. This subcommand is the escape hatch for the cases that the web flow cannot cover: the account’s address does not receive mail, or nothing on the host is delivering any. It sets the same list_user.password_hash as the web flow, with the same Argon2 parameters, and, like a password change made in the browser, it ends every open session of that account. A password given with --password must meet the rule of the member pages: at least ten characters, not on the list of common passwords, and not containing the address.

ban add PATTERN [–list ADDRESS]

Ban an address, or a regular expression beginning with ^. Without --list the ban is site-wide.

ban list [–list ADDRESS]

Show the bans.

ban remove PATTERN [–list ADDRESS]

Lift a ban.

tasks –once

Run the periodic sweeps once and exit: the bounce warn-and-remove passes, the eviction of processed bounce events ([pepsi-list] BOUNCE_EVENT_RETENTION), the max_days_to_hold expiry of held messages, the sweeps of expired confirmation tokens and purge tombstones, and the archive retention sweep. This is what the pepsi-list-tasks.timer unit runs daily. --once is required: without it the command refuses, because the schedule belongs to the timer and not to a loop in this program.

Everything in the subsystem that happens on a schedule is here, and each part is invisible when the timer does not run. The bounce stage disables a member whose bounce score crossed the threshold, but the warnings and the eventual unsubscription come from this sweep. A deployment without the sweep therefore disables members and then neither warns nor removes them.

Note

tasks --once also discards held messages older than their list’s max_days_to_hold. That attribute is inert in GNU Mailman 3 and active here: a grep of upstream’s whole non-test tree finds the column, the style default and the REST validator, but no job that reads it. It is a strict superset: the default 0 means “never expire”, which is what happens upstream, so nothing changes for a list that leaves it alone. Each discard is written to the audit log. The owners and moderators get one notice per list and sweep through [pepsi-list] NOTICE_STAGE, with the count and the sender and subject of each post, because only they can still tell the senders. A list whose moderators cannot be told keeps its posts, with a warning in the log: this needs NOTICE_STAGE and at least one owner or moderator.

Note

The archive retention sweep deletes each list’s archived messages older than its retention, exactly as pepsi-archive expire would (threads and counters are repaired afterwards). The retention is resolved per list: a list’s own archive_retention_days (set with list set-ext) wins, 0 included; a list without one uses [pepsi-list] ARCHIVE_RETENTION, whose own default is 0; and 0 keeps everything. A stored value that is not a day count is skipped with a warning rather than guessed at.

check [–strict]

Validate every list’s configuration and the installation’s. Each finding is an error, a warning or a note, with a remedy where there is an obvious one. Exits non-zero on an error, or on a warning too with --strict.

import mailman3 –rest URL [–user NAME] [–password-file FILE] [–dry-run] [–report FILE]

Import a running GNU Mailman 3 site over its own REST API: domains, lists attribute for attribute, users, addresses (keeping their verified_on), members, bans, header matches and template URIs.

REST is the default path rather than the fallback, because it works against a running site, needs no knowledge of upstream’s schema, and is insulated from upstream’s own migrations. It is also what makes the import checkable: both ends answer /3.1/lists/<id>/config, so a per-attribute comparison is possible.

Prefer --password-file to --password: an argument is visible to every other process on the host.

import mailman21 –listdir DIR [–list NAME] [–domain HOST] [–dry-run] [–report FILE]

Import a Mailman 2.1 site from its lists/ directory, reading each config.pck. The attribute mapping is GNU Mailman’s own: the eighteen renames, the integer-to-enum tables, the two boolean collapses, the DMARC precedence rule and the user_options bitfield are transcribed from upstream’s utilities/importer.py.

Both importers set hold_dmarc_failures to no on every list they create (never on one that already exists), because GNU Mailman holds no post for failing DMARC.

--domain supplies the mail host for a pickle that names none.

Note

Started as root, pepsi-list becomes the ``pepsi`` service user before it connects to the database (it is not setuid, and must not be made so), so a --report path must be one that this user can write. --report /root/import.txt fails with a permission error and produces no report; /tmp or a directory owned by pepsi works. Without --report the full report goes to standard output, so it is never lost.

import mailman21 –json FILE [–list NAME]

The escape hatch. This reader constructs nothing: it looks up no module and calls no callable, because a config.pck comes from somebody else’s server. The price is that the reader is strict. When it refuses a file, run contrib/mm21-export.py on the old server with that server’s own Python and feed the JSON here. Both routes go through the same mapping.

invite send [–list ADDRESS] [–rate N] [–limit N] [–dry-run]

Invite everybody with no password to choose one. import deliberately sends nothing, so with this verb you decide when the largest mailing that the server will ever send actually goes out.

--rate defaults to [pepsi-list] INVITE_RATE (300/hour). --dry-run prints the per-domain histogram, which is a deliverability tool rather than a progress bar: whether one provider holds half of a migrated site is what decides the rate. The run resumes where it stopped and skips every address with a live invitation. Invitations last four weeks, because an invitation competes with a holiday rather than with an evening.

invite status [–list ADDRESS]

How many accounts have a password, how many have an outstanding invitation, and how many are still to invite. Nobody is ever unsubscribed for not answering.

digest send ADDRESS

Send one list’s accumulated digest now, whatever digest_size_threshold says. An operator who asks for an issue has already decided.

digest bump ADDRESS

Advance the volume and reset the number to 1 without sending, which is upstream’s --bump. An owner does this at the start of a year, or after an issue that went out wrong.

digest periodic

Send every list whose digest is due: over digest_size_threshold, or accumulated at all when digest_send_periodic is on. This is the timer’s verb: the pepsi-list-digest.timer unit runs it hourly. Upstream has no periodic runner for digests at all: maybe_send_digest_now fires synchronously on a post once the threshold is crossed, and the operator is meant to run mailman digests --send from cron. Pepsi keeps both halves and puts the second on a systemd timer, the same arrangement as for tasks --once.

digest status [ADDRESS]

What each list has accumulated, and whether it is due.

template list

Show the 28 notice-template names, and any URI overrides set for them at the site, domain or list level.

template set NAME URI [–list ADDRESS | –domain DOMAIN] [–user USER] [–password PASSWORD]

Point one template at a URI: mailman: (the built-in text) or an http(s): URL, optionally with HTTP Basic credentials. Without --list or --domain the override applies to the whole site. An http(s): URI is fetched by the server, under the limits in [pepsi-list] TEMPLATE_FETCH_TIMEOUT, TEMPLATE_FETCH_MAX_BYTES and TEMPLATE_FETCH_ALLOW_INTERNAL. A file: URI is refused, here as on every other write path (the REST API and the importers), because it would make the server read its own disk; a file: row already in the table is never read, and the notice is rendered as if it were absent.

template clear NAME [–list ADDRESS | –domain DOMAIN]

Remove an override, so the built-in text is used again.

unsub-token LIST EMAIL [–serial N] [–uri]

Print the RFC 8058 one-click unsubscribe token that a delivery to EMAIL would carry. The same code that mints the token for deliveries computes it here; --uri prints the whole https: URI instead. --serial signs a given subscription serial rather than the member’s current one. Needs [pepsi-list] UNSUBSCRIBE_SECRET. It discloses nothing that the member does not already have in their mailbox.

95.1.53.1.7. Reserved verbs

Three verbs are listed by --help and answer with where the capability actually is, rather than with “unknown subcommand”:

held, requests

Moderating held messages and subscription requests is not implemented on the command line. It is in the owner console at /lists/<list-id>/admin (pepsi-httpd(1)), over the REST API, and by mail to list-request.

inject

Not implemented, and not needed: post to the list’s own address with pepsi-sendmail(1) or any SMTP client. A message injected by a verb here and a message posted normally would go through the same stage anyway.

These answer before any database connection is attempted, so the answer is the same on a host that has no list schema.

95.1.53.1.8. Styles

A style is a named set of attribute defaults applied when a list is created. GET /3.0/lists/styles reports three, which are upstream’s, with upstream’s names and descriptions:

legacy-default

Ordinary discussion mailing list style. The default. Also accepted as discussion or default.

legacy-announce

Announce only mailing list style — members may not post. Also accepted as announce-only or announce.

private-default

Discussion mailing list style with private archives; not advertised, and subscriptions need both a confirmation and a moderator. Also accepted as private.

One further style is Pepsi’s own and is accepted here but not listed over REST, for the same reason as the Pepsi-only per-list settings:

moderated

A discussion list on which every member’s post is held for a moderator.

95.1.53.1.9. Exit status

0 on success, non-zero on failure. check exits non-zero when it found an error (or, with --strict, a warning). import exits non-zero when anything was not carried over, even though everything else was written.

95.1.53.1.10. Files

/etc/pepsi/pepsi.conf

Configuration; see pepsi.conf(5), section [pepsi-list].

95.1.53.1.11. See also

pepsi.conf(5), pepsi-setup(1), pepsi-settings(1), pepsi-whitelist(1).

GNU Mailman 3’s documentation at <https://docs.mailman3.org>.