19. The GNU Mailman 3 REST API¶

pepsi-httpd serves the GNU Mailman 3 REST API under /3.0/ and /3.1/. This is not an API of Pepsi’s own design: it is somebody else’s interface, reimplemented so that software written for GNU Mailman 3 works against Pepsi unchanged. mailmanclient, Postorius and HyperKitty are the clients it exists for.

The version implemented is 3.3.10. That precision matters: “Mailman 3” is not a contract, because the attribute set drifts between point releases while the /3.1/ path does not. The interface is documented by the GNU Mailman project at https://docs.mailman3.org/ and that documentation — not this chapter — is the specification. What follows is what an operator needs in order to run it, plus the three places where Pepsi has to answer something honest rather than copy something it does not have.

Note

Attribution. The resource tree, the attribute names, the enumeration values, the wire conventions and the error messages are GNU Mailman’s, copyright the Free Software Foundation, and are reproduced here in order to be compatible with them. See vendor/PEPSI-VENDORING.md. Pepsi is not GNU Mailman and is not endorsed by the GNU Mailman project.

This chapter is about the interface. What the resources mean — a list, a member, a moderation action, a digest — is Mailing lists, whose How the subsystem is put together shows where this API sits relative to the operator’s /api/v1 and the public member pages; the archive’s own resources are Archives. Turning the listener on, and the credentials it needs, is Installation; the browser consoles that are clients of the same model are The administration console; and the list of what Pepsi implements at all is Supported Features.

19.1. There are two APIs in this tree¶

An operator will meet both and they have nothing in common but the server:

/api/v1: Pepsi’s own administrative API (The administrative API). Its audience is the operator: the queue, the key store, the logs, the configuration. It has accounts, scopes, sessions, bearer tokens and CSRF, its collections are {items, total, limit, offset} and its errors are {code, hint, detail}.
/3.0/ and /3.1/: This one. Its audience is mailing-list software. It has one shared password and no scopes, its collections are {start, total_size, entries} and its errors are Falcon’s {title, description}.

They are deliberately not unified. The whole value of this one is that its shapes are not ours to improve.

19.2. Where it is served¶

Only on a listener flagged LIST_API = yes. On every other listener the /3.0/ and /3.1/ routes answer a plain 404, byte for byte what any unknown path gets, so a public listener cannot be probed for whether the API is enabled on this deployment.

The same rule the administrative API follows applies here, and for the same reason — the credential is one static password, so it must not cross a cleartext network:

a TLS listener is always allowed;
a UNIX-socket listener is always allowed (it never leaves the host);
a plaintext TCP listener is allowed only on a loopback address;
a systemd-activated socket in cleartext is refused, because the server cannot see where systemd bound it.

A listener that asks for the flag but is bound somewhere else does not serve the API, and pepsi-httpd says so loudly at start-up rather than failing quietly.

[pepsi-httpd-listener-mailman]
# What every existing mailman.cfg-shaped client already points at.
BIND_TO = 127.0.0.1:8001
MODE = plain
LIST_API = yes

[pepsi-list]
API_USER = restadmin
API_PASS = a-long-random-string

Warning

API_USER and API_PASS are one shared credential for the whole site. Anyone who holds it can read and change every list, every subscription and every user record. That is upstream’s model, not a simplification of it; keep the listener on loopback or behind TLS, and do not reuse the password anywhere else. pepsi-httpd refuses to start when a listener asks for the API and the credential is incomplete: an API whose whole authorisation model is that password, serving without one, is not a degraded mode.

19.3. Authentication¶

HTTP Basic, against that one pair, compared in constant time. There are no sessions, no CSRF tokens, no per-user accounts and no token minting: a client sends Authorization: Basic ... on every request. A failure is 401 with

WWW-Authenticate: Basic realm="mailman3-rest",charset="utf-8"

and that realm string is upstream’s, because a client may display it.

Pepsi adds a rate limiter, which upstream has none of. It is a superset and cannot break a correct client, but a compatibility suite fires hundreds of requests in seconds, so the default is deliberately high — 3000 requests per minute per source address, settable with [pepsi-list] API_RATE_LIMIT. A 429 from this API is Pepsi’s, not Mailman’s.

19.4. The two versions differ in four ways¶

Not one, which is the usual mistake:

the six *_uri template attributes are list attributes in /3.0/ and absent in /3.1/, which reaches them through /uris instead;
/templates/... exists only in /3.0/; /uris/..., /lists/<id>/uris, /domains/<host>/uris and /plugins exist only in /3.1/. Each is a 404 on the other version;
identifiers are serialised differently. /3.0/ emits a UUID as uuid.int, a decimal integer of up to thirty-nine digits; /3.1/ emits uuid.hex, a string. This reaches member_id, user_id and every self_link built from them, and it is why /3.1/ exists at all — the integer form is not representable in every JavaScript runtime;
every self_link and Location carries its own version prefix.

19.5. Template URIs¶

A template override written through either version — a *_uri attribute in /3.0/, or /uris in /3.1/ — is a URI the server fetches, so whoever holds the REST credential chooses what the server reads. Pepsi accepts mailman: (the built-in text) and http(s):, fetched under the limits in [pepsi-list] TEMPLATE_FETCH_*, and answers 400 to a file: URI, which upstream would read from its own disk. A file: row that is in the table anyway (written by hand) is never read: the notice is rendered as if the row were absent.

19.6. Three honest analogues¶

Three resources describe things GNU Mailman has and Pepsi does not. None of them is faked.

/queues: Upstream’s queues are directories of .pck files. Pepsi has one table: a message in flight is a row in pepsi.workqueue at some stage. So this resource answers with the real counts — in is every message still in flight, retry those paused for a later attempt, bad and shunt the two terminal states the dispatcher never re-queues — with the workqueue_id standing in for a filebase and the table’s name where a directory would be. POST /queues/<name> injects a message at the list’s posting stage and DELETE /queues/<name>/<id> removes one row. pepsi-queue(1) remains the real tool.
/plugins: Always an empty collection. Pepsi has no plugin interface and no pluggable archivers. /3.0/ has no such resource.
/reserved: Upstream’s own test hook, which upstream marks as not part of the stable API. GET /reserved/reset truncates every list and archive table, so it exists only on a listener that explicitly opted in with LIST_API_TESTING = yes; anywhere else it is a 404 indistinguishable from an unknown path, and a test asserts that default. It is there because the compatibility suites reset state between test classes in-process, which an out-of-process server cannot offer. DELETE /reserved/uids/orphans is a successful no-op: identifiers here are v4 UUIDs and there is no orphan table to cull.

19.7. What needs configuring beyond the credential¶

[pepsi-list] RELEASE_STAGE: The [stage-<name>] section running pepsi-stage-list-post. A moderator accepting a held message, and POST /queues/<name>, both put a message back into the pipeline there, and only the deployment knows what that section is called. Without it those two operations answer 400 and say so — which beats answering 204 and dropping the message, because a moderator who accepts a post and watches it never arrive cannot tell that from a delivery failure.
[pepsi-list] DEFAULT_LANGUAGE: The bottom of the preference chain, reported by GET /<api>/system/preferences. Defaults to en.

19.8. A worked example¶

Keep the credential out of the command line — on a shared machine it is in every process listing and in the shell history — by putting it in a file and letting curl read it:

$ cat > ~/.mailman-netrc <<'EOF'
machine localhost login restadmin password a-long-random-string
EOF
$ chmod 600 ~/.mailman-netrc
$ alias mmcurl='curl -sS --netrc-file ~/.mailman-netrc'

$ mmcurl http://localhost:8001/3.1/system/versions
{"mailman_version":"GNU Mailman 3.3.10 (Tom Sawyer; implemented by Pepsi …)",
 "python_version":"…","api_version":"3.1",
 "self_link":"http://localhost:8001/3.1/system/versions",
 "http_etag":"\"…\""}

$ mmcurl -X POST --data 'mail_host=lists.example.com' \
    http://localhost:8001/3.1/domains

$ mmcurl -X POST --data 'fqdn_listname=devel@lists.example.com' \
    http://localhost:8001/3.1/lists

$ mmcurl 'http://localhost:8001/3.1/lists?count=10&page=1'

mailman_version is worth a second look. It has to begin with the string a client matches on, and it has to be true; so it names the interface version first and says what is actually running in the parenthesis upstream fills with a release code name. Pepsi does not claim to be GNU Mailman.

POST /lists/<id>/digest: bump advances the volume, and send and periodic both go through the same sender the CLI uses, so an issue sent from a client and one sent from a timer are the same code path and carry the same numbering.

19.9. What is not implemented¶

A rejected held message or subscription is dropped without the rejection notice upstream sends. Accepting, discarding and deferring are complete.

19.10. Testing it¶

tests/22-list-api-test.sh runs upstream’s own test suites against a deployed Pepsi: mailmanclient 3.3.5 (344 doctest statements plus 36 test functions) and Postorius 1.3.13 (30 files, 248 test functions), each with only its server-starting fixture replaced. The replacements live in contrib/compat/ and are versioned beside the pinned requirements, so “unmodified upstream tests” stays literally true of everything except the fixture. See Test Suite.