14. The administrative API¶
pepsi-httpd serves an HTTP API under /api/v1 covering everything the
operator command-line tools do: the queue, the health summary, the
configuration, the key store and the logs. It exists so that a browser console
and an unattended script are clients of the same documented surface, rather
than three implementations that drift.
Warning
The API is unstable until Pepsi’s first release. The path carries v1
so the versioning mechanism exists and clients are written against it, but
until there is a tagged release the shape of a request or a response may
change without a migration path — the same disclaimer this manual already
makes about the on-disk and database formats. Freezing the contract before
the console, its first real client, has exercised it would lock in decisions
taken blind; tightening the promise later costs nothing.
14.1. Where it is served¶
Only on a listener flagged ADMIN = yes. On every other listener the
/api/v1 routes answer a plain 404 that is byte-for-byte what any unknown
path gets — so publishing the public HTTPS listener does not publish
administration, and a public listener cannot even be probed for whether
administration is enabled on this deployment.
The shipped configuration flags exactly one listener, a UNIX socket:
[pepsi-httpd-listener-admin]
SERVE = unix
UNIXPATH = /run/pepsi/admin.sock
UNIXPATH_MODE = 660
MODE = plain
ADMIN = YES
pepsi-httpd refuses to serve the administrative routes on a flagged
listener that would carry them in cleartext off this host: plaintext TCP is
accepted only on a loopback address, TLS and UNIX sockets always. A flagged
listener that fails the test logs a warning at start-up and serves the public
endpoints only; pepsi-setup reports the same thing when it validates the
configuration, so the mistake is caught before it matters.
Note
One binary serves both audiences. There is no separate administrative daemon: one thing to build, package, supervise and keep in sync — and, in exchange, a bug in the shared server process is a bug in both surfaces. The isolation is therefore the operator’s to configure, not structural. Bind the administrative listener to a UNIX socket or to loopback, and put a reverse proxy in front of it if it must be reachable from elsewhere.
14.2. Authentication¶
Three mechanisms, one identity model. An explicitly presented credential always
wins: the Authorization header is tried first, then the session cookie, and
SO_PEERCRED only when neither is present. That order matters — if peercred
won, an administrator who deliberately presented a narrowly-scoped token over
the local socket would silently get full authority back.
14.2.1. Local administrators (SO_PEERCRED)¶
A process connecting over a UNIX-socket listener is identified by the
credentials the kernel recorded at connect(2), which the peer cannot forge.
root, or a member of [pepsi-admin] ADMIN_GROUP (default
pepsi-admin), is an administrator with no credential to configure, store
or lose:
# curl --unix-socket /run/pepsi/admin.sock http://localhost/api/v1/status
This is how first-run bootstrap works on a system that has no accounts yet, and
it is why the socket may safely be mode 0660 with a group: the identity
comes from the kernel, not from the file mode.
14.2.2. Bearer tokens¶
For automation. A token is minted through the API and shown exactly once:
# curl --unix-socket /run/pepsi/admin.sock -X POST \
-H 'Content-Type: application/json' \
-d '{"label":"monitoring","scopes":["queue:read","logs:read"]}' \
http://localhost/api/v1/tokens
The presented form is pepsi_<id>.<secret>. Only the SHA-256 of the secret
half is stored, so a token cannot be re-displayed or recovered from a database
backup; the id half is a public selector that indexes the table, which is what
lets the comparison be a constant-time check against one row rather than a scan.
Present it as Authorization: Bearer pepsi_<id>.<secret> — the Bearer
scheme of RFC 6750, carried in the Authorization field RFC 9110 §11.6.2
defines. Scheme names are case-insensitive (RFC 7235 §2.1), so bearer and
BEARER are accepted too. Tokens may carry an expiry (expires_in_days)
and are revoked by deleting them.
The token is not an OAuth 2.0 (RFC 6749) access token and there is no authorization server: it is a locally minted credential that happens to use the same wire form. Pepsi does speak OAuth 2.0 elsewhere — outbound to a smarthost, see pepsi-helper-token-refresh — and the two should not be confused.
14.2.3. Passwords and sessions¶
For a browser. Accounts live in pepsi.admin_account with Argon2id (RFC 9106)
password hashes; POST /api/v1/auth/login exchanges a name and password for a
session cookie (RFC 6265) plus a CSRF token, and every mutating request from a
session must repeat that token in the X-Pepsi-Csrf header. A bearer token
and a local peer need no CSRF token: neither is ever sent by a browser on
somebody else’s behalf.
The cookie carries HttpOnly, SameSite=Strict and the __Host- name
prefix. That prefix is the strongest of the three: the browser refuses the
cookie unless it was set over HTTPS with no Domain attribute and
Path=/, so a sibling host on the same registrable domain cannot plant a
session cookie for this one.
Those requirements are also why the secure-link portal, sharing an origin,
cannot use the same prefix: __Host- mandates Path=/, which would send
the portal’s session cookie to /metrics, /resume and the Web Key
Directory as well. It keeps its path scoping, uses __Secure- instead, and
buys back what the prefix would have given by sealing the cookie’s value under
the server pepper — see The secure-link fallback portal.
Sessions are rows, not process memory, so a restart does not log everyone out and two server processes agree about who is logged in. They expire twice over:
an idle timeout (
SESSION_IDLE, default 30 minutes) pushed forward on every request, anda hard lifetime (
SESSION_LIFETIME, default 12 hours) that is never extended.
There is no “remember me”. This console can revoke a key, read who corresponds with whom and rewrite the pipeline; a forgotten browser tab should not still hold that tomorrow, and a stolen laptop should not be a standing administrative session. The policy is affordable precisely because the people who administer a deployment most often — local operators — never type a password at all.
Note
PAM is deliberately not supported. It would put a privileged authentication path and system-account semantics inside a mail server’s web tier, and make the API’s security depend on host configuration Pepsi does not control. A local administrator does not need it; a remote one gets an account in the database with an explicit scope list.
14.4. Conventions¶
JSON only. Requests and responses are
application/json.One error shape, on every failure:
{"code": "scope_required", "hint": "this endpoint requires the 'queue:write' scope", "detail": {"scope": "queue:write"}}
codeis a stable token to switch on;hintis prose for a human and may change;detailcarries structured extra information when there is any.Uniform authentication failures. A wrong password, an unknown account and a disabled account all produce the same
invalid_credentials, and an unknown account still pays for one password verification, so neither the message nor the response time enumerates accounts.Internal failures never carry a cause. The full error goes to the process log; the client gets
internal_error.Listings answer
{"items": [...], "total": N, "limit": L, "offset": O}, and every one of them means the same thing by it. See Pagination.ISO-8601 timestamps, with an offset.
No trailing slashes.
14.5. Pagination¶
Ten endpoints answer the listing envelope, and all ten behave identically:
Endpoint |
Filters that also apply to |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
— |
|
— |
|
— |
|
— |
limit defaults to [pepsi-admin] PAGE_LIMIT (100) and is clamped to
1–1000; offset is clamped to be non-negative; a value that is not a number is
a 400. The envelope echoes the window that was applied, so a request for
limit=100000 comes back saying "limit": 1000. Both go into the SQL, and
total is a COUNT(*) over the same WHERE clause as the page — never
the length of items, and never the size of an over-fetched window. Every
listing orders by something unique last (its row id), so an OFFSET can
neither skip a row nor show one twice.
GET /api/v1/mail-log adds one member to the envelope, mode, because an
empty page means something different depending on whether [pepsi] MAIL_LOG is
recording anything at all.
14.5.1. total and the page are two queries¶
The count runs first, then the page, and nothing holds a snapshot across the two. A read-only transaction per listing would make them exactly consistent, at the cost of tying up a pooled database connection on a web tier for a guarantee no client needs. So, against a table being written concurrently:
rows deleted in between make
totalan over-estimate — and the queue is drained continuously, so this is the normal case there, not a curiosity. The visible effect is a last page that comes back empty.rows inserted in between make it an under-estimate.
Page until items is shorter than limit; treat total as a good number
to show an operator, not as a loop bound to trust to the row. What total is
no longer is a lower bound that only becomes true on the last page, which is what
the queue and secure-message listings reported when they over-fetched and sliced.
The count is a real COUNT(*), not an estimate, on every one of these tables.
That is affordable because of what they are: pepsi.ingress is the live
backlog — a delivered message’s row is deleted by the stage that delivered it,
so the table holds mail still in flight rather than a history of everything ever
sent — and the key store, the accounts and the tokens are deployment-scale
objects. event_log and mail_log are the two that do grow with traffic, and
they have counted themselves this way since they were written; both are bounded
by the retention sweeps described under The audit log and
The mail log (off by default).
14.5.2. Not this shape¶
GET /api/v1/status takes a limit — how many stuck messages the report
lists — but no offset, and answers pepsi-status’s report rather than a
listing. GET /api/v1/tls-sessions and GET /api/v1/dns-cache answer a bare
{"items": [...]}, bounded by the days parameter (1–365, default from
pepsi-status) and by the size of the DNS failure cache respectively.
14.6. Endpoints¶
The authoritative list for a given build is GET /api/v1/openapi.json, which
is generated from the server’s own route table — including the limit and
offset parameters, which it declares on exactly the endpoints listed under
Pagination. The Notes column below gives each listing’s other parameters;
every listing additionally takes ?limit= and ?offset=.
Method |
Path |
Scope |
Notes |
|---|---|---|---|
|
|
— |
Returns a session cookie and a CSRF token. |
|
|
— |
Idempotent. |
|
|
— |
The first thing to check when something answers |
|
|
|
Password hashes are never returned. |
|
|
|
Deleting an account ends its sessions. |
|
|
|
The secret is in the |
|
|
|
Revocation. |
|
|
|
Identical to |
|
|
|
|
|
|
|
Envelope, stage, status and |
|
|
|
Pending again at its current stage. |
|
|
|
Body |
|
|
|
Deletes the message. |
|
|
|
The audit log. |
|
|
|
|
|
|
|
|
|
|
|
MX addresses currently failing. |
|
|
|
|
|
|
|
One section. |
|
|
|
Validated before it is stored. See below. |
|
|
|
Dry run; answers |
|
|
|
|
|
|
|
|
|
|
|
The public half, base64. |
|
|
|
Refused — see below. |
|
|
|
|
|
|
|
Import a correspondent’s key by hand. See below. |
|
|
|
Queue a discovery lookup; it does not fetch. See below. |
|
|
|
Forget a cached key. |
|
|
|
The S/MIME trust anchors. |
|
|
|
Changes which inbound signatures validate. |
|
|
|
Outstanding secure-link messages, metadata only. |
|
|
|
One message with its access history. Never its content. |
|
|
|
Revoke; this destroys the only copy of the message. |
|
|
|
The records to publish and the last live verdict. See below. |
|
|
|
The setup interview as data. |
|
|
|
The staged answers; credentials masked. |
|
|
|
Merge answers into the staged (draft) set. |
|
|
|
Abandon the staged interview. |
|
|
|
Privileged setup tasks, newest first. |
|
|
|
Ask the applier for one of its closed set of actions. |
|
|
|
One task, with |
|
|
|
Check the environment (ports, resolver DNSSEC). |
|
|
|
Compare live DNS against what setup would publish. |
|
|
|
Obtain the certificates the configuration needs. |
|
|
— |
Generated from the route table. |
The monitoring endpoints call pepsi-status’s and pepsi-queue’s own
functions and serialise the same structs, so pepsi-status --json and
GET /api/v1/status are the same bytes by construction. The configuration
endpoints reuse pepsi-config’s provenance walk and its validation.
14.7. Two things this server deliberately cannot do¶
14.7.1. Generate key material¶
POST /api/v1/identities answers 501 private_material_not_reachable.
Private key material lives in crypto_identity.private_wrapped, which
pepsi-setup grants to the pepsi-crypto role alone and revokes from
every other account, pepsi-httpd included: a web tier that can mint signing
keys is a web tier whose compromise mints signing keys. Use pepsi-keys
identity generate. The endpoint exists, and reports the boundary, so a client
discovers it from the API rather than from a 404 it cannot tell from a typo.
14.7.2. Write the configuration, unless you ask for it¶
Writing pepsi.config_override belongs to the pepsi-config database role,
which pepsi-setup grants and explicitly revokes from every service account —
a component that processes mail must not be able to rewrite the pipeline it runs
in. pepsi-httpd is such an account, and peer authentication keys off the
effective uid, so its own connection cannot be that role.
PUT and DELETE on /api/v1/config therefore answer 503
config_write_unavailable until [pepsi-admin] CONFIG_DB names a connection
that authenticates as pepsi-config — in practice a password in a
secrets.d fragment readable only by pepsi-httpd, or a pg_ident map.
The server checks with SELECT current_user at start-up and refuses anything
else, because a boundary the code merely believes in is not a boundary.
Sections that are read from the configuration file only ([pepsi],
[pepsi-postgres], [paths], the HTTP(S) and ingress listener sections)
are refused with 403 regardless.
14.7.3. Do anything privileged¶
Setup writes /etc/pepsi/pepsi.conf, hands each secrets.d fragment to the
one account that reads it, runs certbot, creates database roles and generates key
material — all as root. pepsi-httpd drops privileges before it accepts a
connection and must never be able to regain them.
So the setup endpoints do not act. They write an intent row into
pepsi.setup_task, and a separate root program, pepsi-setup apply, drains
it. Root is reached through a database table, never through a socket that speaks
a protocol. The full trust model — the closed task list, the two admission gates,
what is deliberately absent, and what the design cannot promise — is in
pepsi-setup(1), “The applier’s trust model”, and is required reading before
deploying the browser setup.
14.8. Correspondent keys¶
POST /api/v1/peers stores a correspondent’s public key by hand — the browser
console’s “import” and pepsi-keys peer import’s sibling. The body names the
address, the protocol (openpgp or smime), the material
(base64, or armored/PEM text verbatim) and an optional pin.
Two refusals are the point of it:
Material that is not a key is a
400. OpenPGP input goes through the hardened parser the discovery methods use, which walks the top level only and rejects a stream carrying a compressed or encrypted packet — a transferable public key never contains one, and a compression bomb stapled to a keyring is not a key worth having.A key that names a different address is a
400naming whose key it really is. Filing one correspondent’s key under another’s address silently misdirects every future message to them. A key that names no address is stored — silence cannot contradict anything — and the response says so in itsbindingfield (matched/no-identity).
The row is stored with source = api, which the conflict rule ranks beside a
hand-entered key: it displaces a discovered one and does not displace a
pinned one. A *@domain entry is refused here; the schema admits one only
for source = manual, so widening a whole domain stays a decision taken at a
terminal.
POST /api/v1/peers/discover queues a lookup and returns at once. It does
not fetch: discovery speaks HTTPS, LDAP and DNS to hosts the correspondent’s
domain chooses, and doing that inside a request handler would let a stranger hold
this server’s connection open — the same reason the encrypt and decrypt stages
park a message rather than looking a key up themselves. The endpoint writes the
same pepsi.key_request row a stage’s park writes, through the same SQL
function, and a pepsi-keydisc service does the work. The answer appears in
GET /api/v1/peers when one has stored it. {"force": true} re-asks an
address whose fresh negative cache entry says there is no key.
14.9. Secure links¶
GET /api/v1/secure-messages lists the messages the secure-link portal is
holding: who they are from and for, when they were created and when they expire,
how many times the PIN was entered correctly and wrongly, whether a lockout is in
force, and how many bytes are stored. GET /api/v1/secure-messages/{token}
adds the access log — the timestamp, the peer address and the outcome of each
attempt. There is deliberately no geolocation and no user agent.
No endpoint returns the message. pepsi-httpd is the portal, so it is
the one database role granted secure_message.ciphertext — which is exactly
why nothing on the operator surface reads it. The queries behind these endpoints
name every column except that one and report only length(ciphertext), so the
property is enforced by the SQL rather than by a field somebody remembered to
leave out. Nor would reading it help: the key is Argon2id over a PIN the server
never sees, salted per row and peppered from a file only the portal reads.
DELETE /api/v1/secure-messages/{token} revokes. It deletes the row, and with
it the only copy of the message — there is no escrow key and the portal keeps no
second copy — so it needs secure:write rather than secure:read. The
sender re-sends if it was a mistake.
14.10. DNS records¶
GET /api/v1/dns-records answers with the records this deployment should
publish, the last live-DNS verdict for each (ok / missing / mismatch
/ lookup-failed), the remedy for each that is wrong, and the zone text
pepsi-setup run would print.
It serves a stored answer rather than computing one. Deriving what should be
published means reading the validated configuration and the DKIM key directory,
and comparing it means querying DNS — work that belongs to pepsi-setup, which
this server cannot call (the dependency already runs the other way) and whose
inputs a web tier should not be reading. So the privileged applier computes it as
part of a run-preflight task and writes it back, and this endpoint reports it
with the time it was computed, always: a verdict with no age on it invites
trust in a stale one. POST /api/v1/setup/dns-check asks for a fresh one, and
needs setup:write accordingly — looking at what was found is a read, asking a
root process to go and look is not.
Before any check has run, the response is empty and names the endpoint that runs one, rather than an empty list that reads like “everything is fine”.
14.11. Online setup¶
The browser path through pepsi-setup. Two mechanisms, deliberately different:
Answers are staged, not applied. PUT /api/v1/setup/answers merges into a
set of draft rows in pepsi.config_override, which every configuration
reader filters out structurally (WHERE NOT draft). Nothing takes effect while
the interview is in progress, so a session that times out half way through has
not half-configured a mail server, and resuming is reading the drafts back. The
identifiers are the ones GET /api/v1/setup/questions publishes, which are
also the keys pepsi-setup --answers reads and the [pepsi-wizard] section
round-trips — so an interview can be started in a browser and finished at a
terminal.
Actions are requested, not performed. POST /api/v1/setup/tasks enqueues
one of seven kinds (write-config, write-secret, obtain-certificate,
install-schema, provision-roles, generate-keys, run-preflight)
with strictly validated parameters. There is no arbitrary-command kind, and
there is no restart, reload or shutdown kind — a change that needs a
component restarted ends with the operator restarting it, which the console says
rather than hiding. The applier refuses anything else, and every refusal,
success and failure is an audit record.
Enqueuing goes through the [pepsi-admin] CONFIG_DB connection, exactly as
configuration writes do and for the same reason: only the pepsi-config role
may INSERT into setup_task, and the applier refuses any row whose
written_by says otherwise. Without that connection the setup endpoints answer
503 setup_write_unavailable; the server’s own account may only SELECT the
queue, so the console can watch privileged work without being able to ask for
it.
GET /api/v1/setup/tasks/{id}?since=<seq> returns the progress lines the
applier has written so far, so a certbot run or a schema install can be watched
line by line without the applier holding an HTTP connection open.
14.12. Hardening¶
Rate limits, per source address (
RATE_LIMIT, default 120/minute) and, on the login path, per account and per source address (LOGIN_RATE_LIMIT, default 10/minute) — many passwords against one account and one password against many accounts are different attacks.Body caps (
MAX_BODY, default 1 MiB).Constant-time comparison of every token and CSRF digest.
No secret is ever returned by
GET /api/v1/config: a value that could be one is replaced by"***"with"secret": true. The masking is deliberately generous — a value wrongly masked costs a look in the file, a value wrongly shown is published to everyone holdingconfig:read.No message content is reachable. The queue endpoints answer the envelope, the stage, the status and the
stateJSON, neverheadersorbody. Reading mail is not an administrative function.Responses carry
X-Content-Type-Options: nosniffand aframe-ancestors 'none'policy.
14.13. The audit log¶
Every configuration change, key operation, login, failed login, account and
token change and administrative queue action is recorded in pepsi.event_log
and readable through GET /api/v1/events:
event_id at actor kind subject severity detail
actor names the principal (peer:root, token:monitoring,
session:alice) or, for a change made at a terminal, the invoking login
(cli:alice). The operator command-line tools write to the same log, so
it is complete regardless of which surface acted; a log that noticed only what
happened over HTTP would invite the wrong conclusion from an absence.
The log is append-only for everything that processes mail: pepsi-setup
grants those roles INSERT and SELECT and revokes UPDATE/DELETE,
then verifies the revocation against the live server. Retention is bounded by
[pepsi-admin] EVENT_RETENTION_DAYS (default 90) and pruned by
pepsi-httpd a few times a day.
Neither a configuration value nor any key material is written to a record: what changed is the auditable fact, not what it changed to.
14.14. The mail log (off by default)¶
Warning
Switching ``[pepsi] MAIL_LOG`` on makes this deployment keep a record of who corresponds with whom. Consider whether you need that evidence, and whether keeping it is lawful where you operate, before you turn it on.
Pepsi deletes a message’s row when the pipeline is finished with it. There is therefore no per-message success log — a property of the design rather than an omission: an ordinary deployment does not accumulate a record of its users’ correspondence, and cannot be compelled to produce one it does not have.
Some deployments genuinely need that evidence. [pepsi] MAIL_LOG provides it,
as a deliberate administrative act with a stated consequence:
offThe default. Nothing is written;
pepsi.mail_logstays empty.summaryOne row as the message leaves the pipeline: the envelope sender and recipients, the direction (
outboundfor locally-submitted mail,inboundotherwise), the stage it ended at, the outcome, and what the pipeline decided about it (the authentication verdict, the spam/paid decision, any next-hop failure detail).fullThe above plus the
Subject:line.
Message content is never recorded at any setting. Rows are readable through
GET /api/v1/mail-log by a principal holding logs:read (or, for one
address, own:<address>), are append-only for the mail-processing accounts
exactly as the audit log is, and are pruned after
[pepsi-admin] MAIL_LOG_RETENTION_DAYS days (default 30).
A row is written when a stage finishes with a message (completed) and when a
message fails permanently (failed) — the two ways it stops moving. The write
rides on the same statement as the terminal that removes or fails the row, so
enabling the log costs no extra database round-trip.
14.15. Bootstrapping¶
A fresh install has no accounts, and does not need one: the UNIX socket plus
SO_PEERCRED always works for a local administrator. From there,
# curl --unix-socket /run/pepsi/admin.sock -X POST \
-H 'Content-Type: application/json' \
-d '{"login":"alice","password":"…","scopes":["queue:read","queue:write"]}' \
http://localhost/api/v1/accounts
creates the first remote account. A lost password is not a lockout: the local socket is still there.
When the socket is not reachable — an operator working from another machine, or a
container without a shell on the host — pepsi-setup bootstrap, run as root,
prints a bearer token that holds setup:write and nothing else, expires within
the hour, and is accepted exactly once: enough for one
POST /api/v1/accounts.
Single use because of where it is printed. A token on a terminal is in a scrollback buffer and a token in the journal is readable by whoever can read the journal, which is usually a wider set of people than “may administer the mail server”. It is consumed on presentation by an update only one racing request can win, so a token two people read admits one of them. Only its digest is stored, so it cannot be recovered — run the command again, which also revokes the previous one.
14.16. The browser console¶
The administration console is a client of this API, mounted under /ui on the same
ADMIN = yes listeners, sharing its authentication, its scopes and its audit
log. It adds no capability: everything it does is reachable with curl here.
Two details of the session mechanism exist for it, and are worth knowing when
writing another client:
POST /api/v1/auth/loginreturns the CSRF token in the response body and sets it as apepsi_csrfcookie (HttpOnly,SameSite=Strict). A program reads the body; the console has no other way to get the token into a form, because it ships no script to read one out of the page. Neither placement is the check — the check is that the digest of what a mutating request presents equals the digest stored with the session.A mutating request may present the token in the
X-Pepsi-Csrfheader (what a program does) or as a_csrfform field (what an HTML form does). A bearer token and a UNIX-socket peer are asked for neither, since a browser never sends them on somebody else’s behalf.