.. This file is part of PEPSI. Copyright (C) 2026 Pepsi contributors PEPSI is free software; you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. PEPSI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. .. _packages: =============== Debian packages =============== One source package builds four binary packages. Three of the splits are ordinary packaging judgement — a stage nobody should have to download unless they use it, a collector that belongs on somebody else's machine. The fourth is a **security control**, and it is why this chapter exists: it contains the one mechanism by which an HTTP request can cause a privileged change on the host, and nothing else. Remove it and the removal *means* something, because what is gone is the mechanism rather than the button that reaches it. :doc:`installation` covers installing and bootstrapping a deployment. This chapter is about what each package contains, why the line was drawn where it was, and how to use the split deliberately. .. contents:: On this page :local: :depth: 2 The package set =============== .. list-table:: :header-rows: 1 :widths: 28 10 62 * - Package - Arch - What it carries * - ``pepsi`` - ``any`` - The mail server: every program except the two split out below, the SQL schema, the templates, the packaged configuration defaults, the man pages, the info manual, ``/etc/pepsi/pepsi.conf``, the ``tmpfiles.d`` snippet, and every systemd unit except the applier's two and the collector's one. * - ``pepsi-httpd-admin`` - ``all`` - ``pepsi-setup-apply.socket`` and ``pepsi-setup-apply.service``. Nothing else — no binary, no configuration, no ELF object at all. * - ``pepsi-stage-detect-language`` - ``any`` - ``pepsi-stage-detect-language``, the ``pepsi-detect-language`` diagnostic tool (the same executable under a second name), and both man pages. * - ``pepsi-telemetry`` - ``any`` - ``pepsi-telemetry``, its systemd unit, its man page, nginx and Apache site files, and the report-formatting helper script. ``pepsi`` --------- The base package, and on a terminal-administered host the only one you need. It installs the release layout described under *Installing* in :doc:`installation`: the multi-call binary at ``/usr/libexec/pepsi/pepsi`` with one ``/usr/bin`` symlink per folded program, and the standalone programs — the setuid and setgid ones, plus the separately-invoked ones such as ``pepsi-setup`` — as real executables of their own. Alongside them go the SQL patch files ``pepsi-setup`` reads, the public-hoster list, the message templates, the ``config.d`` defaults that are parsed *before* ``pepsi.conf``, all of the man pages, and the GNU info manual. It also claims Debian's mail-transport-agent interface — ``Provides``/``Conflicts``/``Replaces: mail-transport-agent`` — so ``/usr/sbin/sendmail``, ``mailq`` and ``newaliases`` become Pepsi's, and no second MTA can be installed beside it. It ``Depends`` on ``postgresql``, ``adduser`` and ``acl``, and its ``postinst`` creates the service accounts, the gating groups, the runtime directories, the ``dpkg-statoverride`` entries for the privileged binaries and (best effort) the database roles. See *Users, groups and privileges* in :doc:`installation` for what those identities are and why each exists. Everything it ships is installed **disabled**. Bringing the pipeline up is ``systemctl enable --now pepsi.target``, after a configuration file exists and ``pepsi-setup`` has run. .. important:: ``pepsi-setup`` itself is in **this** package, not in ``pepsi-httpd-admin``. That is deliberate and load-bearing: a host with no administrative package installed must still be fully configurable from a terminal. ``pepsi-httpd-admin`` --------------------- Two systemd unit files. That is the entire payload — which is also why it is ``Architecture: all`` and needs no ``${shlibs:Depends}``. Those two files are the privileged half of the browser console, and the rest of this chapter is about them. The dependency direction is the point: ``pepsi-httpd-admin`` ``Depends`` on ``pepsi`` (it is useless alone), while ``pepsi`` merely ``Recommends`` ``pepsi-httpd-admin``. A default installation therefore has it — the console's setup interview works out of the box — and ``apt remove pepsi-httpd-admin`` succeeds without taking the mail server with it. A ``Depends`` would make the lever un-pullable; a ``Suggests`` would turn every default installation into one whose console silently cannot apply anything. The units ship disabled like all the others, and the package's ``postinst`` prints that at install time rather than leaving an operator to infer from a package name what has just become possible. It also **empties the setup task queue** as it installs them — see :ref:`packages-install-clears-queue` for why arriving must not mean executing everything asked for while it was away. ``pepsi-stage-detect-language`` ------------------------------- The language-detection stage, split off because it embeds sizable statistical language-recognition models and most pipelines never load it. ``pepsi`` ``Suggests`` it; the stage ``Enhances`` ``pepsi`` and ``Depends`` on the exact same binary version. Validation carries a check written for exactly this split: when the configured pipeline uses the stage and the binary is not on ``$PATH``, ``pepsi-setup`` warns and names the ``apt install`` line — so the omission is diagnosed before start-up rather than discovered when the dispatcher fails to spawn a worker. The package also carries ``pepsi-detect-language``, the offline diagnostic and tuning tool — a ``$PATH`` symlink to the very same executable, which classifies a message read from a file and prints only what the stage would have recorded. It is here rather than in ``pepsi`` because it *is* this binary: it shares the whole classification path and the language models, so shipping it separately would duplicate exactly the bulk this split exists to avoid. See :doc:`programs/pepsi-stage-detect-language` and :doc:`programs/pepsi-detect-language`. ``pepsi-telemetry`` ------------------- The central collector for Pepsi's anonymous, opt-in feature telemetry — a small HTTP server intended to run on one host behind an existing nginx or Apache, which is why the package also ships a site file for each (in ``sites-available/``, disabled) and ``Recommends`` a front server and certbot. Ordinary Pepsi nodes never need it; what *they* run is the telemetry **client**, which is part of the ``pepsi`` package and governed by ``[pepsi] SHARE_TELEMETRY``. That switch is **opt-in and defaults to** ``no``: until an operator sets it to ``yes``, the daemon exits at once and the in-process calls are no-ops, so the client's unit shipping in the main package costs a deployment nothing it did not ask for. Note that the collector package still ``Depends`` on ``pepsi`` at the same binary version: "runs on its own host" is a statement about where you deploy it, not about it being self-contained. See :doc:`programs/pepsi-telemetry`. .. _packages-hardening: The split as a hardening lever ============================== Everything above is inventory. This is the part that changes what an attacker can do. Why there is a removable half at all ------------------------------------ Setup is privileged work: writing ``/etc/pepsi/pepsi.conf``, handing each ``secrets.d`` fragment to the single account that reads it, running certbot, creating database roles, generating key material. ``pepsi-httpd``, which serves the browser console, does none of it. It drops to the ``pepsi-httpd`` account before it accepts a connection and must never be able to regain what it gave up. So it does not act. It records an **intent row** in ``pepsi.setup_task`` describing what should be true, and then connects to a UNIX socket and closes it. That connection is the entire message: nothing is written to it and nothing is ever read from it, so it cannot become a command channel. What it does is let systemd's socket activation start ``pepsi-setup apply`` as root, which takes its work from the database — where every row is validated against a closed task list and audited — and exits again when there is none. Root is reached through a database table, never through a socket that speaks a protocol. The full trust model, including the things the applier deliberately cannot do, is *The applier's trust model* in :doc:`programs/pepsi-setup`, and it is required reading before enabling any of this. That shape is what makes the capability separable. The doorbell socket unit and the applier service unit are the whole mechanism, they are two files, and they are the two files ``pepsi-httpd-admin`` contains. What removing it actually removes --------------------------------- ``apt remove pepsi-httpd-admin`` stops and disables both units and deletes them. From that moment **nothing drains** ``pepsi.setup_task`` on its own. An ``INSERT`` into that table is inert: it sits there, and nothing will act on it unless a human with a root shell chooses to (``pepsi-setup apply --once`` still works — the package carries the units, not the program). That is the distinction worth being precise about. The loop is broken at the **mechanism**, not at the user interface. It does not matter what happens to the web tier afterwards — a hostile administrator with a valid session, a stolen bearer token, a remote-code-execution bug in ``pepsi-httpd`` itself — because none of those confer the ability to start a root process. The web tier never had that ability; it only ever had a doorbell, and the doorbell is gone. Two things are pointedly *not* removed, and confusing them with the applier is the easiest mistake to make here: * **Configuration overrides still work.** ``/api/v1/config`` and the console's configuration pages write ``pepsi.config_override``, the database overlay every component reads at run time — not a file in ``/etc``. They keep their own gate, ``[pepsi-admin] CONFIG_DB``, and are unaffected. The applier has nothing to do with them, and widening the read-only rule to cover them would remove a capability this split was never about. See *Configuration in the database* in :doc:`configuration`. * **Nothing about mail flow changes.** ``pepsi-ingress``, ``pepsi-dispatch``, every stage and ``pepsi-httpd``'s MTA-STS and metrics surfaces are untouched. This is not a reduced-functionality build of the mail server. How ``pepsi-httpd`` notices --------------------------- A console that went on offering forms after the applier was removed would be the worst outcome of the split: an operator fills in an interview, presses apply, sees a ``pending`` row and waits for a change that is never going to happen. So the server asks, before every mutation, whether a privileged change can actually happen from here. It has already dropped privileges, so it cannot ask ``dpkg`` and cannot talk to systemd's private bus — and must not gain a way to. What it can do is look at two files, which between them answer *installed* and *armed*: #. ``pepsi-setup-apply.socket`` exists as a unit file in one of the directories systemd searches (``/etc/systemd/system``, ``/run/systemd/system``, ``/usr/local/lib/systemd/system``, ``/usr/lib/systemd/system``, ``/lib/systemd/system``); **and** #. ``[pepsi-admin] APPLY_SOCKET`` exists, is a socket, and is writable by this account — connecting to a UNIX socket needs write permission, and a doorbell that cannot be rung is not a capability. Two ``stat``\ s and an ``access``, no privilege required. The probe deliberately does **not** connect: connecting *is* the doorbell ring, and would start a root process on every page render. Neither test is redundant, which is the detail most likely to be optimised away by someone reading the code later: * Without the unit-file test, a **stale socket node** reads as a live doorbell. Stale nodes are reachable in practice: systemd's ``RemoveOnStop=`` defaults to off (the shipped unit sets it; an older or hand-edited one may not), and a package whose files are deleted rather than stopped leaves the node behind outright. That is precisely the scenario this feature exists for, and getting it wrong would fail **open**. * Without the socket test, a freshly installed package whose socket has never been enabled would read as ready, and every task would sit ``pending`` for ever. The check **fails closed**. A path that cannot be examined, a path that is not a socket, a permission error, an unreadable parent directory — every uncertainty reads as "no applier" and the console goes read-only. The failure that produces is an operator being told to install something that is already installed; the failure in the other direction is an operator believing a change was made. Only the first is recoverable by reading the message, and the messages are written to be read: they distinguish *not installed* from *installed but not started* from *there is a stale node here*, and each names its own remedy. What read-only looks like ------------------------- **Everything is still rendered.** Every setting the setup interview asks about is shown, with its staged answer or its default; every control that would change one is disabled, under a banner naming the missing package. Hiding the pages would be the easy reading of "read-only" and the wrong one — an operator who deliberately removed the applier still has to be able to see what the deployment is configured to do. The API refuses the corresponding mutations with a status and a code of their own rather than a generic failure: .. code-block:: text HTTP/1.1 503 Service Unavailable {"code": "setup_applier_unavailable", "hint": "the administrative package 'pepsi-httpd-admin' is not installed …", "detail": {"package": "pepsi-httpd-admin", "unit": "pepsi-setup-apply.socket"}} It applies to ``POST /api/v1/setup/tasks``, ``/setup/preflight``, ``/setup/dns-check``, ``/setup/certificates`` and ``PUT /api/v1/setup/answers``, and to the console's own form posts — ``POST /ui/setup/{step}``, ``/ui/setup/tasks`` and ``/ui/domains/check``. They all reach the queue through one enqueuing function that asks first, so the refusal is a single decision rather than one per surface, and the disabled control on the page sits over that check rather than in place of it. Answer *staging* is included on purpose: a draft answer's only purpose is to be applied, and a wizard that banks six steps of them on a host where nothing can apply them is exactly the silent no-op the design is avoiding. ``DELETE /api/v1/setup/answers`` is **not** gated — discarding drafts cannot cause a privileged change, and a read-only console still has to be able to clear a half-finished interview. The code is distinct from ``setup_write_unavailable`` (which means there is no ``CONFIG_DB`` connection) because the two have different remedies, and the applier is reported first when both apply: configuring a database role would not have helped. What still works with the package gone -------------------------------------- All of it, from a terminal. ``pepsi-setup`` is in the ``pepsi`` package, so:: pepsi-setup -c /etc/pepsi/pepsi.conf run pepsi-setup -c /etc/pepsi/pepsi.conf --wizard pepsi-setup -c /etc/pepsi/pepsi.conf check behave exactly as they always did, as does editing ``pepsi.conf`` by hand, generating keys, running certbot yourself and every other administrative task. The applier subcommand is still there too, as noted above: what the package carries is the *units*, not the program. What is removed is one thing only: **the web tier's ability to start it**. Arming it is a separate, deliberate step ---------------------------------------- Installing ``pepsi-httpd-admin`` makes the capability *available*. It does not turn it on. Like every other Pepsi unit the two ship disabled, so a host that has the package but has never run .. code-block:: console # systemctl enable --now pepsi-setup-apply.socket has no doorbell, and the console stays read-only and says which command would change that. Installing a package and arming a privilege are two decisions, and Pepsi does not let one imply the other. .. _packages-install-clears-queue: Installing it clears the queue first ------------------------------------ Removing the applier breaks the loop at the mechanism, and rows can still reach ``pepsi.setup_task`` while it is gone: an operator with a root shell can enqueue one by hand, and a console that is only *told* it is read-only is one release away from a path that forgets to ask. Nothing expires such a row. A request is durable, and the queue is drained in the order it was written. Put those together and an applier that turns up months later would execute every one of them at the first doorbell ring, against a deployment that has moved on — a configuration written from an answer set nobody remembers, a certificate obtained for a host that has been renamed. Consent does not keep. So installing ``pepsi-httpd-admin`` deletes the queue before its units are armed. The ``postinst`` runs .. code-block:: console # pepsi-setup -c /etc/pepsi/pepsi.conf apply --clear which removes every row **without executing any of them** and says so, naming the reason, so an operator who had legitimately queued something knows what became of it and can ask again. What was asked for is not lost either way: each request, refusal, success and failure is an independent durable entry in the audit log, which the reset does not touch. The same command is available by hand at any time — see :doc:`programs/pepsi-setup`. It is best effort, as a maintainer script must be. At install time there may be no configuration file, no schema, no database and no PostgreSQL running; each of those is reported and skipped, never a failed installation. (A database with no Pepsi schema is not even a skip: it has never held a task, so it is an empty queue.) The reset runs on every ``configure``, not only on a first installation, because dpkg reports a re-installation after ``apt remove`` as an upgrade — and remove-then-reinstall is exactly the case this exists for. The same lever from source -------------------------- A ``make install`` deployment gets the identical control:: make install INSTALL_ADMIN_UNITS=no which installs everything except ``pepsi-setup-apply.service`` and ``pepsi-setup-apply.socket``, and prints what it left out. ``make uninstall`` always removes **both** sets regardless of the variable's value on that invocation: a privileged unit surviving an uninstall because a make variable changed is exactly the failure this split is about. Choosing a posture ------------------ The probe, the configuration option and the package set compose, so the question "can this console change this machine?" has more than two answers. ``[pepsi-admin] APPLIER`` is the override, documented in full in :manpage:`pepsi.conf(5)`: .. list-table:: :header-rows: 1 :widths: 30 34 36 * - Posture - How - Result * - Administered from a browser - Package installed, socket enabled, ``APPLIER = auto`` (the default) - The console can apply setup tasks. * - Installed but not armed - Package installed, socket never enabled - Read-only; the refusal names ``systemctl enable --now pepsi-setup-apply.socket``. * - Administered from a terminal - ``apt remove pepsi-httpd-admin`` (or ``INSTALL_ADMIN_UNITS=no``) - Read-only; the refusal names the package, and **nothing drains the task queue unattended**. * - Read-only by policy - ``[pepsi-admin] APPLIER = no``, package left installed - Read-only by configuration rather than by which packages are installed, and the refusal says so. Reversible by editing one option — a policy, not a boundary. * - Applied some other way - ``pepsi-setup apply --once`` from cron, or a hand-written unit, plus ``[pepsi-admin] APPLIER = yes`` - The console offers the forms. ``yes`` is a promise the operator makes; nothing verifies it. Two of those rows deserve their difference spelled out. ``APPLIER = no`` and removing the package both produce a read-only console, and they are not the same defence. ``APPLIER = no`` is a line in a configuration file that ``pepsi-httpd`` consults; anyone who can change that file, or who can make ``pepsi-httpd`` read a different one, undoes it. Removing the package deletes the units that would have run the root process, and no amount of influence over ``pepsi-httpd`` puts them back. Use the option to state an intention; remove the package to enforce it. The last row is a promise, not a check: with ``APPLIER = yes`` the console will happily queue tasks whether or not anything drains them. It exists for the deployments the probe genuinely cannot see — a cron drainer, a host without systemd, unit files installed somewhere systemd does not search — and it is the one place in this mechanism where the code takes an operator's word for it. There is one hand-made state the packaging cannot produce: the socket present and writable while the *service* unit is masked or broken. The probe reads that as available, the task is enqueued, and it stays ``pending``. The task page shows that status rather than pretending otherwise. Both units ship in one package, so reaching this state takes deliberate effort. See also ======== :doc:`installation` — installing either way, and the accounts, groups and privileged bits the ``pepsi`` package's ``postinst`` creates. :doc:`security` — the threat model this split is a mitigation in, in particular what an attacker who compromises ``pepsi-httpd`` does and does not get. :doc:`web-ui` and :doc:`admin-api` — the console and API surfaces that go read-only. :doc:`programs/pepsi-setup` — the applier's trust model, its closed task list and its admission gates. :doc:`programs/pepsi-httpd` — the probe, the gated endpoints and ``[pepsi-admin]`` in reference form.