.. This file is part of PEPSI. Copyright (C) 2026 GNUnet e.V. PEPSI is free software; you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. ============ Installation ============ Pepsi can be installed two ways. Building **from source** out of the Cargo workspace gives you the binaries and leaves it to you to create the service accounts, set the privileged bits and lay out the runtime directories. The prebuilt **Debian package** does all of that for you — its ``postinst`` creates the accounts, groups, directories, privileged-binary overrides and the database — so only the site-specific bootstrap (editing the config and running ``pepsi-setup``) is left to do. Pick the tab for the method you are using. Everything after it — provisioning the schema/keys/DNS, running the services, the privilege model and the schema lifecycle — is the **same for both**. Prerequisites ============= * **PostgreSQL** — one database, into which Pepsi installs the single ``pepsi`` schema. All components connect to the same database. (The Debian package ``Depends`` on ``postgresql`` and pulls it in for you.) * **PostgreSQL's** ``postgresql-contrib`` **— optional, and only for mailing lists.** It carries the ``pg_trgm`` extension, which the mailing-list archive uses for substring and fuzzy search: finding a hostname, a config key, a line from a traceback, or a name the searcher has misspelled. A stemming dictionary throws exactly that material away, so full-text search cannot answer those queries at all. A site without it **installs and runs normally**. The extension is created in a guarded block, so its absence is a warning rather than a failed install; the archive then has full-text search only, and ``pepsi-list check`` says which mode the site is in. Installing it later and re-running ``pepsi-setup run`` enables it, after which the existing archive needs ``pepsi-archive reindex`` to populate the column the index reads. The Debian package ``Recommends`` it, so a default ``apt install`` pulls it in and a site that hosts no lists can remove it. * For real mail flow: public DNS you control for each served domain (to publish DKIM, SPF and MTA-STS records), and the ability to bind the privileged SMTP port 25 (and 465/587 for submission). A development build can run entirely on unprivileged ports and a throwaway database. Installing Pepsi ================ .. tab-set:: .. tab-item:: From source Build the workspace, place the binaries and SQL with ``make install``, and create the system accounts and privileges yourself (the same ones the package would create — see *Users, groups and privileges* below). .. rubric:: Additional prerequisites * A **Rust** toolchain with ``rustc`` 1.93 or later (the floor is set by the vendored ``taler-common``; ``./configure`` warns about an older one), plus the native build dependencies of the crypto and Kerberos crates: ``pkg-config``, ``cmake``, libclang, nettle, GMP and the Kerberos development headers (on Debian: ``libclang-dev``, ``nettle-dev``, ``libgmp-dev``, ``libkrb5-dev``). * The vendored **GNU Taler Rust** submodule, fetched before the first build:: git submodule update --init --recursive .. rubric:: Building Pepsi is a Cargo workspace whose **root package** (``pepsi``) owns every binary under ``src/bin/``; the ``pepsi-*`` member crates are libraries the thin binaries call into. A development build produces one executable per program:: cargo build --workspace A **release** build instead links most programs into a single multi-call binary, ``pepsi``, which selects the program to run from the name it is invoked under (``argv[0]``). This is selected by the ``unibin`` Cargo feature (which the ``make build`` target enables for you); the default ``multibin`` feature keeps the per-program binaries used for development and the test suite:: make build # == cargo build --release --locked \ # --no-default-features --features unibin,crypto-nettle Either way the per-program names are what you run; the release layout installs them as symlinks to the one binary (see *Installing*). Some programs are **not** folded in and stay standalone; the authoritative list is the Makefile's ``STANDALONE_BINARIES``. Two reasons put a program there: * it carries its own **setuid or setgid bit**, which one shared binary cannot do per program — the local-delivery, ``~/.forward`` and wallet helpers and their calling stages, the smarthost relay, ``pepsi-quota``, ``pepsi-whitelist``, and ``pepsi-stage-encrypt`` and ``pepsi-stage-decrypt`` (both setuid ``pepsi-crypto``, so they can reach the private key material) — or a site may give it one (``pepsi-keys``); * it is deployed or invoked separately — ``pepsi-setup``, ``pepsi-config``, ``pepsi-stage-detect-language`` (its language models are large; that binary is itself multi-call, and a second ``$PATH`` symlink, ``pepsi-detect-language``, runs the offline diagnostic tool out of it), ``pepsi-telemetry`` (which ships in its own package and runs on a separate host) and ``pepsi-helper-mailbox-scan`` (which ``pepsi-whitelist`` executes by name). The programs this chapter refers to are listed below. This is a selection, not an inventory: the authoritative lists are the Makefile's ``FOLDED_BINARIES`` and ``STANDALONE_BINARIES`` (``BINARIES`` is derived from those two and is not itself editable). * ``pepsi-ingress`` — inbound SMTP server * ``pepsi-dispatch`` — stage dispatcher * ``pepsi-stage-arc``, ``pepsi-stage-srs``, ``pepsi-stage-encrypt``, ``pepsi-stage-decrypt``, ``pepsi-stage-dkim-sign``, ``pepsi-stage-bounce``, ``pepsi-stage-relay-to-internet``, ``pepsi-stage-relay-to-smarthost``, ``pepsi-stage-discard`` — the stages * ``pepsi-setup`` — provisioning/bootstrap * ``pepsi-queue`` — queue inspection/repair * ``pepsi-status`` — pipeline health summary * ``pepsi-sendmail`` — local submission (installed as ``/usr/sbin/sendmail`` by the Debian package) The quality gates the project keeps green are:: RUSTFLAGS="-D warnings" cargo build --workspace cargo clippy --workspace --all-targets -- -D warnings cargo fmt --all -- --check cargo test --workspace cargo deny check make docs ``cargo test --workspace`` needs a reachable PostgreSQL, because the dispatcher-lifecycle crate spawns the real dispatcher against a database. ``make check`` is the entry point that gates it -- skipping that suite rather than failing when no database is usable -- and it also runs ``cargo deny check``; see *Other make targets* below. .. rubric:: Installing A bare build only produces *binaries*; Pepsi also ships SQL migrations and a sample configuration that must be placed on disk. The ``Makefile`` wraps both — run it as ``root`` so it can also apply the privileged-binary bits (it prints the exact ``chown``/``chmod`` to run by hand otherwise):: make install PREFIX=/usr/local SYSCONFDIR=/etc DESTDIR= Better still, run ``./configure`` first and let it resolve the directory variables into ``config.mk`` (which the ``Makefile`` reads before its own defaults, so what ``configure`` decided wins). Every directory derives from ``--prefix``, so naming a prefix and nothing else relocates the whole installation — binaries, SQL, ``config.d`` defaults and message templates — below it:: ./configure --prefix=/opt/pepsi # templates at /opt/pepsi/share/pepsi/templates make && make install ``--sysconfdir`` is the one exception, because a configuration file has to be where the *system* looks for it: it is ``/etc`` when no ``--prefix`` was given (and when ``--prefix=/usr`` was, since ``/usr/etc`` exists on no system), and ``$(PREFIX)/etc`` otherwise. Pass ``--sysconfdir`` explicitly to override either way. ``./configure --help`` lists everything it accepts. Besides the GNU directory variables (``--prefix``, ``--exec-prefix``, ``--bindir``, ``--sbindir``, ``--libexecdir``, ``--sysconfdir``, ``--datarootdir``, ``--datadir``, ``--mandir``, ``--infodir``, ``--docdir``) it takes: ``--with-templatedir=DIR`` Where the message templates go; must match ``[pepsi] TEMPLATE_DIR``. Defaults to ``$(DATADIR)/templates``. ``--with-systemdunitdir=DIR`` / ``--with-tmpfilesdir=DIR`` / ``--with-sendmail-libdir=DIR`` The unit files, the ``tmpfiles.d`` snippet, and where the legacy ``/usr/lib/sendmail`` link goes. ``--with-maildir-group=NAME``, ``--with-forward-group=NAME``, ``--with-wallet-group=NAME``, ``--with-token-group=NAME``, ``--with-dispatch-group=NAME``, ``--with-crypto-user=NAME``, ``--with-whitelist-user=NAME`` The site names of the accounts and groups that gate the privileged binaries (see *Privileged binaries* below). They default to ``pepsi-maildir``, ``pepsi-forward``, ``pepsi-wallets``, ``pepsi-token``, ``pepsi``, ``pepsi-crypto`` and ``pepsi-whitelist``. ``--enable-docs`` / ``--disable-docs`` Whether the man pages and the info manual are rendered from source or taken pre-built (what a release tarball ships). The default is ``auto``: on when ``sphinx-build``, ``makeinfo`` and Python with ``docutils`` are all present. ``--enable-mta-links`` / ``--disable-mta-links`` Whether ``make install`` claims ``/usr/sbin/sendmail`` and the rest of the Debian mail-transport-agent interface. **Off** by default, so a source install cannot silently displace the system's existing MTA; the Debian package passes ``INSTALL_MTA_LINKS=yes`` because its ``Conflicts: mail-transport-agent`` already guarantees exclusivity. ``--with-crypto-backend=FEATURE`` The sequoia crypto backend Cargo feature (default ``crypto-nettle``). ``CARGO=``, ``INSTALL=``, ``SPHINXBUILD=``, ``MAKEINFO=``, ``INSTALL_INFO=``, ``RST2MAN=`` and ``PYTHON=`` are accepted as ``VAR=VALUE`` arguments or from the environment. Options this build has no use for (``--libdir``, ``--localstatedir``, ``--build``, ``--host``, ...) are accepted and ignored, so a generic packaging driver can call the script unchanged; only a missing ``cargo`` is a hard error. ``make install`` performs the following steps: ``install-bin`` Builds the release (unified) binary and installs it to ``$(DESTDIR)$(PREFIX)/libexec/pepsi/pepsi``, then creates one ``$(DESTDIR)$(PREFIX)/bin`` symlink per folded program name pointing at it (e.g. ``pepsi-ingress -> ../libexec/pepsi/pepsi``). The seventeen standalone programs named in the Makefile's ``STANDALONE_BINARIES`` are installed as their own ``$(DESTDIR)$(PREFIX)/bin`` executables. Because each symlink still lives in ``$(PREFIX)/bin`` (on ``$PATH``), the runtime ``DATADIR`` derivation below is unaffected. ``install-data`` Copies the SQL files from ``pepsi-setup/db/`` to ``$(DESTDIR)$(DATADIR)/sql`` (where ``DATADIR = $(PREFIX)/share/pepsi``). ``pepsi-setup`` reads these when installing the schema; the default ``SQL_DIR`` is ``${DATADIR}/sql``. Also installs ``contrib/hosters.txt`` as ``$(DATADIR)/hosters.txt``, the public-hoster list ``pepsi-whitelist import --auto-wildcard`` never proposes a wildcard for. ``install-templates`` Copies the message templates (``..body`` — the anti-spam payment request, the bounce bodies, the mailing-list notices and the other automatic replies) to ``$(DESTDIR)$(TEMPLATEDIR)``, which defaults to ``$(DATADIR)/templates``. The binaries' ``[pepsi] TEMPLATE_DIR`` default is ``${DATADIR}/templates`` and resolves through the same runtime derivation, so the two ends agree under any prefix without the configuration naming a path. ``install-config`` Installs the reference ``pepsi.conf.sample`` under ``$(DESTDIR)$(SYSCONFDIR)/pepsi/``. It never creates ``pepsi.conf``: the sample wires up optional stages that need further setup and is not a working configuration, so the live file comes from ``pepsi-setup --wizard`` (an existing one is never touched). ``install-config-d`` Copies ``contrib/config.d/*.conf`` to ``$(DESTDIR)$(DATADIR)/config.d``. These are packaged defaults, **not** conffiles, and they are parsed *before* ``pepsi.conf`` — see :doc:`configuration`. Two ship with Pepsi: the per-language vacation notice texts and the S/MIME container setting (``CRYPTO_ALLOW_DOWNGRADE``). ``install-man`` / ``install-info`` Installs the rendered roff man pages under ``$(DESTDIR)$(MANDIR)/manN`` and ``pepsi.info`` under ``$(DESTDIR)$(INFODIR)`` (registered with ``install-info``). With ``BUILD_DOCS=no`` — what ``./configure --disable-docs`` sets, and what a release tarball uses — pre-built pages are installed instead of being rendered, and each step is skipped with a message if none is present. ``install-doc`` Installs ``README.md``, ``NEWS``, ``SECURITY.md``, ``AUTHORS``, ``COPYING`` and the licence texts from ``LICENSES/`` under ``$(DESTDIR)$(DOCDIR)``, which defaults to ``$(DATAROOTDIR)/doc/pepsi``. ``install-systemd`` Installs the systemd units from ``debian/systemd/`` into ``$(DESTDIR)$(SYSTEMDUNITDIR)`` (rewriting ``/usr/bin`` and ``/etc/pepsi`` to the configured ``BINDIR``/``SYSCONFDIR``) plus the ``tmpfiles.d`` snippet that gives ``/run/pepsi`` its owner and ACL -- which is where that directory's permissions come from *entirely*, since no unit may declare it as a ``RuntimeDirectory`` (see :ref:`sec-no-runtimedirectory`). Everything ships **disabled**. ``make install INSTALL_ADMIN_UNITS=no`` omits the two ``pepsi-setup-apply`` units, which is the source-install equivalent of not installing the ``pepsi-httpd-admin`` package. In addition, ``install`` depends on the privileged-bit targets (``install-helper``, ``install-maildir-stage``, ``install-forward-*``, ``install-auto-pay-*``, ``install-smarthost-stage``, ``install-whitelist-suid``, ``install-crypto-suid``, ``install-quota-tool``) described under *Privileged binaries* below; each prints the ``chown``/ ``chmod`` to run by hand when it is not run as root, or when the group it needs does not exist. Override ``PREFIX``, ``SYSCONFDIR`` and ``DESTDIR`` to suit the target system. ``make uninstall`` removes everything ``make install`` placed (always including both ``pepsi-setup-apply`` units, whatever ``INSTALL_ADMIN_UNITS`` says) but leaves the live ``pepsi.conf`` in place. .. rubric:: Other make targets ``make check`` The test entry point that needs no remote accounts: the file-inspection cross-checks of the systemd units and the telemetry socket path, the workspace tests (minus the dispatcher-lifecycle crate, which needs PostgreSQL), then -- best effort, skipped rather than failed when the prerequisite is absent -- the lifecycle suite against a local ``pepsicheck`` database, the setuid-helper contract tests, the authentication interoperability script, ``cargo deny check`` and the manual's cross-reference check. ``make integrationtests`` / ``make benchmarks`` The live-pipeline scripts under ``tests/``, which need passwordless SSH to the hosts named in ``ACCOUNTS`` (default ``tests/test-accounts.ini``, created from ``tests/test-accounts.ini.sample``); ``integrationtests`` first runs the Thunderbird interoperability gate. The benchmarks are a separate target because they deliberately load the MTA. ``make docs`` The manual for the source language -- an alias for ``make docs-en``, which builds its HTML *and* its PDF, plus the man pages and the info manual (both source-language only). ``make docs-`` builds one other language and ``make docs-all`` builds every language in ``DOC_LANGUAGES`` (``en de fr``); ``make html`` builds every language's HTML alone. ``make dist`` / ``make distcheck`` Build the release tarball ``pepsi-$(VERSION).tar.gz`` (every tracked file including the vendored submodule, plus ``configure`` and pre-built documentation), and verify it by unpacking it into a clean tree and running ``configure``, ``make``, ``make check``, ``make install`` and ``make uninstall`` against a throwaway prefix. The tarball is copied from the working tree, so ``make dist`` refuses to run while a tracked file differs from ``HEAD`` or the submodule is not at the commit ``HEAD`` records (untracked files do not count); ``make dist DIST_ALLOW_DIRTY=yes`` builds a development tarball anyway. ``make clean`` / ``make distclean`` ``clean`` runs ``cargo clean`` and removes the rendered documentation; ``distclean`` additionally removes ``config.mk``, ``config.status`` and anything ``make dist`` generated. .. note:: ``DATADIR`` must match what the binaries compute at runtime: ``taler-common`` derives ``${DATADIR}`` as ``/share/pepsi`` from the binary's own location. When running from a source checkout (not installed), set ``[pepsi-postgres] SQL_DIR`` explicitly so ``pepsi-setup`` finds the migrations. .. rubric:: Preparing the system The Debian package's ``postinst`` does the steps below automatically; on a source install you do them yourself, using the **same** account, group, mode and directory layout documented under *Users, groups and privileges* and *Directories and secrets* below. Create the accounts and groups **before** ``make install`` (so it can apply the privileged bits), and the database roles **before** ``pepsi-setup`` (which installs the schema): #. Create the system accounts, each ``--system`` with ``/usr/sbin/nologin`` and home ``/var/pepsi`` and a private group of the same name, plus the gating groups. ``debian/pepsi.postinst`` is the authoritative list; this loop is it, translated:: for u in pepsi pepsi-ingress pepsi-httpd pepsi-owner \ pepsi-helper-token-refresh pepsi-whitelist pepsi-crypto \ pepsi-keydisc pepsi-config; do addgroup --system "$u" adduser --system --home /var/pepsi --no-create-home \ --ingroup "$u" --shell /usr/sbin/nologin \ --disabled-password "$u" done for g in pepsi-maildir pepsi-forward pepsi-token pepsi-admin \ pepsi-telemetry; do addgroup --system "$g" done # The shared-wallet account owns its wallet databases, so unlike the # service accounts it gets a real home. addgroup --system pepsi-wallets adduser --system --home /var/lib/pepsi-wallets --ingroup pepsi-wallets \ --shell /usr/sbin/nologin --disabled-password pepsi-wallets install -d -o pepsi-wallets -g pepsi-wallets -m 0700 /var/lib/pepsi-wallets # Memberships: telemetry producers, and the SRS secret fragment. adduser pepsi pepsi-telemetry adduser pepsi-ingress pepsi-telemetry adduser pepsi pepsi-ingress Each of the last four accounts exists because something refuses to work without it: ``pepsi-whitelist`` is the account the whitelist CLI is setuid to, ``pepsi-crypto`` is the **only** database role granted ``crypto_identity.private_wrapped`` and the owner of the key-encryption fragment, ``pepsi-keydisc`` is the narrowest role in the deployment (it parses key material fetched from the open internet), and ``pepsi-config`` is the only role that may write the configuration overlay. Skip ``pepsi-crypto`` and ``install-crypto-suid`` merely warns — leaving the two crypto stages unprivileged and unable to open a single private key. #. Create the nine PostgreSQL login roles and the database (peer auth, as the ``postgres`` superuser) — the same set ``debian/pepsi.postinst`` creates, every account that authenticates as itself over the peer-auth socket. ``pepsi-telemetry`` is among them even on a node that serves no telemetry: ``pepsi-setup run`` grants it on every host, and it does its database work as ``pepsi-owner``, which cannot create a missing role:: for r in pepsi-owner pepsi pepsi-ingress pepsi-httpd \ pepsi-whitelist pepsi-config pepsi-crypto pepsi-keydisc \ pepsi-telemetry; do createuser "$r" # createuser takes one role name at a time done createdb -O pepsi-owner pepsi #. Run ``make install`` as ``root`` (above) so the setuid/setgid bits are applied; otherwise apply them by hand per *Privileged binaries* below. Each ``install-*`` target only warns when the group it needs is missing, so an account or group skipped above leaves the corresponding program silently unprivileged. #. Create the runtime directories with the ownership and modes listed under *Directories and secrets*. ``pepsi-setup`` creates ``/var/pepsi/keys`` for you; create the smarthost-secret directories only if you configure a smarthost using them. .. tab-item:: Debian package The package is the simpler path: it ships the binaries (installed to ``/usr/bin``), the SQL schema, the message templates, the man pages, the GNU info manual and **socket-activated systemd units**, and its ``postinst`` provisions the whole system for you. .. rubric:: Installing the package Install the ``.deb`` with ``apt`` so its dependencies (including ``postgresql``) are pulled in:: apt install ./pepsi_*.deb On ``configure`` the ``postinst`` automatically: * creates the ten service accounts and the six gating groups — ``pepsi-maildir``, ``pepsi-forward``, ``pepsi-token``, ``pepsi-admin``, ``pepsi-telemetry`` and ``pepsi-wallets`` — with the memberships between them (the same identities the source install makes by hand); * creates the ``/var/pepsi`` runtime directories — ``keys``, ``tokens``, ``token-refresh``, ``tls``, ``krb5`` — and ``/etc/pepsi/secrets`` and ``/etc/pepsi/secrets.d`` with the ownership and modes documented under *Directories and secrets* below; * applies the privileged-binary bits via ``dpkg-statoverride`` (the three setuid-root helpers, the setgid programs that may exec them — the two delivery stages, the wallet stage and ``pepsi-quota`` — the setgid smarthost relay, the setuid ``pepsi-whitelist`` and the two setuid ``pepsi-crypto`` stages — see *Privileged binaries* below); and * best-effort creates the nine PostgreSQL login roles (``pepsi-owner``, ``pepsi``, ``pepsi-ingress``, ``pepsi-httpd``, ``pepsi-whitelist``, ``pepsi-config``, ``pepsi-crypto``, ``pepsi-keydisc``, ``pepsi-telemetry``) and the ``pepsi`` database (owned by ``pepsi-owner``) when a local cluster is reachable — printing the roles to create by hand if it is not. The service is shipped **disabled**: the package installs but does not start Pepsi, leaving the site-specific bootstrap to you. .. note:: ``pepsi`` is one of **four** binary packages built from this source: the language-detection stage and the telemetry collector are split out for size and deployment reasons, and ``pepsi-httpd-admin`` — which ``pepsi`` ``Recommends``, so ``apt`` installs it by default — carries the only mechanism by which the browser console can cause a privileged change on this host. Removing that one package is a supported hardening step for a deployment administered from a terminal. :doc:`packages` describes each package and that lever in full. .. rubric:: After the package is installed On first installation the package copies a placeholder configuration to ``/etc/pepsi/pepsi.conf``; later upgrades never touch that file (it is not a dpkg conffile, so an upgrade never stops to ask about it). Edit it, then run the bootstrap exactly as in the shared steps below — for example:: # 1. edit /etc/pepsi/pepsi.conf (HOSTNAME, ACCEPTED_DOMAINS, domains, TLS) # 2. install the schema + DKIM keys and print the DNS records: pepsi-setup --no-certbot run # or: pepsi-setup --wizard # 3. publish the printed DNS records, then start the pipeline: systemctl enable --now pepsi.target Because the units are socket-activated, ``pepsi.target`` brings up ``pepsi-ingress``, ``pepsi-httpd`` and ``pepsi-dispatch`` together; you do not invoke the ``serve`` subcommands by hand. Migrating from an existing mail server ====================================== If this host already runs Postfix, Exim, Sendmail, qmail or Stalwart, you do not have to re-derive its configuration by hand. With no ``pepsi.conf`` present, ``pepsi-setup --wizard`` detects what is installed and offers to import it: the hostname, accepted domains, smarthost (with the relay credentials read from that server's own password file), local-delivery method, submission SASL socket, trusted networks, message size limit, queue lifetime and TLS certificate paths all arrive as the interview's pre-filled answers, which you then confirm or correct one by one. To see what a migration would produce *before* running the wizard — nothing is written:: pepsi-setup import Routing tables are converted too. ``/etc/aliases``, ``/etc/postfix/virtual``, Sendmail's ``virtusertable``, qmail's ``.qmail-*`` files and their equivalents become a Pepsi alias map beside the configuration (see :doc:`programs/pepsi-stage-aliases`), with bare alias names qualified in the style you choose. Since Pepsi alias keys must carry a domain, ``postmaster: root`` becomes ``postmaster@example.org root@example.org`` for each accepted domain by default. Everything that does **not** carry over is written to ``import-report.txt`` beside the configuration, split into what failed to parse, what Pepsi has no equivalent for (``header_checks``, an alias delivering to a command), what was approximated (mbox delivery becoming Maildir) and what the importer did not recognise. An import never aborts setup — an unreadable file or an unknown directive produces a report entry, not a failure. Two things are deliberately not migrated: **queued mail** (let the old server's queue drain before switching the MX over) and **DKIM keys** (``pepsi-setup run`` generates fresh ones and prints the records to publish). If the old MTA still holds port 25 when the wizard runs, the report says so — stop and disable it before starting ``pepsi-ingress``. See :doc:`programs/pepsi-setup` for the per-server detail, the ``--alias-style`` choices and the ``--expert`` options that expose the settings the interview does not otherwise ask about. Mail filters already on the host -------------------------------- Independently of any migration, the wizard **scans the host for milter daemons** — ``clamav-milter``, ``rspamd``, ``spamass-milter``, ``milter-greylist``, ``milter-regex``, ``mimedefang``, ``amavisd-milter`` — and offers each one it finds as a stage. Finding one means three things in turn: the package is installed, a socket can be read from its own configuration (or its unit, or the paths its distribution ships), and **a real milter option negotiation completes against that socket**. Nothing is offered on the strength of an installed package alone, and the negotiation is also how the generated ``ALLOW_ACTIONS`` comes to grant exactly what that filter asks for rather than a guess. Accepted filters are placed by what they do — policy and greylisting, then virus, then spam, then general-purpose frameworks — after decryption and before the whitelist and paywall gates. Because Pepsi filters *after* accepting a message, a filter's ``REJECT`` is routed to a generated discard stage rather than bounced, so forged senders get no backscatter; see :doc:`programs/pepsi-stage-milter`. Filters whose job Pepsi already does (``opendkim``, ``openarc``, ``opendmarc``, SPF daemons, ``postsrsd``) are named and skipped rather than offered. Provisioning the database, keys and DNS ======================================= With a configuration file in place (see :doc:`configuration`), run the bootstrap tool once:: pepsi-setup -c /etc/pepsi/pepsi.conf run > pepsi-dns.zone This **validates** the whole configuration, **installs** the ``pepsi`` schema, **generates** per-domain RSA-2048 and Ed25519 DKIM keys under ``KEY_DIR`` (only if absent), and **prints** the DNS records to publish (DKIM public keys, an SPF policy from the configured ``PUBLIC_IP`` addresses, and an MTA-STS record). The operation is idempotent and may be re-run at any time; upgrading the schema to a new release is ``pepsi-setup schema`` (see :ref:`upgrading`). ``run --reset`` drops and recreates the schema (destroying all queued mail; keys on disk are kept). Publish the printed DNS records, then verify what is live against what Pepsi expects:: pepsi-setup -c /etc/pepsi/pepsi.conf check See :doc:`programs/pepsi-setup` for the full provisioning workflow. Setting up in a browser ======================= Everything above can also be done from a browser, against the same code. The terminal path is the shortest route on a machine you have a shell on; the browser path is for the ones you do not, and for operators who would rather see the DNS records as a red/green checklist than as a zone file. The split, and why ------------------ Setup is privileged: it writes ``/etc/pepsi/pepsi.conf``, hands each ``secrets.d`` fragment to the single account that reads it, runs certbot, creates database roles and generates key material — all as root. ``pepsi-httpd``, which serves the interface, drops privileges before it accepts a connection and must never regain them. So it does not act. It writes an *intent* row saying what should be true, and a separate root program — ``pepsi-setup apply``, started on demand and gone again when it has nothing to do — decides how, does it, and records what happened. Root is never on the network. **Read "The applier's trust model" in :doc:`programs/pepsi-setup` before enabling this**: it states exactly what the applier will and will not do, including the things it deliberately cannot (there is no restart or shutdown task, and ``install-schema`` refuses ``reset``). Enabling it ----------- The Debian packages ship a minimal but complete configuration, an administrative listener on a UNIX socket, and — in ``pepsi-httpd-admin``, which ``pepsi`` ``Recommends`` and ``apt`` therefore installs by default — the applier's socket unit (see :doc:`packages`). Three steps: .. code-block:: console # systemctl enable --now pepsi-httpd.socket pepsi-setup-apply.socket # pepsi-setup -c /etc/pepsi/pepsi.conf bootstrap The second prints a one-time token and the ``curl`` command that turns it into the first administrator account. On the machine itself you can skip it entirely: a member of the ``pepsi-admin`` group (and root) is identified by ``SO_PEERCRED`` over ``/run/pepsi/admin.sock`` and needs no credential at all. One more setting is needed before anything privileged can be *asked for*: ``[pepsi-admin] CONFIG_DB`` must name a connection that authenticates as the ``pepsi-config`` database role. That is the boundary: only that role may enqueue setup work, and ``pepsi-httpd``'s own account is not it, so a compromise of the web tier is not a compromise of the pipeline definition. Until it is set the setup endpoints answer ``503`` and say so. Driving it ---------- The interview is data. ``GET /api/v1/setup/questions`` returns the ordered steps and, for each question, its shape, default, help text and the condition under which it is asked — the same model ``pepsi-setup questions`` prints and the same one the terminal wizard is described by, asserted equal by the test suite. Answers are staged with ``PUT /api/v1/setup/answers`` into draft rows that no running process can see, so an interrupted session cannot half-configure anything. ``POST /api/v1/setup/tasks`` then asks for the privileged half — writing the configuration, obtaining certificates, installing the schema, generating keys — and ``GET /api/v1/setup/tasks/{id}?since=`` streams what the applier is doing, line by line. What will always need a text editor ----------------------------------- Some settings are read from the configuration file only, and no interface will change that: ``[pepsi]``, ``[pepsi-postgres]``, ``[paths]``, and the HTTP(S) and ingress **listener** sections. They have to work before a database connection exists, or they are a security boundary — a database that could move a listening socket or relax the crypto policy would make a database compromise a compromise of the MTA's own TLS and ports. So **adding a submission port, or moving a listener, is a text-editor-and-restart operation.** So is any change the console marks ``restart`` rather than ``hot``: there is no restart button, because a button that restarts a service on request would hand the web tier's privileges to whoever reaches it. Scripting it ------------ The same answer model drives an unattended install:: pepsi-setup --wizard --answers answers.json -c /etc/pepsi/pepsi.conf ``pepsi-setup questions`` prints the schema that file is written against. This is not a third implementation of the interview: it runs the interactive one with every prompt answered from the file, so the branches and the per-field validation are the same ones a human sees. Running the services ==================== Two long-lived processes need to run continuously: * ``pepsi-ingress serve`` — accepts mail (typically under a service manager, optionally with systemd socket activation for the privileged ports). * ``pepsi-dispatch serve`` — runs **exactly one** instance per system; it is the only process that starts stage programs. The stage programs themselves are spawned by the dispatcher as persistent ``PROGRAM worker`` processes that read message ids on standard input; you do not run them by hand in production (for debugging you can pipe an id to ``worker``, e.g. ``echo 42 | pepsi-stage-srs -c /etc/pepsi/pepsi.conf worker``). .. _upgrading: Upgrading ========= From the first release, 0.0.0, on, every release upgrades the database of any earlier release in place: the queue, the key store, the settings and everything else in it are kept, and mail waiting in the queue is delivered by the new release. Downgrades are not supported (see below). The procedure: 1. Read ``NEWS`` for the release. It lists configuration options that changed, and anything else that needs doing by hand. 2. Take a backup (:ref:`backup-restore`), or at least enable the automatic dump described below. 3. Install the new release. The **Debian packages** do the rest: they upgrade the schema and restart the services that were running. **From source**: ``make install``, then ``pepsi-setup -c /etc/pepsi/pepsi.conf schema`` and ``systemctl try-restart pepsi.target``. 4. Check with ``pepsi-status`` that the queue drains, and look at the journal for warnings (:ref:`operations-logs`). The rest of this section explains what happens during step 3. Every Pepsi program checks, as it connects to the database, that the schema was built from exactly the SQL files the program itself was built with. The installer records the SHA-256 and release of each file it applies in ``pepsi.schema_file``, and a program that finds anything else -- an older schema, a newer one, or one built from different files -- stops at once with exit status 78 and a message naming the fix, rather than failing at its first missing column in the middle of a message. The dispatcher treats a stage worker that stops this way as "cannot run yet", not as a crash: the message it was handed is put back in the queue, never failed. Upgrading the schema is its own step:: pepsi-setup -c /etc/pepsi/pepsi.conf schema It installs the patches this release adds, replaces the stored functions, re-applies the role grants and does nothing else -- no certificates, keys or DNS -- so it needs only the ``[pepsi-postgres]`` section. On a telemetry collector, give it the collector's file, ``/etc/pepsi-telemetry/pepsi-telemetry.conf``. **Debian packages** run it automatically when they are upgraded, before the services are restarted; a new installation is left to ``pepsi-setup run``. If it fails (the database is down, say), the package still installs, a notice says so, and the services refuse to start and are retried by systemd -- backing off to once a minute, without a start limit -- until the schema matches. Once the cause is fixed, running the command above is all it takes; the services come back by themselves. **From source**, after ``make install``:: pepsi-setup -c /etc/pepsi/pepsi.conf schema systemctl try-restart pepsi.target What the refusals mean: .. list-table:: :header-rows: 1 :widths: 30 70 * - The message says - What to do * - no ``pepsi`` schema - A new installation: ``pepsi-setup run`` (or ``pepsi-setup schema`` on a collector host). * - older than this program - ``pepsi-setup -c FILE schema``. * - newer than this program - A downgrade, which is not supported. Install the newer release again, or restore the database backup taken before the upgrade (below). * - built from a different copy of a patch - A released patch never changes, so this is a bug; please report it. The one exception is a database built from a development snapshot (a Git checkout between releases), where the newest patch is still being edited: drain it and re-create it with ``pepsi-setup run --reset``. * - stored functions from a different build - Development builds only: ``pepsi-setup -c FILE schema``. Downgrades ---------- Not supported. ``pepsi-setup`` refuses to install an older release's SQL over a newer schema (exit status 78, nothing changed), and the older release's programs refuse to run on it. Going back to an older release means restoring a database backup taken before the upgrade, then installing the older packages. .. _upgrade-backup: Backing up before a schema upgrade ---------------------------------- No backup is taken unless you enable it. When enabled, the backup is taken only when an upgrade actually changes the schema: ``pg_dump`` saves the ``pepsi`` schema (with the extensions it uses) as ``pepsi--.dump``, readable by root only. If the dump fails, the schema is not upgraded. Backups are never deleted automatically. With the **Debian packages** it is a debconf setting, asked only at priority low:: dpkg-reconfigure -plow pepsi Answer yes, and confirm or change the directory (default ``/var/backups/pepsi``). Unattended, or before the first upgrade:: echo "pepsi pepsi/schema-backup boolean true" | debconf-set-selections echo "pepsi pepsi/schema-backup-dir string /var/backups/pepsi" | debconf-set-selections On a telemetry collector host the questions belong to ``pepsi-telemetry`` (``pepsi-telemetry/schema-backup``, ``pepsi-telemetry/schema-backup-dir``). When both packages are installed and use the same database, whichever upgrades it first takes the backup. **From source**, pass the directory to the upgrade step:: pepsi-setup -c /etc/pepsi/pepsi.conf schema --backup-dir /var/backups/pepsi To restore one, stop the services, re-create the database empty and owned by the schema owner, and restore into it as that owner:: systemctl stop pepsi.target sudo -u postgres dropdb pepsi sudo -u postgres createdb -O pepsi-owner pepsi sudo -u pepsi-owner pg_restore -d pepsi /var/backups/pepsi/pepsi-0.0.0-20261001T000000Z.dump Then install the release the backup came from and start the services again. Everything the database learnt after the backup -- mail accepted since, settings changed since -- is lost. The stored private keys in the dump are encrypted under the key-encryption key in ``secrets.d``, which is not in the database: keep that file, and back it up separately (see :ref:`backup-restore`). Users, groups and privileges ============================= Pepsi never *runs* as ``root``. The long-lived daemons may be **started** as root so they can bind privileged ports and read root-only TLS keys, but each one drops to its own unprivileged service account before serving a single connection; if the drop cannot be completed the process refuses to start rather than risk running as root (see ``pepsi-common::privdrop``). Local delivery — which must write into arbitrary users' mailboxes — is the one operation that needs more than the service user can give, and it is confined to a single narrowly-scoped setuid helper rather than handed to the pipeline at large. Under the shipped systemd units the daemons are never root **at all**: systemd binds the privileged sockets (socket activation) and starts each process as its service account directly. That leaves the TLS keys, which certbot keeps readable by root only — so ``pepsi-setup`` has systemd read those too, writing a ``LoadCredential=`` drop-in per unit: systemd opens each certificate and key as root when it starts the unit and hands the service a private copy under ``$CREDENTIALS_DIRECTORY``, which is where the server looks first. Re-run ``pepsi-setup run`` whenever you add, move or remove a certificate, so the drop-in is regenerated. See :doc:`programs/pepsi-httpd`. The browser setup described above is the one path by which something *other than an operator at a terminal* can cause a root process to run: ``pepsi-httpd`` rings a doorbell socket and systemd starts a short-lived root ``pepsi-setup apply``. That path is the easiest privilege in Pepsi to take away. Its two systemd units are a **separate Debian package**, ``pepsi-httpd-admin``, whose source-install equivalent is ``make install INSTALL_ADMIN_UNITS=no``; without them nothing drains ``pepsi.setup_task`` and no HTTP request can change anything under ``/etc/pepsi``, while ``pepsi-setup`` on a terminal keeps working unchanged. Putting the package back does not retroactively act on what was asked for while it was away: installing it empties that queue before its units are armed. See :ref:`packages-hardening`. The accounts and groups below are created automatically by the Debian package's ``postinst``; on a ``make install`` you create them yourself (the same names). PostgreSQL access is by **peer authentication** over the local socket: each service account logs in as a same-named database role, so no passwords are stored anywhere. A remote database keeps the same model: one login role per account, each authenticated by a client certificate or password file of its own that only that account can read, named per role with the ``{role}`` placeholder in ``[pepsi-postgres] CONFIG`` (see **CONFIG** in :manpage:`pepsi.conf(5)`). The configuration file never holds a password. Service accounts ---------------- These are system accounts (``--system``, ``/usr/sbin/nologin``, home ``/var/pepsi``), each with a private primary group of the same name. The first three are long-lived daemons that drop to their account after binding; the rest are not daemons in the pipeline. .. list-table:: :header-rows: 1 :widths: 24 18 58 * - Account - Runs - Purpose / access * - ``pepsi-ingress`` - ``pepsi-ingress`` - Inbound SMTP server. Socket-activated under systemd (never root, TLS material arrives as a service credential); started by hand it binds 25/465/587 and reads the TLS keys as root, then drops. DB role ``pepsi-ingress``. * - ``pepsi-httpd`` - ``pepsi-httpd`` - HTTP/HTTPS server (MTA-STS policy + ``/metrics``). Socket-activated under systemd, exactly like ``pepsi-ingress``; started by hand it binds 80/443 as root, then drops. DB role ``pepsi-httpd``. * - ``pepsi`` - ``pepsi-dispatch`` **and every stage worker it spawns** - The pipeline's working identity. Reads the DKIM keys (via group ``pepsi``) to sign. DB role ``pepsi`` (owns the queue rows). This is the account that gains the two delivery group-identities below transiently. * - ``pepsi-owner`` - ``pepsi-setup`` (transiently) - **Not a runtime service.** Owns the ``pepsi`` database, schema, tables and functions. When ``pepsi-setup`` runs as root it *temporarily* assumes this identity (``seteuid``) so the objects it creates are owned by the peer-authenticated ``pepsi-owner`` role, then returns to root for the filesystem work (key generation, ownership fix-ups). * - ``pepsi-helper-token-refresh`` - ``pepsi-helper-token-refresh`` service - Only present when a smarthost uses ``AUTH = oauth``. Refreshes OAuth access tokens and writes the token files the smarthost relay reads. Has **no** database role. * - ``pepsi-crypto`` - ``pepsi-keys`` (transiently), and the end-to-end crypto stages - **Custodian of private end-to-end key material.** Owns ``secrets.d/pepsi-crypto.secret``, the key-encryption key that opens ``pepsi.crypto_identity.private_wrapped``, and is the **only** database role granted that column — every other role, ``pepsi`` included, gets a column-level grant that omits it. ``pepsi-keys`` assumes this identity for the duration of a database call and gives it straight back. See :doc:`key-management`. * - ``pepsi-whitelist`` - ``pepsi-whitelist`` (transiently) - Account the setuid whitelist CLI assumes so ordinary users can maintain their own sender whitelists. Its database role is granted ``SELECT``/``INSERT``/``DELETE`` on ``pepsi.whitelist`` and nothing else. * - ``pepsi-keydisc`` - ``pepsi-keydisc@`` services - Key discovery (WKD, DANE, VKS, LDAP). Parses key material fetched from the open internet, so its database role is the narrowest in the deployment. See :doc:`key-management`. * - ``pepsi-config`` - ``pepsi-config`` (as run by an operator) - The only database role that may write the ``pepsi.config_override`` overlay: a component that processes mail must not be able to rewrite the pipeline it runs in. The nine PostgreSQL login roles — ``pepsi-owner``, ``pepsi``, ``pepsi-ingress``, ``pepsi-httpd``, ``pepsi-telemetry``, plus the narrow ``pepsi-whitelist``, ``pepsi-config``, ``pepsi-crypto`` and ``pepsi-keydisc`` roles — are created by the package ``postinst`` (or, on a manual install, by you) before ``pepsi-setup`` installs the schema; ``pepsi-setup`` then grants each what it needs. Groups gating shared resources ------------------------------ Four groups exist purely to *gate* access to privileged binaries and secret files. The ``pepsi`` user is deliberately **not** a permanent member of any of them: the setgid bit on the relevant stage binary grants the worker the group identity only for the lifetime of that process. ``pepsi-maildir`` Gates the local-delivery helper. ``pepsi-helper-maildir-writer`` is setuid ``root`` and mode ``4750`` (``root:pepsi-maildir``), so only this group may execute it. ``pepsi-stage-relay-to-maildir`` is setgid this group (mode ``2550``, ``pepsi:pepsi-maildir`` — owner-execute only, so no other local user can reach the setgid bit); when the ``pepsi`` worker execs the stage it gains ``egid=pepsi-maildir``, exactly the right to launch the helper — which then setuids to the *recipient* and writes ``~/Maildir/new/``. The helper is the only code that ever touches a third party's mailbox. ``pepsi-forward`` Gates the ``~/.forward`` helper, in exactly the shape of the maildir pair above: ``pepsi-helper-dot-forward`` is setuid ``root`` and mode ``4750`` (``root:pepsi-forward``), and ``pepsi-stage-dot-forward`` is setgid this group (mode ``2550``, ``pepsi:pepsi-forward``). Only needed if you run the ``~/.forward`` stage; without the group, ``make install``'s ``install-forward-helper`` / ``install-forward-stage`` targets only print a warning and leave the binaries unprivileged. ``pepsi-wallets`` Gates the wallet helper the same way: ``pepsi-helper-auto-pay`` is ``4750`` ``root:pepsi-wallets`` and ``pepsi-stage-auto-pay`` is setgid this group (mode ``2550``, ``pepsi:pepsi-wallets``). A system account of the same name owns the shared wallet databases under ``/var/lib/pepsi-wallets`` when ``WALLET_MODE = shared``. Only needed if you run the auto-pay stage; ``install-auto-pay-helper`` / ``install-auto-pay-stage`` degrade to a warning otherwise. ``pepsi-token`` Gates the secret material the smarthost relay reads: OAuth token files, the mutual-TLS client cert/key, and the Kerberos credential cache. ``pepsi-stage-relay-to-smarthost`` is setgid this group (mode ``2550``, ``pepsi:pepsi-token``); the directories holding those secrets are setgid ``pepsi-token`` so files dropped into them inherit the group, and the relay can read them without the ``pepsi`` user being a member. Privileged binaries ------------------- These bits are set by ``make install`` when run as root (it prints the exact ``chown``/``chmod`` to run by hand otherwise), and by the Debian package via ``dpkg-statoverride``, which is the authoritative list (``debian/pepsi.postinst``): .. list-table:: :header-rows: 1 :widths: 40 22 38 * - Binary - Mode / owner - Why * - ``pepsi-helper-maildir-writer`` - ``4750 root:pepsi-maildir`` - Setuid root so it can write into any local user's mailbox; group-locked so only ``pepsi-maildir`` may run it. * - ``pepsi-stage-relay-to-maildir`` - ``2550 pepsi:pepsi-maildir`` - Setgid so the ``pepsi`` worker may exec the helper above. Owner-execute and **not** world-execute: the setgid bit is the whole gate, so every other local user must not be able to reach it. ``pepsi`` cannot get the exec right from the group (it is deliberately not a member), so it gets it by owning the file. ``pepsi-quota``, ``pepsi-stage-dot-forward`` and ``pepsi-stage-auto-pay`` carry the same shape for their own groups. * - ``pepsi-quota`` - ``2550 pepsi:pepsi-maildir`` - Same group and same reason: ``measure``/``reconcile`` run the helper above, and the bit is what lets an unattended sweep do it without root. * - ``pepsi-helper-dot-forward`` - ``4750 root:pepsi-forward`` - Setuid root so it can run a user's ``~/.forward`` **as that user**; group-locked to its stage. * - ``pepsi-stage-dot-forward`` - ``2550 pepsi:pepsi-forward`` - Setgid so the ``pepsi`` worker may exec the helper above. * - ``pepsi-helper-auto-pay`` - ``4750 root:pepsi-wallets`` - Setuid root so it can drop to the wallet-owning account (or the recipient's own uid) to run ``taler-wallet-cli``. * - ``pepsi-stage-auto-pay`` - ``2550 pepsi:pepsi-wallets`` - Setgid so the ``pepsi`` worker may exec the helper above. * - ``pepsi-stage-relay-to-smarthost`` - ``2550 pepsi:pepsi-token`` - Setgid so the ``pepsi`` worker may read the ``pepsi-token`` secrets; not world-executable, since the stage accepts ``-c`` and would otherwise send those secrets to any server a local user names. * - ``pepsi-whitelist`` - ``4755 pepsi-whitelist:pepsi-whitelist`` - Setuid, and world-executable on purpose: any local user may manage their own ``/…`` namespace, and the rule stopping them touching anyone else's is inside the program, not the file mode. * - ``pepsi-stage-encrypt``, ``pepsi-stage-decrypt`` - ``4750 pepsi-crypto:pepsi`` - Setuid, not setgid: PostgreSQL peer authentication keys off the **effective uid**, and ``pepsi-crypto`` is the only role granted ``crypto_identity.private_wrapped``. Mode ``4750`` with group ``pepsi`` because these are not user commands — only the dispatcher's account and root may run them. Every other binary is installed unprivileged, including ``pepsi-keys``, which is standalone for the same reasons but ships ``0755``: one binary able to open every private key in a deployment is a much larger prize than ``pepsi-whitelist``'s single table, so making it setuid ``pepsi-crypto`` is a per-site decision rather than the default. Directories and secrets ----------------------- ``pepsi-setup`` and the package create the following under ``/var/pepsi`` (the service home) and ``/etc/pepsi``. The setgid (``2750``) directories cause new files to inherit the directory's group, so token/TLS/Kerberos material is group-readable by the relay without any per-file ``chgrp``. .. list-table:: :header-rows: 1 :widths: 30 24 46 * - Path - Owner / mode - Contents * - ``/var/pepsi`` - ``pepsi-owner:pepsi`` - Service home. * - ``/var/pepsi/keys`` - ``pepsi-owner:pepsi`` ``2750`` - Per-domain DKIM private keys: written by ``pepsi-setup``, read by the ``pepsi`` dispatcher (group ``pepsi``) to sign. * - ``/var/pepsi/tokens`` - ``pepsi-helper-token-refresh:pepsi-token`` ``2750`` - OAuth access-token files: written by the refresher, read by the smarthost relay via group ``pepsi-token``. * - ``/var/pepsi/token-refresh`` - ``pepsi-helper-token-refresh`` ``0700`` - Private rotated **refresh** tokens; never group-readable. * - ``/etc/pepsi/secrets`` - ``root:pepsi-helper-token-refresh`` ``0750`` - The ``@inline-secret@`` token-refresh credentials; only the refresher may read them. * - ``/etc/pepsi/secrets.d`` - ``root:root`` ``0751`` - The fragments ``pepsi-setup`` generates: the SRS key, smarthost/LMTP passwords, merchant and resume tokens, the proof-of-origin key and the ``pepsi-crypto`` key-encryption key. World-**searchable** (not readable) so each service can reach the one fragment it may open; the fragments themselves are ``0640`` owned by their single reader account. ``pepsi-setup`` creates the directory if it is absent. * - ``/var/pepsi/tls`` - ``root:pepsi-token`` ``2750`` - Mutual-TLS client cert + key for the smarthost relay (``TLS_CLIENT_CERT`` / ``TLS_CLIENT_KEY``). * - ``/var/pepsi/krb5`` - ``root:pepsi-token`` ``2750`` - Kerberos credential cache for ``AUTH = gssapi`` smarthosts (point ``KRB5CCNAME`` at a ``FILE:`` cache here; an external keytab refresher populates it). * - ``/run/pepsi`` - ``root:root`` ``0775`` + ACL - The local submission, administrative, HTTP and setup-applier sockets. Created by the ``tmpfiles.d`` snippet, whose ACL grants ``pepsi-ingress`` and ``pepsi-httpd`` write access by name. * - ``/var/lib/pepsi-wallets`` - ``pepsi-wallets:pepsi-wallets`` ``0700`` - Home of the ``pepsi-wallets`` account and the wallet databases ``pepsi-helper-auto-pay`` creates under it (``wallets/.sqlite3``, mode ``0700``). Only with ``pepsi-stage-auto-pay`` in ``WALLET_MODE = shared``. The ``/var/pepsi/tokens``, ``/var/pepsi/tls`` and ``/var/pepsi/krb5`` directories (and the ``pepsi-helper-token-refresh`` account) only matter when you configure a smarthost with the corresponding ``AUTH`` mechanism; a default direct-to-MX or local-delivery deployment uses none of them. .. _migrating-from-mailman: Migrating from GNU Mailman ========================== An existing GNU Mailman installation — 2.1 or 3 — moves onto Pepsi with ``pepsi-list import`` and ``pepsi-archive import``. Read the table below before you start, not afterwards. **The behavioural differences are a different table, and it is not duplicated here**: :ref:`declared-differences` in :doc:`mailing-lists` collects every place Pepsi deliberately does something other than what upstream does — one-click unsubscribe, the always-per-recipient fan-out, the archive's obscured sender addresses, the purge tombstone, the renamed ``X-Mailman-*`` headers, and what is simply absent (no NNTP gateway, no pluggable archivers, no plugin API). Two copies of that table would diverge, and the copy a migrating operator happened to read would be the stale one. This section is only about what the *importer* leaves behind. What does not come across, and why ---------------------------------- .. list-table:: :header-rows: 1 :widths: 26 74 * - - Why * - **Passwords** - Neither Mailman core's ``sha512_crypt`` hashes nor mailman-web's Django ``pbkdf2_sha256`` ones are imported. Upstream's own importer has the code and it is commented out. This is why the importer never has to read the Django database at all, and why there is no legacy hash verifier anywhere in Pepsi. **Verified addresses stay verified**, so nobody re-proves an address — they only choose a password, which is what ``pepsi-list invite`` is for. * - **Bounce scores** - Upstream does not import them either: ``config.pck``'s ``bounce_info`` and Mailman 3's bounce events are both discarded, so a migrated member starts at score zero on *both* systems. * - **In-flight tokens** - A pending confirmation or moderation token is a promise about a workflow that will not exist after the cutover. Anybody mid-subscription asks again. * - **Topics** (2.1) - No Mailman 3 equivalent. * - **Template URIs we refuse to fetch** - A ``file:`` override would read the server's own disk, so it is not imported. The importer reports every such URI **at import time** rather than letting the first notice that needs it fail months later, and the list falls back to the built-in template. Anything else the importer did not understand is in its report. It exits non-zero when something was lost, which is the point: a migration that half worked and exited zero is a migration nobody checks. Nothing is rolled back, and **re-running an import is safe** — every write is keyed on the list id, the address or the ``Message-ID``, so a second run is a no-op. The recommended order --------------------- Do it in this order. The reason is in step 4. 1. **Import the configuration and the roster**, against the running old server:: pepsi-list import mailman3 --rest http://localhost:8001 \ --user restadmin --password-file /etc/mailman3/rest.pw \ --report /root/mailman-import.txt Or, for a Mailman 2.1 site whose server is already off:: pepsi-list import mailman21 --listdir /var/lib/mailman/lists \ --report /root/mailman-import.txt Add ``--dry-run`` first: it reads everything, reports everything and writes nothing. 2. **Import the archive**, per list:: pepsi-archive import --list announce@lists.example.org \ --rejects /root/rejected.txt /var/lib/mailman/archives/private/announce.mbox/*.mbox Threads are reattached and renumbered automatically at the end — an out-of-order import otherwise leaves them split, which is the mistake this command exists not to repeat. 3. **Verify.** ``pepsi-list check --strict``, then spot-check a few lists' settings against the old server. Over REST both ends answer ``/3.1/lists//config``, so the comparison is per-attribute rather than by eye. 4. **Warm the IP with the invitation run, while the old server is still delivering mail.** :: pepsi-list invite send --dry-run # the per-domain histogram pepsi-list invite send --rate 200 --limit 500 pepsi-list invite status This is the step whose *timing* matters. Telling every member to choose a password is the largest single mailing this server will ever send, to an address list it has never validated, from an IP with no reputation — and that is how a new deployment's reputation is destroyed on day one. Doing it before the MX switch means a throttling or blocking problem becomes visible while it is still not load-bearing. The dry run's histogram is a deliverability tool, not a progress bar: one provider holding forty per cent of a migrated site is the fact that decides the rate, and it is invisible until something counts it. Invitations last four weeks, the run resumes where it stopped, and **nobody is unsubscribed for not answering** — a member who never acts keeps their subscription and simply cannot sign in until they use the link or ask for another. 5. **Switch the MX**, and only then stop the old server. When our pickle reader is stricter than theirs ---------------------------------------------- ``pepsi-list import mailman21`` reads ``config.pck`` with its own parser, which **constructs nothing**: no module is looked up and no callable is called, because a ``config.pck`` comes from somebody else's server and Python's ``pickle.load`` on untrusted input is arbitrary code execution. The price of that is strictness. If the reader refuses a file — an opcode it does not implement, a protocol newer than Python 2 could write, an integer too wide for 64 bits — it says so and names the escape hatch:: python2 contrib/mm21-export.py /var/lib/mailman/lists/announce/config.pck \ > /root/announce.json pepsi-list import mailman21 --json /root/announce.json --list announce That script runs on the **old** server, with the Python that server already trusted, and dumps the same dictionary as JSON. Both routes then go through the same mapping, so they cannot disagree about what the file means. A migration must never be blocked by our parser being stricter than the one that wrote the file. Header renames a user will notice --------------------------------- Pepsi emits ``X-Pepsi-List-*`` where Mailman emitted ``X-Mailman-*``; the mapping table is in :doc:`mailing-lists`. This is a migrating *user's* real cost, because procmail and Sieve rules are written against those names, and it is worth telling them before the switch rather than after.