4. Installation¶
Pepsi can be installed two ways. Building from source out of the Cargo
workspace gives you the binaries and leaves it to you to create the service
accounts, set the privileged bits and lay out the runtime directories. The
prebuilt Debian package does all of that for you: its postinst creates
the accounts, groups, directories, privileged-binary overrides and the database.
Only the site-specific bootstrap (editing the configuration and running
pepsi-setup) is then left to do.
Pick the tab for the method that you are using. Everything after it is the same for both: provisioning the schema, keys and DNS, running the services, the privilege model and the schema lifecycle.
4.1. Prerequisites¶
PostgreSQL — one database, into which Pepsi installs the single
pepsischema. All components connect to the same database. (The Debian packageDependsonpostgresqland pulls it in for you.)PostgreSQL’s
postgresql-contrib— optional, and only for mailing lists. It carries thepg_trgmextension, which the mailing-list archive uses for substring and fuzzy search: finding a hostname, a config key, a line from a traceback, or a name that the searcher has misspelled. A stemming dictionary throws exactly that material away, so full-text search cannot answer those queries at all.A site without it installs and runs normally. The extension is created in a guarded block, so its absence is a warning rather than a failed install; the archive then has full-text search only, and
pepsi-list checksays which mode the site is in. Installing it later and re-runningpepsi-setup runenables it, after which the existing archive needspepsi-archive reindexto populate the column that the index reads.The Debian package
Recommendsit, so a defaultapt installpulls it in and a site that hosts no lists can remove it.For real mail flow: public DNS you control for each served domain (to publish DKIM, SPF and MTA-STS records), and the ability to bind the privileged SMTP port 25 (and 465/587 for submission). A development build can run entirely on unprivileged ports and a throwaway database.
A DNSSEC-validating resolver, ideally your own on this host. DNS setup is outside Pepsi’s scope: Pepsi does not validate DNSSEC itself but trusts the
ADbit of the resolver that it is given (/etc/resolv.conf,[pepsi] DNS_SERVERS, or a stage’sDNS_SERVERS), and DANE and DNSSEC-gated key discovery rest on that bit. Run your own validating recursive resolver on the host, for exampleunbound(apt install unbound, thennameserver 127.0.0.1inresolv.conf), or systemd-resolved withDNSSEC=yes. Where that is not possible, name a validating resolver that you trust in[pepsi] DNS_SERVERSas DNS over TLS (tls://192.0.2.53#dns.example.net) or DNS over HTTPS (https://192.0.2.53/dns-query#dns.example.net): the channel is then authenticated by the resolver’s certificate, so nobody on the path can forge theADbit, and every Pepsi program uses it (see pepsi.conf(5)). A resolver reached over plain DNS across a network is a trust you place in its operator and in the path to it;pepsi-setupnames every such resolver by IP address when it runs, and refuses nothing. Either way the resolver’s operator decides what counts as validated.pepsi-statusshows, in its “DNSSEC resolver” section, whether the resolver in use validates; the DANE workers log a warning at start when it does not.
4.2. Installing Pepsi¶
Build the workspace, place the binaries and SQL with make install, and
create the system accounts and privileges yourself (the same ones that the
package would create; see Users, groups and privileges below).
Additional prerequisites
A Rust toolchain with
rustc1.95 or later (the floor is set by the vendoredtaler-common;./configurewarns about an older one), plus the native build dependencies of the crypto and Kerberos crates:pkg-config,cmake, libclang, nettle, GMP and the Kerberos development headers (on Debian:libclang-dev,nettle-dev,libgmp-dev,libkrb5-dev).The vendored GNU Taler Rust submodule, fetched before the first build:
git submodule update --init --recursive
Building
Pepsi is a Cargo workspace whose root package (pepsi) owns every
binary under src/bin/; the pepsi-* member crates are libraries
that the thin binaries call into. A development build produces one executable per
program:
cargo build --workspace
A release build instead links most programs into a single multi-call
binary, pepsi, which selects the program to run from the name under
which it is invoked (argv[0]). This is selected by the unibin Cargo
feature (which the make build target enables for you); the default
multibin feature keeps the per-program binaries used for development and
the test suite:
make build # == cargo build --release --locked \
# --no-default-features --features unibin,crypto-nettle
Either way the per-program names are what you run; the release layout
installs them as symlinks to the one binary (see Installing). Some
programs are not folded in and stay standalone; the authoritative list
is the Makefile’s STANDALONE_BINARIES. Two reasons put a program there:
it carries its own setuid or setgid bit, which one shared binary cannot do per program, or a site may give it one (
pepsi-keys). This applies to the local-delivery,~/.forwardand wallet helpers and their calling stages, the smarthost relay,pepsi-quota,pepsi-whitelist, andpepsi-stage-encryptandpepsi-stage-decrypt(both setuidpepsi-crypto, so that they can reach the private key material);it is deployed or invoked separately:
pepsi-setup,pepsi-config,pepsi-stage-detect-language(its language models are large; that binary is itself multi-call, and a second$PATHsymlink,pepsi-detect-language, runs the offline diagnostic tool out of it),pepsi-telemetry(which ships in its own package and runs on a separate host) andpepsi-helper-mailbox-scan(whichpepsi-whitelistexecutes by name).
The programs this chapter refers to are listed below. This is a
selection, not an inventory: the authoritative lists are the Makefile’s
FOLDED_BINARIES and STANDALONE_BINARIES (BINARIES is derived
from those two and is not itself editable).
pepsi-ingress— inbound SMTP serverpepsi-dispatch— stage dispatcherpepsi-stage-arc,pepsi-stage-srs,pepsi-stage-encrypt,pepsi-stage-decrypt,pepsi-stage-dkim-sign,pepsi-stage-bounce,pepsi-stage-relay-to-internet,pepsi-stage-relay-to-smarthost,pepsi-stage-discard— the stagespepsi-setup— provisioning/bootstrappepsi-queue— queue inspection/repairpepsi-status— pipeline health summarypepsi-reputation— IP blocklists for inbound mailpepsi-sendmail— local submission (installed as/usr/sbin/sendmailby the Debian package)
The quality gates the project keeps green are:
RUSTFLAGS="-D warnings" cargo build --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all -- --check
cargo test --workspace
cargo deny check
cargo vet --locked
make docs
cargo test --workspace needs a reachable PostgreSQL, because the
dispatcher-lifecycle crate spawns the real dispatcher against a database.
make check is the entry point that gates it: it skips that suite,
rather than failing, when no database is usable. It also runs
cargo deny check and cargo vet --locked (the dependency audit
record, supply-chain/; see The supply chain); see Other make
targets below.
Installing
A bare build only produces binaries; Pepsi also ships SQL migrations and a
sample configuration that must be placed on disk. The Makefile wraps
both. Run it as root, so that it can also apply the privileged-binary
bits; otherwise it prints the exact chown/chmod to run by hand:
make install PREFIX=/usr/local SYSCONFDIR=/etc DESTDIR=
Better still, run ./configure first and let it resolve the directory
variables into config.mk (which the Makefile reads before its own
defaults, so what configure decided wins). Every directory derives from
--prefix, so naming a prefix and nothing else relocates the whole
installation below it: binaries, SQL, config.d defaults and message
templates:
./configure --prefix=/opt/pepsi # templates at /opt/pepsi/share/pepsi/templates
make && make install
--sysconfdir is the one exception, because a configuration file has to
be where the system looks for it: it is /etc when no --prefix was
given (and when --prefix=/usr was, since /usr/etc exists on no
system), and $(PREFIX)/etc otherwise. Pass --sysconfdir explicitly
to override either way.
./configure --help lists everything it accepts. Besides the GNU
directory variables (--prefix, --exec-prefix, --bindir,
--sbindir, --libexecdir, --sysconfdir, --datarootdir,
--datadir, --mandir, --infodir, --docdir) it takes:
--with-templatedir=DIRWhere the message templates go; must match
[pepsi] TEMPLATE_DIR. Defaults to$(DATADIR)/templates.--with-systemdunitdir=DIR/--with-tmpfilesdir=DIR/--with-udevrulesdir=DIR/--with-sendmail-libdir=DIRThe unit files, the
tmpfiles.dsnippet, the udev rule that records a virtual machine’s generation-ID change (see Cloned and restored virtual machines), and where the legacy/usr/lib/sendmaillink goes.--with-maildir-group=NAME,--with-forward-group=NAME,--with-wallet-group=NAME,--with-token-group=NAME,--with-dispatch-group=NAME,--with-crypto-user=NAME,--with-whitelist-user=NAMEThe site names of the accounts and groups that gate the privileged binaries (see Privileged binaries below). They default to
pepsi-maildir,pepsi-forward,pepsi-wallets,pepsi-token,pepsi,pepsi-cryptoandpepsi-whitelist.--enable-docs/--disable-docsWhether the man pages and the info manual are rendered from source or taken pre-built (what a release tarball ships). The default is
auto: on whensphinx-build,makeinfoand Python withdocutilsare all present.--enable-mta-links/--disable-mta-linksWhether
make installclaims/usr/sbin/sendmailand the rest of the Debian mail-transport-agent interface. Off by default, so a source install cannot silently displace the system’s existing MTA; the Debian package passesINSTALL_MTA_LINKS=yesbecause itsConflicts: mail-transport-agentalready guarantees exclusivity.--with-taler-wallet-cli=PATHThe only wallet program the setuid
pepsi-helper-auto-paywill run, fixed at build time (default/usr/bin/taler-wallet-cli, where the GNU Taler packages install it). It is never taken from the command line, the environment or the configuration, because the helper’s caller is less privileged than every account the helper can become.--with-crypto-backend=FEATUREThe cryptographic library behind OpenPGP (Sequoia’s backend), selected by Cargo feature. Exactly one is built in;
FEATUREis one of:crypto-nettle(the default)Nettle, with its
hogweedpublic-key half and GMP, linked dynamically from the system (Debian:nettle-dev,libgmp-dev). Nettle has no post-quantum algorithms. Only Nettle 3 will do: the bindings do not build against Nettle 4, which droppednettle/pgp.h, so on a system that ships Nettle 4 (Homebrew, for one) choosecrypto-openssl.crypto-opensslOpenSSL’s
libcrypto, linked dynamically from the system (Debian:libssl-dev). This is the backend to choose for post-quantum OpenPGP: only it, and only with OpenSSL 3.5 or later, can decrypt messages to ML-KEM-768+X25519 keys and handle ML-DSA-65+Ed25519 signatures (see Traditional public-key algorithms broken by quantum computers).
Any other value is refused. The choice affects OpenPGP only: TLS, S/MIME, DKIM and ARC use AWS-LC with either backend, and no program links OpenSSL for TLS (OpenPGP). Without
configure, the same choice ismake build CRYPTO_BACKEND=crypto-openssl.--jobs=NHow many compiler jobs cargo runs at once. Cargo on its own runs one job per CPU, whatever the memory, and a machine with many cores and little RAM then runs out of memory near the end of the build. So by default (
auto),contrib/build-jobs.shdecides just before each cargo command. It allows one job per CPU, but only as many as the memory that is free at that moment can hold, and it leaves a wide margin for other programs and for future growth of the build. The largest job of the release build needs about 5 GB and each further one up to 2.7 GB, so the release builds with one job when 16 GB are free, with two when 18 GB are free and with four when 32 GB are free; the test build (make check) needs a little less per job.Nfixes the number instead. Withoutconfigure,CARGO_BUILD_JOBS=Nin the environment or on themakecommand line does the same.make -jNdoes not raise the number but caps it atN, and the Debian build takes that cap fromDEB_BUILD_OPTIONS=parallel=N.
CARGO=, INSTALL=, SPHINXBUILD=, MAKEINFO=,
INSTALL_INFO=, RST2MAN= and PYTHON= are accepted as
VAR=VALUE arguments or from the environment. Options that this build
has no use for (--libdir, --localstatedir, --build, --host,
…) are accepted and ignored, so a generic packaging driver can call the
script unchanged; only a missing cargo is a hard error.
make install performs the following steps:
install-binBuilds the release (unified) binary and installs it to
$(DESTDIR)$(PREFIX)/libexec/pepsi/pepsi, then creates one$(DESTDIR)$(PREFIX)/binsymlink per folded program name pointing at it (e.g.pepsi-ingress -> ../libexec/pepsi/pepsi). The seventeen standalone programs named in the Makefile’sSTANDALONE_BINARIESare installed as their own$(DESTDIR)$(PREFIX)/binexecutables. Because each symlink still lives in$(PREFIX)/bin(on$PATH), the runtimeDATADIRderivation below is unaffected.install-dataCopies the SQL files from
pepsi-setup/db/to$(DESTDIR)$(DATADIR)/sql(whereDATADIR = $(PREFIX)/share/pepsi).pepsi-setupreads these when installing the schema; the defaultSQL_DIRis${DATADIR}/sql. Also installscontrib/hosters.txtas$(DATADIR)/hosters.txt, the list of public hosters for whichpepsi-whitelist import --auto-wildcardnever proposes a wildcard. (The list of known compromised keys, against which every peer key is checked, is not installed: each deployment fetches it into its database, seepepsi-keys blocklist update.)install-templatesCopies the message templates (
<name>.<lang>.body— the anti-spam payment request, the bounce bodies, the mailing-list notices and the other automatic replies) to$(DESTDIR)$(TEMPLATEDIR), which defaults to$(DATADIR)/templates. The binaries’[pepsi] TEMPLATE_DIRdefault is${DATADIR}/templatesand resolves through the same runtime derivation, so the two ends agree under any prefix without the configuration naming a path.install-configInstalls the reference
pepsi.conf.sampleunder$(DESTDIR)$(SYSCONFDIR)/pepsi/. It never createspepsi.conf: the sample wires up optional stages that need further setup and is not a working configuration, so the live file comes frompepsi-setup --wizard(an existing one is never touched).install-config-dCopies
contrib/config.d/*.confto$(DESTDIR)$(DATADIR)/config.d. These are packaged defaults, not conffiles, and they are parsed beforepepsi.conf— see Configuration. One ships with Pepsi: the per-language vacation notice texts. If that directory holds athunderbird.conf, the target deletes it: every file there is parsed, and that file setsCRYPTO_ALLOW_DOWNGRADE = yesfor every peer, where the S/MIME capability record decides per peer.install-man/install-infoInstalls the rendered roff man pages under
$(DESTDIR)$(MANDIR)/manNandpepsi.infounder$(DESTDIR)$(INFODIR)(registered withinstall-info). WithBUILD_DOCS=no— what./configure --disable-docssets, and what a release tarball uses — pre-built pages are installed instead of being rendered, and each step is skipped with a message if none is present.install-docInstalls
README.md,NEWS,SECURITY.md,AUTHORS,COPYINGand the licence texts fromLICENSES/under$(DESTDIR)$(DOCDIR), which defaults to$(DATAROOTDIR)/doc/pepsi.install-systemdInstalls the systemd units from
debian/systemd/into$(DESTDIR)$(SYSTEMDUNITDIR)(rewriting/usr/binand/etc/pepsito the configuredBINDIR/SYSCONFDIR) plus thetmpfiles.dsnippet that gives/run/pepsiits owner and ACL. That snippet is the only source of the directory’s permissions, since no unit may declare it as aRuntimeDirectory(see No unit may claim it as a RuntimeDirectory). Everything ships disabled.make install INSTALL_ADMIN_UNITS=noomits the twopepsi-setup-applyunits, which is the source-install equivalent of not installing thepepsi-httpd-adminpackage.
In addition, install depends on the privileged-bit targets
(install-helper, install-maildir-stage, install-forward-*,
install-auto-pay-*, install-smarthost-stage,
install-whitelist-suid, install-crypto-suid, install-quota-tool)
described under Privileged binaries below; each prints the chown/
chmod to run by hand when it is not run as root, or when the group it
needs does not exist.
Override PREFIX, SYSCONFDIR and DESTDIR to suit the target
system. make uninstall removes everything that make install placed
(always including both pepsi-setup-apply units, whatever
INSTALL_ADMIN_UNITS says) but leaves the live pepsi.conf in
place.
Other make targets
make checkThe test entry point that needs no remote accounts. It runs the file-inspection cross-checks of the systemd units, the telemetry socket path, the version numbers and the two mail-client extensions, and the workspace tests (minus the dispatcher-lifecycle crate, which needs PostgreSQL). It then runs, best effort, the lifecycle suite against a local
pepsicheckdatabase, the setuid-helper contract tests, the authentication interoperability script,cargo deny check,cargo vet --lockedand the manual’s cross-reference check; each of these is skipped rather than failed when its prerequisite is absent.make integrationtests/make benchmarksThe live-pipeline scripts under
tests/, which need passwordless SSH to the hosts named inACCOUNTS(defaulttests/test-accounts.ini, created fromtests/test-accounts.ini.sample);integrationtestsfirst runs the Thunderbird interoperability gate. The benchmarks are a separate target because they deliberately load the MTA.make docsThe manual for the source language. It is an alias for
make docs-en, which builds its HTML and its PDF, plus the man pages and the info manual (both source-language only).make docs-<lang>builds one other language andmake docs-allbuilds every language inDOC_LANGUAGES(en de fr);make htmlbuilds every language’s HTML alone.make dist/make distcheckBuild the release tarball
pepsi-$(VERSION).tar.gz(every tracked file including the vendored submodule, plusconfigureand pre-built documentation), and verify it by unpacking it into a clean tree and runningconfigure,make,make check,make installandmake uninstallagainst a throwaway prefix. The tarball is copied from the working tree, somake distrefuses to run while a tracked file differs fromHEADor the submodule is not at the commit thatHEADrecords (untracked files do not count);make dist DIST_ALLOW_DIRTY=yesbuilds a development tarball anyway. The tarball is reproducible: entries sorted, every mtime the commit’s date (SOURCE_DATE_EPOCH), numeric uid 0, no gzip timestamp.make dist-signSign an existing release tarball: a detached, ASCII-armoured OpenPGP signature
pepsi-$(VERSION).tar.gz.ascwith gpg’s default key orDIST_SIGN_KEYID, verified before the target succeeds. See Verifying a download for the user’s side and Making a release for the keys.make debBuild the Debian packages into
target/debian/(Debian packages), unsigned;make deb DEB_SIGN_KEYID=<fingerprint>signs the.changesand.buildinfofiles with that key.make clean/make distcleancleanrunscargo cleanand removes the rendered documentation;distcleanadditionally removesconfig.mk,config.statusand anything thatmake distgenerated.
Note
DATADIR must match what the binaries compute at runtime:
taler-common derives ${DATADIR} as <install-prefix>/share/pepsi
from the binary’s own location. When running from a source checkout (not
installed), set [pepsi-postgres] SQL_DIR explicitly so pepsi-setup
finds the migrations.
Preparing the system
The Debian package’s postinst does the steps below automatically; on a
source install you do them yourself, using the same account, group,
mode and directory layout documented under Users, groups and privileges
and Directories and secrets below. Create the accounts and groups
before make install (so it can apply the privileged bits), and the
database roles before pepsi-setup (which installs the schema):
Create the system accounts, each
--systemwith/usr/sbin/nologinand home/var/pepsiand a private group of the same name, plus the gating groups.debian/pepsi.postinstis the authoritative list; this loop is it, translated:for u in pepsi pepsi-ingress pepsi-httpd pepsi-owner \ pepsi-helper-token-refresh pepsi-whitelist pepsi-crypto \ pepsi-keydisc pepsi-config pepsi-greylist; do addgroup --system "$u" adduser --system --home /var/pepsi --no-create-home \ --ingroup "$u" --shell /usr/sbin/nologin \ --disabled-password "$u" done for g in pepsi-maildir pepsi-forward pepsi-token pepsi-admin \ pepsi-telemetry; do addgroup --system "$g" done # The shared-wallet account owns its wallet databases, so unlike the # service accounts it gets a real home. addgroup --system pepsi-wallets adduser --system --home /var/lib/pepsi-wallets --ingroup pepsi-wallets \ --shell /usr/sbin/nologin --disabled-password pepsi-wallets install -d -o pepsi-wallets -g pepsi-wallets -m 0700 /var/lib/pepsi-wallets # Memberships: telemetry producers, and the SRS secret fragment. for u in pepsi pepsi-ingress pepsi-httpd pepsi-keydisc pepsi-greylist; do adduser "$u" pepsi-telemetry done adduser pepsi pepsi-ingress # The greylisting milter's socket belongs to group pepsi-greylist. adduser pepsi-ingress pepsi-greylist # The DKIM signing daemon: its own home holds the signing keys, and # membership of pepsi lets it hand its socket to the stage workers. addgroup --system pepsi-dkim adduser --system --home /var/lib/pepsi-dkim --no-create-home \ --ingroup pepsi-dkim --shell /usr/sbin/nologin \ --disabled-password pepsi-dkim adduser pepsi-dkim pepsi install -d -o pepsi-dkim -g pepsi-dkim -m 0700 /var/lib/pepsi-dkim install -d -o pepsi-dkim -g pepsi-dkim -m 0755 /var/lib/pepsi-dkim-public
Each of the last five accounts exists because something refuses to work without it:
pepsi-whitelistis the account that the whitelist CLI is setuid to,pepsi-cryptois the only database role grantedcrypto_identity.private_wrappedand the owner of the key-encryption fragment,pepsi-keydiscis the narrowest role in the deployment (it parses key material fetched from the open internet), andpepsi-configis the only role that may write the configuration overlay, andpepsi-greylistruns the greylisting milter, whose socket’s group it is. If you skippepsi-crypto,install-crypto-suidmerely warns, and the two crypto stages are left unprivileged and unable to open a single private key.Create the ten PostgreSQL login roles and the database (peer authentication, as the
postgressuperuser). This is the same set thatdebian/pepsi.postinstcreates: every account that authenticates as itself over the peer-authentication socket.pepsi-telemetryis among them even on a node that serves no telemetry, becausepepsi-setup rungrants rights to it on every host, andpepsi-setupdoes its database work aspepsi-owner, which cannot create a missing role:for r in pepsi-owner pepsi pepsi-ingress pepsi-httpd \ pepsi-whitelist pepsi-config pepsi-crypto pepsi-keydisc \ pepsi-telemetry pepsi-greylist; do createuser "$r" # createuser takes one role name at a time done createdb -O pepsi-owner pepsi
Run
make installasroot(above) so that the setuid/setgid bits are applied; otherwise apply them by hand per Privileged binaries below. Eachinstall-*target only warns when the group it needs is missing, so an account or group skipped above leaves the corresponding program silently unprivileged.Create the runtime directories with the ownership and modes listed under Directories and secrets. Create the smarthost-secret directories only if you configure a smarthost that uses them.
The package is the simpler path: it ships the binaries (installed to
/usr/bin), the SQL schema, the message templates, the man pages, the GNU
info manual and socket-activated systemd units, and its postinst
provisions the whole system for you.
Building the package
From a checkout (with the vendor/taler-rust submodule initialised) and
the build dependencies listed in debian/control installed, run:
make deb
It copies the working tree (including uncommitted changes and new files
that git does not ignore) into target/debian/pepsi-<version>/ and runs
dpkg-buildpackage -b -us -uc there (override with
DEB_BUILD_FLAGS, e.g. DEB_BUILD_FLAGS= to build and sign the
source package too). That compiles the release build and writes the
.deb files to target/debian/ (DEB_BUILDDIR chooses another
directory): pepsi, pepsi-httpd-admin, pepsi-telemetry and
pepsi-stage-detect-language. dpkg-buildpackage
always writes its results next to the source directory, which is why the
build does not run in the checkout itself.
Installing the package
Install the .deb with apt so that its dependencies (including
postgresql) are pulled in:
apt install ./pepsi_*.deb
On configure the postinst automatically:
creates the eleven service accounts and the six gating groups (
pepsi-maildir,pepsi-forward,pepsi-token,pepsi-admin,pepsi-telemetryandpepsi-wallets), with the memberships between them; these are the same identities that the source install creates by hand;creates the
/var/pepsiruntime directories (keys,tokens,token-refresh,tls,krb5) and/etc/pepsi/secretsand/etc/pepsi/secrets.dwith the ownership and modes documented under Directories and secrets below;applies the privileged-binary bits via
dpkg-statoverride: to the three setuid-root helpers, to the setgid programs that may exec them (the two delivery stages, the wallet stage andpepsi-quota), to the setgid smarthost relay, to the setuidpepsi-whitelistand to the two setuidpepsi-cryptostages (see Privileged binaries below); andbest-effort creates the ten PostgreSQL login roles (
pepsi-owner,pepsi,pepsi-ingress,pepsi-httpd,pepsi-whitelist,pepsi-config,pepsi-crypto,pepsi-keydisc,pepsi-telemetry,pepsi-greylist) and thepepsidatabase (owned bypepsi-owner) when a local cluster is reachable, and prints the roles to create by hand if it is not.
The service is shipped disabled: the package installs but does not start Pepsi, leaving the site-specific bootstrap to you.
Note
pepsi is one of four binary packages built from this source:
the language-detection stage and the telemetry collector are split out
for size and deployment reasons. pepsi-httpd-admin, which
pepsi Recommends and apt therefore installs by default,
carries the only mechanism by which the browser console can cause a privileged
change on this host. Removing that one package is a supported
hardening step for a deployment administered from a terminal.
Debian packages describes each package and that lever in full.
After the package is installed
On first installation the package copies a placeholder configuration to
/etc/pepsi/pepsi.conf; later upgrades never touch that file (it is not a
dpkg conffile, so an upgrade never stops to ask about it). Edit it, then
run the bootstrap exactly as in the shared steps below, for example:
# 1. edit /etc/pepsi/pepsi.conf (HOSTNAME, ACCEPTED_DOMAINS, domains, TLS)
# 2. install the schema, have pepsi-dkim make the DKIM keys and
# print the DNS records:
pepsi-setup --no-certbot run # or: pepsi-setup --wizard
# 3. publish the printed DNS records, then start the pipeline:
systemctl enable --now pepsi.target
Because the units are socket-activated, pepsi.target brings up
pepsi-ingress, pepsi-httpd, pepsi-dkim and pepsi-dispatch
together; you do not invoke the serve subcommands by hand.
macOS is not yet a supported deployment target: the package has not been
installed as root on a test system yet, and docs/ROADMAP.md lists
what still differs from Linux. The package installs the programs under
/usr/local (the system volume’s /usr is read-only) and the
configuration in /etc/pepsi, as on Linux. It installs launchd jobs
in /Library/LaunchDaemons (net.taler.pepsi.*) instead of systemd
units, one job for each unit, and its scripts do what the Debian
postinst does.
Building the package
On a Mac with the Xcode command line tools, a Rust toolchain and the
native libraries listed under From source (Nettle 3, not 4, or
CRYPTO_BACKEND=crypto-openssl), run:
make macos-pkg
This compiles the release build, stages it with the macOS layout and
writes target/macos/pepsi-<version>-<arch>.pkg with Apple’s
pkgbuild and productbuild. To sign the package, name a “Developer
ID Installer” certificate from your keychain in
MACOS_PKG_SIGN_IDENTITY; an unsigned package installs with:
sudo installer -pkg pepsi-<version>-<arch>.pkg -target /
What the installation does
It creates the same service accounts, groups and memberships as the Debian package, as hidden system accounts with ids from 400 up.
It creates the directories under
/var/pepsi,/var/liband/etc/pepsiwith the same owners and modes.It sets the same setuid and setgid bits on the privileged programs.
It adds
/runto/etc/synthetic.conf, which makes/runa link to/private/var/run. If/rundoes not appear at once, it appears after the next reboot.On an upgrade, it stops the jobs that are running, upgrades the database schema and starts the same jobs again.
It does not create the PostgreSQL roles and the database, because macOS
has no system PostgreSQL. Create them as described under From source,
in a cluster that uses peer authentication for local connections.
After the package is installed
sudo pepsi-setup --wizard
# publish the printed DNS records, then start the pipeline:
sudo /usr/local/libexec/pepsi/pepsi-launchd enable
pepsi-launchd stands in for systemctl and pepsi.target.
Without a job name, enable, disable, start, stop,
restart and status act on the jobs that pepsi.target pulls
in; the optional telemetry-client and helper-token-refresh jobs
are named explicitly (pepsi-launchd enable telemetry-client). Each job
writes its log to /var/log/pepsi/<job>.log. To remove the package, run
sudo /usr/local/libexec/pepsi/pepsi-launchd uninstall. Like
apt remove, this keeps the configuration, the accounts, the database
and the data under /var.
Three things work differently from Linux:
Without socket activation,
pepsi-ingressandpepsi-httpdbind their ports themselves (SERVE = tcp, which the wizard writes on a host without systemd). Both start as root, read their TLS certificates and keys, and then drop to their accounts.pepsi-setuprestarts their launchd jobs when their TLS files change, and the certbot deploy hook restarts them after a renewal./etc/pepsi/postgres.envis read only by the jobs that run as root. Give each account its own credentials through the{role}placeholder in[pepsi-postgres] CONFIGinstead.The setup applier (
pepsi-httpd-adminon Debian) has no launchd job, so the browser console’s configuration pages are read-only; administer the deployment withpepsi-setupin a terminal.
4.3. Verifying a download¶
Release tarballs are signed with the release manager’s OpenPGP key: Christian
Grothoff’s or Florian Dold’s, the same keys that sign GNU Taler’s releases on
ftp.gnu.org. The signature is the detached .asc file published next to
the tarball; verify it before unpacking:
gpg --verify pepsi-X.Y.Z.tar.gz.asc pepsi-X.Y.Z.tar.gz
gpg must hold the signing key, and the check is worth exactly as much as
your reason to believe that the key is the release manager’s. Christian Grothoff’s
key is published at https://grothoff.org/christian/grothoff.asc with the
fingerprint D842 3BCB 326C 7907 0339 29C7 939E 6BE1 E29F C3CC (the same
key that Reporting a vulnerability names); both keys are in the GNU keyring,
https://ftp.gnu.org/gnu/gnu-keyring.gpg, which gpg --keyring
gnu-keyring.gpg --verify … uses without importing. A good signature says
that the tarball is the one that the release manager signed; whether it is also
what the source produces is the subject of Reproducible builds.
Debian packages on deb.taler.net are signed with the taler-systems release
key, which signs every package there, and apt verifies them against the
keyring that you install once:
wget -O /etc/apt/keyrings/taler-systems.gpg https://taler.net/taler-systems.gpg.key
named by signed-by= in the repository’s sources entry. A package built
from the source tree with make deb DEB_SIGN_KEYID=… carries a signed
.changes file that dscverify and debsign --verify check; a package
built without a key is unsigned, which is what a local build is.
4.4. Reproducible builds¶
The release build is reproducible: the same commit, built with the pinned
toolchain from the same Cargo.lock, gives byte-identical binaries and the
same tarball wherever and whenever it is built. A published binary can
therefore be checked against the source by rebuilding it, and this is the one
check that catches a compromised build host or release manager’s machine. The CI job
5-reproducible performs that check on every run, building twice from two
clones at different paths with different crate caches (Continuous integration).
What the recipe pins, and where:
the toolchain —
rust-toolchain.tomlnames the exactrustcversion;rustupuses (and installs) that one in this tree. A toolchain not managed byrustupignores the file, and a different compiler gives a different binary, so userustupto reproduce a release;the crates —
Cargo.lockwith--locked, which everymaketarget passes;the paths —
make buildmaps the checkout, the target directory (CARGO_TARGET_DIR, ortarget/in the checkout) and the crate cache (CARGO_HOME, default~/.cargo) to the fixed names/build,/targetand/cargo, each under its own and its canonical path, forrustc(--remap-path-prefix) and for the C and C++ compilers that the native crates run (-ffile-prefix-map). So neither the location of the tree nor that of the crate cache reaches the binary.strip = trueremoves the symbol tables. A symbolic link inside the crate cache is not covered: keep it a real tree;the time —
SOURCE_DATE_EPOCHis exported as the commit’s date and is what the tarball’s mtimes and the documentation’s dates use; passSOURCE_DATE_EPOCH=<seconds>to override it in a tree that is not a git checkout;the tarball —
make distwrites it withtar --sort=name --mtime=@$SOURCE_DATE_EPOCH --owner=0 --group=0 --numeric-ownerandgzip -n.
To reproduce a release, check out its tag with the submodule, build, and compare:
git clone --recurse-submodules https://git.taler.net/pepsi.git
cd pepsi && git checkout vX.Y.Z
make build # the multi-call binary and the standalone programs
make dist # the tarball
sha256sum target/release/pepsi target/release/pepsi-setup pepsi-X.Y.Z.tar.gz
Then compare the sums with those of the published artefacts. The binaries
inside a .deb (unpacked with dpkg-deb -x) are the files that make
build produces, but built with the toolchain of the package build, which is
Debian’s own rustc and ignores rust-toolchain.toml. So reproduce a
package with the rustc that its .buildinfo file names. A different
RUSTFLAGS or CFLAGS in the environment, a different crate cache
content (a crate whose checksum differs fails --locked anyway), or
another toolchain gives another binary; diffoscope shows where two builds
differ.
4.5. Migrating from an existing mail server¶
If this host already runs Postfix, Exim, Sendmail, qmail or Stalwart, you do not
have to re-derive its configuration by hand. With no pepsi.conf present,
pepsi-setup --wizard detects what is installed and offers to import it: the
hostname, accepted domains, smarthost (with the relay credentials read from that
server’s own password file), local-delivery method, submission SASL socket,
message size limit, queue lifetime and TLS certificate paths all arrive as the
interview’s pre-filled answers, which you then confirm or correct one by one.
The old server’s trusted networks are listed in the migration report instead:
Pepsi lets a network relay only as the addresses that [pepsi-ingress]
SENDER_MAP binds it to, so each network needs a binding that you write by
hand.
To see what a migration would produce before running the wizard, run the following; it writes nothing:
pepsi-setup import
Routing tables are converted too. /etc/aliases, /etc/postfix/virtual,
Sendmail’s virtusertable, qmail’s .qmail-* files and their equivalents
become a Pepsi alias map beside the configuration (see
pepsi-stage-aliases), with bare alias names qualified in the
style you choose. Since Pepsi alias keys must carry a domain, postmaster:
root becomes postmaster@example.org root@example.org for each accepted
domain by default.
Everything that does not carry over is written to import-report.txt
beside the configuration, split into what failed to parse, what Pepsi has no
equivalent for (header_checks, an alias delivering to a command),
what was approximated (mbox delivery becoming Maildir) and what the importer did
not recognise. An import never aborts setup: an unreadable file or an unknown
directive produces a report entry, not a failure.
Two things are deliberately not migrated: queued mail (let the old server’s
queue drain before switching the MX over) and DKIM keys (pepsi-setup run
generates fresh ones and prints the records to publish). If the old MTA still
holds port 25 when the wizard runs, the report says so; stop and disable it
before you start pepsi-ingress.
See pepsi-setup for the per-server detail, the
--alias-style choices and the --expert options that expose the settings
that the interview does not otherwise ask about.
4.5.1. Mail filters already on the host¶
Independently of any migration, the wizard scans the host for milter daemons
(clamav-milter, rspamd, spamass-milter, milter-greylist,
milter-regex, mimedefang, amavisd-milter) and offers each one that
it finds. Finding one means three things in turn: the package is
installed, a socket can be read from its own configuration (or its unit, or the
paths that its distribution ships), and a real milter option negotiation completes
against that socket. Nothing is offered on the strength of an installed
package alone, and the negotiation is also how the generated ALLOW_ACTIONS
comes to grant exactly what that filter asks for rather than a guess.
pepsi-ingress consults every accepted filter during the SMTP session,
after its own checks, as an MTA would: a filter that refuses mail answers the
sending server with its own 4xx or 5xx, so nothing that was accepted has to be
bounced later. The filters are consulted in an order that depends on what they
do: policy and greylisting, then virus, then spam, then general-purpose
frameworks. Milters that an MTA import found (Postfix smtpd_milters,
sendmail INPUT_MAIL_FILTER) are carried over the same way, with the
failure action, timeouts and macros that the old MTA used. The wizard also
writes [stage-quarantine], which holds a message that a filter asks to
quarantine until you release it. See Milters in the SMTP session for the
generated sections and pepsi-ingress for how the session uses
them.
Two kinds of filter run as pipeline stages of pepsi-stage-milter
instead, after the message was accepted: a filter that ran on locally
submitted mail (Postfix non_smtpd_milters), and, with --expert, a
filter that you place after decryption so that it sees plaintext. Filters whose
job Pepsi already does (opendkim, openarc, opendmarc, SPF daemons,
postsrsd) are named and skipped rather than offered.
The wizard also asks whether to greylist with Pepsi’s own greylisting milter,
pepsi-greylist. The answer defaults to no, because greylisting delays the
first message from every new correspondent. pepsi-setup check negotiates
with every configured milter and reports one that does not answer, or that asks
for an action that its ALLOW_ACTIONS does not grant.
4.6. Provisioning the database, keys and DNS¶
With a configuration file in place (see Configuration), run the bootstrap tool once:
pepsi-setup -c /etc/pepsi/pepsi.conf run > pepsi-dns.zone
This validates the whole configuration, installs the pepsi schema,
has the signing daemon pepsi-dkim generate the DKIM keys for
each domain (RSA and Ed25519, for every domain that has none), and prints
the DNS records to publish. These are the DKIM public keys, an SPF policy from
the configured PUBLIC_IP addresses, an MTA-STS record and, for each TLS
ingress listener, a DANE TLSA record. That record pins the mail
certificate’s public key, which is why certbot certificates are obtained with
--reuse-key (see Changing the key behind a TLSA record). The mail listeners and
pepsi-httpd’s HTTPS listener get separate certbot lineages, and so separate
keys: /etc/letsencrypt/live/<HOSTNAME>/ for mail and
/etc/letsencrypt/live/<HOSTNAME>-web/ for the web server. The web server
therefore never holds the key that the TLSA record pins. (An installation
whose HTTPS listener still uses the mail certificate is moved to its own at the
next pepsi-setup run, once that certificate has been obtained.) The
operation is idempotent and may be re-run at any time; upgrading the schema to
a new release is pepsi-setup schema (see Upgrading). run --reset
drops and recreates the schema; this destroys all queued mail but keeps the
DKIM keys, which are not in the database.
Publish the printed DNS records, then verify what is live against what Pepsi expects:
pepsi-setup -c /etc/pepsi/pepsi.conf check
See pepsi-setup for the full provisioning workflow.
4.7. Setting up in a browser¶
Everything above can also be done from a browser, against the same code. The terminal path is the shortest route on a machine where you have a shell; the browser path is for machines where you do not, and for operators who would rather see the DNS records as a red/green checklist than as a zone file.
4.7.1. The split, and why¶
Setup is privileged: it writes /etc/pepsi/pepsi.conf, hands each
secrets.d fragment to the single account that reads it, runs certbot, creates
database roles and generates key material — all as root. pepsi-httpd, which
serves the interface, drops privileges before it accepts a connection and must
never regain them.
So it does not act. It writes an intent row that says what should be true. A
separate root program, pepsi-setup apply, decides how, does it, and records
what happened; it is started on demand and exits again when it has nothing to
do. Root is never on the network. Read “The applier’s trust model” in
:doc:`programs/pepsi-setup` before enabling this: it states exactly what the
applier will and will not do, including the things that it deliberately cannot
do (there
is no restart or shutdown task, and install-schema refuses reset).
4.7.2. Enabling it¶
The Debian packages ship a minimal but complete configuration, an administrative
listener on a UNIX socket, and the applier’s socket unit. The socket unit is in
pepsi-httpd-admin, which pepsi Recommends and apt therefore
installs by default (see Debian packages). Two commands enable it:
# systemctl enable --now pepsi-httpd.socket pepsi-setup-apply.socket
# pepsi-setup -c /etc/pepsi/pepsi.conf bootstrap
The second prints a one-time token and the curl command that turns it into
the first administrator account. On the machine itself you can skip it entirely:
a member of the pepsi-admin group (and root) is identified by
SO_PEERCRED over /run/pepsi/admin.sock and needs no credential at all.
One more setting is needed before anything privileged can be asked for:
[pepsi-admin] CONFIG_DB must name a connection that authenticates as the
pepsi-config database role. That is the boundary: only that role may enqueue
setup work, and pepsi-httpd’s own account is not it, so a compromise of the
web tier is not a compromise of the pipeline definition. Until it is set the
setup endpoints answer 503 and say so.
4.7.3. Driving it¶
The interview is data. GET /api/v1/setup/questions returns the ordered steps
and, for each question, its shape, default, help text and the condition under
which it is asked. This is the same model that pepsi-setup questions prints
and that describes the terminal wizard; the test suite asserts that they are
equal.
Answers are staged with PUT /api/v1/setup/answers into draft rows that no
running process can see, so an interrupted session cannot half-configure
anything.
POST /api/v1/setup/tasks then asks for the privileged half: writing the
configuration, obtaining certificates, installing the schema and generating
keys. GET /api/v1/setup/tasks/{id}?since=<seq> streams what the applier is
doing, line by line.
4.7.4. What will always need a text editor¶
Some settings are read from the configuration file only, and no interface will
change that: [pepsi], [pepsi-postgres], [paths], and the HTTP(S) and
ingress listener sections. They have to work before a database connection
exists, or they are a security boundary: if the database could move a listening
socket or relax the crypto policy, a database compromise would become a
compromise of the MTA’s own TLS and ports.
So adding a submission port, or moving a listener, is a
text-editor-and-restart operation. So is any change the console marks
restart rather than hot: there is no restart button, because a button
that restarts a service on request would hand the web tier’s privileges to
whoever reaches it.
4.7.5. Scripting it¶
The same answer model drives an unattended install:
pepsi-setup --wizard --answers answers.json -c /etc/pepsi/pepsi.conf
pepsi-setup questions prints the schema against which that file is written.
This is not a third implementation of the interview: it runs the interactive one
with every prompt answered from the file, so the branches and the per-field
validation are the same ones that a human sees.
4.8. Running the services¶
These long-lived processes run continuously (pepsi.target starts them all
under the shipped units):
pepsi-ingress serve— accepts mail (typically under a service manager, optionally with systemd socket activation for the privileged ports).pepsi-dispatch serve— runs exactly one instance per system; it is the only process that starts stage programs.pepsi-dkim serve— holds the DKIM and ARC private keys and signs for the signing stages; the dispatcher’s unit pulls it in.pepsi-httpd serve— the MTA-STS policy, the Web Key Directory, the secure-link portal and, on administrative listeners, the API and console.pepsi-keydisc serve METHOD— one instance per key-discovery method (pepsi-keydisc@dane,@wkd-advanced,@wkd-directand@vksby default), answering the crypto stages’ key requests.
One more runs only where greylisting is turned on, from its own unit outside
pepsi.target:
pepsi-greylist serve— the greylisting milter thatpepsi-ingress(or a Postfix in front of it) asks atRCPT.
The timers that pepsi.target also wants (failure bouncing, queue and log
garbage collection, key refresh and retirement, quota reconciliation, the
mailing-list tasks and digests) run one-shot jobs; systemctl list-timers
'pepsi-*' lists them.
The stage programs themselves are spawned by the dispatcher as persistent
PROGRAM worker processes that read message ids on standard input; you do not
run them by hand in production (for debugging you can pipe an id to worker,
e.g. echo 42 | pepsi-stage-srs -c /etc/pepsi/pepsi.conf worker).
4.9. Upgrading¶
From the first release, 0.0.0, on, every release upgrades the database of any earlier release in place: the queue, the key store, the settings and everything else in it are kept, and mail waiting in the queue is delivered by the new release. Downgrades are not supported (see below).
The procedure:
Read
NEWSfor the release. It lists configuration options that changed, and anything else that needs doing by hand.Take a backup (Backup and restore), or at least enable the automatic dump described below.
Install the new release. The Debian packages do the rest: they upgrade the schema and restart the services that were running. From source:
make install, thenpepsi-setup -c /etc/pepsi/pepsi.conf schemaandsystemctl try-restart pepsi.target.Check with
pepsi-statusthat the queue drains, and look at the journal for warnings (Logs).
A release may also refuse a configuration that an earlier one accepted, when the
earlier behaviour was a security weakness (NEWS says which); the services
then do not start. Run pepsi-setup -c /etc/pepsi/pepsi.conf validate before
restarting them: it applies the same checks and names what to change.
The rest of this section explains what happens during step 3.
Every Pepsi program checks, as it connects to the database, that the schema was
built from exactly the SQL files that the program itself was built with. The
installer records the SHA-256 and release of each file that it applies in
pepsi.schema_file. A program that finds anything else (an older schema, a
newer one, or one built from different files) stops at once with exit status
78 and a message naming the fix, rather than failing at its first
missing column in the middle of a message. The dispatcher treats a stage worker
that stops this way as “cannot run yet”, not as a crash: the message it was
handed is put back in the queue, never failed.
Upgrading the schema is its own step:
pepsi-setup -c /etc/pepsi/pepsi.conf schema
It installs the patches that this release adds, replaces the stored functions
and re-applies the role grants. It does nothing else (no certificates, keys or
DNS), so it needs only the [pepsi-postgres] section. On a telemetry collector,
give it the collector’s file, /etc/pepsi-telemetry/pepsi-telemetry.conf.
Debian packages run it automatically when they are upgraded, before the
services are restarted; a new installation is left to pepsi-setup run. If
it fails (the database is down, say), the package still installs, a notice says
so, and the services refuse to start. systemd retries them until the schema
matches, backing off to once a minute, without a start limit. Once the
cause is fixed, running the command above is all it takes; the services come
back by themselves.
From source, after make install:
pepsi-setup -c /etc/pepsi/pepsi.conf schema
systemctl try-restart pepsi.target
What the refusals mean:
The message says |
What to do |
|---|---|
no |
A new installation: |
older than this program |
|
newer than this program |
A downgrade, which is not supported. Install the newer release again, or restore the database backup taken before the upgrade (below). |
built from a different copy of a patch |
A released patch never changes, so this is a bug; please report it. The
one exception is a database built from a development snapshot (a Git
checkout between releases), where the newest patch is still being
edited: drain it and re-create it with |
stored functions from a different build |
Development builds only: |
4.9.1. Upgrading from 0.0.0: the DKIM keys¶
Release 0.0.0 keeps its DKIM and ARC keys as files under [pepsi] KEY_DIR,
readable by the pipeline. Later releases hold them in the signing daemon
pepsi-dkim, under its own account and in its own directory.
The daemon does not take over the 0.0.0 key files: it generates fresh keys
under new selectors (pepsi-YYYYMM and pepsi-YYYYMM-ed25519) and signs
with them at once. It signs only for the domains that pepsi-setup run hands
it; until that has happened, outbound mail
waits at the signing stage and is retried, so run it right after installing:
Install the new release as above.
Run
pepsi-setup -c /etc/pepsi/pepsi.conf run. It hands the signing domains topepsi-dkim, which makes the keys, deletes the old key files underKEY_DIRand prints the new DKIM records, with the oldpepsiandpepsi-ed25519records as removable comments.Publish the new records. Until they are published, receivers cannot verify the signatures and DMARC rests on SPF alone;
pepsi-setup check,pepsi-dkim statusand the daemon’s mail to the postmaster say what is still missing.Remove the old
pepsi._domainkeyandpepsi-ed25519._domainkeyrecords once mail signed with the old keys has been delivered, a week after the upgrade at most.
Leave /var/lib/pepsi-dkim out of backups from then on
(Backup and restore). Rotation stays off unless you turn it on
(Rotating DKIM keys).
4.9.2. Downgrades¶
Not supported. pepsi-setup refuses to install an older release’s SQL over a
newer schema (exit status 78, nothing changed), and the older release’s
programs refuse to run on it. Going back to an older release means restoring a
database backup taken before the upgrade, then installing the older packages.
No Pepsi interface can install an older release, so a rollback to a vulnerable
version needs root acting deliberately (see
Signed firmware updates accepted without version check).
4.9.3. Backing up before a schema upgrade¶
No backup is taken unless you enable it. When enabled, the backup is taken only
when an upgrade actually changes the schema: pg_dump saves the pepsi
schema (with the extensions it uses) as
pepsi-<previous version>-<UTC time>.dump, readable by root only. If the dump
fails, the schema is not upgraded. After each new dump, only the two newest are
kept in the directory (pepsi-setup schema --backup-keep; 0 keeps them
all): a dump holds the whole queue, including the plaintext of decrypted mail
waiting in it, so old dumps are copies of mail that would otherwise have left
the host long ago. Copy a dump elsewhere if it has to outlive two upgrades.
With the Debian packages it is a debconf setting, asked only at priority low:
dpkg-reconfigure -plow pepsi
Answer yes, and confirm or change the directory (default
/var/backups/pepsi). Unattended, or before the first upgrade:
echo "pepsi pepsi/schema-backup boolean true" | debconf-set-selections
echo "pepsi pepsi/schema-backup-dir string /var/backups/pepsi" | debconf-set-selections
On a telemetry collector host the questions belong to pepsi-telemetry
(pepsi-telemetry/schema-backup, pepsi-telemetry/schema-backup-dir).
When both packages are installed and use the same database, whichever upgrades
it first takes the backup.
From source, pass the directory to the upgrade step:
pepsi-setup -c /etc/pepsi/pepsi.conf schema --backup-dir /var/backups/pepsi
To restore one, stop the services, re-create the database empty and owned by the schema owner, and restore into it as that owner:
systemctl stop pepsi.target
sudo -u postgres dropdb pepsi
sudo -u postgres createdb -O pepsi-owner pepsi
sudo -u pepsi-owner pg_restore -d pepsi /var/backups/pepsi/pepsi-0.0.0-20261001T000000Z.dump
Then install the release that the backup came from and start the services
again. Everything that the database learnt after the backup is lost, such as
mail accepted since then and settings changed since then. The stored private keys in the dump are encrypted
under the key-encryption key in secrets.d, which is not in the database: keep
that file, and back it up separately (see Backup and restore). The keys are
sealed with an AEAD, so a modified blob in a dump never opens (see
Unauthenticated private-key encryption lets the storage holder maul keys); without the file they are lost
(see Key management too hard: end-to-end mail clients error-prone or unused).
4.10. Users, groups and privileges¶
Pepsi never runs as root. The long-lived daemons may be started as
root so that they can bind privileged ports and read root-only TLS keys, but each one
drops to its own unprivileged service account before serving a single
connection; if the drop cannot be completed the process refuses to start rather
than risk running as root (see pepsi-common::privdrop). Acting as another
local user, such as writing into their mailbox, running their ~/.forward or
using their wallet, is the one kind of operation that needs more rights than
the service user has. Each such operation is confined to a narrowly scoped
setuid helper rather than handed to the pipeline at large. The privilege split
describes the split as a security mechanism, and Trust boundaries the
boundaries that it upholds.
Under the shipped systemd units the daemons are never root at all: systemd
binds the privileged sockets (socket activation) and starts each process as its
service account directly. That leaves the TLS keys, which certbot keeps readable
by root only. pepsi-setup therefore has systemd read those too, by writing a
LoadCredential= drop-in per unit: systemd opens each certificate and key as
root when it starts the unit, and hands the service a private copy under
$CREDENTIALS_DIRECTORY, which is where the server looks first. Re-run
pepsi-setup run whenever you add, move or remove a certificate, so that the
drop-in is regenerated. See pepsi-httpd.
The browser setup described above is the one path by which something other
than an operator at a terminal can cause a root process to run: pepsi-httpd
rings a doorbell socket and systemd starts a short-lived root pepsi-setup
apply. That path is the easiest privilege in Pepsi to take away.
Its two systemd units are a separate Debian package, pepsi-httpd-admin,
whose source-install equivalent is make install INSTALL_ADMIN_UNITS=no.
Without them, nothing drains pepsi.setup_task and no HTTP request can change
anything under /etc/pepsi, while pepsi-setup on a terminal keeps working
unchanged. Putting the package back does not act on what was requested while it
was absent: installing it empties that queue before its units are armed. See The split as a hardening lever.
The accounts and groups below are created automatically by the Debian package’s
postinst; on a make install you create them yourself (the same names).
PostgreSQL access is by peer authentication over the local socket: each
service account logs in as a same-named database role, so no passwords are
stored anywhere. A remote database keeps the same model: one
login role per account, each authenticated by a client certificate or password
file of its own that only that account can read, named per role with the
{role} placeholder in [pepsi-postgres] CONFIG (see CONFIG in
pepsi.conf(5)). The configuration file never holds a password.
4.10.1. Service accounts¶
These are system accounts (--system, /usr/sbin/nologin, home
/var/pepsi), each with a private primary group of the same name. The first
three are long-lived daemons that drop to their account after binding; the rest
are not daemons in the pipeline.
Account |
Runs |
Purpose / access |
|---|---|---|
|
|
Inbound SMTP server. Socket-activated under systemd (never root, TLS
material arrives as a service credential); started by hand it binds
25/465/587 and reads the TLS keys as root, then drops. DB role
|
|
|
HTTP/HTTPS server (MTA-STS policy, Web Key Directory, secure-link
portal and, on administrative listeners, |
|
|
The pipeline’s working identity. Has messages signed by |
|
|
Not a runtime service. Owns the |
|
|
Only used when a smarthost uses |
|
|
Custodian of private end-to-end key material. Owns
|
|
|
Account that the setuid whitelist CLI assumes, so that ordinary users can maintain
their own sender whitelists. Its database role is granted
|
|
|
Key discovery (WKD, DANE, VKS, LDAP). Parses key material fetched from the open internet, so its database role is the narrowest in the deployment. See Key management. |
|
|
The only database role that may write the |
|
|
The greylisting milter. Its group is the group of its socket
|
|
|
Custodian of the DKIM and ARC private keys, 0600 files in its home
|
The package postinst (or, on a manual install, you) creates the ten
PostgreSQL login roles before pepsi-setup installs the schema:
pepsi-owner, pepsi, pepsi-ingress, pepsi-httpd,
pepsi-telemetry, plus the narrow pepsi-whitelist, pepsi-config,
pepsi-crypto, pepsi-keydisc and pepsi-greylist roles. pepsi-setup then grants each
role what it needs.
4.10.3. Privileged binaries¶
These bits are set by make install when run as root (it prints the exact
chown/chmod to run by hand otherwise), and by the Debian package via
dpkg-statoverride, which is the authoritative list
(debian/pepsi.postinst):
Binary |
Mode / owner |
Why |
|---|---|---|
|
|
Setuid root so it can write into any local user’s mailbox; group-locked
so only |
|
|
Setgid so the |
|
|
Same group and same reason: |
|
|
Setuid root so it can run a user’s |
|
|
Setgid so the |
|
|
Setuid root so it can drop to the wallet-owning account (or the
recipient’s own uid) to run |
|
|
Setgid so the |
|
|
Setgid so the |
|
|
Setuid, and world-executable on purpose: any local user may manage their
own |
|
|
Setuid, not setgid: PostgreSQL peer authentication keys off the
effective uid, and |
Every other binary is installed unprivileged, including pepsi-keys, which is
standalone for the same reasons but ships 0755: one binary able to open every
private key in a deployment is a much larger prize than pepsi-whitelist’s
single table, so making it setuid pepsi-crypto is a per-site decision rather
than the default.
4.10.4. Directories and secrets¶
pepsi-setup and the package create the following under /var/pepsi (the
service home) and /etc/pepsi. The setgid (2750) directories cause new
files to inherit the directory’s group, so token/TLS/Kerberos material is
group-readable by the relay without any per-file chgrp.
Path |
Owner / mode |
Contents |
|---|---|---|
|
|
Service home. |
|
|
The DKIM and ARC private keys ( |
|
|
The daemon’s public |
|
|
The signing socket |
|
|
The greylisting milter’s socket |
|
|
OAuth access-token files: written by the refresher, read by the smarthost
relay via group |
|
|
Private rotated refresh tokens; never group-readable. |
|
|
The |
|
|
The fragments |
|
|
Mutual-TLS client cert + key for the smarthost relay
( |
|
|
Kerberos credential cache for |
|
|
The local submission, administrative, HTTP and setup-applier sockets.
Created by the |
|
|
Home of the |
The /var/pepsi/tokens, /var/pepsi/tls and /var/pepsi/krb5 directories
(and the pepsi-helper-token-refresh account) only matter when you configure a
smarthost with the corresponding AUTH mechanism; a default direct-to-MX or
local-delivery deployment uses none of them.
4.11. Migrating from GNU Mailman¶
An existing GNU Mailman installation — 2.1 or 3 — moves onto Pepsi with
pepsi-list import and pepsi-archive import. Read the table below before
you start, not afterwards.
The behavioural differences are a different table, and it is not duplicated
here: Declared differences from GNU Mailman 3 in Mailing lists collects every place
where Pepsi deliberately does something other than what upstream does. These
are one-click unsubscribe, the always-per-recipient fan-out, the archive’s
obscured sender addresses, the purge tombstone, the renamed X-Mailman-*
headers, and what is simply absent (no NNTP gateway, no pluggable archivers, no
plugin API). Two copies of that table would diverge, and the copy that a
migrating operator happened to read would be the stale one. This section is only about what the importer
leaves behind.
4.11.1. What does not come across, and why¶
Why |
|
|---|---|
Passwords |
Neither Mailman core’s |
Bounce scores |
Upstream does not import them either: |
In-flight tokens |
A pending confirmation or moderation token is a promise about a workflow that will not exist after the cutover. Anybody mid-subscription asks again. |
Topics (2.1) |
No Mailman 3 equivalent. |
Template URIs we refuse to fetch |
A |
One Pepsi setting is decided by the import rather than carried over: a list the
importer creates gets hold_dmarc_failures = no, so it keeps distributing
posts that fail DMARC as GNU Mailman did; the report says so per list. Turn it
on with pepsi-list list set-ext <list> hold_dmarc_failures yes once the
list’s posters are known to pass (Settings Pepsi invented).
Anything else the importer did not understand is in its report. It exits
non-zero when something was lost, which is the point: a migration that half
worked and exited zero is a migration nobody checks. Nothing is rolled back, and
re-running an import is safe: every write is keyed on the list id, the
address or the Message-ID, so a second run is a no-op.
4.11.2. The recommended order¶
Do it in this order. The reason is in step 4.
Import the configuration and the roster, against the running old server:
pepsi-list import mailman3 --rest http://localhost:8001 \ --user restadmin --password-file /etc/mailman3/rest.pw \ --report /root/mailman-import.txt
Or, for a Mailman 2.1 site whose server is already off:
pepsi-list import mailman21 --listdir /var/lib/mailman/lists \ --report /root/mailman-import.txt
Add
--dry-runfirst: it reads everything, reports everything and writes nothing.Import the archive, per list:
pepsi-archive import --list announce@lists.example.org \ --rejects /root/rejected.txt /var/lib/mailman/archives/private/announce.mbox/*.mbox
Threads are reattached and renumbered automatically at the end. Otherwise an out-of-order import would leave them split, which is the mistake that this command is designed not to repeat.
Verify.
pepsi-list check --strict, then spot-check a few lists’ settings against the old server. Over REST both ends answer/3.1/lists/<id>/config, so the comparison is per-attribute rather than by eye.Warm the IP with the invitation run, while the old server is still delivering mail.
pepsi-list invite send --dry-run # the per-domain histogram pepsi-list invite send --rate 200 --limit 500 pepsi-list invite status
This is the step whose timing matters. Telling every member to choose a password is the largest single mailing that this server will ever send. It goes to an address list that the server has never validated, from an IP address with no reputation, and that is how a new deployment’s reputation is destroyed on day one. If you do it before the MX switch, a throttling or blocking problem becomes visible while the new server does not yet carry the mail.
The dry run’s histogram is a deliverability tool, not a progress bar: one provider holding forty per cent of a migrated site is the fact that decides the rate, and it is invisible until something counts it. Invitations last four weeks, the run resumes where it stopped, and nobody is unsubscribed for not answering: a member who never acts keeps their subscription and simply cannot sign in until they use the link or ask for another.
Switch the MX, and only then stop the old server.
4.11.3. When our pickle reader is stricter than theirs¶
pepsi-list import mailman21 reads config.pck with its own parser, which
constructs nothing: no module is looked up and no callable is called, because
a config.pck comes from somebody else’s server and Python’s pickle.load
on untrusted input is arbitrary code execution.
The price of that is strictness. The reader may refuse a file, for example because of an opcode that it does not implement, a protocol newer than Python 2 could write, or an integer too wide for 64 bits. It then says so and names the escape hatch:
python2 contrib/mm21-export.py /var/lib/mailman/lists/announce/config.pck \
> /root/announce.json
pepsi-list import mailman21 --json /root/announce.json --list announce
That script runs on the old server, with the Python that server already trusted, and dumps the same dictionary as JSON. Both routes then go through the same mapping, so they cannot disagree about what the file means. A migration must never be blocked by our parser being stricter than the one that wrote the file.
4.11.4. Header renames a user will notice¶
Pepsi emits X-Pepsi-List-* where Mailman emitted X-Mailman-*; the mapping
table is in Mailing lists. This is a migrating user’s real cost, because
procmail and Sieve rules are written against those names, and it is worth telling
them before the switch rather than after.