.. This file is part of PEPSI. Copyright (C) 2026 Pepsi contributors PEPSI is free software; you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. PEPSI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. =========== Test Suite =========== Pepsi ships an **end-to-end test suite** under ``tests/`` that exercises a live deployment over SMTP, asserting the *observable* behaviour of the whole pipeline rather than any single function in isolation. It is deliberately a black-box suite: messages are sent with the ordinary ``mail`` command or by raw SMTP, and the assertions read the recipients' mailboxes and the ``pepsi.ingress`` queue. This complements — it does not replace — the in-tree unit tests (``cargo test --workspace``), which cover the pure logic (SRS, MIME downgrade, DSN construction, language scoring, DNS classification, …) without a network. Two things are covered in between, by integration tests that need neither a network nor a database. The **migration** suites (``pepsi-setup/tests/import_*.rs`` and ``tests/cli_import.rs``) run each MTA importer against a fixture ``/etc`` tree and then run the real ``pepsi-setup`` binary end to end, up to a ``--wizard --import`` that must produce a configuration passing Pepsi's own validation; see :ref:`the extending chapter ` for their layout. The **dispatcher lifecycle** suite (``pepsi-test-stages``) needs a PostgreSQL and is run by ``make check``. .. _testing-doc-builds: The documentation builds are checks too ======================================= ``make docs`` runs all of them for the source language, and is the one documentation target in the general quality gate. It is an alias for ``make docs-en``; ``make docs-de`` and ``make docs-fr`` build the same artifacts from the translated catalogs and ``make docs-all`` builds every language. Four separate builds sit under it, and passing one says little about the others: ``make docs-html-en`` Sphinx HTML. Catches broken cross-references, unknown roles and malformed tables. ``make man`` The section-1/5 man pages through ``docs/rst2man.py``. Catches title over/underlines that do not match the title length — a docutils error the HTML build tolerates. Source language only: the roff pages are rendered straight from ``docs/*.rst`` and never go through Sphinx, so they have no translation catalog. ``make info`` The GNU info manual, through Sphinx's texinfo builder and ``makeinfo``. Source language only, for the same reason ``install-info`` registers one ``pepsi.info`` in one info ``dir`` index. ``make docs-pdf-en`` LaTeX. **The one most easily forgotten, and the one that fails hardest.** pdflatex has no glyph for an arbitrary Unicode character and does not degrade when it meets one: it aborts with *"Unicode character … not set up for use with LaTeX"* and produces no PDF at all. HTML and man render the same character without complaint, so a new arrow, box-drawing character or Greek letter in the prose passes both and breaks only this build. Every non-ASCII character used in the manual must be declared in ``latex_elements["preamble"]`` in ``docs/manual/conf.py``. The box-drawing set used by the ASCII-art pipeline diagrams is mapped to plain ASCII on purpose: those diagrams sit inside ``code-block:: text``, so the replacement is expanded in a verbatim environment where math mode is unavailable. The second failure mode is **a table that does not fit on a page**. Sphinx renders a ``list-table`` into LaTeX's ``longtable`` — which splits across pages and repeats the header — only past a row-count threshold; below it the table becomes a ``tabular``, which cannot break at all. The heuristic counts *rows*, but what overflows a page is *height*, and those diverge badly for a wide table whose cells wrap: the 113-row RFC summary table splits happily while the 17-row, seven-column comparison table in :doc:`introduction` did not, and ran 397 pt off the bottom of the page. The fix is to say so explicitly rather than rely on the heuristic:: .. list-table:: :header-rows: 1 :widths: 20 14 14 14 14 14 14 :class: longtable Unlike a missing character this does **not** fail the build — it is an ``Overfull \vbox`` warning and a PDF with content off the page — so grep the log for ``Overfull \vbox`` and treat anything beyond a point or two as a real defect. Sub-point overshoots are ordinary typographic rounding. When reading its output, ignore the ``Hyper reference … undefined`` warnings from the first passes — latexmk resolves forward references on later ones. The final pass, in ``docs/manual/_build/pdf/en/latex/pepsi.log``, is the one that must be clean. The translations are checked the same way, and the check is not redundant: a character a *translator* introduces — a guillemet, a German quotation mark, a non-breaking space — fails the same pdflatex build in the same way, and no English-only run would ever meet it. ``make docs-all`` is therefore a release gate rather than a per-commit one; each language's PDF lands in ``docs/manual/_build/pdf//latex/pepsi.pdf``, and a German or French build additionally needs that language's babel support installed (Debian: ``texlive-lang-german``, ``texlive-lang-french``). Keeping the translations current is ``make update-po``, which re-extracts the message templates and merges them into ``docs/manual/locale//LC_MESSAGES`` so that changed paragraphs come back marked *fuzzy* and new ones untranslated. Only the ``.po`` files are under version control; Sphinx compiles them at build time. Topology ======== The suite drives three real mail accounts across three hosts, described in a small INI file (``tests/test-accounts.ini``): .. list-table:: :header-rows: 1 :widths: 12 30 58 * - Role - Account (ssh target = e-mail) - Function * - ``ADMIN`` - ``root@`` - Runs Pepsi (ingress, dispatch, httpd), PostgreSQL and systemd; also the smarthost the served domain relays through. * - ``ALICE`` / ``BOB`` - ``alice@`` / ``bob@`` - Local accounts on the served domain. Their MTA relays through ADMIN and is trusted by it (``MYNETWORKS``), so their mail is **local origin** (the outbound path). * - ``CAROL`` - ``test@`` - An account on an independent host. Her mail reaches ADMIN's public MX on port 25 as a foreign MTA — the **inbound** path. * - ``DAVE`` - ``dave@`` *(optional)* - A local unix user **on the ADMIN host** that receives mail by direct local delivery into its Maildir (the **local-delivery** path; no onward relay). Unlike the others it is never ssh'd into directly — its mailbox is read *as root* over the ADMIN connection. Only ``07-maildir-test.sh`` needs it; ``01-deploy.sh`` provisions the ``dave``/``dave-broken`` users and the ``pepsi-maildir`` group when it is set. Every host is reached over passwordless ``ssh``; the harness installs nothing. ``ALICE``/``BOB`` use an mbox (``/var/mail/``); ``CAROL`` and ``DAVE`` use a Maildir. The mailbox helpers are format-agnostic and locate a message by a run-unique token across both layouts. The same file carries one non-mail account, in a ``[demobank]`` section: .. code-block:: ini [demobank] URL = https://bank.demo.taler.net USER = pepsi-ci PASS = pepsi-pass This is a bank account at the public GNU Taler **demo** bank, whose currency (``KUDOS``) is play money. ``12-wallet-topup-test.sh`` needs it because a *manual* wallet top-up only tells the user where to wire money — nothing credits the wallet until that bank transfer actually happens. The harness therefore plays the user's bank: it logs in and makes the transfer the e-mailed instructions ask for, which is the only way the test can assert that the withdrawal completes rather than merely that instructions were sent. Register an account at https://bank.demo.taler.net/ (a new one starts with ``KUDOS:100`` and may go ``KUDOS:500`` into debt — roughly 120 runs of that case); ``DEMOBANK_USER``, ``DEMOBANK_PASS`` and ``DEMOBANK_URL`` in the environment override the file. No other script uses it, and with the section absent only that one credit assertion skips. The scripts =========== ``tests/lib.sh`` Shared library *sourced* by every numbered script. It parses the accounts INI, provides the ``ssh``/dependency/database/service probes used by the common pre-flight, the result counters and coloured ``ok``/``no``/``skip`` output, the ADMIN-host SQL and reset helpers, the mailbox send/receive helpers, the Taler wallet helpers, the RFC 3464 DSN assertion, and — for the extended scripts — ``send_smtp`` (a raw-SMTP sender that can set the ESMTP and DSN parameters the ``mail`` command cannot) and ``hop_caps`` (an EHLO capability probe). It is not executable on its own. ``01-deploy.sh`` Provisions a complete Pepsi deployment on the ADMIN host from a clean checkout: service users, PostgreSQL role/database, the configuration file (including the full stage pipeline), ``pepsi-setup`` (TLS, schema, DKIM keys, DNS records), the demo payment merchant, and the systemd units. It only *verifies* required tools and bails with a precise list if any are missing — it never installs OS dependencies. ``02-pipeline-test.sh`` The core happy-path suite (T1–T6): the pay-to-send anti-spam stage (unpaid → bounce, paid → delivered, paying centrally from the runner's Taler wallet), the language filter (blacklisted French → bounce), the outbound signing/relay path (DKIM + ARC verified at the recipient), whitelist learning with SRS envelope rewriting, and a second local user. ``03-dsn-test.sh`` RFC 3461 DSN extensions and bounce variants, using ``send_smtp``: ``NOTIFY=NEVER`` suppresses the DSN (D1); ``ENVID``/``ORCPT`` are echoed back (D2); ``RET=HDRS`` returns headers only (D3); ``NOTIFY=SUCCESS`` with ``ORIGINATE_SUCCESS_DSN`` yields a positive DSN (D4); an NXDOMAIN recipient routes to the default bounce (D5); an unreachable relay produces a DELAY DSN (D6); and a null-sender bounce to an ``SRS0=…`` address is reverse-decoded by ingress back to the original sender (D7). ``04-settings-test.sh`` The per-address override layer: a ``pepsi-settings`` row changes one recipient's outcome but not another's (S1); an account edits its own overrides by e-mail and is acknowledged (S2); an edit to a non-``EDITABLE_STAGES`` section is rejected and stores nothing (S3); and one ``DATA`` with divergent per-recipient overrides is split into separate ingress rows (S4). ``05-mime-test.sh`` 8BITMIME / SMTPUTF8 handling: an 8-bit body and a UTF-8 ``Subject`` survive the relay intact when the delivery hop advertises the extension, are downgraded (quoted-printable/base64, RFC 2047) when it does not, and a non-ASCII recipient address that the hop cannot represent is permanently failed. The script probes the hop's EHLO first and takes the matching branch. ``06-cli-test.sh`` The operator CLIs, run directly over ``ssh``: ``pepsi-config`` (get/dump/pathsub), ``pepsi-whitelist`` (add/list/remove and the dkim/signature flags), ``pepsi-settings`` (set/get/list/unset/remove), ``pepsi-tlsrpt`` (report ``--dry-run`` / prune), and the ``pepsi-httpd`` ``/metrics`` endpoint. ``07-maildir-test.sh`` Direct (local Maildir) delivery via the ``pepsi-stage-relay-to-maildir`` stage, which drives the setuid-root ``pepsi-helper-maildir-writer``. A message to the local ``DAVE`` user lands in its Maildir with the prepended ``Return-Path``/``Delivered-To``/``Received`` headers (L1); a single message to one local and one remote recipient is split, delivering ``dave`` locally while ``alice`` is relayed onward (L2); and a message to a deliberately undeliverable account (``dave-broken``, no usable home) gives up past ``MAX_LIFETIME`` and returns a failure DSN to the sender (L3). Because ``[stage-local]`` sits after the inbound filters (``… → anti-spam → local → srs``), the sender is whitelisted first to clear the pay-to-send gate. The whole script skips cleanly when ``DAVE`` is unset, the deployment predates ``[stage-local]``, or the target users are missing. ``12-wallet-topup-test.sh`` The by-e-mail wallet **self-service** half of ``pepsi-stage-auto-pay`` (the ``[stage-wallet]`` section): a local account tops up its own per-user GNU Taler wallet by sending ``pepsi-wallet@`` a ``Subject:`` command. All three methods are exercised end-to-end against the demo exchange (KUDOS), with the funded local ``taler-wallet-cli`` as the genuine peer: ``withdraw`` (manual bank-wire instructions, the wire then actually made from the ``[demobank]`` account), ``pull`` (a ``taler://pay-pull/`` URI the harness settles) and ``push`` (the harness pushes; the user accepts the ``taler://pay-push/`` URI). Each case asserts the control message is consumed and the wallet balance grows. ``13-auto-pay-test.sh`` The demand-paying half: ``pepsi-stage-auto-pay`` settles a returning pay-to-send demand from the sender's own wallet. The demand is produced **genuinely** — a local user sends to a paywalled local recipient (``DAVE``), the outbound is internet-relayed (stamping a real ``Pepsi-Origin``), loops back into this host's inbound MX where ``pepsi-stage-anti-spam`` holds it and e-mails the sender a real demand. That demand is replayed through the auto-pay ``worker``. The two cases use **separate senders** (hence separate per-user wallets/orders): ``BOB``'s wallet is left empty, so auto-pay forwards the demand and charges nothing (A1); ``ALICE``'s wallet is **pre-funded ahead of time** (so the slow withdrawal is out of the paywall's short 30s payment window), so auto-pay settles the demand within it — the order flips to paid, the wallet is debited, and the held message is released and delivered to ``DAVE`` (A2). ``14-multi-recipient-test.sh`` Recipient fan-out, the failure mode where a message addressed to several people reaches only some of them. Both cases assert *positive* delivery to **every** intended recipient. T1 sends one envelope to two remote recipients and requires both to arrive — a relay that takes ``rcpt_to.first()`` and then terminates the row passes nothing else on. T2 covers the ``~/.forward`` fan-out: one envelope addressed to a local user who forwards (``DAVE``) *and* a second plain recipient, where the forwarded copy and the passthrough must each arrive exactly once — a stage that reduces the database row server-side without mutating the in-memory message leaves a fused successor working from the stale recipient list. T2 needs the optional ``DAVE`` account and a deployed ``~/.forward`` stage. ``16-whitelist-import-test.sh`` The setuid ``pepsi-whitelist`` and its mailbox importer, against the installed deployment — everything about that program that only exists once it is *installed*. It checks the installation itself (mode ``4755`` owned ``pepsi-whitelist``, the scan helper carrying **no** privilege bits, the table-scoped database role, the hoster list); the namespace rule, by creating two throwaway accounts and confirming one may use ``/…`` and nothing else, that ``list`` does not disclose the operator's shared whitelist, and that ``-c`` is refused; the import itself, which must yield exactly the recipients of the mail that user **sent** and ignore received mail; the wildcard decision, where a busy domain collapses to one ``*@domain`` row while ``gmail.com`` with just as many correspondents does not; the privilege drop, by substituting a helper that reports its own credentials and asserting the caller's uid appears in **all three** id slots; ``--user`` seeding another account's own namespace; and finally that ``pepsi-stage-check-whitelist`` consults the per-user name its ``{localpart}`` placeholder expands to. Individual checks skip when the deployment predates the feature. Beside the numbered scripts, ``tests/whitelist-suid.sh`` covers the same setuid contract **without** SSH or a deployment: run as root on any host with a local PostgreSQL, it creates a throwaway database, installs the binary setuid into a suid-honouring directory and runs the same namespace/privilege-drop assertions. ``make check`` runs it (and the other ``*-suid.sh`` scripts) best-effort — they SKIP rather than fail when they cannot get root, a database or a suid-capable filesystem. Running ======= .. code-block:: sh # One-time, on the MX host (from a clean checkout): ssh root@ 'bash -s' < tests/01-deploy.sh # Then, repeatedly, from the test runner: tests/02-pipeline-test.sh tests/test-accounts.ini tests/03-dsn-test.sh tests/test-accounts.ini tests/04-settings-test.sh tests/test-accounts.ini tests/05-mime-test.sh tests/test-accounts.ini tests/06-cli-test.sh tests/test-accounts.ini tests/07-maildir-test.sh tests/test-accounts.ini # needs the optional DAVE account tests/12-wallet-topup-test.sh tests/test-accounts.ini # needs taler-wallet-cli on the MX host tests/13-auto-pay-test.sh tests/test-accounts.ini # needs DAVE + taler-wallet-cli tests/16-whitelist-import-test.sh tests/test-accounts.ini # needs root on the MX host Each script is self-managing: it runs a pre-flight (``ssh`` reachability with the real error on failure, required remote tools, a database probe, and the three services' state with the journal tail of any that is down), ensures a short ``PAYMENT_DEADLINE`` and brisk ``POLL_INTERVAL`` for fast turnaround, and resets the whitelist/queue/settings it depends on. A missing assumption fails the pre-flight with an actionable message; a message that does not arrive triggers a diagnostics dump (the live ingress queue plus the dispatcher journal). Tunables such as ``PAYMENT_DEADLINE_SECS``, ``DELIVER_TIMEOUT`` and ``WITHDRAW_AMOUNT`` are environment overrides. Each script exits ``0`` only when no case failed (skips are allowed). Skips and gated cases ===================== A case **skips** (rather than fails) when a precondition it cannot satisfy is absent, always with a one-line reason. The common ones: * **No usable Taler wallet.** The paid leg of ``02`` skips if the local ``taler-wallet-cli`` is missing or incompatible with the demo exchange. All payments are made centrally from the runner's wallet (in production each sender would pay for their own message). * **Deployment predates a feature.** ``03``/``04`` skip the cases that need pipeline features added later to ``01-deploy.sh`` (``ORIGINATE_SUCCESS_DSN``, a ``[stage-internet]`` ``BOUNCE_STAGE``, the ``[stage-delaytest]`` blackhole relay, the ``[stage-edit-settings]`` stage) when run against an older deployment — re-run ``01-deploy.sh`` to enable them. ``06`` skips the ``pepsi-tlsrpt`` case unless a ``[pepsi-tlsrpt]`` section is configured. ``07`` skips entirely when the optional ``DAVE`` account is unset, the deployment predates ``[stage-local]``, or the ``dave``/``dave-broken`` target users are missing on the ADMIN host. ``14``'s T2 skips when ``DAVE`` is unset or the deployment has no ``~/.forward`` stage. * **Hop capability.** ``05`` takes its pass-through, downgrade or unrepresentable-address branch according to whether the delivery hop advertises 8BITMIME / SMTPUTF8; the cases that require the hop to *lack* an extension skip when it advertises it. * **Wallet / auto-pay not deployed.** ``12``/``13`` skip entirely when the ``[stage-wallet]``/``[stage-pay]`` sections, ``taler-wallet-cli`` or the ``pepsi-wallets`` account are absent (re-run ``01-deploy.sh``). ``12``'s ``withdraw`` credit leg skips without a ``[demobank]`` account (see above); its ``pull``/``push`` cases skip without a usable local wallet. ``13`` skips when ``DAVE`` is unset, the genuine demand is not produced (paywalled hop or merchant down), or — for A2 — the per-user wallet cannot be funded. Constraints =========== Testing must never send real mail to third-party MX hosts: the suite uses only the operators' own domains and reserved/``.invalid`` names (e.g. the DELAY test relays to an unrouteable ``192.0.2.0/24`` TEST-NET-1 address via a recipient in ``blackhole.invalid``). The development uid cannot bind the privileged ports 25/53, so the privileged listeners are exercised only on the deployed host.