.. This file is part of PEPSI. Copyright (C) 2026 GNUnet e.V. PEPSI is free software; you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. ============= pepsi-archive ============= *Search, export, import, purge and repair the mailing-list archive.* Role ==== ``pepsi-archive`` is the operator's command-line tool for the mailing-list archive. It connects through the shared ``[pepsi-postgres]`` section and, started as ``root``, continues as the ``pepsi`` service account before connecting, so no ``sudo -u pepsi`` is needed. It is **not** setuid and must not be made so. Reference: :manpage:`pepsi-archive(1)`. The archive is a reimplementation of GNU Mailman 3's HyperKitty: its data model, its threading rules, its Message-ID hash and its URL scheme are upstream's, copyright the Free Software Foundation and its contributors, under the GPL. See :doc:`../archives`, which says so at greater length and names what was taken. The one it exists for ===================== ``purge`` is the command this tool exists for: remove a message somebody posted by accident from a public archive. It deletes the message, its attachments and its votes, renumbers the thread, repairs the counters, and writes an ``event_log`` row naming who did it. It leaves a **tombstone** — the ``Message-ID``, kept for ``[pepsi-list]`` ``TOMBSTONE_RETENTION`` days (default 90) — during which the archive refuses to store that message again. There are exactly two ways a purged message comes back, a re-delivery and a re-run import, and both are accidents. ``--forget`` skips the tombstone. Commands ======== * **search** *QUERY* — search as the operator, who sees every list. ``--list`` scopes the search, which is both faster (one partition) and better (the list's own language dictionary). The output says which mechanism answered, and says so when the list does not index message bodies for substring search. * **show** *LIST* *ID* — print one archived message. *ID* may be a ``Message-ID``, a Message-ID hash, or an archive URL containing one. * **export** ``--list`` *ADDRESS* — write the list's archive to **standard output** as an ``mboxrd`` mbox, oldest first; ``--since`` / ``--until`` take RFC 3339 timestamps. * **import** ``--list`` *ADDRESS* *FILE*… — import mbox files; see below. * **expire** ``--list`` *ADDRESS* ``--before`` *TIMESTAMP* — delete everything older than the timestamp, in batches. Batched because the predicate is a date while the partitioning is by list, so it cannot prune: one statement over a year of a busy list would hold a long transaction. The retention sweep of ``pepsi-list tasks --once`` runs the same deletion daily for every list with a retention (see :doc:`../archives`). * **reindex** — rebuild ``search_text`` and ``trgm_text``. **This is how a changed trigram tier takes effect**, in both directions and with no schema change. * **recount** — rebuild the counter columns from the rows themselves; the repair for the one documented way to break them, which is writing to the archive tables directly. * **rebuild-threads** — reattach replies whose parent arrived after them and renumber every thread. Upstream leaves those threads split for ever; this is the repair. Import, and the round trip ========================== ``import`` is the universal path into the archive and the other half of the ``export`` round trip — which is the cheapest possible test of both. **The mbox repairs are in the reader**, not in a separate script an operator has to remember. A body line beginning ``From `` does not split a message: a separator must follow a blank line *and* carry a four-digit year. A missing ``Message-ID`` is generated deterministically from the message's own bytes, so re-running an interrupted import produces the same id rather than a duplicate. The `` (added by postmaster@…)`` form is repaired; a missing ``Date`` falls back to the mbox envelope line; a folded ``Subject`` is joined; and ``Content-Length`` is dropped, because it described a framing that does not exist once the message is out of the mbox. **Threads are reattached and renumbered at the end, automatically** (unless ``--no-rebuild``). An out-of-order import otherwise leaves them split — which is what happens with upstream's batch mode, and which nobody notices for years. A message that cannot be imported is **reported**: to ``--rejects`` when a file is named and to standard error when one is not, and the command exits non-zero unless ``--skip-bad`` says the loss is expected. Re-running is safe, because every message is idempotent on its ``Message-ID``, so resumption keys on content rather than on a file's timestamp. Configuration ============= ``[pepsi-list]``: ``SEARCH_TRIGRAM`` — the site **ceiling** on the substring-search tier (``off``, ``short`` by default, or ``full``), which a list chooses up to with ``pepsi-list list set-ext``; ``TOMBSTONE_RETENTION`` in days, swept by ``pepsi-list tasks --once``; ``ARCHIVE_RETENTION`` in days, the default the retention sweep of ``pepsi-list tasks --once`` applies to a list without its own ``archive_retention_days`` (``0``, the default, keeps everything); and ``ARCHIVE_PARTITIONS``, used by ``pepsi-setup`` to create the partitions (and compared against them by ``pepsi-list check``) and **fixed at install**, because PostgreSQL has no online re-partitioning. See also ======== :doc:`../archives`, :doc:`../mailing-lists`, :doc:`pepsi-list`, :doc:`pepsi-stage-list-post`, :doc:`pepsi-httpd`, :manpage:`pepsi-archive(1)`.