70.1.1. pepsi-ingress

receive incoming e-mail over SMTP and store it for Pepsi

Manual section:

1

70.1.1.1.1. Name

pepsi-ingress - SMTP server that stores incoming e-mail in PostgreSQL.

70.1.1.1.2. Synopsis

pepsi-ingress [GLOBAL-OPTIONS] serve

70.1.1.1.3. Description

pepsi-ingress is the mail-receiving component of Pepsi. It runs an SMTP server (a Message Transfer Agent) that accepts incoming e-mail over cleartext, implicit TLS and STARTTLS. A message body may be sent with DATA or, for clients that announce it, with CHUNKING/BDAT (RFC 3030) — advertised in the EHLO response. It stores each accepted message — together with its envelope, a small amount of parsed metadata, and the SMTP-origin metadata of the delivering connection (client IP, HELO/EHLO name, TLS, listener, reverse-DNS/iprev; see pepsi.conf(5)) — in the ingress table of the pepsi schema of a PostgreSQL database. The message is stored split into a headers column (everything up to the blank line) and a body column, so that downstream stages which only touch the headers need not load the body. The message enters the stage pipeline at the initial stage (stage = init). After the row has been committed, a NOTIFY is issued on the ingress channel to wake the downstream processors.

The server accepts mail for a configured set of domains. Within those domains it accepts every local-part without trying to verify that the mailbox exists; recipients outside the configured domains are rejected as relaying (550). The domainless reserved mailbox <Postmaster> is always accepted regardless of the configured domains (RFC 5321 §4.5.1) and rewritten to a routable address — [pepsi-ingress] POSTMASTER when set, otherwise postmaster@HOSTNAME — so the rest of the pipeline (aliases, local delivery, relay) routes it like any ordinary recipient. To make postmaster mail reach a person, either point POSTMASTER at a real mailbox or add an alias for the rewritten address in a pepsi-stage-aliases ALIASES map. Such a message is also committed with state.spam = false so that the language and pay-to-send gates (pepsi-stage-block-language(1), pepsi-stage-anti-spam(1)) forward it, keeping the postmaster mailbox reachable as RFC 5321 §4.5.1 requires. When SRS is configured (the shared [pepsi-srs] section; see pepsi-stage-srs(1)), a recipient in the SRS domain whose local-part is a valid Sender Rewriting Scheme token is reverse- decoded to the original sender and accepted for relay there — even though that sender’s domain is not one we serve, because the valid signature authorises it; a forged or expired token is rejected (550). This is how bounces to mail Pepsi forwarded find their way back to the original senders. Receipt of a message is confirmed to the client (250) only after the message has been durably committed to the database; if the database is unavailable the client receives a transient error (451) and is expected to retry.

As required by RFC 5321 §4.4, pepsi-ingress prepends a Received: trace header to every accepted message, naming the connecting client (HELO/EHLO name, reverse-DNS and IP), this server, the transport used (SMTP, ESMTP or ESMTPS per RFC 3848) and the time of receipt; a for clause names the recipient only for a single-recipient transaction. The header is added before authentication so that the ARC seal (added later by pepsi-stage-arc(1)) covers it.

For an encrypted session the ESMTPS keyword on its own does not record which TLS version and cipher were negotiated. As recommended by RFC 8314 §4.3, pepsi-ingress therefore appends the negotiated parameters as a comment after the with keyword, e.g.:

by mail.example.org (pepsi-ingress) with ESMTPS
    (version=TLSv1_3 cipher=TLS13_AES_128_GCM_SHA256)

These values are the same TLS parameters recorded in the message state under origin.tls (see pepsi.state(7)); whichever of the version/cipher the TLS stack reports is included.

70.1.1.1.3.1. Authentication

Before a message is stored, pepsi-ingress authenticates it so that downstream — after Pepsi has relayed the mail and broken the original SPF and DKIM alignment — receivers can still see the verdict Pepsi reached at the boundary. For each accepted message it:

1. verifies SPF (on the MAIL FROM identity, using the connecting client’s IP), DKIM and DMARC; and

2. prepends an Authentication-Results header reporting these results, with the [pepsi-ingress] HOSTNAME as the authserv-id, and records the rendered result text under state.origin.ar_results.

ARC is handled separately: ingress no longer verifies the ARC chain or seals the message. The pepsi-stage-arc(1) pipeline stage verifies any inbound ARC chain and adds the fresh ARC set (AAR/AMS/AS) as the [pepsi] ARC_DOMAIN identity, reading the session context (authserv-id, client IP, HELO/EHLO, and the ar_results fragment) ingress recorded in the row’s state. Reusing that fragment lets the stage build the AAR without re-running SPF/DKIM/DMARC, so the authentication checks happen exactly once. This keeps the ARC signing keys out of the SMTP front end.

70.1.1.1.3.2. Delivery Status Notifications (DSN)

pepsi-ingress advertises the DSN extension (RFC 3461) in its EHLO response and accepts its parameters: RET and ENVID on MAIL FROM and NOTIFY and ORCPT on RCPT TO. Their syntax is validated — an invalid RET/NOTIFY value, a NOTIFY=NEVER combined with other keywords, or a malformed ENVID/ORCPT xtext is rejected with 501 — and the accepted values are stored in the row’s state.dsn (message-level ret/envid plus a per-recipient array of notify/orcpt). The stage pipeline honours them: the relay stages propagate the parameters to the next hop when it also advertises DSN, and pepsi-stage-bounce(1) generates a failure bounce only when the recipient’s NOTIFY requests it (FAILURE, the default when absent), echoing ENVID/ORCPT into the DSN. Every stage must preserve state.dsn.

70.1.1.1.3.3. 8-bit and internationalised content

pepsi-ingress advertises 8BITMIME (RFC 6152) and SMTPUTF8 (RFC 6531) in its EHLO response, so a message may carry raw 8-bit octets in its body and UTF-8 in its headers and envelope. The BODY= parameter on MAIL FROM is accepted and recorded (state.origin.body_8bit); only 7BIT and 8BITMIME are valid — BINARYMIME is not advertised and is rejected with 501. The relay stages re-advertise BODY=8BITMIME / SMTPUTF8 to a next hop that supports them, and downgrade the message when it does not: an 8-bit body towards a non-8BITMIME hop is MIME-re-encoded to a 7-bit content-transfer-encoding, and UTF-8 headers towards a non-SMTPUTF8 hop are rewritten as RFC 2047 encoded-words (a non-ASCII address there cannot be downgraded and is bounced). See pepsi-stage-relay-to-internet(1).

70.1.1.1.3.4. ESMTP parameter validation

pepsi-ingress validates the MAIL FROM/RCPT TO parameters strictly. The recognised parameters are SIZE and BODY (and AUTH) on MAIL FROM and the DSN parameters above; a SIZE= value that is not a non-negative integer is rejected with 501 (RFC 1870 §6.1) rather than silently dropped. A parameter the server does not implement is rejected with 555 5.5.4 (RFC 5321 §4.1.1.11) — it is not silently ignored. The one exception is the RFC 4954 AUTH= parameter on MAIL FROM (which an authenticated client may attach because AUTH is advertised): it is accepted and ignored — Pepsi does not propagate the asserted identity onward — rather than rejected.

70.1.1.1.3.5. Client authentication

Each listener can authenticate its clients, marking accepted mail as locally originated. A session is authenticated when any one of three per-listener mechanisms accepts it:

• MYNETWORKS — the connecting IP is within one of a whitespace/comma separated list of trusted IPv4/IPv6 CIDR networks (a bare address is taken as a /32 or /128 host).

• SASL — a SASL_TYPE backend (currently dovecot, via the SASL_PATH auth-client socket) accepts an AUTH exchange. AUTH (mechanisms PLAIN and LOGIN) is advertised and accepted only on a TLS-protected session (implicit TLS or after STARTTLS); a cleartext AUTH attempt is refused with 538.

• TLS client certificate — the client presents a certificate whose SubjectPublicKeyInfo SHA-256 (64 hex chars) is listed in TLS_AUTH_CLIENT, or whose chain signature-validates up to a listed CA key.

SASL_TYPE and TLS_AUTH_CLIENT require a TLS listener (MODE = tls or starttls). An authenticated session is recorded with state.local_origin = true (otherwise false) and is permitted to relay to any domain, not only the served domains; an authenticated AUTH also sets the RFC 3848 ESMTPSA keyword in the trace header. An unrecognised client certificate never aborts the handshake — it simply leaves the session unauthenticated.

70.1.1.1.3.6. Message submission (RFC 6409)

A listener carrying SUBMISSION = yes is treated as a Message Submission Agent (RFC 6409) rather than an MX:

• Authentication is mandatory. An unauthenticated MAIL FROM is refused with 530 5.7.0 Authentication required; the client must first satisfy one of the Client authentication mechanisms above. The flag therefore requires a TLS listener and at least one configured mechanism (SASL_TYPE, TLS_AUTH_CLIENT or MYNETWORKS), and pepsi-setup rejects a submission listener that has none.

• Submission fixups are applied to every accepted message: a missing Date: header is added (§8.2) and a missing Message-ID: is generated as <random@HOSTNAME> (§8.3). Headers that are already present are left untouched, and the fixups run before the Received: trace header is prepended, so the added fields are covered by the downstream ARC seal and DKIM signature.

The flag has no effect on an MX listener; leave it off (the default) for port 25.

70.1.1.1.3.7. Submission identity (RFC 6409 §6 / §8.1)

A SASL-authenticated session is bound to the set of e-mail addresses its username is allowed to use. By default the username alice is mapped to the single address alice@HOSTNAME (the [pepsi-ingress] HOSTNAME domain); the optional USERNAME_MAP file overrides this mapping. Each non-blank, non-comment line is:

username: addr1@example.org addr2@example.org

a # begins a comment, the key is the SASL username (before the first :), and the value is the whitespace/comma separated list of allowed addresses, which may be empty. Usernames are matched case-insensitively and addresses are lower-cased. The file is re-read whenever its modification time changes (checked on each authentication), so edits take effect without a restart; a missing or unreadable file is treated as empty (every user falls back to the default).

An allowed address may use a * wildcard (a *-glob matched against the candidate address), so alice: *@example.org lets alice use any address at example.org and a bare * permits any address at all. A wildcard entry names no single canonical identity, so it is skipped for the Sender: fixup below (the first concrete, wildcard-free address is the canonical identity, if any). pepsi-setup syntax-checks a configured USERNAME_MAP at install time (see pepsi-setup(1)). The wildcard is a glob, not a regular expression: write *@example.org, never .*@example.org (a literal dot followed by a glob — it matches nothing, so the user would be denied every address it was meant to grant). Setup rejects such a pattern and names the glob you meant.

The mapping has three outcomes for a username:

• listed with addresses — exactly those addresses (and wildcard patterns) are allowed; the first concrete (wildcard-free) one is the canonical identity used for the Sender: fixup;

• listed with no addresses — the user is denied: even though SASL accepted the credentials, the AUTH command is refused with 535 5.7.8 and the session stays unauthenticated;

• not listed (or no USERNAME_MAP configured) — the default username@HOSTNAME.

Given that set, two checks run on a SASL-authenticated submission:

• Envelope and header identity (§6). The MAIL FROM envelope sender and the message From: address must each be one of the allowed addresses, otherwise the message is refused with 550 5.7.1 (at MAIL FROM and at end-of-DATA respectively). The null sender <> and a message with no parseable From: address are exempt.

• Sender fixup (§8.1). When the From: address is an allowed address that differs from the canonical identity, a Sender: header naming the canonical identity is prepended (unless the message already carries one). Like the other fixups it is added before the trace header, so it is covered by the ARC seal and DKIM signature.

These checks apply only to sessions authenticated by SASL AUTH (which carry a username). A MYNETWORKS or TLS-client-certificate session has no SASL username and is not subject to the identity mapping.

70.1.1.1.4. Mailbox quota at RCPT

When a recipient is on one of the served domains, pepsi-ingress may refuse it in the SMTP session because the mailbox is known to be full — 452 4.2.2 or 552 5.2.2 according to [pepsi] MAILBOX_OVER_QUOTA. Refusing here rather than accepting and bouncing later is the point: the sending MTA is still on the line, so it tells its user, and Pepsi generates no backscatter to an envelope sender that may well be forged.

Four properties make that safe to do from a database row.

No resolution. The recipient’s local part — lower-cased and stripped of any sub-address at [pepsi] RECIPIENT_DELIMITER — is compared to a passwd login exactly. Aliases, ~/.forward files and the passwd database itself belong to the delivery stage; pepsi-ingress guesses at none of them, so an address that is not spelled like a login simply matches nothing and is accepted.

No measurement. pepsi-ingress cannot read a 0700 Maildir, and must never gain the pepsi-maildir group that would let it run the helper that can. It reads figures somebody else wrote, and holds only SELECT on the table.

Only a fresh measurement counts. The running usage estimate is deliberately not consulted: it over-counts, and refusing on it would shut a mailbox whose owner has since emptied it — permanently, since with every message refused no delivery would ever run to re-measure it. A measurement older than [pepsi] MAILBOX_QUOTA_MAX_AGE accepts the message and lets the delivery path take a fresh look; pepsi-quota(1) reconcile keeps at-limit accounts measured.

Every uncertainty accepts. No row, no measurement, a database error — all mean accept, in keeping with the rest of inbound processing.

The domainless <Postmaster> mailbox is accepted regardless, because RFC 5321 §4.5.1 requires it to stay reachable.

Set [pepsi] MAILBOX_QUOTA_RCPT_CHECK = no to skip the lookup entirely and leave all enforcement to delivery time.

The verification is fail-open: a DNS failure or unparseable message never rejects the mail; it is recorded as temperror/none and accepted. Only when DMARC_ENFORCE is set does a definite DMARC failure under a quarantine/reject policy cause a 550 rejection; by default a forwarder accepts such mail (and the ARC stage seals the failing result) for the final receiver to judge. The SPF/DKIM/DMARC verdicts are stored in the row’s state.auth object (as spf/dkim/dmarc/arc), and the authserv-id under state.origin; the arc verdict starts as none and is filled in later by the ARC stage. The DNS lookups happen synchronously in the DATA phase, so a slow resolver delays the 250.

Multiple SMTP streams are processed concurrently. Several listeners can be configured, each bound to its own TCP port, UNIX-domain socket, or a socket inherited from a systemd parent through socket activation, and each with its own transport mode. All behaviour is driven by an INI-style configuration file (see pepsi.conf(5)).

70.1.1.1.4.1. Overload protection

Two mechanisms keep a flood of connections — including clients that complete the TCP handshake and then go silent without ever closing — from exhausting the server. Both operate on a source range: a single IPv4 address (a /32), or the /32 prefix of an IPv6 address. Local (UNIX-socket) connections are exempt.

• Per-command timeouts (RFC 5321 §4.5.3.2). A session that does not send its next command within COMMAND_TIMEOUT (default 300 s), or the next block of a DATA/BDAT body within DATA_TIMEOUT (default 180 s), is closed with a 421. The same COMMAND_TIMEOUT bounds a stalled STARTTLS (or implicit-TLS) handshake.

• Connection ceiling with eviction. The server admits at most min(MAX_CONNECTIONS, MAX_OPEN_SOCKETS) concurrent connections. MAX_OPEN_SOCKETS guards against file-descriptor exhaustion; when unset it is derived from the soft RLIMIT_NOFILE minus RESERVED_SOCKETS (default 64, held back for the database pool, listeners, stdio and DNS sockets). When the ceiling is reached and a new connection arrives, the server makes room by closing the longest-idle connection in the range whose total idle time is largest (a 421, then close), so the greediest source loses its most-idle slot first. “Idle” is time a connection spends blocked awaiting client input between commands — not time spent receiving a body or processing.

In addition, accepted new connections are rate-limited per source range by a token bucket: CONN_RATE_PER_SECOND (default 1) refill rate with a CONN_RATE_BURST (default 1) burst. A connection over the rate gets a 421 and is closed without occupying a slot. (On an implicit-TLS listener the refusing 421 is omitted, since no plaintext may precede the handshake; the connection is simply closed.)

The database schema is installed by pepsi-setup(1), which applies the migrations of every component to the shared pepsi schema; pepsi-ingress itself has no schema-initialization command.

70.1.1.1.4.2. Privilege dropping

The SMTP ports (25/465/587) are privileged, so pepsi-ingress is often started as root. It does not, however, serve as root: once every configured listener has been bound (and any TLS key material read), the process drops to the unprivileged service account pepsi-ingress, dropping all supplementary groups and switching to that account’s group and user id before accepting a single connection. The account must therefore exist before the server is started as root; create it (for example useradd --system --no-create-home --shell /usr/sbin/nologin pepsi-ingress) as part of installation.

If the drop cannot be completed while running as root — most often because the pepsi-ingress account does not exist — the server logs an error and exits without serving, rather than risk running as root. When started by a non-root user no privilege change is performed (the server simply runs as the invoking user); it will never run as root.

70.1.1.1.4.3. TLS key material

The TLS_CERT/TLS_KEY of every MODE = tls/starttls listener is read once, at start-up, before the privilege drop above.

Under systemd the unit runs as the unprivileged pepsi-ingress user from the outset and never holds root, so it cannot open certbot’s root-only /etc/letsencrypt/{live,archive}. pepsi-setup(1) therefore writes a drop-in /etc/systemd/system/pepsi-ingress.service.d/10-tls-credentials.conf with one LoadCredential= entry per file: systemd opens them as root at unit start and hands the service private copies under $CREDENTIALS_DIRECTORY, which pepsi-ingress consults before falling back to reading the configured path itself. Re-run pepsi-setup run after adding, moving or removing a certificate so the drop-in is regenerated. Because credentials are materialised when the unit starts, a renewed certificate is served only after a restart — the certbot deploy hook pepsi-setup installs performs it. Started directly (as root, dropping privileges after binding), the server reads the files itself and no credential is involved. See pepsi-httpd(1), which describes the same mechanism in full.

Unlike pepsi-httpd, which serves many certificates and skips one it cannot load, a listener here has exactly one: if it cannot be read there is nothing to fall back to, and the server exits rather than silently offer no TLS on a port the configuration says is encrypted.

70.1.1.1.5. Commands

serve

Bind every configured [pepsi-ingress-listener-*] socket and serve SMTP connections until interrupted. Requires that the schema has been installed with pepsi-setup(1).

70.1.1.1.6. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations (see FILES).

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace (default: info).

-v, –verbose

Show log messages from all sources, including third-party libraries that are filtered out by default.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

70.1.1.1.7. Signals

SIGINT, SIGTERM

Initiate shutdown. The process stops accepting new connections and exits.

70.1.1.1.8. Exit Status

0

Successful completion (including a clean shutdown of serve).

1

An error occurred, for example a malformed configuration file, a failed database connection, or a listener that could not be bound. The reason is written to the log.

70.1.1.1.9. Environment

XDG_CONFIG_HOME, HOME

Used to locate the default configuration file when –config is not given (see FILES).

LISTEN_FDS, LISTEN_PID, LISTEN_FDNAMES

Honoured for systemd socket activation. A listener configured with SERVE = systemd adopts the file descriptor passed by systemd at the index given by its FD_INDEX option.

The shipped pepsi-ingress.socket passes four: ports 25, 465 and 587, which the listener sections claim by FD_INDEX 0, 1 and 2, and the local submission socket /run/pepsi/submission.sock, which is claimed differently — see below. Binding that UNIX socket in the unit rather than in the server is deliberate: systemd creates it as root before the service starts, so the unprivileged pepsi-ingress never needs write access to /run/pepsi, a runtime directory pepsi-httpd and the setup-apply doorbell also declare and that systemd re-chowns to whichever of them started last.

The local submission socket has no FD_INDEX and no listener section. pepsi-ingress serves it unconditionally (from [pepsi-sendmail] SOCKET) and finds the descriptor by asking the kernel which of the passed sockets is bound to that path. An index would be a position among the unit’s ListenStream= entries and would therefore shift whenever an SMTP port is added or removed; the bound address cannot drift out of step with anything. If no passed descriptor matches, the path is bound by the server instead — which is what happens off systemd, and what makes failing to bind it fatal there.

Every other descriptor passed must be claimed by some listener section. One that is not is reported with a warning at start-up naming its index, because systemd keeps accepting connections on it that nothing will ever answer — a client then blocks until its own read timeout expires (two minutes, for pepsi-sendmail(1)) rather than being refused at once. Both are a deferral, but one of them stalls the caller for every message. If you remove a listener section, remove its ListenStream= from the unit too.

JOURNAL_STREAM

When set (i.e. when running under systemd), timestamps are emitted in UTC without a local time-zone offset.

70.1.1.1.10. Files

When –config is not given, the first existing file from the following list is used:

• $XDG_CONFIG_HOME/pepsi.conf

• $HOME/.config/pepsi.conf

• /etc/pepsi/pepsi.conf

• /etc/pepsi/pepsi.conf

Every Pepsi component shares the same configuration file, so this is the same list each one searches.

70.1.1.1.11. Examples

Install the schema (once, with pepsi-setup) and start serving:

pepsi-setup   -c /etc/pepsi/pepsi.conf
pepsi-ingress -c /etc/pepsi/pepsi.conf serve

Check which domains mail is accepted for (with pepsi-config(1)):

pepsi-config -c /etc/pepsi/pepsi.conf get pepsi-ingress ACCEPTED_DOMAINS

Dump the merged configuration with source annotations:

pepsi-config -c /etc/pepsi/pepsi.conf dump --diagnostics

70.1.1.1.12. See Also

pepsi-config(1), pepsi.conf(5), pepsi-setup(1), systemd.socket(5)

70.1.1.1.13. Bugs

Report bugs to the Pepsi issue tracker.