95.1.46. pepsi-reputation

IP reputation: blocklists for inbound mail, and this host’s own

Manual section:

1

95.1.46.1.1. Name

pepsi-reputation - IP reputation: blocklists for inbound mail, and this host’s own.

95.1.46.1.2. Synopsis

pepsi-reputation [GLOBAL-OPTIONS] iplists update [–force]

pepsi-reputation [GLOBAL-OPTIONS] iplists status [–json]

pepsi-reputation [GLOBAL-OPTIONS] check [–json] [–no-store]

95.1.46.1.3. Description

pepsi-reputation keeps the IP blocklists that inbound mail is checked against (iplists), and checks whether this host itself is on one (check).

Each [pepsi-reputation-list-NAME] section names one list. iplists update downloads every enabled list whole, over HTTPS or rsync, into the pepsi.ip_blocklist table. pepsi-ingress(1) looks up the client of every unauthenticated session in that table, records the lists that name it under state.origin.ip_blocklist, and, for a list with ACTION = reject, refuses its recipients. pepsi-stage-if(1) can route on the mark: to a discard stage, to the pay-to-send gate, to a quarantine folder.

Nobody is asked about a client while it is connected. A DNS blocklist query per connection, which is how most MTAs use these lists, would tell the list operator which servers mail this host, and would make every session wait on a third party. A downloaded list costs neither, and the session never waits on the network for it. The price is coverage: the most widely used lists (Spamhaus ZEN, SpamCop, Barracuda) are offered for free only as DNS queries, so they cannot be used this way.

The lists that ship in ${DATADIR}/config.d/reputation.conf are:

spamhaus-drop-v4, spamhaus-drop-v6 (enabled)

Spamhaus DROP, networks hijacked or run by spam and malware operations. About 1,700 networks; free for any use.

blocklist-de-mail (enabled)

blocklist.de’s addresses that attacked a mail service in the last 48 hours. About 14,000 addresses, rebuilt every 30 minutes.

uceprotect-1 (disabled)

UCEPROTECT level 1, addresses that sent spam to its traps in the last 7 days, by rsync. Enable it with ENABLED = YES in pepsi.conf(5); UCEPROTECT asks for a donation.

All of them start with ACTION = mark, which changes nothing until a stage acts on the mark.

95.1.46.1.3.1. Downloading

iplists update is what pepsi-reputation-iplists.timer runs, hourly (a few minutes past the hour, the same minute every hour on a given host). For each enabled list it

  1. skips the list if its last download started less than MIN_INTERVAL ago. List operators ask for no more than about one download an hour, and UCEPROTECT locks out, and lists, a host that fetches more than four files an hour;

  2. downloads it. Over HTTPS the request carries the ETag and Last-Modified of the last download, so an unchanged list costs one 304 Not Modified. Over rsync, the last copy is kept under CACHE_DIR, so rsync transfers only the difference;

  3. when the file is new (by SHA-256), parses it and replaces the list’s entries in one transaction. A listing expires, so nothing of the previous version is kept;

  4. records when the list was last confirmed current.

A list that cannot be downloaded or parsed keeps its entries and records the error. pepsi-ingress(1) consults a list only while its last confirmation is younger than MAX_AGE, so a list that stops updating stops being used. It is then treated as unknown, not as “nobody is listed”, and unknown admits mail. A list that is disabled, no longer configured, or fetched by rsync on a host without rsync loses its entries at the next run.

Entries wider than a /8 (IPv4) or a /16 (IPv6) are dropped: a list that suddenly lists that much is broken or hostile, and lists have listed the whole internet before when their operators gave up. A file in which more than a tenth of the lines are not entries is refused as not being in the configured FORMAT.

HTTPS downloads verify the server against the system CA roots and refuse loopback and private addresses. rsync is run with an empty environment, the URL after --, connection and I/O timeouts and --max-size, and is killed after ten minutes.

95.1.46.1.3.2. rsync

The rsync program is an optional dependency (the Debian package only suggests it). A list whose URL starts with rsync:// is not used on a host without it: iplists update removes its entries and logs a warning; pepsi-setup(1) run warns about it and removes its entries at once, and the wizard’s IP_BLOCKLISTS expert option refuses to enable it.

95.1.46.1.3.3. Checking this host

An address inherited from a cloud provider may arrive with a bad reputation, and a clean one gets listed after an account is compromised. pepsi-reputation-check.timer runs check daily. It asks the DNS blocklists that the large receivers consult about this host’s own addresses and domains, and never about a correspondent, so the lists learn nothing that they do not already learn from every receiver that this host mails. It checks:

  • every PUBLIC_IP of a pepsi-stage-relay-to-internet stage, the addresses that [pepsi-ingress] HOSTNAME resolves to, and CHECK_ADDRESSES, leaving out addresses that are not globally routable;

  • the [pepsi-ingress] ACCEPTED_DOMAINS.

Answers are not taken at face value. Per RFC 5782, a list must list its test entry (127.0.0.2, ::ffff:7f00:2, the domain test) and must not list 127.0.0.1 or invalid. When a list fails that test for an address family, because it does not serve the family, refuses this resolver or answers everything, it is reported as unchecked for that family, never as clean or listed. The same holds for an answer in 127.255.255.0/24, which Spamhaus returns for a refused query: .254 means the query came through a public resolver such as 8.8.8.8, which Spamhaus does not serve; run a local resolver (Pepsi needs one for DANE anyway). A listing is logged as a warning, the answers are stored in pepsi.reputation_check, and pepsi-status(1) shows them. pepsi-setup(1) check reports them under blocklists, beside the requirements and sign-ups of the large mailbox providers.

95.1.46.1.4. Commands

iplists update [–force]

Download the lists that are due and load new versions. –force downloads every enabled list now, ignoring MIN_INTERVAL, the cache validators and the cached copy’s hash. Use it sparingly: list operators block hosts that download too often.

iplists status [–json]

For every configured list: whether it is current, stale (older than MAX_AGE, so ingress ignores it), never-loaded, disabled or no-rsync; its ACTION; how many entries it has; when it was last confirmed current; and the last error.

check [–json] [–no-store]

Ask the DNS blocklists about this host, print every listing and every list that could not be asked, and replace the stored answers. –no-store prints without storing; –json prints the targets and every answer.

95.1.46.1.5. Configuration

[pepsi-reputation]
CACHE_DIR

Where rsync keeps its copy of each list (default /var/lib/pepsi-reputation, the timer unit’s StateDirectory=).

CHECK_IP_ZONES

The address lists that check asks (default zen.spamhaus.org bl.spamcop.net b.barracudacentral.org psbl.surriel.com bl.mailspike.net dnsbl-1.uceprotect.net ix.dnsbl.manitu.net; none for none).

CHECK_DOMAIN_ZONES

The domain lists that check asks (default dbl.spamhaus.org multi.surbl.org).

CHECK_ADDRESSES

Further addresses to check, for instance a NAT address that the configuration does not name.

GOOGLE_SITE_VERIFICATION

DOMAIN:TOKEN pairs: the token that Gmail Postmaster Tools shows when a domain is added there. pepsi-setup(1) run prints the google-site-verification=TOKEN TXT record to publish at the domain, and check verifies it. Pepsi never contacts Google.

[pepsi-reputation-list-NAME], one per list. NAME is made of lower-case letters, digits and -.

URL

https://… or rsync://HOST/MODULE/FILE. Required, except for the four lists above, whose URL and FORMAT default to the shipped values.

FORMAT

ip-list (the default): one entry per line, as rbldnsd’s ip4set and ip6trie datasets or a plain address list have it. An entry is an address, a network (a.b.c.d/n), a range (a.b.c.d-e.f.g.h, or a.b.c.d-h for the last octet) or rbldnsd’s short form (a.b.c for a /24); ! before an entry excludes it. Comments, rbldnsd’s : value lines and $ directives are ignored. spamhaus-drop: Spamhaus DROP’s JSON lines.

ENABLED

Whether the list is downloaded and consulted (default YES).

ACTION

mark (default): record the listing. reject: also refuse the session’s recipients with 554 5.7.1. postmaster is still accepted, as RFC 5321 asks, so that a listed sender can reach a human about it.

MIN_INTERVAL

The least time between two downloads (default 45 m).

MAX_AGE

How long ingress still consults the list after its last successful check (default 12 h).

MAX_SIZE

The largest file accepted, in bytes (default 64 MiB).

95.1.46.1.5.1. Turning the downloads off

ENABLED = NO turns off a single list. To stop all downloads, mask the timer; pepsi.target stays enabled and keeps everything else running:

systemctl mask --now pepsi-reputation-iplists.timer

95.1.46.1.6. Running as root

Started as root, the tool continues as the pepsi service account before it connects to the database, as pepsi-status(1) does. The configuration is read before the switch; downloads happen after it. Both timer units run as pepsi.

95.1.46.1.7. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace. When the flag is absent the global [pepsi] LOG option applies, and failing that info.

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

95.1.46.1.8. Exit Status

0

Successful completion.

1

An error occurred: a malformed list section, a failed database connection, or at least one list that could not be updated (the others are still updated).

78

The database schema does not match this build; run pepsi-setup(1) schema first.

95.1.46.1.9. Examples

Whether this host is listed anywhere, without storing the answer:

pepsi-reputation -c /etc/pepsi/pepsi.conf check --no-store

What the lists look like now:

pepsi-reputation -c /etc/pepsi/pepsi.conf iplists status

Refuse mail from the addresses that blocklist.de lists, in pepsi.conf(5):

[pepsi-reputation-list-blocklist-de-mail]
ACTION = reject

Send inbound mail from a listed client to a quarantine stage instead of the inbox, with pepsi-stage-if(1):

[stage-blocklisted]
PROGRAM = pepsi-stage-if
STATE_PATH = origin.ip_blocklist.listed
VALUE = true
TRUE_STAGE = quarantine
FALSE_STAGE = deliver

95.1.46.1.10. See Also

pepsi-ingress(1), pepsi-stage-if(1), pepsi-setup(1), pepsi-status(1), pepsi.conf(5), pepsi.state(7).