95.1.47. pepsi-greylist

greylisting milter for pepsi-ingress, Postfix and sendmail

Manual section:

1

95.1.47.1.1. Name

pepsi-greylist - greylisting milter for pepsi-ingress, Postfix and sendmail.

95.1.47.1.2. Synopsis

pepsi-greylist [GLOBAL-OPTIONS] serve

pepsi-greylist [GLOBAL-OPTIONS] list [–pending | –passed | –never-retried] [–domain DOMAIN] [–ip ADDRESS] [–rcpt ADDRESS] [–limit N] [–json]

pepsi-greylist [GLOBAL-OPTIONS] allow KIND VALUE [–expires DURATION] [–note TEXT]

pepsi-greylist [GLOBAL-OPTIONS] deny KIND VALUE [–expires DURATION] [–note TEXT]

pepsi-greylist [GLOBAL-OPTIONS] rules [–json]

pepsi-greylist [GLOBAL-OPTIONS] unrule ID

pepsi-greylist [GLOBAL-OPTIONS] prune [–json]

pepsi-greylist [GLOBAL-OPTIONS] stats [–hours N] [–json]

95.1.47.1.3. Description

pepsi-greylist is a milter: a filter that an MTA asks about each SMTP command. An MTA that uses it (pepsi-ingress(1), Postfix, sendmail) asks it at every RCPT TO of a session that did not authenticate. The first attempt of a sender at a recipient from an unknown client is answered with 451 4.7.1 Greylisted, please try again later. A mail server that follows RFC 5321 queues the message and tries again; a retry after DELAY (5 minutes) and within RETRY_WINDOW (48 hours) is accepted, and so is later mail from the same client, sender and recipient for PASSED_TTL (35 days) after it was last seen. Software that sends each message once and never retries does not get through.

The cost is a delay of the first message from each new correspondent. The bypasses below keep that delay off mail from correspondents a user already knows and from the large providers.

serve runs the daemon, as pepsi-greylist.service does. The other commands inspect and steer it. pepsi-greylist-prune.timer runs prune hourly.

95.1.47.1.3.1. Who the client is

Large providers send from pools of hosts, and the retry of a message often comes from another host than the first attempt. So the triplet is not (client address, sender, recipient) but (client identity, sender, recipient), and the identity is found by the first rung of IDENTITY that applies (default spf ptr network):

spf

The SPF record of the envelope sender’s domain (the HELO name for the null sender) authorises the client: the identity is that domain. Every host that the domain’s SPF record names is then the same client.

ptr

The client’s reverse DNS name resolves back to the client address (forward-confirmed), and the organisational domain of that name, found the way DMARC finds it (the DMARC record tree walk of RFC 9989), authorises the client by its SPF record or lists it as one of its MX hosts. The identity is that organisational domain. mail-7.out.example.net alone proves nothing; example.net saying that the host is one of its own does.

network

The client’s network, the address masked to IPV4_MASK (24) or IPV6_MASK (64) bits. This rung always applies and is the last resort.

When a retry comes from a host whose own identity is different (the first attempt was made by a host the ptr rung named example.net, the retry comes from a host without a reverse name), the earlier triplets of the same sender and recipient are compared with the new client: an spf or ptr triplet matches when its domain’s SPF record or MX hosts include the new client, a network triplet when the new client is inside its network. A match counts as that triplet’s retry. Only the domain is stored; the check runs when the retry arrives, and its answer is kept for DNS_CACHE_TTL.

95.1.47.1.3.2. The decision

For each recipient, in this order:

  1. Never greylisted: a session that authenticated (the MTA sends the {auth_authen} macro), a client on a UNIX socket or a loopback address, a recipient outside RECIPIENT_DOMAINS (when set) and a recipient in EXEMPT_RECIPIENTS (postmaster and abuse).

  2. Rules (allow, deny): a deny rule refuses the recipient with 550 5.7.1; an allow rule lets it through.

  3. Bypasses, each with its own option: the sender is on the recipient’s whitelist (WHITELIST_BYPASS); the client belongs to a big sender (BIG_SENDERS); SPF passed (SPF_BYPASS, off by default); the client identity has passed greylisting often enough (AUTO_WHITELIST_PASSES).

  4. The triplet, as described above.

A database error, or a DNS step that takes longer than DNS_TIMEOUT, lets the recipient through: greylisting fails open, logs a warning and counts an error. The daemon asks for no message content and makes no change to a message.

95.1.47.1.4. Commands

serve

Listen on SOCKET and answer the MTAs that connect. Started as root, it binds the socket first and then continues as the pepsi-greylist account. SIGTERM stops it after it has written its counters out.

list [–pending | –passed | –never-retried] [–domain DOMAIN] [–ip ADDRESS] [–rcpt ADDRESS] [–limit N] [–json]

Show triplets, the most recently seen first (at most N, default 50). –pending: not passed yet. –passed: passed. –never-retried: deferred once and not tried again. –domain: the client identity domain or the sender’s domain is DOMAIN or a subdomain of it. –ip: the last client address is inside ADDRESS (an address or a network), or the identity is a network that holds it. –rcpt: the recipient.

allow KIND VALUE [–expires DURATION] [–note TEXT]

Never greylist what VALUE names. KIND is one of:

ip

the client address;

network

a network holding the client address (192.0.2.0/24);

domain

the client identity domain (spf or ptr rung) or the envelope sender’s domain, each also as a subdomain of VALUE. The sender’s domain is not authenticated: an allow rule for a domain also lets through a forger who writes that domain into the envelope. Prefer a network rule for a sender that has no SPF record;

sender

the envelope sender;

rcpt

the recipient.

–expires takes a duration (30 days, 12 h); without it the rule stays until unrule. Adding the same rule again renews it. Every change is recorded in the audit log.

deny KIND VALUE [–expires DURATION] [–note TEXT]

Refuse what VALUE names with 550 5.7.1. A deny rule wins over an allow rule; neither applies to the exempt recipients.

rules [–json]

Show the rules that have not expired, with their id.

unrule ID

Remove rule ID.

prune [–json]

Remove pending triplets past RETRY_WINDOW (counting those deferred once and never tried again), passed triplets not seen for PASSED_TTL, client records that are no longer auto-whitelisted, expired rules and hourly counters older than 400 days.

stats [–hours N] [–json]

Whether the daemon answers on its socket, and the counts of the last N hours (default 24): deferred, passed on retry, passed from another host (by rung and by how the host matched), never retried, known, bypassed (by reason), exempt, denied, and the errors that let a recipient through.

95.1.47.1.5. Configuration

[pepsi-greylist]
SOCKET

Where the daemon listens (default unix:/run/pepsi-greylist/milter.sock). inet:HOST:PORT (or inet:PORT@HOST) listens on TCP, for an MTA on another host; it has no access control of its own, so bind it to an address that only the MTAs reach, or filter the port.

SOCKET_MODE

The mode of the UNIX socket, in octal (default 0660).

SOCKET_GROUP

The group of the UNIX socket (default pepsi-greylist, the daemon’s own group). A member of the group may ask the daemon; the group grants nothing else. The package makes pepsi-ingress a member. A group the daemon is not in can be set only when it is started as root.

DELAY

How long a first attempt is refused (default 5 min). Durations here may be written in days and weeks (35 days).

RETRY_WINDOW

How long after the first attempt a retry still passes (default 48 h). Must be longer than DELAY.

PASSED_TTL

How long a passed triplet is remembered after it was last seen (default 35 days).

AUTO_WHITELIST_PASSES

After this many passes of one client identity (a retry that passed, or later mail of a passed triplet) the identity is not greylisted any more (default 5; 0 turns the auto-whitelist off).

AUTO_WHITELIST_TTL

How long an auto-whitelisted identity stays exempt after it was last seen (default 35 days).

IDENTITY

The identity ladder, in order of precedence (default spf ptr network). A rung left out is not used; network is always the last.

IPV4_MASK, IPV6_MASK

The prefix lengths of the network rung (default 24 and 64).

DNS_TIMEOUT

The bound on each DNS step of a decision: one SPF evaluation, one reverse lookup with its forward confirmation, one MX lookup with the hosts’ addresses (default 2 s). A step that takes longer lets the recipient through.

DNS_CACHE_TTL

How long the answer to “is this client one of that domain’s hosts?” is kept (default 10 min).

DNS_SERVERS

The resolvers to ask, as in [pepsi] DNS_SERVERS, which applies when this is unset.

EXEMPT_RECIPIENTS

Recipients never greylisted: local parts (any domain, a sub-address included) or whole addresses (default postmaster abuse; none for none).

RECIPIENT_DOMAINS

When set, only mail to these recipient domains is greylisted; mail to any other domain passes. Unset (the default), every recipient is.

WHITELIST_BYPASS

Let a sender through that is on one of the recipient’s whitelists (default YES): the same match pepsi-stage-check-whitelist(1) makes, against the envelope sender. A row that requires DKIM or DMARC counts when SPF passes for the sender’s domain. Which whitelists: WHITELIST_NAME.

WHITELIST_NAME

The whitelist names (templates with {localpart} and {login} as for the whitelist stage, with LOCAL_DOMAINS, RECIPIENT_DELIMITER and TARGETS read from this section). Unset, the names of every stage that runs pepsi-stage-check-whitelist are used, so a sender a user whitelisted is never delayed. The console’s Allow this sender (pepsi-httpd(1), /ui/deferred) writes to the first of these names that is in the user’s own namespace (<login>/…), and is not offered when there is none.

BIG_SENDERS

Let the senders of the [pepsi-greylist-senders-*] groups through (default YES).

SPF_BYPASS

Let every client through for which SPF passes (default NO). Off, an SPF pass only chooses the identity; on, it is a full bypass, and every sender whose domain publishes SPF is never delayed.

REPLY

The answer to a first attempt (default 451 4.7.1 Greylisted, please try again later). It must be one line with a 4xx code, and an enhanced status code, when there is one, of class 4: greylisting asks for a retry (RFC 5321 section 4.5.4.1).

DB_POOL_SIZE

Database connections the daemon holds at most (default 4).

MAX_CONNECTIONS

MTA connections served at once (default 256). An MTA connection beyond it is closed, which the MTA treats as a milter failure; with pepsi-ingress(1)’s default ON_FAILURE = accept the mail is accepted.

[pepsi-greylist-senders-NAME], one per group of senders that is never greylisted. NAME is made of lower-case letters, digits and -. The groups google, microsoft, yahoo, apple and transactional (Amazon SES, SendGrid, Mailgun, Mailchimp, Postmark, SparkPost) ship in ${DATADIR}/config.d/greylist.conf; a pepsi.conf section of the same name changes them option by option.

DOMAINS

A client whose identity domain or forward-confirmed PTR name is one of these, or a subdomain of one, passes. The envelope sender’s domain alone never matches.

NETWORKS

A client address inside one of these networks passes.

ENABLED

Whether the group applies (default YES).

95.1.47.1.6. Using it from an MTA

From pepsi-ingress(1), in pepsi.conf(5):

[pepsi-ingress-milter-greylist]
SOCKET = unix:/run/pepsi-greylist/milter.sock

[pepsi-ingress]
MILTERS = greylist

From Postfix, in main.cf:

smtpd_milters = unix:/run/pepsi-greylist/milter.sock
milter_default_action = accept

Postfix’s smtpd runs as postfix and must be in the socket’s group:

adduser postfix pepsi-greylist
systemctl restart postfix

On Debian, smtpd runs chrooted in /var/spool/postfix, where /run/pepsi-greylist does not exist. Either listen on TCP (SOCKET = inet:127.0.0.1:10031 here, smtpd_milters = inet:127.0.0.1:10031 there), or put the socket inside the chroot (SOCKET = unix:/var/spool/postfix/pepsi-greylist/milter.sock, and smtpd_milters = unix:/pepsi-greylist/milter.sock) and let the unit write there with a drop-in:

# systemctl edit pepsi-greylist.service
[Service]
ReadWritePaths=/var/spool/postfix/pepsi-greylist

Postfix sends the {auth_authen} macro with MAIL FROM by default, which is what exempts its authenticated submissions. Do not list the milter in non_smtpd_milters: locally submitted mail is never greylisted anyway.

95.1.47.1.7. The service

pepsi-greylist.service runs the daemon as pepsi-greylist, with the socket directory /run/pepsi-greylist (RuntimeDirectory=). It is not part of pepsi.target: greylisting is a choice, and pepsi-setup(1)’s wizard enables the unit when it is made. By hand:

systemctl enable --now pepsi-greylist.service

pepsi-greylist-prune.timer is part of pepsi.target; on a host that does not greylist it finds nothing to prune.

The database role pepsi-greylist reaches the greylist_triplet, greylist_client, greylist_rule (to read and to prune) and greylist_stats tables, reads whitelist, and appends to event_log. Rules are added and removed through the functions greylist_rule_add and greylist_rule_remove, which normalise the value; they are also the only way in which pepsi-httpd(1) may change a rule. The console’s /ui/greylist and the API’s /api/v1/greylist show the counters, the triplets and the rules, and change the rules with origin = console.

95.1.47.1.8. Running as root

Started as root, the command line continues as the pepsi-greylist account before it connects to the database, and serve binds its socket first. stats asks the socket before the switch, so that root can probe a socket whose group it is not in.

95.1.47.1.9. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace. When the flag is absent the global [pepsi] LOG option applies, and failing that info.

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

95.1.47.1.10. Exit Status

0

Successful completion.

1

An error occurred: a malformed section, a refused rule value, an unknown rule id, a failed database connection.

78

The database schema does not match this build; run pepsi-setup(1) schema first.

95.1.47.1.11. Examples

Which first contacts from example.net were never retried:

pepsi-greylist -c /etc/pepsi/pepsi.conf list --never-retried --domain example.net

Never delay mail from a partner’s network, for a year:

pepsi-greylist -c /etc/pepsi/pepsi.conf allow network 198.51.100.0/24 \
    --expires '1 year' --note 'partner relay'

Greylist only the mail to one domain of a shared host, in pepsi.conf(5):

[pepsi-greylist]
RECIPIENT_DOMAINS = example.org

Greylist the transactional senders’ mail after all:

[pepsi-greylist-senders-transactional]
ENABLED = NO

95.1.47.1.12. See Also

pepsi-ingress(1), pepsi-stage-check-whitelist(1), pepsi-status(1), pepsi-httpd(1), pepsi-setup(1), pepsi.conf(5).