72. pepsi-greylist

Greylist first contacts: a milter that asks an unknown client to come back later.

72.1. Role

pepsi-greylist is a milter, a filter that an MTA asks about the SMTP commands of a session. pepsi-ingress asks it at every RCPT TO of a session that did not authenticate, and so can a Postfix or sendmail that receives mail in front of Pepsi. The first attempt of a sender at a recipient from an unknown client gets 451 4.7.1 Greylisted, please try again later. A mail server queues the message and tries again, as RFC 5321 section 4.5.4.1 requires, and the retry gets through; spam software that sends each message once and never retries does not. Greylisting is off until the operator turns it on: pepsi-setup --wizard asks, with no as the default answer, and the daemon’s unit is not part of pepsi.target. Reference: pepsi-greylist(1).

72.2. Why it is a separate daemon

Greylisting needs a store of what was seen, a decision on the hot path of every session, and its own bypass policy. Done as a milter, it sits behind the socket that MTAs already use for filters, so the same daemon serves pepsi-ingress and a Postfix in front of it, and ingress gains no greylisting code of its own. The daemon runs as its own account with a database role that reaches its own tables only, and every failure of it ends in accepting the mail (Greylisting).

72.3. What greylisting costs, and how Pepsi keeps it small

Greylisting delays the first message from every new correspondent, and the mail that people wait for is hit hardest: password resets, confirmation links, one-time codes. Three things keep the delay off mail that does not need it:

  • Bypasses. A sender on the recipient’s whitelist is never delayed (WHITELIST_BYPASS, on by default), nor are the large mailbox providers and the services that send transactional mail for other organisations (BIG_SENDERS, the groups in config.d/greylist.conf). A client identity that has passed five times is auto-whitelisted for 35 days (AUTO_WHITELIST_PASSES, AUTO_WHITELIST_TTL). Authenticated sessions, local clients and the postmaster and abuse addresses are never greylisted.

  • A client identity instead of an address. Large providers retry from another host of their pool. Greylisting keyed on the client address would delay their mail until a retry happens to come from the first host, which can take hours. Pepsi keys a triplet on the client’s identity: the domain whose SPF record authorises the client, else the organisational domain of its forward-confirmed reverse DNS name when that domain’s SPF record or MX hosts include the client, else the client’s /24 (IPv4) or /64 (IPv6) network. A retry from any host of the same identity passes.

  • A short delay. Five minutes (DELAY); most servers retry after five to fifteen.

The protection is small too. Greylisting stops software that does not retry; spam sent through real mail servers (a hijacked account at a provider, a compromised host running an ordinary MTA) retries and passes.

72.4. The identity ladder

IDENTITY = spf ptr network is an ordered list. The first rung that applies names the client:

spf

SPF of the envelope sender’s domain (of the HELO name for the null sender) passes for the client. Every host that the domain authorises is then one client. With SPF_BYPASS = yes an SPF pass lets the mail through instead; by default it only chooses the identity, because an SPF record is cheap for a spammer to publish.

ptr

The client’s reverse name forward-confirms, its organisational domain is found by the DMARC tree walk (the method Pepsi’s DMARC evaluation uses), and that domain’s SPF record or MX hosts include the client. A reverse name alone is never trusted: whoever holds an address chooses it.

network

The masked network. It always applies.

When the retry comes from a host of a different identity, the first attempt’s triplet is found anyway: the earlier triplets of the same sender and recipient are compared with the new client, an spf or ptr triplet by asking whether its domain’s SPF record or MX hosts include the new client, a network triplet by whether the new client is inside it.

72.5. Monitoring and whitelisting

Greylisting describes what to watch. In short:

  • pepsi-status has a Greylisting section with the last day’s counts and a warning when the daemon does not answer;

  • pepsi-greylist list --never-retried --domain example.net shows first contacts that were never retried. A domain that sends from an SPF identity and never retries is a provider that does not retry at all (some transactional senders do not); allow it;

  • pepsi-greylist allow domain example.net (or network, sender, rcpt) stops the delay for good, --expires for a while; a user stops it for a correspondent by whitelisting them (When the first message from someone new arrives late).

72.6. Security

  • The daemon is a principal of its own (Principals and how far each is trusted) behind the greylisting socket (B8, Trust boundaries). Membership of the socket’s group, pepsi-greylist, is what lets an MTA ask, and it grants nothing else.

  • Its database role reaches greylist_triplet, greylist_client, greylist_rule and greylist_stats, reads whitelist and appends to event_log; it cannot read the queue. pepsi-httpd reads the four tables and changes a rule only through greylist_rule_add and greylist_rule_remove.

  • The tables record who wrote to whom for up to PASSED_TTL; the prune timer removes the rest hourly.

  • A database error or a slow DNS step lets the recipient through (Availability).