85.1.45. pepsi-keydisc

find a correspondent’s public key or certificate

Manual section:

1

85.1.45.1.1. Name

pepsi-keydisc - asynchronous key discovery, one service per method.

85.1.45.1.2. Synopsis

pepsi-keydisc [GLOBAL-OPTIONS] serve METHOD

pepsi-keydisc [GLOBAL-OPTIONS] probe ADDRESS

85.1.45.1.3. Description

pepsi-keydisc finds a correspondent’s OpenPGP key or S/MIME certificate, records where it came from and how much that source is worth, and caches it in pepsi.peer_key (see pepsi-keys(1)). Finding a key is easy; knowing whether to believe it is the whole problem, so every result carries its source and whether the lookup that produced it was DNSSEC-validated, and the consuming stage’s policy decides what is good enough.

Discovery is asynchronous and lives outside the stages. A stage that needs a key it does not have in cache never performs network I/O of its own: it commits whatever work it may keep, pauses the message and enqueues a request in pepsi.key_request; a pepsi-keydisc instance answers the request and, in the same round-trip that stores the key, releases every message parked on that address:

stage                cache hit         -> use the key
                     fresh negative    -> the no-key path, at once
                     cache miss        -> park: commit, pause, enqueue

pepsi-keydisc@...    LISTEN key_request -> look up -> store the key AND
                                           release every parked message

What the park commits is the stage’s decision, not this program’s. In particular pepsi-stage-decrypt(1) parks an S/MIME message with the decrypted plaintext committed — the nested signature survives as an ordinary MIME layer — but parks an OpenPGP one unchanged, because an OpenPGP signature lives only inside the packet stream and committing the plaintext would destroy the evidence the key is being fetched to check.

That removes “a slow key server holds a stage worker” entirely rather than bounding it. The request is keyed on the address, so a hundred messages to one correspondent cost one lookup and are released together. A request that settles without a key releases its waiters exactly like one that found a key: the services find keys, the stages decide about mail.

It is not a stage. It connects to the shared database through the [pepsi-postgres] section and is configured entirely by [pepsi-keydiscovery] (see pepsi.conf(5)).

85.1.45.1.4. One instance per method

There is one binary and one configuration section, but one process per discovery method, started from the systemd template unit pepsi-keydisc@.service:

pepsi-keydisc@dane

DNS OPENPGPKEY (RFC 7929) and SMIMEA (RFC 8162), under a hashed local part. Both are refused without the resolver’s DNSSEC AD bit.

pepsi-keydisc@wkd-advanced

The Web Key Directory at openpgpkey.<domain>, under /.well-known/openpgpkey/<domain>/hu/<hash>.

pepsi-keydisc@wkd-direct

The Web Key Directory at the domain itself, under /.well-known/openpgpkey/hu/<hash>.

pepsi-keydisc@vks

The VKS_SERVERS verifying key servers, at /vks/v1/by-email/<address>.

pepsi-keydisc@ldap

The configured LDAP directory; only in a build with the ldap feature.

pepsi.target starts the first four, which are the default SOURCES.

The split is what buys the isolation: a hung LDAP bind cannot stall WKD, and turning a method off is systemctl mask --now pepsi-keydisc@vks (and dropping it from SOURCES) rather than a restart of something shared. The two Web Key Directory methods are separate ranks and therefore separate instances run concurrently — never a fallback chain, since running direct only “if advanced fails” would silently accept the weaker answer whenever the stronger one was merely slow.

This is a deliberate deviation from draft-koch-openpgp-webkey-service §3.1, which requires the advanced method to be tried first and the direct one to be used only where the openpgpkey sub-domain does not exist at all. The draft’s ordering is about trust rather than speed: the apex /.well-known/openpgpkey/ is served by whatever runs the domain’s website, often a different party from its mail operator, so a domain that publishes correctly under openpgpkey.<domain> still has its apex consulted here — and an answer from there is stored at the wkd-direct rank, above LDAP, VKS and anything harvested from mail. Where that matters, SOURCES can list wkd-advanced without wkd-direct.

An instance whose method does not appear in [pepsi-keydiscovery] SOURCES refuses to start, with a message naming what SOURCES does list. That is not pedantry: the rank-grace stop rule waits only for the methods SOURCES names, so an unlisted instance would be answering a request nobody is waiting for.

Harvesting keys from inbound mail (application/pgp-keys parts, S/MIME signer certificates, the Autocrypt: header) is not a discovery service and has no instance. It runs inline where the message already is, in pepsi-stage-autocrypt-learn(1), costs no network I/O, and writes through the same path — so a harvested key also releases anything parked on that address. inbound is consequently rejected in SOURCES.

The same goes for gossip, the keys that stage reads out of the Autocrypt-Gossip: fields inside a message that arrived encrypted (Autocrypt Level 1 §5.3). It is a second inline rung rather than a variant of inbound, because those keys are third parties’ rather than the sender’s own, and the ladder has to be able to tell them apart.

Note where “inline” is: pepsi-stage-decrypt(1) extracts the certificates a message carries in order to check its signature and then drops them — it stores nothing. A pipeline with no pepsi-stage-autocrypt-learn stage never learns a key from mail at all, however much material arrives, and no amount of [stage-decrypt] configuration changes that.

85.1.45.1.5. Which key wins a conflict

A key whose fingerprint is already stored for the same address and protocol is not a conflict at all: the row is updated in place (its source, validity, expiry and Autocrypt effective date are carried forward, the source only ever improving) and the answer is refreshed. That happens before every test below, so re-seeing a pinned key still refreshes what is known about it.

When a discovered key differs from everything stored for that address and protocol, the store decides in one server-side step, in this order:

  1. a pinned row is never displaced, whatever arrives;

  2. a strictly better-ranked source replaces everything stored — so WKD displaces a harvested key, an operator entry displaces WKD, and a gossiped key displaces nothing at all;

  3. newest-wins inside the Autocrypt tier (below);

  4. otherwise the stored key is kept and the new one is not stored, and the answer is reported as rejected.

85.1.45.1.5.1. Newest-wins in the Autocrypt tier

Autocrypt Level 1’s peer-state update is youngest-wins: a client stores the key from the youngest message it has seen carrying an Autocrypt: header. Rule 2 alone cannot express that — it would refuse the second Autocrypt key ever seen for an address and keep the first for ever, so a correspondent who reinstalls their client, changes device or rotates their key would go on receiving mail encrypted to a key they no longer hold, which is exactly the unreadable-mail failure Autocrypt exists to avoid.

So rule 3 lets a newer key win, and it is confined so that it can only ever decide between claims of the same kind. It applies when all of:

  • the incoming source is inbound or gossip; and

  • every key currently stored for that address and protocol is inbound or gossip too — nothing an owner or an operator published is ever at stake; and

  • the incoming source ranks at least as high as the stored one, so inbound replaces inbound and gossip replaces gossip, but a third party’s introduction can never take the row off what the address’s owner said about themselves; and

  • the incoming message’s effective date is strictly newer than the stored one.

The effective date is the message’s Date: header, clamped so that it can never be in the future, and the time of receipt when the header is absent or unparseable — Level 1’s own definition. Equal dates are therefore first-wins (a redelivered message cannot flip the row), and a key that carries no effective date at all — every network method — never displaces anything.

The price of the rule is what it hands an attacker. Harvesting is keyed on the From: header, which the sender writes, and inbound authentication is fail-open, so a forged From: can displace a key that was itself only ever trust-on-first-use. That is Autocrypt’s own trade-off — it defends against passive collection, not against an active attacker on the path — and it is bounded on all four sides above: nothing published by the domain (WKD, DANE), by a key server, by a directory or by the operator can be touched, and neither can a pinned row. An operator who does not want it sets [pepsi-keydiscovery] MIN_TRUST above the tier, or pins the keys that matter with pepsi-keys peer pin (see pepsi-keys(1)); note that an address this deployment holds an identity for is answered from that identity and never from the peer cache at all.

A rule-3 replacement is reported as rotated rather than replaced and writes a key.peer.rotate row to the audit log in the same statement, so a rotation cannot happen without its record. The only caller that can trigger one — inbound learning, pepsi-stage-autocrypt-learn(1) — also mails the local recipients of the message, and can restrict the rule with ACCEPT_ROTATION = expired (every stored key must be recorded revoked or expired), under which a refused rotation is reported as held and audited as key.peer.rotate.held. The network methods never reach rule 3.

The same fingerprint re-seen is a refresh, not a conflict, and the row takes over the better of the two sources. One such refresh is reported separately: a key stored from gossip that its owner’s own message (source inbound) now carries is reported as promoted and audited as key.peer.promote, never as a rotation — Autocrypt Level 1 §5.3’s promotion out of gossip.

85.1.45.1.6. Commands

serve METHOD

Run one method’s service instance until interrupted. METHOD is one of dane, wkd-advanced, wkd-direct, vks or ldap — the systemd template’s instance name — and must appear in [pepsi-keydiscovery] SOURCES.

The instance LISTENs on the key_request channel, whose payload is the request id, answers each request this method still owes, and sweeps once on start-up and on every listener reconnect so nothing raised while it was away is lost. Between notifications it ticks a short settle sweep: a rank-grace window is a point in time, so something has to look at the clock, and a deployment running any instance at all therefore has the sweep. A single failing lookup is recorded on the request (which is what selects the short ERROR_TTL over the long NEGATIVE_TTL) and never tears the listener down.

probe ADDRESS

Run the whole fan-out for one address once and print what each method found — the protocol, the fingerprint, the size, and dnssec for an AD-validated DANE answer — or the error it hit, or that it found nothing.

Deliberately read-only: it stores nothing, neither the key nor a negative-cache entry. An operator diagnosing “why does this correspondent have no key” wants to see each method’s answer, not to have a diagnostic write a negative entry that then suppresses the real lookup for a day. To look an address up and store the result, use pepsi-keys peer refresh --address (see pepsi-keys(1)).

An address excluded by ALLOW_DOMAINS/DENY_DOMAINS is reported as such and not queried.

85.1.45.1.7. Privileges

The services run as their own unprivileged account, pepsi-keydisc — not the pepsi service user and not pepsi-crypto. This daemon parses key material fetched from the open internet (a hostile WKD body, a key-server answer, a DNS record), which makes it the likeliest process here to be broken into, so pepsi-setup(1) gives it the narrowest database role in the deployment:

  • SELECT/INSERT/UPDATE/DELETE on pepsi.peer_key — the cache filling it is its whole job, and the conflict rule has to be able to replace a row;

  • SELECT/INSERT/UPDATE on pepsi.key_request — the work queue it answers;

  • EXECUTE on the keydisc_* functions and the two crypto_* helpers they call;

  • on pepsi.workqueue, SELECT plus a column-level UPDATE (status, timeout) — and nothing else.

Those two columns are exactly enough to move a parked row from paused back to pending, which happens inside keydisc_resolve/keydisc_sweep. With a table-level grant this account could rewrite a message’s recipients or its body; with these two columns it can only wake a message up. It has no reach into pepsi.crypto_identity at all, and therefore none into any private key.

The program carries no setuid or setgid bit and is folded into the multi-call pepsi binary. Started as root it becomes the pepsi-keydisc account before connecting (the configuration, and any @inline-secret@ fragment, is read first, as the invoking user), so it can be launched from a root shell for testing without sudo -u.

85.1.45.1.8. LDAP is a compile-time feature

The LDAP client is behind the ldap cargo feature and is off by default: it adds a dependency, and a deployment with no directory should not link one. A build without the feature refuses to start pepsi-keydisc@ldap with a message saying so, rather than silently answering “no key” for ever, and pepsi-setup(1) rejects a configuration that lists ldap in SOURCES on such a build — as it does one that lists ldap without LDAP_URL and LDAP_BASE.

85.1.45.1.9. Global Options

These global options precede the subcommand (a trailing flag is rejected).

-c FILE, –config FILE

Read the configuration from FILE instead of searching the default locations.

-L LOGLEVEL, –log LOGLEVEL

Set the logging verbosity. LOGLEVEL is one of error, warn, info, debug or trace (default: info).

-v, –verbose

Show log messages from all sources, including third-party libraries.

-h, –help

Print a usage summary and exit.

-V, –version

Print the version and exit.

85.1.45.1.10. Signals

SIGINT, SIGTERM

Stop taking new requests and exit. Nothing is lost: an unanswered request stays pending in pepsi.key_request and is picked up by the next instance’s start-up sweep, or settles on its own deadline.

85.1.45.1.11. Exit Status

0

Clean shutdown, or a completed probe.

1

An error occurred: a malformed configuration, a method that [pepsi-keydiscovery] SOURCES does not list, ldap in a build without the feature, or a failed database connection. Individual lookup failures are not errors — they are recorded on the request and the service keeps running.

85.1.45.1.12. Examples

pepsi.target starts the default set of methods (dane wkd-advanced wkd-direct vks). Without the target, start them by hand:

systemctl enable --now pepsi-keydisc@dane pepsi-keydisc@wkd-advanced \
                       pepsi-keydisc@wkd-direct pepsi-keydisc@vks

Turn one method off without touching the configuration of the others:

systemctl mask --now pepsi-keydisc@vks

(then set [pepsi-keydiscovery] SOURCES = dane wkd-advanced wkd-direct, so the stop rule stops waiting for an answer that will never come). mask rather than disable: pepsi.target starts a merely disabled instance again.

Find out what each method says about one address, changing nothing:

pepsi-keydisc -c /etc/pepsi/pepsi.conf probe bob@example.com

Run one instance in the foreground, with full logging, to see why an address yields no key:

pepsi-keydisc -c /etc/pepsi/pepsi.conf -L debug serve wkd-advanced

85.1.45.1.13. See Also

pepsi-keys(1), pepsi-setup(1), pepsi.conf(5), pepsi.state(7), pepsi-dispatch(1), pepsi-config(1)

85.1.45.1.14. Bugs

Report bugs to the Pepsi issue tracker.