- Zig 94%
- Nix 6%
zig-spf #
Sender Policy Framework — RFC 7208 — for Zig.
SPF answers one question: may this host send mail claiming to come from this domain? The domain publishes a TXT record listing the hosts that may, and a receiver evaluates that record against the address that connected to it.
The library does no I/O of its own. Record and macro are parsers over
bytes, and the evaluator takes a resolver from its caller. That is not an
accident of layering: SPF's evaluation is a recursive DNS walk — what
gets looked up next is decided by what the last lookup returned — so the
resolver cannot be hoisted out the way a DKIM key lookup can. Handing it
in is what makes the whole of RFC 7208's conformance suite, which is
defined as a table of zone data and expected results, run offline as
ordinary unit tests.
The API reference is published
from the default branch and is generated from the doc comments, which is
where most of the explanation in this library lives. zig build docs-serve builds the same pages and serves them at
http://127.0.0.1:8000/; a server is needed rather than opening the
files, because the viewer fetches its data at runtime and a browser
refuses that from a file:// page.
Where this lives #
- https://git.jcollie.dev/jeff/zig-spf — the Forgejo repository.
- https://tangled.org/jcollie.dev/zig-spf — the Tangled mirror.
rad:z2oUNxKQBmukgxhHEmHWF3mLwj7oQ— on Radicle, where a repository is findable only by its ID.
git clone https://git.jcollie.dev/jeff/zig-spf.git
rad clone rad:z2oUNxKQBmukgxhHEmHWF3mLwj7oQ
Status #
check_host() works, against a resolver you supply. What is here:
checkHost,checkMailFrom,checkHelo— the evaluator of §4 and §5: every mechanism,redirect=,exp=, the §4.6.4 query and void lookup budgets, and the recursion throughinclude:.Received— theReceived-SPFheader field of §9.1, which is how the answer travels with the message.Resolver— the interface the evaluation reads the DNS through.StubResolver— that interface over the real DNS, asking through zig-dns-client.spf— a tool that runs a check against it.Zone— a resolver over a table of records, for tests.Record— thev=spf1grammar of §4.6 and §5.macro— the macro language of §7.
All 203 cases of the openspf.org conformance suite pass — see below.
For a worked example as an SMTP handler, see
examples/spf_policy.zig in
zig-smtp: connect records the client's address, helo the greeting,
and mailFrom puts the two together and asks — which is the first moment
the question can be asked, and before the message has cost anything to
receive.
Checking a client #
const outcome = spf.checkMailFrom(
io,
resolver,
client_address, // a z46.Address
"mail.example.net", // what the client said in HELO
"alice@example.com", // the envelope sender
.{},
);
switch (outcome.result) {
.pass => {}, // authorized
.fail => {}, // the domain says no
.softfail, .neutral, .none => {},// no useful opinion
.temperror => {}, // defer; §8.6
.permerror => {}, // the policy is broken; §8.7
}
checkHelo answers the same question about the greeting alone, which can
be done before the envelope arrives and is the only identity a client with
nothing else to claim still has. checkHost is the RFC's own entry point
underneath both.
An empty reverse-path — the MAIL FROM:<> of a bounce — has no domain to
check, so §2.4 says to check postmaster at the HELO domain instead;
checkMailFrom does that rather than leaving each caller to remember it.
Nothing allocates. Pass Options.explanation a buffer to have a fail's
exp= fetched and expanded into it — leaving it null skips the lookup
entirely, which is a query saved on the path where nobody may read the
text. The explanation is written by the domain being checked, so §6.2 is
explicit that it is untrusted text headed for a human.
Recording the answer #
A check nobody records is a check nobody can act on later. §9.1's
Received-SPF field is how the answer travels with the message, for a
filter, a mailbox rule, or somebody reading a header two years from now:
const received: spf.Received = .{
.result = outcome.result,
.receiver = "mybox.example.org",
.identity = .mailfrom,
.client_ip = client_address,
.envelope_from = "myname@example.com",
.helo = "foo.example.com",
};
try writer.print("{f}", .{received});
Received-SPF: pass (mybox.example.org: domain of myname@example.com designates 192.0.2.1 as permitted sender)
receiver=mybox.example.org; identity=mailfrom; client-ip=192.0.2.1;
envelope-from="myname@example.com"; helo=foo.example.com
The wording of the comment is the one every deployed implementation uses, because these strings get grepped for by filters written against Postfix and libspf2 that would not recognize better prose.
Every value in the field was supplied by the client being judged. The
sender, the greeting, and a problem= taken from a domain's own
explanation are all somebody else's bytes going into a structured header
field above a message they do not control — so the escaping is the whole
job, not tidiness:
(,)and\become quoted-pairs inside the comment, so a sender cannot close it and write prose of their own.- A value that is not a
dot-atomis written as aquoted-string, with"and\escaped. That is every envelope sender, since@is notatext, and every IPv6 address, since:is not either — a field that left one bare would parse asclient-ip=2001and a reader would believe it. - CR and LF are dropped rather than escaped. There is no spelling of them that belongs in a header field, and the alternative is a sender who can end the field and begin one of their own.
Each of those has a test, and each test was checked by removing the rule it covers.
The budgets are the security property #
§4.6.4 caps an evaluation at ten DNS-querying terms and two lookups that
come back empty, and Limits is where those live. They are not tuning
knobs. An SPF record is a small program written by the domain being
checked, every include: in it is another program written by somebody
else again, and a receiver runs all of it on demand for anyone who opens
a connection. Without the budget that is a DNS amplifier with a mail
server attached.
They are also where implementations quietly differ, so the rules that are easy to get backwards are worth naming:
mxwith more than ten exchangers is apermerror, whileptrwith more than ten names ignores the rest and carries on. Same section, two different answers.exp=does not count, because its lookup happens after the record has already been evaluated.- An included
failis not a fail. §5.2 takes onlypassfor an answer; every other result simply does not match, and evaluation carries on to the next term. An implementation that propagated the fail would refuse the mail of every domain that writesinclude:a provider and then its own addresses — which is the common shape. - An
include:of a domain with no policy is apermerror, not anone: the record pointed somewhere and there was nothing there. So is aredirect=to one. - An IPv4-mapped IPv6 client is an IPv4 client, which §5 says
outright. A receiver on a dual-stack socket sees every IPv4 client that
way, so without it every
ip4:mechanism in the world stops matching.
Supplying the DNS #
SPF's evaluation is a recursive DNS walk, so the resolver is a parameter rather than something the library reaches for:
const resolver: spf.Resolver = .{ .context = my_resolver, .vtable = &.{
.txt = myTxt,
.addresses = myAddresses,
.mx = myMx,
.ptr = myPtr,
} };
An answer is written into storage the caller owns; nothing here allocates.
A resolver that runs out of room leaves what it already wrote in place and
returns error.NoSpace, because for ptr running out of room is not an
error at all — §4.6.4 asks for the first ten names to be used and the rest
ignored.
StubResolver is that interface over the real DNS. The asking is
zig-dns-client's — the
socket, /etc/resolv.conf, the identifier that has to be unguessable, the
EDNS0 buffer size, the move to TCP when an answer did not fit, and the
retry when a server does not answer — and what is here is the seam between
that and SPF: the five record types §5 asks about, and the error
distinctions §4.4 turns on.
const stub: spf.StubResolver = try .init(io); // whoever the host asks
const outcome = spf.checkMailFrom(io, stub.resolver(), client, helo, from, .{});
asking takes a list of servers instead, which is how a receiver points
this at the resolver it trusts rather than at whatever the host is
configured for. The search list is read and thrown away: every name SPF
looks up comes out of a record or an envelope and is already absolute, so
a check that answered out of a neighbouring zone would be worse than one
that failed.
Two things the client does that a first attempt does not, and the second was a live bug here before it was a paragraph:
- It advertises an EDNS0 response size. Without that record a server
must keep a UDP answer inside 512 octets, and 512 octets is nothing:
google.compublishes seventeen TXT records and about six fit. - It reads the truncation bit and asks again over TCP. An
implementation that ignored it would read those six — a different six
each time, because servers rotate a record set — and report that a
domain with a policy has none. That is not a rare corner.
apple.compublishes a TXT record whose content is a base64 blob saying, in as many words, that it is there to truncate UDP responses for TXT queries to apple.com;amazon.compublishes sixty-four records and 4.6 kilobytes of them, andspf txt amazon.comreturns all sixty-four.
It does not cache, which matters more here than it looks: §4.6.4
allows ten queries per check and a busy receiver runs a check per
message. A real deployment wants a cache underneath, and the whole point
of Resolver being an interface is that it can have one without this
library knowing. It also does not make the answer trustworthy: an off-path
forgery has to guess the identifier, the source port and the question to
be believed, and nothing here validates DNSSEC. What can be done about
that from here is the client's transport — one handed
dns_client.tls.Client.transport() speaks DNS over TLS or DNS over HTTPS
and authenticates the server against a certificate rather than against a
route.
Zone is that same interface over a table, which is how the tests run:
const zone: spf.Zone = .{ .entries = &.{
.{ .name = "example.com", .data = .{ .txt = "v=spf1 mx -all" } },
.{ .name = "example.com", .data = .{ .mx = "mail.example.com" } },
.{ .name = "mail.example.com", .data = .{ .a = "192.0.2.1" } },
} };
const outcome = spf.checkHelo(io, zone.resolver(), client, "example.com", .{});
RFC 7208's own conformance suite is written as zone data and expected results, so a resolver that reads a table runs the whole of it with no network, no nameserver and no virtual machine. That is what the seam is for.
The tool #
$ spf check 209.85.220.41 mail-sor-f41.google.com someone@gmail.com
pass
Received-SPF: pass (spf-cli: domain of someone@gmail.com designates 209.85.220.41 as permitted sender)
receiver=spf-cli; identity=mailfrom; client-ip=209.85.220.41;
envelope-from="someone@gmail.com"; helo=mail-sor-f41.google.com
spf txt <name> prints what the resolver reads out of the DNS, marking
the records §4.5 would select, and spf parse <record> takes a record
apart without asking anything — including saying when a redirect= is
dead because the record has an all.
It exists because a resolver is worth exactly what a live nameserver says it is worth: the unit tests and the conformance suite both read a table, which is what makes them fast and complete and is also the one thing they cannot check.
The conformance suite #
zig build test -Dconformance
conformance: 203 of 203 cases pass, 1 explanation text exempt
The openspf.org test suite is 203
cases written as zone data and expected results, and it is the only way
to find out whether an implementation agrees with everybody else's rather
than with itself. It is a lazy dependency — it lives in
pyspf's tree as
test/rfc7208-tests.yml — and nothing from it is copied into this
repository. src/conformance.zig holds a reader for the subset of YAML
the file uses, which is less work than it sounds and much less than the
general parser a test suite has no business depending on.
Three things the suite expects of a driver rather than of an implementation, each of which is a rule in that file with the reasoning next to it:
- A
SPF:record becomes a TXT record unless the name already has a TXT line of its own. RFC 7208 withdrew the SPF resource record type, but the suite still writes most of its zone data asSPF:with a note saying drivers are expected to duplicate those. The one section that tests the withdrawal publishes both types for a name and expects the TXT to win. - A
TIMEOUTbeside a real TXT record is ignored. It means the type SPF query times out — the case the suite's own comment calls "a popular braindead DNS server" — and a library that never asks for that type answers from the TXT. ATIMEOUTwith no TXT beside it is the TXT query itself failing, and stays atemperror. TXT: NONEis a marker, not a record. It says the TXT type has been accounted for at that name, which is what stops the duplication above and makes a type-SPF-only domain read as no policy at all.
The one exemption is v-macro-ip6, and it is a disagreement between the
suite and the RFC rather than a bug: the suite expects %{ir} to expand
an IPv6 address to nibbles in uppercase hexadecimal, while RFC 7208
§7.4's own worked example writes them in lowercase — which is what this
library produces and what its unit test transcribes. The suite writes DNS
names in uppercase throughout, which costs nothing where a name is
compared, because names are compared without case, and costs exactly this
where a name is compared as part of a sentence. The case's result is
still checked; only the string comparison is skipped, and the driver
fails if an exemption stops being reached.
Parsing a record #
const spf = @import("spf");
const record = try spf.Record.parse("v=spf1 a mx ip4:192.0.2.0/24 -all");
var terms = record.iterate();
while (try terms.next()) |term| switch (term) {
.directive => |directive| { /* qualifier + mechanism, in order */ },
.modifier => |modifier| { /* redirect, exp, or one to ignore */ },
};
Nothing allocates: every domain, name and value is a slice of the record text the caller still owns.
Two things the type knows that an iterator alone could not, and both change the answer:
record.has_all— §6.1 says aredirect=is ignored when the record has anallmechanism, becauseallalways matches and the redirect could never be reached. A record with both is not an error; it is a record whose redirect is dead.error.NotSpfis not a syntax error. §4.5 selects a record from a domain's TXT records by itsv=spf1prefix, so a TXT record that is something else entirely means the domain has no SPF policy — the resultnone. A record that claimed to be SPF and then did not parse is apermerror. Conflating the two reports a broken policy for every domain that publishes a DKIM key.
Expanding a macro #
var buffer: [spf.macro.max_domain_len]u8 = undefined;
const name = try spf.macro.expand(&buffer, "%{ir}.%{v}._spf.%{d2}", .{
.sender = .{ .local_part = "strong-bad", .domain = "email.example.com" },
.domain = "email.example.com",
.client_ip = .{ .ipv4 = .init(192, 0, 2, 3) },
.helo = "email.example.com",
}, .{}, true);
// 3.2.0.192.in-addr._spf.example.com
That example is from §7.4, and so is the test that checks it. The macro language is where a surprising amount of the conformance suite lives, and it has more corners than it looks:
- A transformer's digits count parts from the right, after an optional
reversal, and the parts are rejoined with
.whatever delimiter they were split on — the delimiter list says how to take a value apart, not how to put it back together. - An uppercase macro letter expands identically and is then URL escaped, which happens after the transformation rather than before.
%{i}is the address in the form a reverse lookup uses — for IPv6 that is 32 nibbles separated by dots — while%{c}is the readable form and is allowed only in an explanation. They are two macros because they are two different jobs.- Explanation text is
*( macro-string / SP ), so a space is a literal there and a syntax error everywhere else.
Addresses #
Addresses, networks and the comparison between them come from
z46, which with
zig-dns-client — the same
z46 underneath it, so an address out of a reply is an address here — is
what this depends on.
That is not a convenience: z46's parsing is already as strict as §5.6's
ABNF asks for — four decimal octets, no leading zeros anywhere including
in a prefix length, none of the inet_aton shorthands, no zone
identifier — and being strict here is a security property rather than a
nicety, since two implementations that disagree about whether
192.168.01.1 is octal disagree about which network an address is in.
Network.contains is then the whole of what an ip4: or ip6:
mechanism does, z46's RFC 5952 output is what %{c} expands to, and
Ipv6Address.nibbles is what %{i} expands an IPv6 client to — the 32
nibbles the right way round, since reversing them is the r transformer's
job and not the address's.
An ip4: network may be written with its host bits set — ip4:192.0.2.1/24
is a legal record — so the parser truncates rather than refusing, which is
the difference z46 draws between a network and an interface.
Licence #
MIT. The project follows REUSE; reuse lint
passes.
Standards #
- RFC 7208 — Sender
Policy Framework (SPF) for Authorizing Use of Domains in Email, Version
- The specification this implements.
- RFC 4291 — IPv6
addressing, which §5.6's
ip6:networks are written in. - RFC 3986 — URIs, whose unreserved set is what §7.3's uppercase macros escape against.
References cited #
- [RFC7208] Kitterman, S., "Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1", RFC 7208, April 2014, https://www.rfc-editor.org/info/rfc7208.
- [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006, https://www.rfc-editor.org/info/rfc4291.
- [SPF-TESTS] Gathman, S., Mehnle, J., et al., "The openspf.org SPF
test suite", release 2014.04,
http://www.open-spf.org/Test_Suite/. Read from its copy in
pyspf as
test/rfc7208-tests.yml; see The conformance suite. - [RFC1035] Mockapetris, P., "Domain names - implementation and
specification", STD 13, RFC 1035, November 1987,
https://www.rfc-editor.org/info/rfc1035. The query and response
format
StubResolverasks in, and §4.1.1's truncation bit. - [RFC6891] Damas, J., Graff, M., and P. Vixie, "Extension Mechanisms for DNS (EDNS(0))", STD 75, RFC 6891, April 2013, https://www.rfc-editor.org/info/rfc6891. The OPT record that says how large an answer a resolver can take.
- [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322,
October 2008, https://www.rfc-editor.org/info/rfc5322. The comment,
dot-atomandquoted-stringforms §9.1's header field is built from. - [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, January 2005, https://www.rfc-editor.org/info/rfc3986.
- [RFC5952] Kawamura, S. and M. Kawashima, "A Recommendation for IPv6
Address Text Representation", RFC 5952, August 2010,
https://www.rfc-editor.org/info/rfc5952. The canonical form
%{c}expands to, by way of z46. - [Z46] Ollie, J., "z46 — IPv4 and IPv6 addresses, networks and endpoints for Zig", MIT, https://git.jcollie.dev/jeff/z46. Parses, formats and compares every address in this library.
- [DNS-CLIENT] Ollie, J., "zig-dns-client — a DNS stub resolver for
Zig", MIT, https://git.jcollie.dev/jeff/zig-dns-client. The asking
half of
StubResolver: sockets,/etc/resolv.conf, EDNS0, and the move to TCP when an answer did not fit.
Building it with Nix #
nix build # the library and the tool
nix build .#checks.x86_64-linux.conformance # the openspf.org suite, in the sandbox
nix flake check
A Nix build has no network and the Zig package manager wants one, so the
dependencies are materialized by zon2nix
into build.zig.zon.nix and handed to zig build --system. That file is
generated and committed; regenerate it after any zig fetch --save:
nix develop -c zon2nix --16 --nix=build.zig.zon.nix build.zig.zon
The conformance suite runs inside the sandbox because it reads a table rather than a network — which is the same property that lets it be a unit test at all.
Tests #
zig build test # the unit tests
zig build test -Dconformance # and the openspf.org suite, fetched on demand
The unit tests are checked for vacuity by breaking the behaviour they
cover and confirming the right one fails — which has already earned its
keep: the test for §5.2's "an included fail is not a fail" rule passed
with the rule inverted, because the record it used ended in -all either
way.