An HTTP library for Zig 0.17, server and client, HTTP/1.0 through HTTP/3, with the protocol in a module that owns no socket.
zig-http doc https-records.md
10 kB · 197 lines
Markdown
at main

HTTPS records for http_io.Client: a plan #

Status. Implemented, all three steps. Step 1 — the hook and everything Client does with it — is in src/io/Client.zig, tested with a hook written in the test. Step 2 is zig-dns-client's ServiceLookup, from ef4f807. Step 3 is interop/https-records/, which runs the two together against a DNS server publishing HTTPS records. The three questions this plan ended on are settled; see Decisions.

This is what it takes for http_io.Client to learn from DNS — RFC 9460's HTTPS resource records — that an origin speaks HTTP/3, so that even the first request to it can go over QUIC. Alt-Svc, which is implemented, can only say so on a response, which means the first request has already gone over TCP.

What an HTTPS record says #

RFC 9460 §9 defines the HTTPS RR type (65), SVCB (64) made specific to the https scheme. A record is either AliasMode (priority 0), which says "look at that name instead" and is how a zone apex gets something a CNAME could never be, or ServiceMode (priority ≥ 1), which describes an endpoint:

example.com. 300 IN HTTPS 1 . alpn=h3,h2 port=8443 ipv4hint=192.0.2.1

— a TargetName (. meaning the owner's own name, §2.5.2), and parameters (§7): alpn and no-default-alpn, port, ipv4hint and ipv6hint, ech, and mandatory, which lists the keys a client must understand or else ignore the record (§8). The default ALPN set for HTTPS is http/1.1 alone (§9).

Why this is a hook, and where each half lives #

std.Io.net.HostName.lookup returns addresses and nothing else; it cannot ask for record type 65. zig-dns-client can — zig-dns already decodes SVCB and HTTPS records, mandatory check included (src/rdata/svcb.zig) — but http_io cannot depend on it, because zig-dns-client speaks DNS over HTTPS with this library. That circle is why Client.resolver is a function pointer, and the same answer applies here: a second hook, supplied by the program, with zig-dns-client providing an implementation beside its HostLookup.

RFC 9460 draws the line between the two halves itself:

  • The resolver's half — "SVCB resolution" (§3). Build the query name (§9.1: the host itself for port 443, _<port>._https.<host> otherwise), follow AliasMode records and CNAMEs with a chain limit (§3, §3.1), drop records with a mandatory key it does not understand (§8), and hand back the ServiceMode records sorted by priority. None of this needs to know what HTTP is. It lives in zig-dns-client, as a ServiceLookup next to HostLookup.
  • The client's half — using the endpoints (§3, §7.1.2, §9). Decide per endpoint which transports its alpn set allows, connect, and fall back. That is HTTP policy, and it lives in http_io.Client.

The hook #

Shaped like HostName.lookup, which Client.resolver already copies, so that it streams, closes its queue on return, and must not block on a queue of sixteen:

/// One ServiceMode endpoint, after SVCB resolution.
pub const Service = struct {
    priority: u16 = 1,
    /// Empty for the name the lookup was for (§2.5.2's "."); `target()` and
    /// `setTarget()`.
    target_buf: [255]u8 = undefined,
    target_len: u8 = 0,
    /// Null for the origin's own port.
    port: ?u16 = null,
    alpn: Alpn = .{}, // { http_1_1, h2, h3 }
    /// Whether §9's default, `http/1.1`, is withdrawn.
    no_default_alpn: bool = false,
    /// `ipv4hint` and `ipv6hint` together; `hints()` and `addHint()`. Used
    /// only when the target's own addresses cannot be had.
    hints_buf: [8]net.IpAddress = undefined,
    hints_len: u8 = 0,
};

pub const ServiceLookupOptions = struct { port: u16 };

services: ?*const fn (
    host_name: net.HostName,
    io: Io,
    resolved: *Io.Queue(Service),
    options: ServiceLookupOptions,
) ServiceLookupError!void = null,

The arguments are in HostName.lookup's order, as Client.resolver's are.

Service is zig-http's type, and zig-dns-client's copy of zig-http is always another pin — so the type cannot cross between them directly. zig-dns-client's ServiceLookup produces a type of its own, and a program that uses both writes the few lines that copy one into the other. The subproject in step 3 is where those lines live, and doubles as the example of them.

ServiceLookupError needs one distinction beyond "nothing found": ProtectedResolutionFailed. §3.1 says that when DNS is cryptographically protected (DNSSEC, or DoT/DoH — which zig-dns-client offers) and SVCB resolution fails with SERVFAIL, a transport error or a timeout, the client "SHOULD abandon its attempt", since otherwise an attacker downgrades the connection by dropping the query. Only the resolver knows whether its channel was protected, so it has to say so.

An alternative shape — the hook returns raw record data and http_io decodes it with zig-dns, which has no dependency on zig-http — was considered and rejected: AliasMode chasing needs more queries, so the resolver has to run the procedure anyway, and splitting the parsing from the querying buys a zig-dns dependency for nothing.

What Client does with it #

In .advertised mode, since an HTTPS record is the origin advertising in DNS what Alt-Svc advertises over HTTP — RFC 9460 §9.2 draws the comparison:

  1. For an https URL fetched directly, ask services. §3 would have it asked in parallel with the A/AAAA lookup ("SHOULD issue in parallel any other DNS queries"); the first version asks it first — see below.
  2. Walk the endpoints in priority order. For each, per §7.1.2: if its ALPN set holds h3, try QUIC offering h3; if it holds h2 or http/1.1 — or the default applies — try TLS over TCP offering everything this client speaks over TCP (h2, http/1.1), not just the record's set, since §7.1.2 step 3 builds the ProtocolNameList "without regard to the SVCB ALPN set". The address is the TargetName's A/AAAA, with the hints to start from. SNI, the certificate check and :authority are the origin's (§9.1, §9.4).
  3. When every endpoint fails, fall back to connecting to the origin as if there were no record (§3, SVCB-optional), unless the hook reported ProtectedResolutionFailed, in which case the fetch fails.
  4. Through a proxy, not at all. §3.2: an SVCB-optional client using a domain-oriented proxy such as CONNECT, with no private way to resolve SVCB, "MUST disable SVCB resolution entirely". The proxy resolves names; asking DNS here would tell a second party where the request is going.
  5. With a cached Alt-Svc, §9.3 asks for the HTTPS records of the alt-authority and for attempts consistent with both. A first version can prefer the HTTPS record for the first request and Alt-Svc thereafter, and leave full consistency checking as a follow-up.

Decisions #

  • The end-to-end test lives in zig-http, in a subproject with its own build.zig.zon that can depend on zig-dns-client without the main build ever fetching it. zig-http's own .paths does not include it, so a project depending on zig-http does not receive it either.
  • An HTTPS record upgrades http to https (§9.5), with Client.https_record_upgrade to turn it off. The upgrade is a redirect nothing was sent for: it spends none of the caller's redirect budget, and privileged headers are kept, as a browser's HSTS upgrade keeps them — nothing went out in cleartext for them to leak through.
  • .advertised covers both Alt-Svc and HTTPS records. A cached Alt-Svc alternative is tried first, being the more specific; then the records' endpoints; then the origin.

Deliberately out of the first version #

  • ECH (the ech key, draft-ietf-tls-esni). tls.zig and zig-quic have no Encrypted ClientHello, so a record with ech among its mandatory keys is dropped by the resolver, and an optional ech is ignored.
  • Asking in parallel with A/AAAA (§3's SHOULD). The records are asked for first, then the addresses; one extra round trip, where a second task would save it.
  • Caching. The records carry a TTL, and the resolver is where DNS caching belongs; Client would ask once per fetch. If that proves costly, AltSvc is the table to extend.
  • Happy Eyeballs (RFC 8305). Endpoints and addresses are tried in order, as they are now.

Steps #

  1. zig-http: Service, the services hook, and the endpoint walk in Client, tested with a hook written in the test — as Client.resolver is — against the HTTP/3 fixture: an h3 record taking the first request to QUIC, a TargetName on another name, a port, no-default-alpn, fallback when every endpoint fails, ProtectedResolutionFailed aborting, and nothing asked through a proxy.
  2. zig-dns-client: ServiceLookup, SVCB resolution over its Resolver with alias chasing, the §9.1 query names and the mandatory check, tested against its loopback server. It pins zig-http, so this lands after step 1.
  3. Proving it together. The demonstration binary no longer links zig-dns-client, so the end-to-end test — a real DNS server publishing an HTTPS record, the client finding HTTP/3 on the first request — belongs where both libraries meet: a NixOS test in zig-dns-client, or in a program that depends on both, such as push-container.

References #

  • RFC 9460, Service Binding and Parameter Specification via the DNS (SVCB and HTTPS Resource Records), 2023. https://www.rfc-editor.org/info/rfc9460. §§2.3, 2.5, 3, 3.1, 3.2, 7, 7.1.2, 8, 9, 9.1–9.5.
  • RFC 7838, HTTP Alternative Services, 2016 — implemented; see client.md.
  • RFC 8305, Happy Eyeballs Version 2, 2017. Named as out of scope.
  • draft-ietf-tls-esni, TLS Encrypted Client Hello. Named as out of scope.