An HTTP library for Zig 0.17, server and client, HTTP/1.0 through HTTP/3, with the protocol in a module that owns no socket.
zig-http doc client.md
40 kB · 709 lines
Markdown
at main

zig-http as a client #

Making requests with zig-http: http_io.Client, which takes a URL and works out the rest, and underneath it the three drivers that each speak one version of HTTP over a connection handed to them. The API documentation at https://jeff.jcollie.page/zig-http/ is generated from the doc comments; this is the account of how the pieces behave together, and of what testing them against other implementations found.

The server side is server.md.

Fetching a URL #

http_io.Client is for a caller holding a URL rather than a connection. It has the shape of std.http.Client.fetch — a location, a method, an optional payload, a writer for the body, a status back — and does the parts the drivers below leave to whoever holds them: it parses the URL with zig-uri, resolves the name, dials, negotiates the version, and follows redirects.

var client: http_io.Client = .{ .io = io, .gpa = gpa };
defer client.deinit();

var body: std.Io.Writer.Allocating = .init(gpa);
defer body.deinit();
var result = try client.fetch(.{
    .location = .{ .url = "https://example.com/" },
    .response_writer = &body.writer,
});
defer result.deinit();
// result.status == .ok, result.version == .http_2,
// result.headers.get("content-type").? == "text/html"

The version is the server's choice. An https URL is dialled offering h2 ahead of http/1.1 in ALPN, and whichever the server selects is the driver that speaks — §3.2 of RFC 9113 makes that the only way HTTP/2 over TLS is agreed. A server that answers with no ALPN extension at all is not taking part in the negotiation (§3.1 of RFC 7301 makes answering optional), and gets HTTP/1.1, which is what an https URL meant before there was anything to negotiate. Cleartext has no handshake to negotiate in and RFC 9113 deprecated the Upgrade that used to stand in for one, so an http URL is HTTP/1.1 unless http2_prior_knowledge says the server speaks HTTP/2 from the first octet. http2 = false offers http/1.1 alone.

The name is resolved through std.Io.net.HostName.lookup, or through Client.resolver when one is given, and the addresses are tried in the order the resolver gave until one accepts the connection — tcp.connectAny and tls.connectAny, which are there for any caller with a list of addresses. A certificate that fails to verify at one address is not retried at the next: every address a name resolves to is meant to be the same server, and trying them until one says something different is the shape of an attack rather than a fallback. Host names are ASCII, so an internationalised one has to arrive in its xn-- form.

The target is sent as it was written. zig-uri keeps a path and a query decoded, and decoding forgets which reserved characters were escaped: a%2Bb and a+b both come back as a+b. §2.2 of RFC 3986 says those are different URIs, and a server that reads a query as a form — Django, PHP, Rails, Go's ParseQuery — takes a bare + for a space. So the path and query of a .url, and of a Location that is followed, go out exactly as written, after a check that they read back as the URI the client resolved. A target that is not written that way already — a space in it, an IRI, a dot segment that resolution removed — and a .uri location, which has no written form, are encoded again from the parsed URI.

The body can be routed by the status. response_writer gets every body whatever its status, a 404's error page as much as a 200's content. response_writer_for is the alternative for a caller who cares — a download going into a file, say: its choose is called once with the status, version and fields of the response fetch will return, before any of its body is read, and returns the writer for the body or null to discard it.

const OnSuccess = struct {
    file: *std.Io.Writer,
    fn choose(context: *anyopaque, head: http_io.Client.ResponseHead) std.Io.Writer.Error!?*std.Io.Writer {
        const self: *@This() = @ptrCast(@alignCast(context));
        return if (head.status.class() == .success) self.file else null;
    }
};
var on_success: OnSuccess = .{ .file = &file_writer.interface };
var result = try client.fetch(.{
    .location = .{ .url = "https://example.com/big.iso" },
    .response_writer_for = .{ .context = &on_success, .choose = OnSuccess.choose },
});

The certificate is checked against the system's store unless root_ca names a bundle. That is the opposite default from tls.ClientOptions, which trusts nothing until told, and deliberately so: that is a dialler handed an address by a caller who has already decided what it is talking to, and this is what a caller reaches for when it has a URL and wants what is at the other end.

Redirects are followed as std.http.Client follows them: up to three for a request without a payload and none for one with, since resending a body to wherever a server points is the caller's decision — redirects = .{ .follow = n } makes it. Location is resolved against the URL that was fetched by RFC 3986 §5's algorithm, a 303 becomes a GET (§15.4.4 of RFC 9110), a POST answered 301 or 302 becomes one too (§15.4.2 and §15.4.3 allow it and every browser does it), and a 307 or 308 keeps the method and the content. Headers come in two kinds: extra_headers go wherever the request goes, and privileged_headers — an Authorization, a Cookie — stop at the first redirect to another origin, the scheme, host and port triple of §4.3.1 of RFC 9110. A URL carrying userinfo is refused outright, as §4.2.4 of RFC 9110 asks of a recipient; a credential belongs in privileged_headers, where a redirect cannot carry it off. A redirect from https to http is followed, as browsers, curl and std.http.Client follow it, unless refuse_insecure_redirects is set — then it is error.InsecureRedirect, before anything goes out in cleartext, since whatever follows a downgrade crosses the network readable by anybody on the path, who may also be the one who asked for it.

What it does not do is keep anything between calls. Every fetch dials, sends, reads and closes, so HTTP/2's multiplexing is not used across calls and a redirect to the same origin opens a second connection; a caller wanting many requests on one connection uses the drivers directly, which is what they are for. A Client is not safe to share between tasks — it holds a random number generator and a lazily loaded certificate bundle, so one each is cheap. Over HTTP/2 the body is bounded by http2_config.max_body_len, because that driver gathers a body before handing it over; over HTTP/1.1 it streams into the writer.

HTTP/3 is asked for, not negotiated. Over TCP, ALPN chooses the version inside the one handshake; QUIC is another transport on another protocol, so trying it is a decision made before anything is sent. http3 = .only fetches an https URL over HTTP/3 or fails with the handshake's error, and .prefer tries QUIC first and falls back to TCP — before anything was sent, so a request is never sent twice. Each address the name resolved to is tried in turn, and a handshake is given http3_handshake_timeout, three seconds by default: the usual reason for no answer is UDP blocked on the way, which is silence rather than a refusal and costs the whole wait. A certificate refused over QUIC is never retried at another address or over TCP, as over TLS. An http URL is never fetched over HTTP/3, since RFC 9114 has no cleartext form, and neither is anything going through a proxy, which would be RFC 9298's CONNECT-UDP. Against the internet: Cloudflare, Google and this repository's Forgejo answer over HTTP/3, and example.com, which has none, fails .only and falls back under .prefer.

Or HTTP/3 where the origin has said so, which is http3 = .advertised and how a browser finds it. RFC 7838's Alt-Svc: h3=":443"; ma=86400 on any response says the origin can be had over QUIC there for the next day; the first fetch to an origin goes over TCP, since nothing has been said yet, and what its response advertises is remembered in an http_io.AltSvc — a fixed table of 32 origins, allocating nothing, with a lock so that Clients on several tasks can share one through Client.alt_svc and what any of them learns the rest use. Later fetches go to the alternative until it stops being fresh: ma, less the response's Age, as §3.1 does the arithmetic. A new Alt-Svc replaces what the origin said before, clear forgets it, and only h3 is kept — the drafts' h3-29, which Google still sends, is skipped for the h3 beside it. The alternative may name another host, and the connection goes there, but SNI and the certificate check are the origin's — §2.1's "reasonable assurances that the alternative service is under control of and valid for the whole origin" are exactly a certificate for it. So the cache learns only from responses over TLS made directly: not over cleartext, where anybody could have said it, and not through a proxy, which §2.4 says a request should keep going through. An alternative that fails falls back to TCP before anything was sent and is left alone for five minutes, though every response goes on advertising it, so that a network dropping UDP costs one handshake timeout rather than one per fetch. AltSvc.clear forgets everything, which §2.2 wants when the network changes and which only the caller can see.

And HTTP/3 from DNS, which Alt-Svc cannot do: an Alt-Svc arrives on a response, so the first request to an origin has already gone over TCP. RFC 9460's HTTPS records say the same thing in DNS before anything is sent, and Client.services is a hook — HostName.lookup's shape, as resolver is — through which .advertised asks for them. The standard library cannot ask for record type 65, and zig-dns-client, which can, cannot be a dependency of http_io, so the program supplies it. The hook does SVCB resolution — query names, aliases, mandatory keys — and hands back endpoints; Client tries them in priority order, over QUIC where an endpoint's ALPN set holds h3 and over TLS where it holds anything that runs over TCP, with the origin's name for SNI and the certificate wherever the connection goes, and falls back to the origin when none answers. A cached Alt-Svc is tried before the records. A lookup that fails over DNS the hook knows to be protected abandons the fetch, since carrying on without the records is the downgrade §3.1 warns of; nothing is asked through a proxy (§3.2). And an http URL whose origin has a record is fetched as https, as §9.5 asks — https_record_upgrade turns that off. https-records.md is the plan this came from.

zig-dns-client's ServiceLookup is the resolver's half, and interop/https-records/ is where the two meet: a project of its own, with a build.zig.zon of its own that depends on zig-http by path and on zig-dns-client by URL, so that neither this build nor any project depending on zig-http ever fetches zig-dns-client. Its https_records module is the glue a program using both needs — Client.Service and zig-dns-client's Service are separate types, since each library pins its own copy of the other, so the endpoints are copied across — and its test runs a DNS server on loopback publishing HTTPS records, zig-http serving over TCP and QUIC, and a client that takes its very first request to HTTP/3 on a port only DNS named, follows an alias, upgrades an http URL, and stays on TCP for a name with no records:

$ cd interop/https-records
$ nix develop ../.. -c zig build test --summary all

A body too large to hold goes in payload_reader rather than payload: a reader and a length. The length is declared as the Content-Length, and exactly that many octets are then read from the reader straight into the connection — after the head over HTTP/1.1, and over HTTP/2 as DATA frames sized by both flow-control windows — with nothing buffered beyond the connection's own buffers, so a 32 MiB upload in tests/fetch.zig runs in 15 MB of memory. A reader that ends before the length is error.RequestBodyShort and one that fails is error.RequestBodyReadFailed, never a shorter body: over HTTP/1.1 the connection is abandoned mid-message, and over HTTP/2 the stream is reset with CANCEL so the server stops waiting. A reader is read once, so a redirect that would send the body again — a 307 or 308, or a 301 or 302 to anything but a POST — is error.StreamedPayloadRedirected, while a 303, which is answered with a GET, is still followed. Underneath it is client.Request.source, which all three drivers take: HTTP/3 sends it as DATA frames of sixteen kibibytes, each filled before its header is written, with QUIC's flow control deciding the pace, and resets a stream it cannot finish with H3_REQUEST_CANCELLED. That last part found a bug in zig-quic — a stream reset while blocked on flow control declared a final size larger than the peer had allowed, and the peer closed the whole connection — which is fixed at the pin this uses.

A timeout is Client.timeout, a std.Io.Timeout — a duration or a deadline — and it is one budget for the whole fetch: the lookup, the connection, the handshake, every redirect and all of the body. Not one per read, because a server trickling a byte a second never trips one of those, and because the standard library has none to give portably: Zig 0.17 times a socket read only on Io.Threaded, panicking on its other Ios, and its connect timeout is a @panic("TODO"). So a timed fetch runs as a task of its own beside one that sleeps until the deadline, std.Io.Select reports whichever finishes first, and the other is cancelled — which interrupts a blocked read at once rather than at the next octet, and closes the connection on the way out like any other failure. It is error.Timeout, and it needs an Io that can run two tasks together; zig-http --fetch URL --fetch-timeout 30 sets one.

result.headers are the last response's fields — the one that was not followed — in the order they arrived, with a repeated field kept as the separate lines it came as, and looked up without case, since HTTP/2 sends every name in lowercase and HTTP/1.1 however the server wrote it. They are copied out of the connection, which fetch closes before it returns, so a result owns them and deinit is what frees them. This is where FetchResult parts company with std.http.Client's, which has a status and nothing to free.

result.status is std.http.Status, as std.http.Client.FetchResult has it: non-exhaustive, so a code nothing has a name for still arrives, and class() is what §15 of RFC 9110 has a client act on for one. It is never an interim response's: a 100 Continue or a 103 Early Hints ahead of the answer is read past, and options.interim is how to see them on the way — see Interim responses below. Over a redirect it is told of every hop's, in the order they came.

Client.resolver is a function with std.Io.net.HostName.lookup's signature, and that function is its default — so anything written to stand in for the standard library's lookup stands in for it here too. The contract is that function's: addresses go into the queue carrying the port asked for, the queue is closed on return even on error, and a queue of sixteen must not block. There is no context pointer, so a replacement that needs configuration keeps it at container level. The demonstration binary uses the default.

Through a proxy #

http_proxy, https_proxy and no_proxy on Client, and initDefaultProxies to fill them from the environment the way curl and std.http.Client read it. A Proxy is a host, a port, whether the connection to it is TLS, and a Proxy-Authorization value; Proxy.parse makes one out of http://user:secret@proxy:3128, turning the userinfo into a Basic credential as RFC 7617 spells it, and takes a value with no scheme as http, which is how a good many environments write it.

Which way through a proxy goes depends on what is being fetched:

  • An http URL is forwarded. The request goes to the proxy with its target in absolute form — GET http://origin/ HTTP/1.1, §3.2.2 of RFC 9112 — and the proxy fetches it. It is HTTP/1.1 to the proxy whatever the origin speaks, and a 407 is simply the response, since it came from the only server this end spoke to.
  • An https URL is tunnelled. CONNECT origin:443 asks the proxy for a byte stream (§9.3.6 of RFC 9110), and the TLS handshake runs inside it. So the certificate checked is the origin's, the proxy sees ciphertext and nothing else, and ALPN reaches the origin through the tunnel and comes back h2 exactly as it would without one. A tunnel that is refused is an error rather than a response — error.ProxyAuthenticationRequired for a 407, error.ProxyRefused for anything else — because nothing a proxy says in answer to CONNECT came from the origin.

The connection to the proxy is either plain or TLS, independently of both: an https URL through an https:// proxy is TLS inside TLS, which is why the TLS session's own read buffer is one whole record rather than a round 16 KiB. The origin's name is resolved by the proxy and never by this end, which is half the reason proxies are used.

The pieces are public for a caller holding the drivers rather than Client: http_io.proxy.tunnel asks for a CONNECT over any transport.Connection and hands the same connection back carrying the tunnel, and http_io.tls.connectOver runs a handshake over a connection that is already open. http_io.proxy's documentation has the four lines that put an HTTP/2 driver behind a proxy. What there is not is a way through a proxy for HTTP/3, which would be RFC 9298's CONNECT-UDP.

zig-http --fetch URL --fetch-proxy http://user:secret@proxy:3128 uses one, and without --fetch-proxy the environment's is used. tests/nixos/proxy.nix puts tinyproxy in front of an origin whose firewall admits nothing else, and asks both ways through it, with and without the right credential.

Trying it #

zig-http --fetch URL, with no --fetch-version, is this client, and tests/fetch.zig runs it against this library's own server over loopback sockets, with and without TLS, looking localhost up for real.

A client, for HTTP/1.0 and HTTP/1.1 #

http.client is the mirror of http.server: a Request is what server.Request describes seen from the other end, and a Response the same for server.Response, so a caller who has written a handler already knows both. http1.client.Connection puts them on a byte stream, over the same transport.Connection the server is served over — which means it runs over TCP, over TLS, and over a slice of memory in a test without knowing the difference.

const conn = try http_io.tcp.connect(io, gpa, address, .{});
var c = try http.http1.client.Connection.init(io, gpa, conn, .{});
defer c.deinit();

const res = try c.send(.{ .host = "example.com", .target = "/" });
const body = try c.readAll(res, 1 << 20);
defer gpa.free(body);

Almost every rule in the driver is about framing, because framing is where an HTTP/1.1 client is dangerous to itself. A response whose length the client reads differently from the server that sent it ends in the wrong place, and what follows it on a kept-alive connection is then read as the next response — which is response splitting, done to oneself. So §6.3's eight rules are applied in order by protocol.responseFraming, a message that frames itself two ways at once is refused outright rather than resolved, and a connection whose framing was uncertain is never reused. canReuse is the one question a caller has to ask, and a body left unread answers it false.

Two things worth knowing about lifetimes. Every slice in a response — the reason phrase, every field name and value — points into the connection's head buffer, so a response is valid until the next send or until the connection goes away. And deinit closes the connection it was given, the way the server closes the one it accepted; release is the way out for a caller that wants the socket afterwards, which is what a 101 leaves behind.

What a request says about how long its body is #

Over HTTP/1.1 a Content-Length is the only thing delimiting content, so the driver has always written one. Over HTTP/2 and HTTP/3 it is optional — §8.1 of RFC 9113 and §4.1 of RFC 9114 delimit the content with the stream — and both of those drivers used to leave it out on that ground, which was a mistake in two ways.

§8.6 of RFC 9110 asks for it: "A user agent SHOULD send Content-Length in a request when the method defines a meaning for enclosed content and it is not sending Transfer-Encoding." Recipients use it to check that what arrived is what was sent, and a server that has it can read a body without waiting to find out how long it is — where a recipient of an HTTP/3 body without one learns the length only when it stops arriving, which is why req.body.length comes back unknown there and none over HTTP/2. And a server may simply insist: Cloudflare's DNS-over-HTTP/3 endpoint answers 400 to a POST without one, which is how this was found — ten requests without the field failed, ten with it did not.

So both multiplexed drivers now send it when the length is known, and Request.declaredContentLength is the one place that decides what "known" means, so the two cannot drift: a .exact body declares its length, a .none body declares nothing (not Content-Length: 0 — a request with no body is not a request with an empty one), and an .unknown one declares nothing either, because the caller left the question open and a field filled in from whatever is in hand would be the driver answering it.

It is also where the length is checked against the content in hand. A Content-Length that disagrees with the DATA frames makes the message malformed — §8.1.1 of RFC 9113 and §4.1.2 of RFC 9114 both say so — so a request whose two halves disagree is error.InvalidRequest before anything goes out, which is the same thing Request.reserved does for a caller trying to set the field by hand.

HTTP/2, where a request is begun and awaited separately #

http2.client is the same idea over RFC 9113, and the difference is not the plumbing but the shape of an exchange. HTTP/2 is multiplexed, so a request is begun and awaited as two steps, and several may be outstanding between them:

var c = try http.http2.client.Connection.init(io, gpa, conn, .{});
defer c.deinit();
try c.handshake();

const first = try c.begin(.{ .host = "example.com", .target = "/one" });
const second = try c.begin(.{ .host = "example.com", .target = "/two" });

const a = try c.await(first);   // frames for `second` are filed as they arrive
const b = try c.await(second);

The pump is the caller's, not a task's. The server runs a reader task and a task per stream, because it does not know when its next request arrives and a handler must not block the connection. A client knows exactly what it asked for, so nothing needs to run on its own: await reads frames until the stream it was asked about is finished, filing frames for other streams into their own slots as it goes. Two requests begun before either is awaited are genuinely in flight together, and it costs no Io.Group, no mutex and no second task. What it does cost is a buffer per outstanding stream, bounded by the window the client advertises — which is what flow control is for.

Flow control is real on both sides: a request body larger than the 65535-octet initial window and the 16 KiB frame limit is split and waits for WINDOW_UPDATE, which tests/tcp.zig exercises with 200 KiB against this library's own server.

Interim responses #

§15.2 of RFC 9110 lets a server send any number of 1xx responses ahead of the final one, and requires that "a client MUST be able to parse one or more 1xx responses received prior to a final response, even if the client does not expect one". Every driver here reads them and returns the final response; none of them is ever what send or await hands back. A caller that wants to see them on the way — the Link fields of a 103 Early Hints, to start fetching a stylesheet before the page is ready, which is what RFC 8297 is for — gives the request a hook:

const Hints = struct {
    fn told(context: ?*anyopaque, head: http.client.Request.InterimHead) void {
        const self: *@This() = @ptrCast(@alignCast(context.?));
        if (head.headers.get("link")) |link| self.preload(link);
    }
    // ...
};

var hints: Hints = .{};
const res = try c.send(.{
    .host = "example.com",
    .target = "/",
    .interim = .{ .context = &hints, .call = Hints.told },
});

fetch takes the same thing as FetchOptions.interim. The head it is shown is borrowed: its fields point into the connection and are gone once the hook returns, so copy what has to outlive the call. Over HTTP/2 the hook is called by whoever is reading the connection, which may be an await for a different request on it — frames are filed as they arrive, whatever stream they are for — so its context has to stay valid until that request's own response has been returned.

A 101 Switching Protocols is not one of these. It is informational by number and final in meaning: it ends HTTP on the connection, so over HTTP/1.1 it is the response send returns, and over HTTP/2 and HTTP/3, which removed it, it is MalformedResponse. So is an interim response that ends the stream, which §8.1 of RFC 9113 and §4.1 of RFC 9114 both forbid.

§15.2 puts no number on them, so each driver's Config does: max_interim_responses, sixteen by default, past which the request fails with TooManyInterimResponses rather than waiting for ever on a server that only ever says 100. Each one is let go of once it has been told — the HTTP/1.1 driver reads the next head into the same buffer, and the multiplexed ones start the stream's arena again — so sixteen cost what one does.

Before the drivers knew about them, each got them wrong in its own way, and the ways are worth recording. The HTTP/1.1 client returned the 103 as the response and left the final one unread on the wire, on a connection it still called reusable, so the next request on it was answered with this one's response. The HTTP/2 client returned the 103 with the final response's fields appended to it, a literal :status among them; it appended a trailer section the same way, unchecked. The HTTP/3 client took the final response for a trailer section and the body after it for a malformed message. The HTTP/2 client now tells a stream's HEADERS apart — interim, final, and at most one trailer section, which ends the stream and is dropped as the other two drivers drop theirs — and a malformed response on one stream is reported by that stream's await rather than by whichever await happened to read it.

Holding the content back: Expect: 100-continue #

A large upload to a server that is going to refuse it — a 401 for a credential that has expired, a 413 for a file too large — costs the whole upload before the refusal arrives, and over HTTP/1.1 the only way to stop a body once it has started is to close the connection. §10.1.1 of RFC 9110 is the way round: the request says Expect: 100-continue and goes out without its content, and the server answers the head alone — a 100 Continue to ask for the content, or the final response that refuses it.

const res = try c.send(.{
    .method = http.protocol.Method.put,
    .host = "example.com",
    .target = "/backups/disk.img",
    .body = .{ .exact = size },
    .source = &file_reader.interface,
    .expect_continue = true,
});
if (!res.content_sent) {
    // Refused before any of it went: `res.status` says why, and the reader
    // has not been touched.
}

expect_continue is a field rather than a header — Expect is one of the fields Request.reserved keeps for the driver — because the driver has to know to wait, and a field set by hand would say one thing while the driver did another. It is dropped where it cannot apply: a request with no content has nothing to hold back, and HTTP/1.0 has no 100 to answer it with.

What happens once the head is out:

  • A 100 Continue, and the content goes, and the final response is read as usual. The 100 is an interim response like any other: interim is told of it and max_interim_responses counts it.
  • A final response first, and the content never goes: content_sent is false. Over HTTP/1.1 the connection is then not reused, since the server may be waiting for the content or may be about to close, and there is no telling which. Over HTTP/2 the request's half of the stream is closed with RST_STREAM of NO_ERROR once the response is complete, and over HTTP/3 with a reset of H3_NO_ERROR — §8.1 of RFC 9113 and §4.1 of RFC 9114's way of saying the content is not coming, rather than ending the stream as though it had been empty.

Over HTTP/2, begin sends the head alone and the content goes out from that request's own await once its 100 has arrived — even if some other request's await was the one that read the 100. So the content slice or source reader has to stay valid until then.

There is no timer. The usual practice — curl's — is to wait about a second for the 100 and then send the content anyway, for the sake of a server that ignores the field. Zig 0.17 times a socket read only on Io.Threaded and never through TLS, so a driver here waits for as long as the server takes. §10.1.1 has an origin server answer such a request "immediately", so a conforming one never leaves it waiting; a server or an intermediary that ignores the field leaves it waiting for ever. That is why it is something a caller asks for and not something that happens by default.

fetch takes it as FetchOptions.expect_continue — .never, the default; .always; or .{ .above = n } for a payload longer than n octets — and uses timeout as the bound the drivers do not have, so the two belong together:

var result = try client.fetch(.{
    .location = .{ .url = "https://example.com/upload" },
    .method = http.protocol.Method.put,
    .payload_reader = .{ .reader = &file_reader.interface, .length = size },
    .expect_continue = .{ .above = 1 << 20 },
});

Two things follow from fetch owning the whole exchange. A redirect that arrives before the content was asked for is followed with the content still unread, so a 307 sends a payload_reader on to where it is wanted rather than ending in error.StreamedPayloadRedirected. And a 417 Expectation Failed — what an intermediary that does not understand the field may answer — is met by sending the request once more without the expectation, as §15.5.18 of RFC 9110 suggests.

Against a server this library did not write #

tests/nixos/client.nix points the client at nginx, which answers HTTP/1.1 and cleartext HTTP/2 on one port by looking at the first octets — the same thing this library's own server does, so one guest gets a second opinion on both — and HTTP/3 on another, with nginx's own QUIC stack. It fetches over 1.0, 1.1, 2 and 3, asks for a body larger than one frame and one window over each, and checks that a 404 comes back as a 404.

It earned its place immediately. Two faults survived every in-process test and died here:

  • The connection window is not the stream window. §6.9.2 starts the connection's flow-control window at 65535 and SETTINGS_INITIAL_WINDOW_SIZE does not change it — only a WINDOW_UPDATE on stream 0 does. The client accounted against a window nginx had never agreed to, so nginx stopped after 65535 octets and waited for credit while the client waited for the rest. Neither end was at fault, and against this library's own server it never showed, because both halves made the same assumption.
  • A frame payload is not necessarily in the buffer. §4.2 puts the floor for SETTINGS_MAX_FRAME_SIZE at 16384, which is larger than a connection's default read buffer, so a payload has to be streamed rather than taken whole. The server had always done this; the client had not, and would have worked against peers that send small frames and failed against those that do not.

The demonstration binary fetches as well as serves — --fetch URL with --fetch-version 1.0, 1.1 or 2 — which is what the guest drives. Naming a version means one driver pointed at one socket, so that form takes an address rather than a name; IpAddress.resolve sounds like it would take a hostname and does not, since it parses a literal and the "resolve" in the name is about an IPv6 interface scope. Leaving the version out is http_io.Client, which looks the name up with HostName.lookup and lets the server choose the version — and the guest asks it that too, by name, over cleartext and over TLS.

HTTP/3, where most of the work is absent #

http3.client is written against transport.Session and nothing else — the same seam the server driver uses, which is what made it possible without a second QUIC implementation. http_io.http3.Dialer dials and hands one over.

Most of what the HTTP/2 client does has no counterpart here, and every absence is §4.6 giving the job to QUIC. No connection preface, because the handshake is one. No stream table, because streams belong to the transport. No flow control and no WINDOW_UPDATE — the two faults nginx found in the HTTP/2 client were both in code this file does not have. No shared decoder state to lose, because §3.2.2 of RFC 9204 with a table capacity of zero means no field section can depend on an insertion. What is left is: open the control stream, say what the settings are, put a HEADERS frame and some DATA frames on a bidirectional stream, and read the same shape back.

Concurrency is the caller's, for the same reason: several requests are several QUIC streams, which is what §4.6 hands to the transport.

A request ends with its last octets rather than behind them. transport.Session has a finishWith beside its finish, and Stream.Writer.finish hands the buffered tail to it instead of flushing first -- so a request goes out as one STREAM frame with the FIN bit rather than as the frame and then an empty frame carrying only the bit. §19.8 of RFC 9000 permits both and every other implementation sends the first, which turned out to matter: Quad9's DNS-over-HTTP/3 server answers a request of that shape and silently ignores one of the other, acknowledging the packets either way. It was found by a DNS client built on this library, and the transport half of the fix is in zig-quic.

One thing to know about the session underneath — quic_io.Client.run wants a task of its own. A stream read blocks until a datagram arrives and the pump is what makes one arrive, so a caller doing both on one task waits for itself.

http3.client.Connection.run wants one too, for the streams the server opens: its control stream, which lasts as long as the connection, and its two QPACK streams. Responses arrive without it, on the streams send opened, but the server's GOAWAY arrives only through it — after one, send refuses with error.GoingAway rather than open a request the server has said it will not process — and so do the §6.2 and §7.2 rules a server can break on those streams, each of which closes the connection. http_io.Client runs it beside the pump.

The demonstration binary fetches over all four versions:

$ zig-http --fetch https://127.0.0.1:8443/ --fetch-version 3
200
content-type: text/plain; charset=utf-8

tests/nixos/h3client.nix points it at aioquic's HTTP/3 server, which is aioquic.nix turned around: that one puts aioquic's client in front of this server, and says what this library writes can be read by somebody else. What it can read is a separate claim, and the one the HTTP/2 client failed until nginx was asked.

The paths are chosen for where a client goes wrong rather than for coverage: a 204 with no body at all, where a client waiting for a DATA frame stops for ever; a response larger than one stream window; a second field section after the content, which §4.1 allows and which a client must consume rather than mistake for another response; and a field section too large for one packet, which is read whole or read wrong — a failure that depends on timing, so it is absent on a loopback socket and present here.

tests/nixos/traefik.nix asks two further things. Traefik's HTTP/3 is quic-go, a third independent QUIC implementation after nginx's own and aioquic's. And Traefik is a proxy, not an origin: it reads the request over HTTP/3, writes it again as HTTP/1.1 to static-web-server, reads that answer and writes it back — so every field in the response has been parsed and re-serialised by something in the middle, which is where a length, a coding or a connection-specific field goes wrong, and is the shape almost every real deployment has.

Nothing was found, which is worth saying plainly rather than claiming as vindication: this client has far less surface to be wrong on than the HTTP/2 one, because §4.6 gave the flow control and the stream bookkeeping to QUIC — and QUIC's own interop suite has been exercising that half all along.

What fuzzing the clients found #

The three clients are fuzzed against arbitrary server bytes — transport.memory for HTTP/1.1 and HTTP/2, and transport.memory_session for HTTP/3, which is the seam paying for itself again: an adversarial HTTP/3 response needs no QUIC handshake, no socket and no port. Three real faults in the first minutes:

  • A 204 that declares a body. §6.3 rule 1 outranks rule 3 — a 204 has no body "regardless of the header fields present" — so HTTP/1.1 204 with a Content-Length, a Transfer-Encoding and five octets is correctly framed as bodiless. But those five octets are still on the wire, and the next request on that connection would be answered by them. The connection is no longer offered for reuse when a bodiless response declared framing.
  • An HTTP/2 stream that ends with no field section handed back the zero its status was initialised to — a response that never arrived. §8.1 requires a message to have a HEADERS frame, so this is now MalformedResponse.
  • A leak in the HTTP/3 client, where the arena had an errdefer and the body list did not, so any error after a DATA frame arrived lost it.

Seeds with interim responses in them found one more, in both multiplexed clients: a :status of 013. Each checked for three digits and not for a first digit of at least 1, so a status with no class at all was handed back. It is MalformedResponse now.

A fourth finding was the property's fault rather than the code's, and is worth recording for that reason: it asserted a status of 100–599, and the fuzzer produced HTTP/1.1 804 within seconds. §15 of RFC 9110 defines five classes and this library deliberately carries a code outside them rather than refusing it — so the property was wrong, and now checks that such a code reports class unknown instead.

What is not there yet: a connection pool and cookies. Each is a decision rather than an omission, and the seam they would be built on is the one above.