zttp tests conformance README.md
3.9 kB · 63 lines
Markdown
at main

HTTP client qualification #

Run just conformance-tools, then just conformance and just conformance --optimize ReleaseSafe. Docker, uv, and just are required. Tool preparation downloads pinned public dependencies. Test execution has no external network: every origin is a bounded loopback fixture inside the container.

The gate also runs 61 outgoing TLS shutdown checks: default/explicit shutdown, retirement, pool eviction, zero pool capacity, active requests, competing closers, cancellation, expired budgets, blocked writes, and concurrency exhaustion. Healthy peers must observe close_notify; forced aborts require their exact error. Tiny write backpressure is established independently of TLS framing.

The gate runs formatting, the library tests, real TLS certificate rotation and hostname checks, deadline/cancellation contracts, and raw HTTP/1 response vectors. Runtime checks cover Threaded, zio with one and four executors, and zio from native callers. They exercise stalled headers and bodies, TLS handshake stalls, backpressured uploads, external cancellation with guards enabled and disabled, absolute deadlines across caller delays, unread-response cleanup, reused progress, concurrency exhaustion, and allocation failure cleanup.

corpus.json contains 50 projected framing vectors and 320 local regression vectors. The upstream projection comes from httnope, revision f99fd64a19acd20f53dedd51ab5ced524671482a (corpus version 2026-09-02.2), and additional RFC 9110/9112 regression vectors. Projected cases retain their upstream source_id; local cases have a null source ID. The upstream ISC license is included. This is a selected response-client profile, not the full httnope suite.

The TLS scheduling fixture timestamps receipt of ClientHello and measures the heartbeat during the subsequent silent network wait. This separates synchronous cryptographic setup from I/O scheduling while retaining the 100 ms heartbeat bound. A separate deadline covers the entire TLS attempt.

The wire tests cover framing ambiguity, malformed fields, chunk extensions and trailers, informational/bodyless responses, fragmented delivery, decompression, resource limits, TLS truncation, and connection reuse after failure. Valid responses must produce the exact status and decoded bytes. Invalid responses must produce an explicitly allowed protocol error; crashes, generic I/O errors, harness timeouts, and fixture failures cannot count as successful rejection. Stateful cases issue a second request through the same client and verify whether the connection was reused or replaced. No failures are allowlisted.

The driver prints the source SHA-256 and immutable tool image ID. Full reports include corpus/probe hashes and per-case observations inside the disposable scratch filesystem. A nonzero exit fails the gate. Source and tools are read-only; only bounded scratch space is writable. Credentials, host home, Docker socket, and external networking are unavailable. CPU, memory, processes, files, and phase wall time are bounded.

To extend coverage, add a case with its source and expected RFC behavior, verify that it fails the affected revision, and run both optimization modes. Retain valid controls alongside rejection cases. Refresh the pinned upstream projection by reviewing the upstream wire bytes and expectations; do not silently reinterpret unexpected results as acceptable. Runtime dependency updates require changing the pin in both build.zig.zon here and install_tools.py.

This profile does not qualify HTTP/2, proxies, browser cookie/cache semantics, every typed response field, or every TLS implementation behavior. Passing it is a reproducible regression baseline, not a claim that the library has no HTTP vulnerabilities. Deadlines require cooperative I/O; synchronous CPU work and a custom sink that ignores cancellation cannot be forcibly preempted.