zat
atproto utils for zig zat.dev
atproto sdk zig
Star 37
Fork 3
Watch

add STAR v1 streaming reads and public repository profiles #31

Merged zzstoatzz.io
targeting main from star-lite
Merged opened by and assigned to zzstoatzz.io
targeting main from star-lite

adds streaming reads for the current STAR v1 draft, with a separate ATProto public-repository profile. generic archives carry canonical metadata and arbitrary byte pairs; repository consumers additionally get path, ordering, commit, and MST verification.

var reader = try zat.star.PublicRepoReader.init(allocator, input, .{});
defer reader.deinit();
try reader.verifyCommit(signing_key, expected_did);
while (try reader.next()) |record| {
    // accumulate provisional results; slices borrow the reader
}
// EOF confirms the reconstructed root; publish results now
format and API boundaries

implements STAR v1 and its public-repository profiles at 512d0be. this is a pinned draft, not a claim of a finalized standard. there is no v0 compatibility path.

  • star.Reader: four-byte v1 magic, canonical DRISL metadata with a string $type, and minimally encoded length prefixes. next() returns borrowed key bytes and a value length; readValue(buffer) streams the value without allocating its declared size. empty, unordered, duplicate, and non-UTF-8 keys are valid at this layer. configurable limits can be stricter than the format's ceilings.
  • star.PublicRepoReader: supports atproto-public-repo and atproto-public-repo-subtree, enforces record paths and order, and buffers at most one 1 MiB record. supplied metadata is validated even without authentication. normal EOF is returned only after the body reproduces the metadata's MST root.
  • commit authentication excludes the profile's added $type before verifying the signature. callers supply the account's resolved key and expected DID. unsigned subtrees cannot authenticate; neither profile proves freshness.
  • mst.SortedBuilder: reusable sorted key/CID reconstruction, sharing canonical node encoding with mutable MSTs. CAR and sparse firehose diff APIs retain their roles.
validation and failure behavior

582 native tests pass; full build, formatting, and 32-bit ARM test compilation pass. the executable verifies the checked-in 300-record v1 fixture. its MST root is generated independently by Microcosm's Rust implementation; the accompanying generator writes v1 framing directly because the upstream Rust archive writer still implements the older framing at this revision.

regressions cover duplicate-signature metadata, malformed metadata without authentication, canonical encoding, nested CID tags, DRISL negative zero and CID constraints, both signature curves, incorrect keys and DIDs, invalid revisions, profile selection, record ordering, corruption, truncation at a complete pair boundary, and allocation failures. a 64 GiB declared value is handled with an 8 KiB fixed allocator; attempting to read its absent body fails without allocating a value-sized buffer.

parse failures remain fatal for that reader. attempting to advance before consuming a value is a recoverable caller error. all streamed records remain provisional until final root verification.

Commits 0