Interfaces #

The internal contract between the build step, the serving core, the shells, and the verifier. This file is normative for implementation: modules may only assume what is written here about each other. It describes internals and is not part of the public API surface documented in ARCHITECTURE.

Language and dependency rules #

• Plain ESM JavaScript, no compile step. Types are JSDoc, checked with npm run typecheck.
• src/serve/** and src/worker/** import nothing — no npm packages, no node: builtins. Web-standard globals only (Request, Response, URL, TextEncoder, atob, JSON).
• src/node/** may import node: builtins only.
• src/build/** and src/verify/** may import anything; they run offline in Node.
• src/message.js belongs to no subsystem: it imports nothing, and src/build/**, src/verify/**, and bin/** may import it without depending on each other. It exists because catch (err) binds unknown everywhere and verify states in its own header that it does not import the build.
• src/dispatch.js belongs to no subsystem either: it is the CLI's argv router, imports only src/message.js, and reaches every subcommand through import() at routing time. It must never gain a static import of a subcommand — see CLI.
• Tests use node:test and node:assert/strict.

Validating lexicon source #

Two independent validators exist and they do not agree. Run both.

lexiconDocumentSchema.parse() from @atproto/lex-document (a transitive dependency, already installed) is the official TypeScript parser and the one the reference stack actually uses. It is the right in-process gate for the build.

It is not sufficient on its own. Its permission schema is {type, resource} intersected with an open dictionary of arbitrary keys to scalars-or-arrays, so it accepts any field name with any scalar shape inside a permission. goat lex lint has concrete Go structs instead and rejects things the TypeScript parser waves through — verified: "aud": ["*"] parses fine under lexiconDocumentSchema and fails under goat with cannot unmarshal array into ... aud of type string. goat is right; aud is a single string.

This matters most exactly where the stakes are highest. Permission sets drive the OAuth consent screen, and they are the document type the generic parser checks least. Bluesky's own permission sets (app.bsky.authFullApp, chat.bsky.authFullChatClient) are the reference for correct shape: lxm is an array of NSIDs, and a permission set uses "inheritAud": true rather than naming an audience, because the set does not know the audience — the requesting client supplies it.

So: corpora build gates on @atproto/lex-document (in-process, no external binary), and the publish loop in PUBLISHING keeps goat lex lint as a second opinion. Neither subsumes the other.

dist/ layout #

corpora build emits exactly this tree. Every shell consumes it; nothing else is contractual.

dist/
  did.json                  # DID document
  repo.car                  # full repo CAR bytes
  commit.json               # sync.getLatestCommit response body
  describeRepo.json         # repo.describeRepo response body
  records/<nsid>.json       # repo.getRecord response body, one per lexicon
  proofs/<nsid>.car         # sync.getRecord response bytes, one per lexicon
  list/page-1.json          # repo.listRecords response body, page 1 (page-N for later pages)
  manifest.json             # build metadata
  data.js                   # generated ESM module embedding all of the above

<nsid> is the literal NSID (e.g. com.example.thing.json). NSIDs are already filename-safe.

dist/data.js — the embedded bundle #

A generated, dependency-free ESM module with one default export. Byte payloads are embedded as base64 strings and decoded at module load by an inlined helper using atob, which exists in both Node >= 18 and workerd. The decoded values are Uint8Array.

/** @type {import('../src/serve/types.js').CorporaData} */
export default {
  did: 'did:web:lexicon.example.com',
  rev: '3lxyz...',                       // commit rev (TID)
  commitCid: 'bafyrei...',               // commit CID, string
  collection: 'com.atproto.lexicon.schema',
  didDoc: { /* did.json, parsed */ },
  commit: { cid: 'bafyrei...', rev: '3lxyz...' },
  describeRepo: { /* describeRepo.json, parsed */ },
  order: ['com.example.thing'],          // rkeys in DESCENDING order — see below
  records: { 'com.example.thing': { uri, cid, value } },
  proofs: { 'com.example.thing': Uint8Array },
  repoCar: Uint8Array,
  manifest: { /* manifest.json, parsed */ },
}

manifest.json carries build metadata, not response bodies:

{
  "corporaVersion": "0.1.0",
  "did": "did:web:lexicon.example.com",
  "collection": "com.atproto.lexicon.schema",
  "rev": "3lxyz...",
  "commitCid": "bafyrei...",
  "mstRoot": "bafyrei...",
  "signingKey": "did:key:zQ3sh...",
  "didDocHash": "sha256-...",
  "nsids": ["com.example.thing"],
  "authorityDomains": { "_lexicon.example.com": ["com.example.thing"] },
  "sources": { "com.example.thing": "sha256-..." },
  "recordCids": { "com.example.thing": "bafyrei..." },
  "totalBytes": 12345
}

didDocHash is the sha256 of the DID document as served — JSON.stringify with no whitespace and no trailing newline, the bytes /.well-known/did.json returns — not of the pretty-printed dist/did.json file. It is what corpora verify --expect-did-doc <hash> compares against, and it exists to be published away from the serving host; see BUILD. No serving shell reads it.

manifest.json must be byte-stable for identical inputs and key — no timestamps, no absolute paths, keys sorted. Reproducibility is an M1 acceptance criterion.

order is a contract between build and serve #

A reference PDS returns listRecords in descending rkey order by default — the opposite of MST order. Verified live against jarrett.app on 2026-08-03: 25 records, io.atcr.tag first and io.atcr.authFullApp last.

So corpora build must emit order — and list/page-N.json — in descending rkey order. The handler treats that sequence as canonical reverse=false order and never sorts. Emit it ascending and Corpora silently disagrees with a real PDS over the same repo, which is exactly the class of near-miss this project is supposed to eliminate.

order holds rkeys, not records. The bundle carries each record body exactly once, under records, and the handler pages over order per request rather than serving a prebuilt page shape. list/page-N.json still exists in the static tree for readers walking dist/ directly; the bundle does not embed it, because doing so meant a second copy of every record — about 27% of data.js and 21% of the heap a Worker holds. test/build/build.test.js guards the invariant structurally and test/worker/bundle-size.test.js guards it by size.

reverse=true is served by reading order from the other end, not from a mirrored copy: a record at index i is at length - 1 - i reversed, and cursors translate through the same arithmetic.

corpora/serve #

/**
 * @param {CorporaData} data
 * @param {{ maxAge?: number, didDocMaxAge?: number, landing?: boolean }} [opts]
 * @returns {(req: Request) => Response}
 */
export function createHandler(data, opts)

Synchronous, no I/O, no async. Routing table and response shapes are specified in RESOLUTION. Behavioural requirements:

• GET and HEAD only; anything else is 405 with an XRPC error body.

• Unknown paths and unknown NSIDs are 404 XRPC errors: {"error":"...","message":"..."}.

• Query parameters are validated: repo/did must equal data.did, collection must equal data.collection, rkey must be a known NSID. Wrong values produce the same error names and status codes a reference PDS returns (RepoNotFound, RecordNotFound, InvalidRequest) — verified live against jarrett.app on 2026-08-03. Note the asymmetry, which is real rather than an oversight: sync.* and describeRepo declare RepoNotFound in their lexicons, while repo.getRecord and repo.listRecords declare no repo error and surface the same condition as InvalidRequest. An unknown rkey is 400 RecordNotFound, matching the reference, not 404. A wrong collection on listRecords returns {"records":[]}, again matching the reference.

  One divergence is deliberate. A real PDS answers sync.getRecord for an absent rkey with 200 and a non-existence proof CAR — verified: 2,259 bytes, comparable to an inclusion proof. Corpora returns 404 RecordNotFound, a declared error of the same lexicon.

  Note that precomputing these is not actually impossible, as one might assume. Absent keys are unbounded, but the exclusion proofs they map to are not: an exclusion proof is the MST path to the gap where the key would sit, and a tree over N records has O(N) such gaps. You could emit one per gap and route any unknown rkey to the right one.

  It is not worth it. Nothing on the resolution path asks for an NSID it does not expect to exist; a resolver meeting an unpublished lexicon needs to fail, and 404 fails it. Building a gap-routing table to return a cryptographically precise "no" to a question nobody asks would add the most intricate code in the project to serve a case with no consumer. Revisit only if a real client turns out to distinguish the two.

• CAR responses use content-type: application/vnd.ipld.car.

• JSON responses use content-type: application/json; charset=utf-8.

• / is an HTML landing page for humans, text/html; charset=utf-8, rendered from data once at handler construction — never per request, and never from anything the request supplies. It is the only response that is composed rather than replayed, so everything it interpolates is HTML-escaped and every link it prints is followed by a test. { landing: false } drops the route back to the standard 404. /favicon.svg and /favicon.ico return the same SVG bytes with image/svg+xml; a browser asking for .ico unprompted gets a working icon rather than a 404, since renderers dispatch on content type, not extension.

• did.json is served with cache-control: public, max-age=300, as is /, whose content follows the deploy; everything else public, max-age=86400, immutable.

• No redirects, ever, on any path.

• CORS: access-control-allow-origin: * on every response, plus an OPTIONS preflight answer.

corpora/node #

/** @param {(req: Request) => Response} handler @returns {import('node:http').RequestListener} */
export function createRequestListener(handler)

/** @param {CorporaData} data @param {{port?: number, host?: string}} [opts] */
export function serve(data, opts)   // → Promise<{ url, port, server, close() }>

corpora/worker #

export function createWorker(data)  // → { fetch(req): Response }

corpora/verify #

Verifies an authority independently of the code that built it — by fetching over HTTP, never by reading build internals.

/**
 * @param {{ origin: string, did?: string, sourceDir?: string, expectDidDoc?: string }} opts
 * @returns {Promise<VerifyReport>}
 */
export async function verifyAuthority(opts)

VerifyReport is { ok: boolean, checks: Array<{name, ok, detail}>, nsids: string[] }.

src/verify/index.js is the only file package.json maps to corpora/verify; it also exports verifyProof, authorityDomain, and authorityDomains. Alongside it, src/verify/format.js holds the verifier's pure formatting and comparison helpers — canonical, byKey, short, bytes, equalBytes, rkeyOf, sameOrigin, describeType, errCode — and is internal: not in exports, imported only by src/verify/**, and importing nothing but src/message.js. The split is what lets those functions be tested as units without the package's public surface growing to accommodate a test. A helper earns a place there by being a total function of its arguments; anything that fetches, reads the filesystem, resolves DNS, verifies a signature, or touches report state stays in index.js.

CLI #

bin/corpora.js is a shebang and one line: process.exit(await dispatch(process.argv.slice(2))). The routing lives in src/dispatch.js, which is importable and therefore testable.

/**
 * @param {string[]} argv                    process.argv.slice(2)
 * @param {CommandTable} [commands]          defaults to the real four
 * @returns {Promise<number>}                the exit code; dispatch never exits the process
 */
export async function dispatch(argv, commands)

Each subcommand lives in its owning module and exports run(argv) returning an exit code. A run that resolves to nothing is dispatched as 0. The table's values are () => import(...) thunks and exactly one is called per invocation: corpora verify must not load the build.

corpora build   --lexicons <dir> --out <dir> --did <did> [--key-file <f>|CORPORA_SIGNING_KEY]
                [--service <url>] [--dns-check strict|cutover|off] [--no-dns-check]
                [--rev <tid|time>] [--against <origin>] [--force] [--quiet]
corpora verify  --origin <url> [--did <did>] [--source <dir>] [--json]
corpora serve   --dist <dir> [--port 2583] [--host 127.0.0.1]
corpora keygen  [--out <file>]

Local end-to-end #

The whole point is being able to run the real resolution chain against localhost. See LOCAL_E2E, owned by the verify/e2e work.