A self-hostable npm metadata gateway. It answers dev.nibling.npm.getClosure: every abbreviated packument that an install of some specs reaches, streamed as NDJSON while it walks the graph. A client that resolves its own tree, such as the nibling kernel, then needs one request instead of one per package, and starts resolving on the first lines.

Anyone can run one. nibling's worker is one gateway among any, and the kernel installs without one.

Run #

npx nib-npm-gateway --port 8787                 # Node 24+
npx nib-npm-gateway --registry https://npm.example.com --did-web

On other runtimes, pass the handler to the server:

import { createGateway } from "@nibling/npm-gateway";

const gateway = createGateway({ userAgent: "my-gateway (+mailto:ops@example.com)" });
Deno.serve(gateway.fetch);              // Deno
export default { fetch: gateway.fetch }; // Cloudflare Workers, Bun

createGateway options (nib-npm-gateway takes --budget too):

registry: the upstream registry. Default https://registry.npmjs.org.
fetch: every upstream request goes through it.
cache: { get(key), put(key, value, ttlSeconds) }. The default is in memory. Packuments and closures are kept for 300 seconds, the registry's own max-age. Version documents are kept for a year.
userAgent: sent upstream, so a registry can reach you.
budget: the most upstream documents one walk loads. Default 900, under a Worker's 1000 subrequests. A gateway with no such limit can raise it.
didWeb: serve /.well-known/did.json naming this host as a #nibling_npm service of type NiblingNpmGateway.

Use from a client #

A client names its gateways when it is built, as a repo names its knot: by DID (did:web:npm.example.com, did:plc:...), by hostname (read as its did:web), or by endpoint URL. At boot it resolves the names to their #nibling_npm endpoints, so a gateway's operator can move one without a client release:

import { resolveGateways } from "@nibling/npm-gateway/resolve";

const kernel = await boot({ npm: { gateways: await resolveGateways(["api.nibling.dev"]) } });

The kernel asks each gateway in order. It gives each 5 seconds to send its first line, and stops reading one that goes quiet for 5 seconds. It installs as usual when none answers. The page's CSP must allow the names' DID documents and their endpoints: pass await gatewaySources(names) as contentSecurityPolicy({ npmGateways }). A process can override the list with NIB_NPM_GATEWAYS (space-separated endpoints, or empty for none). examples/slopbox bakes the names in at build time from NIB_NPM_GATEWAYS.

Protocol #

The lexicon is packages/lexicons/lexicons/dev/nibling/npm/getClosure.json.

GET /xrpc/dev.nibling.npm.getClosure?spec=<name>@<spec>&...[&alias=<pkg>:<substitute>][&binding=<suffix>]

spec: 1 to 50 root specs, as a dependent wrote them.
alias: when the walk picks <pkg>@<v>, it also walks <substitute>@<v>.
binding: when a picked version lists two or more <prefix>-<platform> napi-rs bindings at one exact version, and no <prefix>-<suffix>, the walk also walks that binding.

The walk follows dependencies, optional dependencies, and peers. An optional peer is not installed, but a resolver reads its packument and its peers' packuments, so they come along without their dependencies. The response is application/x-ndjson, one object per line, in the order the walk finds them:

{"t":"closure","v":1,"registry":"https://registry.npmjs.org"}
{"t":"spec","name":"express","raw":"^5","distTags":{"latest":"5.1.0"},"versions":{"5.0.0":{},"5.1.0":{}}}
{"t":"spec","name":"debug","raw":"^4.4.0","distTags":{"latest":"4.4.3"},"versions":{"4.4.0":{},"4.4.3":{}}}
{"t":"meta","id":"express@5.1.0","license":"MIT","funding":{"type":"opencollective","url":"..."}}
{"t":"absent","name":"esbuild-wasm32-wasi"}
{"t":"end"}

spec: with the versions this name's earlier lines sent, versions now holds the abbreviated packument's entry for every version raw accepts, plus every dist-tag's version. A name's first line carries its distTags. npm's version picking reads only a range's own versions and the default tag's, so for that spec the cut copy picks what the whole packument would. Any other spec must be fetched from the registry.
meta: the full version document's fields that the abbreviated packument leaves out, for each version the walk picked.
absent: a name the registry answered 404 for. missing: a name the gateway could not load, which the client fetches itself.
wait: the gateway is still walking while its upstream is slow. It comes at least every 2 seconds while no other line does, so a client can tell a slow registry from a gateway that is gone. A cached replay has none.
end: the walk finished. A stream without it was cut short, and only what arrived holds.

The walk has no levels. It follows each package's dependencies as soon as that package loads, so one slow packument holds back only its own subtree.

One walk loads at most 900 upstream documents, packuments and version documents together (budget); a name past that is missing. A finished stream is cached for 300 seconds and replayed whole.

Trust #

A client should trust a gateway the way it trusts a registry mirror, for freshness and nothing more. The kernel:

ignores a closure whose registry is not the one it installs from;
ignores a packument whose tarball URLs are not under <registry>/<name>/-/, or whose version documents name another package or version;
checks every tarball against its integrity, as it always does.

A dishonest gateway can still hide a release. It cannot pick versions for the client or swap code.