- Rust 99%
- Nix <1%
- Shell <1%
yumak (ball of yarn in turkish) is a tangled knot with no daemon on the read path. yumak only runs when something changes, compiling your git repos and atproto records into immutable files that nginx serves straight from disk.
- clones and fetches work over smart http and ssh. nginx matches the request
body against prebuilt replies, including
depth-1shallow clones (deeper--depthrequests get the full history). - pushes go over ssh as normal, or over http when
[writes]is enabled (git push -oand git lfs are not supported). - issues, pull requests, comments, and attachments work like on a normal knot
when
[writes]is on, each write updates the repo and activates a new generation.
git.klbr.net runs yumak and hosts this repo
(did:plc:amxsvho4opuwe53msnkbwa5j, on tangled at
tangled.org/ptr.pet/yumak).
performance #
hosting the linux kernel (torvalds/linux: 12.0M objects in one 6.5 GB pack,
953 refs) on git.klbr.net itself: 4 vcpus of amd epyc genoa, 7.9 GB of
memory, btrfs. measured on 2026-11-06 with yumak limited to 4 GB without
swap, at nice 19 and a low cpu and io weight, next to the box's other services
(a pds, knot2, navidrome, metrics, logs, bun server, etc.).
| what | time | memory |
|---|---|---|
| push of one commit | 1.9 s | 0.03 GB |
| push of 6 commits | 1.9-2.0 s | 0.03 GB |
| cold generate | 31-32 s | 0.7 GB |
| nightly repack and bitmap, built on the last one | 1.4-1.6 min | 0.8 GB |
| first night, the bitmap built from nothing | 3.9-4.1 min | 1.3 GB |
note the memory here is peak anon memory, rest of the memory usage depends on how much goes through the page cache, for a big repo like the kernel it will matter more.
installation #
there is a nix flake:
nix profile add git+https://git.klbr.net/did:plc:amxsvho4opuwe53msnkbwa5j
or build it with cargo (cargo build --release). the host needs git, an
nginx built with pcre jit, and fcgiwrap. for best performance (and disk
usage on big repos), put <output.root> on a filesystem that supports reflinks
(btrfs, xfs, or apfs).
configuration #
copy knot-static.toml.example to
knot-static.toml and fill it in. --config or KNOT_STATIC_CONFIG points
at another path. the master key that seals sealed_key_file comes from
KNOT_MASTER_KEY (base64 of 32 bytes), or from the variable
secrets.master_key_env names.
then:
- run
knot-static initonce. it creates the meta repo and seals the knot's signing key. - run
knot-static generate, thenknot-static activate <generation>, which writes<output.root>/active.confand reloads nginx. - run nginx against
<output.root>/active.conf, behind a tls reverse proxy withclient_max_body_size 200m. - run
fcgiwrap -f -c 4 -s unix:/run/knot-static/fcgiwrap.sockwithKNOT_STATIC_CONFIGandKNOT_MASTER_KEYin its environment. one worker would let a slow push hold up every write.
note: fcgiwrap is needed even with [writes] off.
ssh #
point your ssh user's authorized_keys at a wrapper that exports
KNOT_MASTER_KEY and runs
knot-static --config /etc/knot-static/config.toml git-ssh "$@". a key you add
there yourself is the operator's and can push anywhere. git-ssh refuses to
run without --operator or --actor, so a key like that says it:
command="/etc/knot-static/git-ssh --operator",restrict ssh-ed25519 AAAA...
fetches over ssh get the same canned replies nginx serves, and those need git
protocol v2. git 2.26+ and gix ask for it in GIT_PROTOCOL, which sshd only
passes on with AcceptEnv GIT_PROTOCOL.
for collaborators, sshd asks knot-static keys list for their keys. keys sync reads them into <output.root>/ssh-keys.json, and a push or a grant
reads accounts it hasn't seen yet. every key is forced to run --command,
with --actor DID added:
Match User op
AuthorizedKeysCommand /etc/knot-static/keys-list
AuthorizedKeysCommandUser op
/etc/knot-static/keys-list is a script that runs
knot-static --config /etc/knot-static/config.toml keys list --command /etc/knot-static/git-ssh.
sshd refuses the command when anyone but root can write to it or a directory
above it, which rules out /nix/store. on nixos, copy it out with
environment.etc.<name>.mode.
usage #
knot-static init create the meta repo and knot signing key
knot-static generate build a new generation (--repo to rebuild one)
knot-static activate switch active.conf to a generation and reload
knot-static gc remove old generations and unreferenced data
knot-static maintenance repack, bitmap, and prune hosted repos
knot-static repo create, import, rename, and configure repos
knot-static issue / pull / label manage issues, pull requests, and labels
knot-static oauth log in to a pds to publish repo declarations
knot-static keys sync collaborator ssh keys for sshd
knot-static plc retry pending plc genesis operations
knot-static git-ssh / cgi sshd ForceCommand and fcgiwrap entry points
run knot-static <command> --help for flags on any subcommand.
tests #
the dev shell has cargo-nextest:
cargo nextest run # the default set
NGINX_BINARY=$(command -v nginx) cargo nextest run --run-ignored all
the second adds the tests that need a real nginx (cargo test -- --include-ignored
without nextest). .config/nextest.toml turns retries off, and a flaky test
fails the run.
how it works #
- smart http git in nginx (
src/nginx/):git-upload-packis aPOST, which nginx refuses to serve static files for, and nginx only reads a request body when proxying. yumak makes nginx proxy thePOSTback to itself as aGETand uses regexmaptables on the request body to pick the right prebuilt pack (ls-refs, full clone, incremental fetch, ordeepen 1). if the client gzips its request body,fcgiwrapdecompresses it and hands the chosen file back to nginx to stream. - spare filler slots in big packs
(
src/git_export/spare.rs,src/git_export/hasher.rs): a git packfile has an object count in its header and a sha1/sha256 checksum at the end, so appending a push would normally mean rehashing a multi-gigabyte file. instead, big replies reserve extra object slots at the tail filled with tiny dummy blobs that no branch points to, and save the hash state right before them. a push clones the previous reply file with a reflink (src/git_export/reply_pack.rs; falls back to a copy on filesystems likeext4), overwrites the next free slots in place, and finishes the hash over the last few kilobytes. - depth-1 clones by slicing the full pack
(
src/git_export/depth_one.rs): instead of runninggit pack-objectsfor shallow clones, yumak indexes where each entry lives in the full pack and slices out just the tip's objects, patching up any delta headers whose base wasn't in the slice. - custom bitmap engine (
src/bitmap/): issues and pull requests live in the same git repo as the code (refs/cobs/*), so yumak has to keep them out of public clones. it has its own multi-pack-index bitmap reader and parallel writer that streams object types from pack headers in disk order and diffs root trees as raw bytes before parsing them. - static xrpc tables (
src/xrpc/,src/atproto/): every read xrpc response and pagination page is prebuilt into a content-addressed pool, and nginxmaptables route query params straight to the file on disk. no_stdfirehose daemon (firehose/,src/firehose.rs): activating a generation appends pre-framed websocket messages to a flat file, and a tiny zero-allocationno_stddaemon streams it to subscribers withsendfile.
credits #
this was made possible thanks to LLMs! thanks to opus / sonnet 5.5, gemini 4 argon, gemini 3.8 flash, gpt 6.1 sol!
thanks to tangled and specifically @oyster.cafe for awesome knot2 :3
the yarn-ball favicon is drawn fresh after twemoji's 🧶 (original by twitter, inc. and contributors under cc-by 4.0).