# walknot
<sub>disclaimer: slop</sub>

A self-hostable [Tangled](https://tangled.org) knot whose storage is an S3 bucket.

walknot is [knot2](https://tangled.org/tangled.org/core/tree/master/knot2) — Tangled's git
+ atproto knot — with its storage rebuilt on the design of
[walgit](https://github.com/tobi/walgit): **every repository is a write-ahead log in an
object store, and every pod's disk is a disposable cache.** Run one replica or many
(Kubernetes, with Redis); they are interchangeable, and losing any of them — or all their
disks — loses nothing.

It speaks exactly knot2's protocol (XRPC, smart HTTP and SSH git, `subscribeRepos`,
records and COBs), so the Tangled appview, knotmirror, hydrant and spindle work with it
unchanged. Without a bucket configured it *is* knot2.

## How it works

- **Repositories are WALs.** `repos/<shard>/manifest.json` is rewritten with
  compare-and-swap (`If-Match`) on every commit — the only consensus there is. Ref edits are
  immutable log entries; objects are content-addressed packs; checkpoints fold the log.
  Lost responses are resolved by re-reading, so every acknowledged write is durable exactly
  once. → `crates/knot-wal`, `crates/knot-store`
- **Local directories are caches.** Opening a repository revalidates it with one
  conditional GET and folds in what other replicas committed; writes go through the WAL
  first and are applied locally after. → `crates/knot-git/src/storage.rs`
- **One writer per repository at a time**, by a lease that moves to whichever replica needs
  to write (re-entrant, renewed while used, released when idle). The CAS still guards
  correctness if a lease is lost.
- **Knot-wide state** (members, registry, outbox, sidecars, LFS, sealed keys, SSH host key)
  lives in the bucket too; a leader lease elects the replica that runs singleton jobs.
- **Redis** (only with several replicas) sequences the firehose knot-wide and shares
  single-use token nonces. It holds nothing durable.

`docs/DESIGN.md` has the details; `UPSTREAM.md` how this tree maps to Tangled's.

## Running it

```toml
# config.toml — the knot2 settings you already know, plus:
[storage]
backend = "s3"
s3_bucket = "my-knot"
# s3_endpoint = "https://minio.example.com"   # any S3-compatible store with conditional writes

[redis]
url = "redis://redis:6379"                     # required for more than one replica
```

- Credentials come from the usual `AWS_*` environment (or the pod's cloud identity).
- `KNOT_MASTER_KEY` must be the same on every replica: it seals the knot's identity, which
  lives in the bucket.
- At boot the knot verifies the store really enforces `If-None-Match`/`If-Match` and refuses
  to start otherwise.

**Kubernetes:** `deploy/kubernetes/base` (Deployment, Services, PodDisruptionBudget);
`deploy/kubernetes/overlays/dev` adds MinIO and Redis for a local cluster.

**Image:** `docker build -t walknot .` (`--build-arg PROFILE=container` for a faster build).

**Migrating an existing knot2:** stop it, set `storage.backend = "s3"`, then
`knot-server import --repos <old scan_path> --lfs <old lfs store_path>` publishes every
repository (the meta repo included) and LFS object. The old directory is copied, never
modified. Keep the same `KNOT_MASTER_KEY` and `secrets.sealed_key_file` / SSH host key
files for the first boot: they are sealed into the bucket then.

## Testing

```sh
cargo test --workspace                       # knot2's suite, local storage
KNOT_TEST_STORAGE=wal cargo test --workspace # the same suite, every repository in a bucket
ci/test-linux.sh                             # in a Linux container, as CI does
```

With real services (`docker run quay.io/minio/minio server /data`, `redis`):

```sh
export WALKNOT_TEST_S3_BUCKET=… WALKNOT_TEST_S3_ENDPOINT=http://127.0.0.1:9000 \
       AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=… WALKNOT_TEST_REDIS=redis://127.0.0.1:6379
cargo test -p knot-store -p knot-wal -p knot-redis    # contract, protocol, chaos on real S3
cargo test -p knot-server --test replicas_e2e         # three real replicas
ci/kind-smoke.sh                                      # 3 replicas on a kind cluster
```

What is covered: the store contract (CAS, create-if-absent, conditional reads) on MemoryStore
and MinIO; the WAL protocol under every injected fault, seeded chaos, racing writers and a
proptest model; leases; materialization checked by `git fsck`; maintenance and compaction
racing other writers; multi-replica behaviour in-process (pushes, races, firehose order,
tickets, ACL changes, token replay) and with real processes; and knot2's own suite in both
storage modes.

## Not yet

- Repositories larger than a pod's cache (walgit's remote range reader).
- Pruning unreachable *git objects* (repacks keep everything). Retired packs, superseded
  sidecars and orphaned log entries are garbage-collected.
- Firehose replay from a cursor scans every repository's chain (fine for small knots).
- Multi-pack index and bitmaps are not built for bucket-backed repositories.