Something went wrong. Try again.
fork of hydrant
Something went wrong. Try again.
# Spec: db layer restructuring
Status: draft
Tracking: beads epic (see `bd show`, filed alongside this spec)
## Motivation
The db layer works, but its correctness rests on conventions that every
callsite must independently remember. Concretely:
- `Db` (`src/db/mod.rs:48`) exposes every keyspace as a public `Keyspace`
field plus `pub inner: Arc<Database>`. Any module can write raw bytes to
any keyspace with hand-built keys; `db.inner.batch()` appears in 27+ files.
- Four access styles coexist: half-dead static async helpers (`Db::get`,
with `insert`/`remove`/`contains_key` marked `#[allow(dead_code)]`),
direct sync calls inside `spawn_blocking` closures, direct sync calls on
dedicated blocking threads, and free functions taking `(batch, db, ...)`.
- Key formats live in free functions returning `Vec<u8>` (`src/db/keys/`)
with prefix constants scattered per module — `db/filter.rs` maintains its
own `SEP`/prefix constants separate from `keys::SEP`. The authoritative
documentation of key formats is comments (duplicated again in AGENTS.md).
- Serialization helpers exist (`ser_repo_state` etc.) but ~31 files call
`rmp_serde` directly, several deserializing `RepoState` raw and bypassing
the helpers (`backfill/worker/process.rs:58`, `backfill/manager.rs:122`,
`backfill/worker/task.rs:187`).
None of these are hypothetical risks:
1. **The record-import invariant is duplicated three times.** A record write
must keep in sync: `records` ks, `blocks` ks, per-collection counts,
backlinks (cfg), a `StoredEvent`, and a jetstream event (cfg). This is
implemented once properly in `ops::apply_commit` (via `RecordEmitter`),
then re-implemented by hand in `backfill/sparse.rs` (`persist_sparse_backfill`
builds `StoredEvent` + event key + `jetstream::stage_event` manually, and
carries its own cfg'd backlinks block) and twice more in
`backfill/worker/process.rs`. Divergence between these is silent index
drift — missing events, missing backlinks, wrong counts.
2. **Three manual registries…