about things notes.zzstoatzz.io
notes systems
1 folder · 14 files

README.md

systems #

design patterns that compose the concrete pieces documented elsewhere — queues, delivery semantics, cursors, batch/serving splits. the layer above any specific database, language, or protocol.

the boundary against its neighbours:

  • storage/ and atproto/ — concrete systems (redis, sqlite, turso, turbopuffer, the AT Protocol)
  • data-structures/ — implementation tradeoffs in reusable structures
  • languages/ — language idioms (python, typescript, zig)
  • interfaces/ — what people see and touch
  • operations/ — running and shipping what you built
  • systems/ — the patterns that compose those things

if a note is mostly about what to do when something is already deployed and misbehaving, it belongs in operations/.

topics #

  • socket lifetimes — shutdown before close, partial writers, descriptor reuse and timeout contracts
  • background-tasks — the celery→sidekiq→arq→docket lineage, queues that remember deliveries, perpetuals, migrating workers (streams mechanics live in storage/redis/streams)
  • at-least-once-forwarder — outbox, ack-after-durable-commit, and a cursor pinned to min(inflight); why ack-on-receipt plus shedding is silent loss
  • leases and expiry — a holder that dies keeps what it holds; leases with server-owned expiry, and counts that stay tied to one owner, one release, and one store
  • serving-from-snapshots — build offline, verify against a manifest, adopt atomically; the overlay is authoritative, and batch cadence sets the freshness floor
  • change-timestamps — "when we last wrote" and "when the content last changed" are different columns; pinning the wrong one turns a correctness gate into a measurement of write traffic
  • watermark-semantics — one cursor, one meaning; the cost of a gate scales with the quantum of the thing it waits on
  • publication across two stores — intent before bytes, one atomic publish, abandon-commit-before-delete, tombstones re-swept; the same shape as a CoW B+ tree's alternating superblocks and free list
  • reusing prior art — separate the problem, the unstated constraints, and the portable primitives; eBird checklists against a single-sighting record
  • policy-enforced-by-accident — a rule enforced by a dependency's implementation detail has no owner; enforce at a layer you own, and test the policy rather than the mechanism
  • cache-lifetimes — a cache is bounded by the keys a process can observe, not its working set; derived state lives with its owner
  • homa — a datacenter RPC transport: message-based, receiver-driven SRPT scheduling, and why TCP is the wrong shape for RPC
  • write batching — a maximum batch size is a limit, not the size a pipeline achieves; producers and the work around each write set it

testing #

  • stateful-property-testing — op histories against an oracle; small key pools, forked exploration, catch rates from planted bugs, refused operations must change nothing, and why a quiet run proves little
  • deterministic-simulation-testing — a whole service in one process with the generator owning time, faults, delivery, and interleavings; the original implementation as oracle, and what a simulator cannot see