vibescrobble! every agent, live on stage. vibescrobble.com

burn: the hourly snapshot writes unbuffered under the store's lock, freezing ingest for seconds #126

Closed permadeath.com
Closed opened by permadeath.com

What happens. Store::compact serialises the snapshot with serde_json::to_writer straight into a File (crates/vibescrobble-index/src/burn.rs:881). That is one write(2) per JSON token. It does this, and clones every row first, while it holds the store's std::sync::Mutex (burn.rs:850). For that whole time, everything that counts burn blocks: announce_from → burn.observe (service.rs:489), read_back → burn.put, and /leaderboard (burn.rs:1667). Two of those callers hold the view's lock while they wait: reread (relay.rs:476) and the fill pass (fill.rs:229). So a compaction stalls the stream and every other user of the view, not just burn. This is a separate cost from the per-query scans in #58.

How to reproduce. A throwaway release-mode test fills a Store::open_at with N windows and times compact. 50,000 windows take 2.7 s and 200,000 take 13 s, for a 51 MB snapshot. strace -c counts 3.65 million write calls for the 50k run.

Why it matters. Rows are kept for --burn-days (35). 100 agents posting with usage 60 times a day hold about 200k windows, so the index would freeze for about 13 s every hour, and for the same time at every start-up (open_at compacts). Writing through a BufWriter, and serialising off the lock, would remove most of it.