Perf fixture: the vault-scale datastore #
Every perf spec measures Peek against one corpus: the Firefox-vault import
uploaded to flit for Peek item 3e3e926d. This directory documents how that
corpus is restored into a scratch desktop profile, so each later perf run starts
from the same loaded state instead of re-importing ~1M bookmarks.
What the corpus is #
ffvault-scratch-datastore.sqlite— a 1.8 GB Peek datastore (the Firefox import): 982,631urlitems, 2.8Mitem_visits, ~1.8ktags, ~1.0Mitem_tags.ffvault-scratch-datastore.sqlite.zst— its 342 MB zstd twin.
Expected row counts (measured on flit, 2026-11-10):
| table | rows |
|---|---|
items |
982,631 |
item_visits |
2,835,530 |
tags |
1,829 |
item_tags |
1,043,512 |
Only items is enforced (a restore refuses under 900,000 — a wrong or partial
corpus must fail loudly rather than silently produce meaningless measurements).
The other counts are printed by every restore and should match the table; the
tags count is 1,829 as measured (the perf item's summary rounds it to 1,828).
Where it lives #
~/peek-perf/ on flit:
~/peek-perf/ffvault-scratch-datastore.sqlite(plain) and its.zsttwin.- Stray
…sqlite-shm/…sqlite-walfiles beside it belong to the corpus and are left alone.
The corpus is never copied into the repo and never modified. The restore
opens it read-only and writes only under its --out directory.
How to restore #
One command:
scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite
Options:
# choose the scratch root (default: ~/peek-perf/profile) and profile name
scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite \
--out ~/peek-perf/profile --profile perf
# restore + verify only; do not launch the app
scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite --no-launch
# headless measurement launch (inherited by the app)
PEEK_HEADLESS=1 scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite
What it does:
- Picks its source: the plain sqlite when it is current, otherwise the
.zsttwin is decompressed. It never writes beside the corpus — a decompressed copy goes straight into the scratch root. - Verifies the datastore opens read-only, prints the row counts for
items,item_visits,tags,item_tags, and refuses loudly ifitemsis under 900,000 or any table is unreadable. - Installs it at
<out>/<profile>/datastore.sqlite, which is exactly where the app resolves--user-data-dir=<out>+PROFILE=<profile>(apps/desktop/main/runtime-entry.ts); the profile directory is created there and nothing is written outside<out>. - Launches the built app detached —
PROFILE=<profile> PEEK_PROFILE=<profile> <electron> <repo> --user-data-dir=<out>— never argv--profile, which the app ignores. Output goes to<out>/launch.log, and the pid to<out>/launch.pid.scripts/perf/wait-ready.sh(PROF=<profile>) polls it until the session is up.
It is idempotent: a second run skips the decompression/copy when
<out>/<profile>/datastore.sqlite is already at least as new as its source, and
skips the launch when that profile's SingletonLock names a live process. A
refused corpus leaves no datastore in <out>.
Requirements: python3 (row-count verification) and zstd (only when the
.zst twin is the source), plus a built app (yarn workspace @peek/desktop build) for the launch step.
Consumers #
perf-baseline-capturedepends on this track (it assumes the corpus is already restored into~/peek-perf/profile).perf-budget-testsmeasured a proportional fixture instead (below); it needs neither the corpus nor a restore.perf-fix-item-tagsdepends onperf-baseline-capture.
Vault-scale budget test (default-on) #
apps/desktop/tests/desktop/perf-startup-budget.spec.ts turns the tags-page
numbers from item 3e3e926d into a guarded assertion: it boots a fresh tags
window and checks the three phases the vault-scale readings identified —
queryItems, the item-tags loop, and render-done — against budgets derived
from that baseline. It runs in the default desktop electron suite: the file
lives in apps/desktop/tests/desktop/, which the playwright config's desktop
project matches, so yarn test:desktop:electron (and
yarn test:desktop:electron:source) run it with no flag and no skip. It seeds
its own fixture, so the suite needs nothing prepared beforehand.
The owner accepted the flakiness risk this carries (the budgets come from
cross-machine baselines, and a desktop suite runs several Electron instances at
once). The rule that came with that decision: if this spec turns out to be
unstable, file an issue and re-gate it — do not silently loosen HEADROOM
or the fixture size to make a red run green.
The fixture, and why it is not the 982k corpus #
A gate-sized run cannot build or query 1.8 GB, so the spec measures a small but proportional fixture instead: 50,000 url items with visits and tags in the corpus ratios, built by a fixed seed so every run measures the same rows.
# Optional: build the fixture by hand (the spec also does this itself)
node scripts/perf-budget-seed.js ~/peek-perf/budget-profile
# -> ~/peek-perf/budget-profile/datastore.sqlite
# items=50000 item_visits=144283 tags=93 item_tags=53098
--items N and --seed N override the size and seed; same seed and size ⇒
byte-identical datastore (the unit test asserts this). The spec seeds its own
throwaway profile (<tmp>/peek-perf-budget-*/test-perf-budget) before launch,
so there is nothing to prepare and the real corpus is never read, copied or
modified.
Running it #
It runs as part of the desktop suite — yarn test:desktop:electron, or
yarn test:desktop:electron:source for the two Electron Playwright projects
alone. To run just this spec on a Linux dev box:
HEADLESS=1 BACKEND=electron \
./scripts/playwright-with-display.sh test \
--config apps/desktop/playwright.config.ts \
--project=desktop apps/desktop/tests/desktop/perf-startup-budget.spec.ts
On a macOS box use the packaged single-spec route in apps/desktop/CLAUDE.md
("One spec without taking focus"), which rebuilds first — a bare
test:desktop:electron:spec runs whatever out/ already holds.
The run prints the seeded fixture's row counts and one budget line:
PERF[tags-budget] fixture: seeded .../datastore.sqlite: items=50000 item_visits=144283 tags=93 item_tags=53098
PERF[tags-budget] {"fixtureItems":50000,"scale":0.050884,"headroom":5,
"measuredMs":{"queryItems":1116,"itemTagsLoop":1329,"renderDone":2553},
"budgetMs":{"queryItems":6987,"itemTagsLoop":11958,"renderDone":21998}}
(Those are the numbers from the packaged desktop-project run of this round —
HEADLESS=1 PACKAGED=1 BACKEND=electron, --project=desktop, the shape
yarn test:desktop:electron uses. A source-mode run on the same tree measured
1184 / 1347 / 2632 ms.)
A failure carries the same budget line in the assertion message, so the measured numbers travel with the failure.
Where the budgets come from #
Baseline: [tags-boot:perf] phase=queryItems-done n=982631 t=27463 (≈27.5 s
queryItems on flit over the real corpus), with ~47 s for the item-tags loop
and ~59 s to render-done from the same perf item's laptop reading. The
cumulative render-done baseline is taken as the sum of its parts
(27,463 + 47,000 + 12,000 ms ≈ 86.5 s), so the cumulative marker can never sit
tighter than its own phases.
Each budget is baseline × (fixtureItems / 982,631) × 5. The 5× is deliberate
headroom: the target is order-of-magnitude re-regression (the N+1 item-tags
loop, full-content queryItems, a full-table scan), not milliseconds.
| phase | baseline @982,631 | scaled @50,000 | budget (×5) |
|---|---|---|---|
queryItems |
27,463 ms | 1,397 ms | 6,987 ms |
| item-tags loop | 47,000 ms | 2,392 ms | 11,958 ms |
render-done |
86,463 ms | 4,400 ms | 21,998 ms |
Budgets scale with whatever the fixture actually contains (the spec counts the
seeded rows), so changing --items moves the budgets instead of invalidating
them. Tighten the headroom only after perf-baseline-capture records precise
numbers on the target machine.
What a failure means #
Because the budgets are scaled from cross-machine readings and the spec now gates every desktop run, a failure is first a question, not a verdict:
measuredMs.renderDonenear its budget whilequeryItemsand the item-tags loop sit far under theirs is load, not the commit — the desktop project runs its specs in parallel, and another worker's Electron can hold the CPU.- A phase at several times its scaled baseline, reproducibly on a re-run, is
the re-regression the spec exists to catch.
perf-fix-item-tagsis the track that consumes it.
The line travels with the failure either way, so the judgement is made from the numbers rather than from the exit code. If the spec proves unstable rather than correct, the answer is an issue plus a re-gate, not a looser budget.