experiments in a post-browser web
README.md

Perf fixture: the vault-scale datastore #

Every perf spec measures Peek against one corpus: the Firefox-vault import uploaded to flit for Peek item 3e3e926d. This directory documents how that corpus is restored into a scratch desktop profile, so each later perf run starts from the same loaded state instead of re-importing ~1M bookmarks.

What the corpus is #

  • ffvault-scratch-datastore.sqlite — a 1.8 GB Peek datastore (the Firefox import): 982,631 url items, 2.8M item_visits, ~1.8k tags, ~1.0M item_tags.
  • ffvault-scratch-datastore.sqlite.zst — its 342 MB zstd twin.

Expected row counts (measured on flit, 2026-11-10):

table rows
items 982,631
item_visits 2,835,530
tags 1,829
item_tags 1,043,512

Only items is enforced (a restore refuses under 900,000 — a wrong or partial corpus must fail loudly rather than silently produce meaningless measurements). The other counts are printed by every restore and should match the table; the tags count is 1,829 as measured (the perf item's summary rounds it to 1,828).

Where it lives #

~/peek-perf/ on flit:

  • ~/peek-perf/ffvault-scratch-datastore.sqlite (plain) and its .zst twin.
  • Stray …sqlite-shm / …sqlite-wal files beside it belong to the corpus and are left alone.

The corpus is never copied into the repo and never modified. The restore opens it read-only and writes only under its --out directory.

How to restore #

One command:

scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite

Options:

# choose the scratch root (default: ~/peek-perf/profile) and profile name
scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite \
    --out ~/peek-perf/profile --profile perf

# restore + verify only; do not launch the app
scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite --no-launch

# headless measurement launch (inherited by the app)
PEEK_HEADLESS=1 scripts/perf-restore-datastore.sh ~/peek-perf/ffvault-scratch-datastore.sqlite

What it does:

  1. Picks its source: the plain sqlite when it is current, otherwise the .zst twin is decompressed. It never writes beside the corpus — a decompressed copy goes straight into the scratch root.
  2. Verifies the datastore opens read-only, prints the row counts for items, item_visits, tags, item_tags, and refuses loudly if items is under 900,000 or any table is unreadable.
  3. Installs it at <out>/<profile>/datastore.sqlite, which is exactly where the app resolves --user-data-dir=<out> + PROFILE=<profile> (apps/desktop/main/runtime-entry.ts); the profile directory is created there and nothing is written outside <out>.
  4. Launches the built app detached — PROFILE=<profile> PEEK_PROFILE=<profile> <electron> <repo> --user-data-dir=<out> — never argv --profile, which the app ignores. Output goes to <out>/launch.log, and the pid to <out>/launch.pid. scripts/perf/wait-ready.sh (PROF=<profile>) polls it until the session is up.

It is idempotent: a second run skips the decompression/copy when <out>/<profile>/datastore.sqlite is already at least as new as its source, and skips the launch when that profile's SingletonLock names a live process. A refused corpus leaves no datastore in <out>.

Requirements: python3 (row-count verification) and zstd (only when the .zst twin is the source), plus a built app (yarn workspace @peek/desktop build) for the launch step.

Consumers #

  • perf-baseline-capture depends on this track (it assumes the corpus is already restored into ~/peek-perf/profile).
  • perf-budget-tests measured a proportional fixture instead (below); it needs neither the corpus nor a restore.
  • perf-fix-item-tags depends on perf-baseline-capture.

Vault-scale budget test (default-on) #

apps/desktop/tests/desktop/perf-startup-budget.spec.ts turns the tags-page numbers from item 3e3e926d into a guarded assertion: it boots a fresh tags window and checks the three phases the vault-scale readings identified — queryItems, the item-tags loop, and render-done — against budgets derived from that baseline. It runs in the default desktop electron suite: the file lives in apps/desktop/tests/desktop/, which the playwright config's desktop project matches, so yarn test:desktop:electron (and yarn test:desktop:electron:source) run it with no flag and no skip. It seeds its own fixture, so the suite needs nothing prepared beforehand.

The owner accepted the flakiness risk this carries (the budgets come from cross-machine baselines, and a desktop suite runs several Electron instances at once). The rule that came with that decision: if this spec turns out to be unstable, file an issue and re-gate it — do not silently loosen HEADROOM or the fixture size to make a red run green.

The fixture, and why it is not the 982k corpus #

A gate-sized run cannot build or query 1.8 GB, so the spec measures a small but proportional fixture instead: 50,000 url items with visits and tags in the corpus ratios, built by a fixed seed so every run measures the same rows.

# Optional: build the fixture by hand (the spec also does this itself)
node scripts/perf-budget-seed.js ~/peek-perf/budget-profile
# -> ~/peek-perf/budget-profile/datastore.sqlite
#    items=50000 item_visits=144283 tags=93 item_tags=53098

--items N and --seed N override the size and seed; same seed and size ⇒ byte-identical datastore (the unit test asserts this). The spec seeds its own throwaway profile (<tmp>/peek-perf-budget-*/test-perf-budget) before launch, so there is nothing to prepare and the real corpus is never read, copied or modified.

Running it #

It runs as part of the desktop suite — yarn test:desktop:electron, or yarn test:desktop:electron:source for the two Electron Playwright projects alone. To run just this spec on a Linux dev box:

HEADLESS=1 BACKEND=electron \
  ./scripts/playwright-with-display.sh test \
  --config apps/desktop/playwright.config.ts \
  --project=desktop apps/desktop/tests/desktop/perf-startup-budget.spec.ts

On a macOS box use the packaged single-spec route in apps/desktop/CLAUDE.md ("One spec without taking focus"), which rebuilds first — a bare test:desktop:electron:spec runs whatever out/ already holds.

The run prints the seeded fixture's row counts and one budget line:

PERF[tags-budget] fixture: seeded .../datastore.sqlite: items=50000 item_visits=144283 tags=93 item_tags=53098
PERF[tags-budget] {"fixtureItems":50000,"scale":0.050884,"headroom":5,
                  "measuredMs":{"queryItems":1116,"itemTagsLoop":1329,"renderDone":2553},
                  "budgetMs":{"queryItems":6987,"itemTagsLoop":11958,"renderDone":21998}}

(Those are the numbers from the packaged desktop-project run of this round — HEADLESS=1 PACKAGED=1 BACKEND=electron, --project=desktop, the shape yarn test:desktop:electron uses. A source-mode run on the same tree measured 1184 / 1347 / 2632 ms.)

A failure carries the same budget line in the assertion message, so the measured numbers travel with the failure.

Where the budgets come from #

Baseline: [tags-boot:perf] phase=queryItems-done n=982631 t=27463 (≈27.5 s queryItems on flit over the real corpus), with ~47 s for the item-tags loop and ~59 s to render-done from the same perf item's laptop reading. The cumulative render-done baseline is taken as the sum of its parts (27,463 + 47,000 + 12,000 ms ≈ 86.5 s), so the cumulative marker can never sit tighter than its own phases.

Each budget is baseline × (fixtureItems / 982,631) × 5. The 5× is deliberate headroom: the target is order-of-magnitude re-regression (the N+1 item-tags loop, full-content queryItems, a full-table scan), not milliseconds.

phase baseline @982,631 scaled @50,000 budget (×5)
queryItems 27,463 ms 1,397 ms 6,987 ms
item-tags loop 47,000 ms 2,392 ms 11,958 ms
render-done 86,463 ms 4,400 ms 21,998 ms

Budgets scale with whatever the fixture actually contains (the spec counts the seeded rows), so changing --items moves the budgets instead of invalidating them. Tighten the headroom only after perf-baseline-capture records precise numbers on the target machine.

What a failure means #

Because the budgets are scaled from cross-machine readings and the spec now gates every desktop run, a failure is first a question, not a verdict:

  • measuredMs.renderDone near its budget while queryItems and the item-tags loop sit far under theirs is load, not the commit — the desktop project runs its specs in parallel, and another worker's Electron can hold the CPU.
  • A phase at several times its scaled baseline, reproducibly on a re-run, is the re-regression the spec exists to catch. perf-fix-item-tags is the track that consumes it.

The line travels with the failure either way, so the judgement is made from the numbers rather than from the exit code. If the spec proves unstable rather than correct, the answer is an issue plus a re-gate, not a looser budget.