ci: add an opt-in --affected run of the native tests a branch can affect #307

targeting main from claude/ci-affected-optin
Open opened by topos-backsword.test.pds.did.bot
targeting main from claude/ci-affected-optin

scripts/ci.sh --affected runs only the native tests a branch can affect. It is opt-in: plain scripts/ci.sh is unchanged.

scripts/affected-tests.py <base> turns the diff against the merge base, plus uncommitted and untracked files, into a nextest filterset:

  • A file inside a workspace crate selects that crate and every crate that depends on it (rdeps()). It also selects protokill's it binary, whose architecture and font tests read every crate.
  • docs/CONTROLS.md selects protokill, whose key-binding test reads it.
  • Files in trees no Rust reads (devlog, plan, msgs, design, docs, brainstorm, tools/pk, top-level Markdown) select nothing. The native tests and their build are then skipped.
  • Any other file, Cargo.lock, Cargo.toml, definitions/, lexicons/, scripts and config included, selects every test. So does a shallow clone or a base that does not resolve.

Every native package stays on cargo's command line, so features unify as in a whole run and the build is reused; only the test selection changes. check-targets.py test --filter <set> is the new step.

Measurements #

Each row is a scratch commit on top of this branch with --affected against its parent, warm, on the shared 16-core box at load 43-55. The plain row runs scripts/ci.sh on the same tree as the server-only row.

Change Filterset Native tests run Whole ci.sh
devlog only none() 0 (skipped, with their build) 16.9 s
one microslice (job-board) rdeps(job_board) + protokill it 16 49.9 s
protokill_server only rdeps(protokill_server) + protokill it 126 212.1 s
same tree, plain ci.sh every test 1126 372.7 s

The server row is bounded by strategic::bots_keep_the_loop_across_a_server_restart, which takes 150-300 s on its own.

Over the last 200 commits on main, the script picks no tests for 48, a subset for 92 and every test for 60. That is 56% of the suite's test time on average, weighted by measured per-test times. Most of the "every test" cases come from Cargo.lock or Cargo.toml changes.

🤖 Generated with Claude Code

Commits 1

ci: add an opt-in --affected run of the native tests a branch can affect
Sign up or Sign in to add to the discussion.