An auto tiler for macOS
parcel Benchmarks RESULTS.md
10 kB · 169 lines
Markdown
at main

Performance results #

Baseline status #

There is no valid checked-in baseline yet. The v1 report was recorded before the layout hot-path fixes and sampler changes; it was removed, not migrated by changing its version. Final recording is deliberately deferred because this Mac is heavily loaded by other agents. Run make bench-record on a quiet machine, review its host-keyed report, then run make bench before committing it.

Schema v5 gates synchronous work on retired instructions (+3%), with CPU/time (+25%) only on verified quiet hosts. It records host, OS build, compiler, configuration, counter source, retained malloc deltas, and proof counters. P99 is absent below 1,000 samples. IPC remains wall-time-only. Hosted CI explicitly runs smoke mode and cannot certify budgets. See MEASUREMENT.md for this rollout.

Historical hot-path measurements #

Observed on a 14-core Apple M3 Max, macOS 15.7.7, Swift 6.2.4, using earlier samplers. These are context only, not valid baselines or controlled p99 claims.

Workload Before median After median
Layout + diff, accordion, 200 windows 40,119.62 µs 178.90 µs
Layout + diff, columns, 200 windows 2,816.72 µs 146.36 µs
Snapshot/menu model, 200 windows 105.88 µs 20.41 µs

A later process-CPU candidate observed at most 27.89 µs through layout and 37.76 µs for layout plus diff with 50 windows. Its wall p99 through layout was 7.28 ms under contention, so it did not establish the 1 ms live target. Those process-CPU measurements are incompatible with the current thread-CPU sampler.

Placement uses per-call divider, minimum-size, and focus summaries, with no stored cache to invalidate. Compressed local gaps retain the exact minimum recurrence; new 200-window oversized-gap cases cover that remaining quadratic path.

Live verification still needed #

Headless CPU tests cannot certify the 1 ms / 0.5 ms live p99 targets, window-server detection, AX latency, visible settling, rendered frame drops, memory, hangs, or 60-second idle CPU/wake-ups. Those remain opt-in harness/Instruments work on a quiet host. No real application is launched by these review regression tests.

Platform idle tests cover notification lifetimes, display debounce, permission observation, and AX watchdog scheduling in addition to engine/display-link idle proof. Watchdog one-shot sources now cancel on completion, replacement, process exit, and shutdown; their callbacks cannot rearm. The uncancellable-deadline follow-up is resolved. An AX call already executing still cannot be interrupted.

Event callback measurements #

EventInstructionBenchmarks.nativeEventPaths writes a headless report when PARCEL_EVENT_BENCH_OUTPUT is set. Use release optimization and deterministic hashing; no live windows or quiet-host certification are required:

SWIFT_DETERMINISTIC_HASHING=1 PARCEL_EVENT_BENCH_OUTPUT=/tmp/parcel-bench-events.json \
  swift test --scratch-path /tmp/parcel-benchmark-build -c release \
  -Xswiftc -enable-testing --no-parallel --filter EventInstructionBenchmarks

Motion uses one fake implementation in both policies. eager reads the frame before offering a notification to the production mailbox; deferred uses its production dirty-marker/materialization seam. Each burst drains the consumer and acknowledgements after 25 notifications. Paced input drains after every notification. Neither scenario is the old all-500-before-consumption best case. Counts come from synthetic frame accessors, including the two attribute methods per frame read; they are not measured AX IPC calls. Paced input may save no reads. consumer-turns counts explicit drain checkpoints, not actor or executor hops.

Created placement invokes the same created-callback placement seam as the app thread, the engine-published target and the real frame writer against counting accessors. It checks that the write happens before engine delivery and that its readback matches the admitted layout. It does not time a manually issued world.write, infer executor hops, or certify a single visible move. The report covers callback through admission and fixture cleanup, not an isolated native callback interval. Fact-method counts are not batched native IPC counts.

Current reports use workload version 3 and compare only matching host, case and policy sets. Completion-driven waits replaced variable polling work; versions 1 and 2 are not numerically compatible with version 3. Eager/deferred numbers are a same-binary synthetic policy comparison, not a production-revision A/B: revisions before callback facts/dirty mailbox hooks cannot compile this identical fixture. Do not compare earlier version-1 reports or hide that incompatibility behind a different fake. The common-core A/B command remains separate and excludes these hook-dependent workloads.

Initial version-2 review run — 2026-10-31 #

Release, 14-core Mac15,10, OS build 24G720, Swift 6.2.4, process rusage counters. Each report records the median of 11 samples after three warmups. Instructions below cover all 500 notifications, not one notification. Both reports used the same executable; the repeat used --skip-build.

Input First eager → deferred instructions Repeat eager → deferred instructions Synthetic frame methods Synthetic frame attribute operations
Burst, drain every 25 12,634,083 → 11,796,808 (-6.63%) 13,957,920 → 12,871,112 (-7.79%) 500 → 40 1,000 → 80
Paced, drain each event 103,508,982 → 102,717,257 (-0.76%) 116,994,458 → 117,210,149 (+0.18%) 500 → 500 1,000 → 1,000

Both policies delivered the final frame with zero application enumerations. Paced input has no frame-read saving and no stable instruction improvement. Created placement measured 1,215,843, then 1,351,192 instructions: one speculative write before engine delivery, three synthetic frame-method reads, 11 frame attribute operations and 15 fact-method reads. Its callback readback matched the admitted layout in both reports. The earlier explicit-write workload is not a valid numeric "before" for this callback measurement.

The same-policy +3% repeat gate failed six of eight cases. Motion cases rose 9.11–14.11%, created placement rose 11.13%, and hung-app focus rose 28.86%. Only focus and Space-change cases passed. Asynchronous scheduling changed work as well as wall time: hung-app focus had one versus two focus/window-list reads. This is not a gated performance success, quiet-host certification or a proven production-revision speedup. Owned debug and release proofs each passed 14 tests in four suites; the first release measurement run passed, while the repeat correctly reported six instruction-budget failures. No repeat-until-pass measurement was performed.

Raw evidence: /tmp/parcel-bench-events-review-v2.json and /tmp/parcel-bench-events-review-gate-v2.json; corresponding .log files retain the passing measurement and failed gate output. Behavior validation logs are /tmp/parcel-bench-debug-passed.log and /tmp/parcel-bench-release-proofs.log.

Completion-driven version-3 run — 2026-10-31 #

The fixture now awaits observable completion rather than polling main-dispatch turns until a predicate becomes true. Scheduling, delivery and acknowledgements signal progress; a single 30-second fixture deadline bounds a missing callback. This changes measurement work, not application behavior or the +3% gate. The failed version-2 evidence above is retained and is not used as a baseline.

Input First eager → deferred instructions Repeat eager → deferred instructions Synthetic frame methods Synthetic frame attribute operations
Burst, drain every 25 14,573,834 → 13,867,461 (-4.85%) 14,460,437 → 13,631,963 (-5.73%) 500 → 40 1,000 → 80
Paced, drain each event 145,695,383 → 145,021,408 (-0.46%) 145,859,807 → 145,212,989 (-0.44%) 500 → 500 1,000 → 1,000

All eight same-policy +3% instruction gates passed on the first version-3 baseline/repeat pair. The largest increase was 0.14%. Created callback placement measured 1,361,337 → 1,359,897 instructions with one pre-engine write, three synthetic frame-method reads and matching admitted geometry. Paced input still has no frame-read reduction; its sub-percent instruction difference is not a meaningful speedup claim. These remain same-binary synthetic policy measurements, not native IPC latency or a production-revision speedup.

Raw reports: /tmp/parcel-perf-events-v3-first.json and /tmp/parcel-perf-events-v3-repeat.json; corresponding logs contain the passing gate output. The completion-driven behavioral proofs passed 14 tests in four suites in /tmp/parcel-perf-completion-tests.log.

Five interleaved production A/B rounds also passed the common-core scope: 43 gated cases plus one reporting-only case, with no regressions. The largest recorded instruction ratio was 1.02194, below +3%. Evidence is build/bench-ab/summary.json and its per-round reports. Both revisions used the same measurement overlay; these common-core cases exclude native callback hooks.

Review validation #

After merging adaptive layouts, swift build, swift test --no-parallel (2,460 tests in 204 suites), make lint, explicit name/format checks, and all 11 Scripts/tests/*.sh suites passed. The 12 targeted oracle/performance tests in four suites also passed with release optimization, exercising all 44 headless workloads. Compiler logs contain no warnings.

The earlier missing-machine-baseline failure from make bench was verified separately; no final baseline was recorded. See REVIEW.md for every finding, integration follow-up, and public API change.

Packaging follow-up: concurrent image checks in other worktrees have observed background aliases resolving to another test-image mount; the verifier rejected the wrong target. Run image checks serially across worktrees until mount/alias isolation is fixed. This task does not change packaging code.