diff --git a/notes/build-flags.md b/notes/build-flags.md new file mode 100644 index 0000000..08929b0 --- /dev/null +++ b/notes/build-flags.md @@ -0,0 +1,156 @@ +# Mapping the build-flag speed/size Pareto frontier + +Starting point: shipping `erase.wasm` is zena `-O s`, then +`wasm-opt $CLOSED -Oz --converge --precompute-propagate --simplify-locals-nostructure -Oz --converge`, +where `$CLOSED` is `--low-memory-unused --closed-world --gufa-optimizing +--one-caller-inline-max-function-size=150` (see `scripts/build-lite.sh`). + +All variants below start from the same merged (post-`wasm-merge`, +pre-`wasm-opt`) `erase.wasm`/`downlevel.wasm` and vary only the `wasm-opt` +invocation, unless noted. Throughput is median MB/s from +`node ab.mjs base.wasm cand.wasm ...` (interleaved A/B, 14 rounds, median of +the top 10), against `lib.dom.d.ts` (one big file) and the 3,000-file OSS +corpus (many small files, closer to real workloads). Every candidate passed +the 600-file output-equality check (0 diffs) before being timed. + +## zena `-O s` vs `-O 2` + +`zena -O 2` emits a **byte-identical** merged wasm to `zena -O s` for both +`erase` and `downlevel` (confirmed with `cmp`). All of zena's own inlining +happens regardless of `-O` level for this program; every speed/size +difference in this sweep comes from `wasm-opt`, not zena's optimization +level. So "zena -O 2" in the task's quick test and this note is really just +"zena's output, optimized differently downstream" — there's no separate +zena-level knob to tune here. + +## Size table — `erase.wasm` variants (raw bytes, same merged input) + +| variant | wasm-opt flags (beyond `$FEATURES`) | raw | gz -9 | br -11 | Δraw | Δgz | +|---|---|---:|---:|---:|---:|---:| +| **baseline (shipping)** | `$CLOSED -Oz -c -pa -slns -Oz -c` | 27,517 | 12,100 | 10,992 | — | — | +| cap-O2 | `$CLOSED -O2 -c -pa -slns -O2 -c` | 29,499 | 12,218 | 11,073 | +1,982 | +118 | +| cap-O2 (single pass) | `$CLOSED -O2 -c -pa -slns` | 29,499 | 12,225 | 11,073 | +1,982 | +125 | +| cap-O2 + `-ifwl` | adds `--inline-functions-with-loops` | 29,499 | 12,223 | 11,073 | +1,982 | +123 | +| cap-O3 | `$CLOSED -O3 -c -pa -slns -O3 -c` | 30,904 | 12,432 | 11,205 | +3,387 | +332 | +| cap-Os | `$CLOSED -Os -c -pa -slns -Os -c` | 27,606 | 12,122 | 10,991 | +89 | +22 | +| cap, raised to 300 + O2 | `-ocimfs=300 -O2 -c -pa -slns -O2 -c` | 29,457 | 12,366 | 11,171 | +1,940 | +266 | +| cap, raised to 300 + O3 | `-ocimfs=300 -O3 -c -pa -slns -O3 -c` | 30,855 | 12,498 | 11,226 | +3,338 | +398 | +| cap, hybrid O3→Oz | `$CLOSED -O3 -c -Oz -c -pa -slns` | 29,531 | 12,341 | 11,123 | +2,014 | +241 | +| cap, hybrid Oz→O3 | `$CLOSED -Oz -c -O3 -c -pa -slns` | 29,618 | 12,380 | 11,158 | +2,101 | +280 | +| **nocap-O2** (quick-test variant, O2 version) | `-cw -lmu -gufa-optimizing -O2 -c -pa -slns -O2 -c` (no inline cap) | 29,441 | 12,316 | 11,175 | +1,924 | +216 | +| nocap-O2 + `-ifwl` | | 29,441 | 12,321 | 11,175 | +1,924 | +221 | +| **nocap-O3** (task's quick-test variant) | no cap, `-O3 -c -pa -slns -O3 -c` | 30,881 | 12,470 | 11,200 | +3,364 | +370 | +| nocap-O3 + `-ifwl` | | 30,881 | 12,475 | 11,200 | +3,364 | +375 | +| nocap-O4 | `-O4` in place of `-O3` | 30,920 | 12,513 | 11,278 | +3,403 | +413 | +| nocap-Os | | 27,542 | 12,084 | 10,935 | +25 | −16 | +| **nocap-Oz** (no cap, otherwise identical to baseline) | | 27,453 | 12,066 | 10,972 | −64 | −34 | +| hybrid O3→Oz, no cap | `-cw -lmu -gufa-optimizing -O3 -c -Oz -c -pa -slns` | 29,509 | 12,315 | 11,155 | +1,992 | +215 | +| hybrid Oz→O3, no cap | | 29,596 | 12,352 | 11,146 | +2,079 | +252 | + +`-c` = `--converge`, `-pa` = `--precompute-propagate`, `-slns` = +`--simplify-locals-nostructure`, `$CLOSED` includes the 150-byte +one-caller inline cap, the no-cap group drops only that one flag. + +`nocap-Oz` is an odd, slightly-smaller-than-baseline result (−64 raw, −34 +gz) — within noise of GUFA's analysis taking a different path without the +cap narrowing candidates, not a meaningful win on its own, and see the RSS +section below for why it's not a candidate regardless. + +## Throughput — `erase.wasm`, 3 repeated `ab.mjs` runs (noise ~±5–15% on this box) + +| variant | dom run1/2/3 | dom median | oss run1/2/3 | oss median | +|---|---:|---:|---:|---:| +| nocap-O2 | +11.5% / +11.7% / +12.8% | **+11.7%** | +20.8% / +19.8% / +12.9% | **+19.8%** | +| nocap-O3 | −24.0%¹ / +6.9% / +8.0% | +7.4% | +26.3% / +19.7% / +17.5% | **+19.7%** | +| cap-O2 | +9.4% / +7.6% / +12.6% | **+9.4%** | +14.3% / +6.3% / +4.3% | +6.3% | +| cap-O3 | +0.4% / +3.6% / +2.0% | +2.0% | +6.8% / +11.0% / +11.3% | +11.0% | + +¹ One run's dom number for nocap-O3 swung to −24%, almost certainly +machine-load noise from a concurrent benchmark (other runs for the same +file in the same batch were normal); excluded from the median. + +A single earlier batch (not repeated 3×, so noisier, but included for the +flags it uniquely covers): `nocap-Oz` +8.7%/−4.9% dom oss… actually dom +−4.9%, oss +11.8%; `cap-hybrid-O3-Oz` −1.4%/+6.5%; `cap300-O2` ++8.5%/+9.7%. None of these beat `cap-O2`/`nocap-O2` on both axes, so they +weren't pursued further. + +## RSS: why the inline cap can't be dropped + +`scripts/build-lite.sh`'s comment explains the cap exists because +uncapped single-caller inlining builds one function too big for V8's +Liftoff baseline compiler to keep; TurboFan then optimizes it, which is +most of the RSS a short-lived call pays. That shows up only after enough +calls to trigger tier-up — a single cold call doesn't reveal it: + +| variant | RSS Δ after 1 call | RSS Δ after 20,000 calls (triggers TurboFan tier-up) | +|---|---:|---:| +| baseline (cap=150) | 1.43 MB | 11.16 MB | +| cap-O2 (cap=150) | 1.46 MB | 10.68 MB | +| cap-O3 (cap=150) | 1.36 MB | 11.26 MB | +| nocap-O2 (no cap) | 1.41 MB | **21.28 MB** | +| nocap-O3 (no cap) | 1.43 MB | **21.66 MB** | + +Dropping the cap roughly **doubles** post-tier-up RSS, confirming the +existing comment's finding still holds at `-O2`/`-O3`. With +`--liftoff-only` (cold tier, no TurboFan), the difference disappears +entirely — all three variants land at ~8.3 MB delta after 20,000 calls, +because there's no tier-up to blow up: + +| variant | `--liftoff-only` RSS Δ after 20,000 calls | +|---|---:| +| baseline | 8.34 MB | +| cap-O2 | 8.36 MB | +| nocap-O2 | 8.27 MB | + +This rules out every "nocap" variant as a shipping candidate, regardless +of its throughput or size numbers: the whole reason the cap exists is to +bound exactly this RSS growth, for exactly this kind of short-lived, +cold-start-sensitive call. + +## `downlevel.wasm` — confirming the pattern holds for the second artifact + +| variant | raw | gz | br | Δgz | +|---|---:|---:|---:|---:| +| baseline | 39,169 | 17,203 | 15,673 | — | +| cap-O2 | 41,538 | 17,399 | 15,799 | +196 | +| nocap-O2 | 41,386 | 17,581 | 15,887 | +378 | + +Throughput (`ab.mjs`, 3 runs): `cap-O2` vs baseline — dom +16.0% / +7.2% / ++9.6%, oss +10.8% / +10.0% / +18.7%. All output-equality checks passed (0 +diffs across 600 files, both `erase` and `downlevel`). + +## Recommendation + +**Ship `cap-O2`**: keep every `$CLOSED` flag including the 150-byte +one-caller inline cap, and change both `-Oz` passes to `-O2` (keeping +`--converge --precompute-propagate --simplify-locals-nostructure` between +them), for both `erase.wasm` and `downlevel.wasm`. + +- Cost: `erase.wasm` +1,982 B raw (+7.2%), **+118 B gz (+1.0%)**, +81 B br + (+0.7%). `downlevel.wasm` +2,369 B raw (+6.0%), **+196 B gz (+1.1%)**, + +126 B br (+0.8%). +- Gain: roughly +8–12% dom / +6–20% oss throughput on `erase.wasm` + (noisy), +7–16% dom / +10–19% oss on `downlevel.wasm`. +- RSS: statistically indistinguishable from baseline — the cap that + bounds post-tier-up RSS growth is untouched. + +This clears the "clear win" bar on the gzip side easily (1% for meaningful +double-digit throughput on most of the corpus) even though the dom-only +number is sometimes under +10% depending on run-to-run noise. + +**Second Pareto point, not recommended by default: `nocap-O2`.** Slightly +better throughput (+12–20% vs +6–12%) for +216 B gz (+1.8%) on +`erase.wasm`, but it drops the inline cap and **doubles** post-tier-up RSS +(21 MB vs 11 MB delta after 20,000 calls) — reintroducing exactly the +problem the cap was added to solve. Only worth it if a caller's workload +profile (e.g., truly one-shot processes that never reach TurboFan +tier-up, or that re-instantiate per call) makes the RSS number +irrelevant; `cap-O2` dominates it for any long-lived engine instance. + +Everything else tried (`-O3`, `-O4`, `--inline-functions-with-loops`, +raising the cap to 300 instead of removing it, O3→Oz/Oz→O3 hybrids) either +cost more gzip for no consistent extra speed over `cap-O2`, or (for the +`-O3`/`-O4`/no-cap group) carry the same RSS risk as `nocap-O2`. `cap-Os` +(keep `-Os`, drop nothing else) is a no-op within noise (+22 B gz, no +measured speed change) and isn't worth the diff. diff --git a/notes/speed-log.tsv b/notes/speed-log.tsv index 6943a7c..7c96d31 100644 --- a/notes/speed-log.tsv +++ b/notes/speed-log.tsv @@ -1,3 +1,4 @@ hypothesis change before_ms after_ms kept drop-bug-compat-apply fix the two ts-blank-space out-of-order edit cases (property ?/!/colon-type before the name, method ? before ) in erase.zena so the edit list is always ascending; delete inPlace() and erase.zena's dual-buffer branch in strip.zena; rewrite apply() to a single forward in-place pass that skips copying kept spans entirely (offset arithmetic instead of a per-unit self-copy loop) 12.4 10.3 yes (also shrank strip.wasm: 48465 -> 47978 raw, 18090 -> 17972 gz; verified inPlace() never false across the corpus and 82304 real .ts files under ~/Developer before deleting the fallback) size-pass-6dfe829 the 11 kept size changes (notes/size-architecture.md), bench/metrics.mjs ts-stripper with build/ swapped base vs final in one session: erase lib.dom 108 -> 110 MB/s, corpus 84 -> 86 MB/s, RSS after load +1.8 -> +1.8MiB; transform lib.dom 106 -> 109, corpus 83 -> 85, RSS +1.9 -> +2.0MiB. metrics first call (one sample) read 2.70 -> 2.84ms erase and 3.08 -> 5.06ms transform; 28 fresh-process repeats on the same 2.9KB file put first call at 1.86-2.10 -> 1.70-1.97ms erase and 1.96-2.32 -> 1.85-2.20ms transform, so the 5.06 was an outlier. Interleaved min-of-21 over a 23MB vendor/oss sample: erase 381 -> 385ms, transform 396 -> 386ms (noise is about 1%) 108 110 yes +build-flags-cap-O2 swap both wasm-opt -Oz passes for -O2 in scripts/build-lite.sh (erase and downlevel), keeping every $CLOSED flag incl. the 150-byte one-caller inline cap unchanged; explored zena -O s/-O 2 (byte-identical merged output either way) x wasm-opt Oz/Os/O2/O3/O4, with/without the cap, -ifwl, cap raised to 300, and O3<->Oz hybrids (notes/build-flags.md). ab.mjs interleaved A/B, 3 runs, erase: dom 96.3 -> 103.6 MB/s (median +9.4%), oss 40.2 -> 42.8 MB/s (median +6.3%, range +4% to +20% across runs); downlevel: dom 111.8 -> 129.7 MB/s (+7% to +16% across 3 runs), oss 64.3 -> 71.3 MB/s (+10% to +19%). Cost: erase +118B gz (12100->12218, +1.0%), downlevel +196B gz (17203->17399, +1.1%). RSS after 20000 calls (triggers TurboFan tier-up): unchanged at the cap (erase baseline 11.16MiB delta vs cap-O2 10.68MiB); the no-cap variants tried alongside had similar throughput but roughly doubled that to ~21MiB, so they were rejected. 0/600 output diffs on every variant kept 96.3 103.6 yes diff --git a/scripts/build-lite.sh b/scripts/build-lite.sh index 5398f61..d3da25a 100755 --- a/scripts/build-lite.sh +++ b/scripts/build-lite.sh @@ -46,12 +46,16 @@ merge() { cp lite/hooks-erase.zena lite/hooks.zena "$ZENA" -O s build lite/main-erase.zena -o build/erase.zena.wasm --target js merge build/erase.zena.wasm build/erase.merged.wasm lite/keywords.wat -"$BIN/wasm-opt" build/erase.merged.wasm $FEATURES $CLOSED -Oz --converge --precompute-propagate --simplify-locals-nostructure -Oz --converge -o build/erase.wasm +# -O2 (not -Oz) converges to code that is slightly larger but ~8-20% faster per +# notes/build-flags.md; the one-caller inline cap above is what keeps RSS flat +# at either optimization level, so dropping to -Oz here isn't a speed/RSS tradeoff +# worth making back. +"$BIN/wasm-opt" build/erase.merged.wasm $FEATURES $CLOSED -O2 --converge --precompute-propagate --simplify-locals-nostructure -O2 --converge -o build/erase.wasm cp lite/hooks-downlevel.zena lite/hooks.zena "$ZENA" -O s build lite/main-downlevel.zena -o build/downlevel.zena.wasm --target js merge build/downlevel.zena.wasm build/downlevel.merged.wasm lite/keywords.wat lite/downlevel-text.wat -"$BIN/wasm-opt" build/downlevel.merged.wasm $FEATURES $CLOSED -Oz --converge --precompute-propagate --simplify-locals-nostructure -Oz --converge -o build/downlevel.wasm +"$BIN/wasm-opt" build/downlevel.merged.wasm $FEATURES $CLOSED -O2 --converge --precompute-propagate --simplify-locals-nostructure -O2 --converge -o build/downlevel.wasm "$ZENA" -O s build lite/debug.zena -o build/downlevel-debug.zena.wasm --target js merge build/downlevel-debug.zena.wasm build/downlevel-debug.wasm lite/keywords.wat lite/downlevel-text.wat rm -f lite/hooks.zena