typestrip docs architecture.md
34 kB · 456 lines
Markdown
at main

Architecture #

A contributor's map of typestrip: how a call gets from JavaScript into the engine and back, how the engine is built, and why the build and the code look the way they do.

Terms follow glossary.md.

Repository layout #

Path Contents
src/ The wrapper: core.js (platform-neutral), index.js (Node entry), browser.js (browser/worker/Deno/workerd entry).
engine/ The engine's Zena source and the generated WebAssembly Text data segments it is merged with.
scripts/build.sh Compiles and links build/erase.wasm and build/downlevel.wasm.
scripts/gen/ Generators for engine/keywords.{zena,wat}, engine/chartable.{zena,wat}, engine/downlevel-text.wat, and shared memory layout constants (layout.mjs).
scripts/debug/ Single-input debug tools (try.mjs, minimize.mjs, bisect.mjs, and others). They load build/erase.wasm by default; bisect.mjs and classify.mjs also read build/downlevel-debug.wasm's dbg() probe. See CONTRIBUTING.md.
build/ Build output. Not checked in.
test/ erase.mjs and downlevel.mjs are the judges; the rest cover the wrapper and types.
corpus/ Small checked-in test inputs, findings, and regressions.
vendor/ Fetched sources: ts-blank-space (the erase-mode oracle), TypeScript 6's test suites, corpus inputs, and the OSS corpus. Not checked in.
bench/ Benchmarks against other tools. See benchmarks.md.
docs/ Reference docs.

engine/ files #

File Role
strip.zena The ABI entry points and the stripping machine: regions, frame stack, emitter, anchors, errors, the statement/expression/class state machine.
lex.zena The byte lexer.
types.zena The type recognizer.
keywords.zena, keywords.wat Keyword ids, keyword sets, and the perfect-hash lookup table. Generated.
chartable.zena, chartable.wat The byte-class table for the lexer. Generated.
downlevel.zena Downlevel mode's emitters: enums, namespaces, parameter properties, import x =, export =.
downlevel-text.wat The text templates downlevel.zena emits, as a data segment. Generated.
hooks-erase.zena, hooks-downlevel.zena The two implementations of the hook seam between strip.zena and downlevel.zena (see "Downlevel mode in the engine" below).
main-erase.zena, main-downlevel.zena The two shipping entry points: the ABI exports, nothing else.
debug.zena main-downlevel.zena plus the dbg() probe, built as build/downlevel-debug.wasm.
rt.wat SIMD scan loops and memory.copy, called from Zena via @external("rt", ...).

How a call flows #

  1. transform/transformSync (src/core.js) validate mode and resolve onError, then either process a whole string or open a stream.
  2. The first call for a given engine slot (0 erase, 1 downlevel) compiles that engine's .wasm module. Node reads it from disk; the browser entry streams it from fetch. Compilation is cached per slot, shared between the sync and async paths.
  3. Opening a stream acquires an idle WebAssembly.Instance for that slot, or creates one (an instance holds its own linear memory and globals, so it serves one stream at a time), and calls init(mode).
  4. Each chunk goes through inbuf/push. A string chunk is UTF-8 encoded straight into the instance's memory with TextEncoder.encodeInto. A Uint8Array chunk is copied in directly, with no decode/encode round trip, as long as it is valid UTF-8 for the whole stream; the first invalid chunk falls back to the decode path for the rest of that stream.
  5. push returns a byte count; the wrapper reads that many bytes from outbuf() and UTF-8 decodes them back to a string (except on the raw-bytes path, which still decodes output, since the ABI's output is always UTF-8 text).
  6. Errors are read from errorsPtr()/errorCount() as (start, end) i32 pairs in UTF-16 units and passed to onError, or collected to throw UnsupportedSyntaxError if there is none.
  7. close() releases the instance back to its slot's idle pool, which keeps every released instance, so a later stream reuses one without paying instantiation again and repeated bursts of N concurrent streams settle on N instances. The most recently released instance is handed out first. Trimming is lazy: once a slot holds more than IDLE = 4 idle instances, an unref'd TRIM_MS = 1000 timer runs. Each tick keeps enough idle instances to reach the highest concurrency seen since the previous tick again, and never fewer than IDLE. It drops the least recently used rest, but always keeps the first instance. One to two seconds after the last stream opens, a pool is back to IDLE and the timer stops.

A whole-string call is a stream under the hood: wholeString opens one, feeds the string in 64 KiB slices so memory stays at the streaming bound instead of growing to three bytes per input character, and closes it.

The engine #

One forward pass, frame stack, resumable streaming #

The engine is a pushdown automaton, not a recursive-descent parser. Recursive descent cannot suspend mid-construct without re-parsing from a checkpoint when input runs out, which is quadratic for a construct spanning many chunks. Instead strip.zena keeps an explicit stack of 16-byte frames ([state, a, b, c]) in linear memory, growable with growRegion. step() dispatches on the top frame's state through an ordered if chain (ordered by frequency: S_EXPR, S_POSTFIX, S_BINARY are the hot states). A frame below the top is read with belowSt/belowA/belowB, computed as an offset from fp, the top frame's own address (not a separate stack-pointer-plus-index calculation).

Streaming correctness rests on one rule.

Scan-then-commit #

Every step first does read-only lookahead: it marks a token position (let m = tb), scans ahead any number of tokens including a whole type-recognizer call, and if the lexer runs out of input (more == 1) it calls back(m) and returns having changed nothing. Only after lookahead succeeds does the step commit: it advances the lexer by exactly one token (adv()), or by exactly the tokens the lookahead already verified. Because a step either fully commits or fully backs out, the lexer failing on the very next token after a commit still leaves every frame consistent, and the next push simply resumes lexing from pos. Frame fields must only ever be set from a token the lookahead verified, never from a stale field: pushFrame conventions like c = 1 meaning "refresh a from the current token on entry" exist specifically so a frame can be pushed before its position is known to be safe.

push retries a step that ran out of input only once the input buffer has grown by at least as many bytes as the failed attempt scanned ahead (retryAt), not on every chunk. A single-chunk-at-a-time retry is quadratic in the construct's size across many chunks; the geometric retry schedule makes it linear.

Lexer #

lex.zena is a streaming byte lexer over UTF-8 input in linear memory. Positions are stream offsets, not buffer-relative addresses (delta converts between them). When a token touches the end of buffered input before fin, scan() sets more, leaves pos at the token's pre-trivia start, and produces nothing, so the same scan can be retried after more bytes arrive. The lexer caches its last result per position (LEXC, a 256-entry direct-mapped table keyed by tb & 255) because lookahead ends in a reseek back to the mark token at least as often as it advances past it; scan() checks the cache before re-lexing.

Context-dependent rescans (rescanRegex, rescanTemplate, rescanGreater, rescanLess) are driven by the parser, not the lexer, because only the parser knows whether / starts a regex or a division, or whether >> should split.

Type recognizer #

types.zena only finds where a type ends; it never emits. Every function takes the lexer's current token, consumes a type (or type parameter list, type argument list, heritage clause, object type body), and leaves the lexer on the first token after it with typeEnd at the end of the last token consumed. A false return means either the input doesn't match a type, or the lexer ran out of input (callers check more). The parser then blanks [start, typeEnd) as one edit.

Because every input is valid TypeScript, the recognizer resolves cover-grammar ambiguity positionally instead of by speculation: inside (...), a : after a complete assignment expression is a type annotation, and ? followed by :, ,, ) or = is an optional marker, so arrow parameters and parenthesized expressions share one parse. The one real speculation is < after an operand: the engine scans a type argument list, then applies TypeScript's own "can this follow a type argument list in an expression" rule, falling back to less-than; the scan is bounded by the argument list itself, so there is no rollback cost. <T>( at an operand's start is decided by a balanced skip of the parentheses followed by =>/: (generic arrow) or not (legacy cast).

Keywords and byte classes #

keywords.zena's keywordAt(p, n) is a perfect hash: one i64 multiply (w0 * KW_MUL >> 56) into a 256-slot table gives a candidate keyword id, which is then verified against the keyword's own stored bytes and length. This replaces what was, on an older design, a chain of if (n == N && bytes == X) return K comparisons per keyword. Keyword membership tests (keywordIn, keywordInHi) and token-type membership tests (tokenIn) are precomputed i64 bitmasks (KS_*/TS_*), generated from named lists in scripts/gen/keywords.mjs, instead of long || chains; a mask test is one and against a constant.

chartable.zena's CLASS table is a 256-byte array of bit flags (ID_START, DIGIT, ID_CHAR = ID_START | DIGIT, WS, NL). One getU8(CLASS + c) load and a mask classifies a byte, replacing range-check chains for identifier start/ continue, digit, and the whitespace/line-terminator if-chain in the lexer's hot loops.

Both tables are data segments (engine/keywords.wat, engine/chartable.wat), written by scripts/gen/keywords.mjs and scripts/gen/chartable.mjs and merged into every build by wasm-merge (see "Build pipeline"). Zena itself cannot declare a data segment; before this, both tables were code (compare chains, or array.new_fixed constants built in the start function), which was both bigger and less compressible.

Statement heads #

A statement's first token decides whether it is a declaration, and TypeScript's own isStartOfDeclaration plus parseModifiers do this with one lookahead rather than a same-line rule per contextual keyword. The engine follows the same shape: for the tokens in the KS_HEAD mask (async, interface, type, module, namespace, declare, const, enum, export, import, the member/statement modifiers, global, using, await), one lookahead decides hdKind (the declaration keyword, or HD_NONE/HD_AT) and hdFlags (HD_DECLARE, HD_EXPORT), with hdMods recording the modifier tokens already consumed. declare, abstract, and the other ASI-sensitive modifiers need their next token on the same line; interface/type need an identifier on the same line; module/namespace need an identifier or string; export looks past an optional type.

Once a statement's head is decided it stays decided (the frame is "committed"), because TypeScript does not re-run isDeclaration after the first modifier token: declare type\nX = 1 is a type alias even though declare alone, at end of line, would otherwise look like a plain identifier statement. From hdKind/hdFlags the engine picks ts-blank-space's action: blank the whole statement (interface, type alias, most declare forms), blank only the removed modifiers (class, function with a body), or keep everything (variables, enums, imports, exports).

Downlevel mode in the engine #

strip.zena imports ./hooks.zena, never ./downlevel.zena directly. engine/hooks.zena does not exist in the repository; scripts/build.sh copies either engine/hooks-erase.zena (every hook a no-op stub) or engine/hooks-downlevel.zena (every hook forwards to downlevel.zena) to that path before each compile. Because the import graph is decided by which file is copied in, not by a runtime mode == 1 check, build/erase.wasm's compile never reaches downlevel.zena at all, and wasm-opt's dead code elimination has nothing to chase there. An earlier version of this guard forced mode == 1 to a literal false at every call site and relied on wasm-opt to fold the dead branches; that only saved part of the cost, because about 19 of strip.zena's downlevel call sites read state that only downlevel.zena's own gated hooks ever set, so they aren't behind a foldable mode == 1 test. Swapping the whole hook target removes the module from the build's graph regardless of those sites.

The hook functions themselves (init, enumStart, enumStep, namespaceStart, namespaceEnd, exportModifier, dropPending, hookSp, blockHook, functionHead, className, varFlags, declNames, importEquals, exportAssign, classOpen, classEnd, paramProp, ctorBody, castErase, ambientVar, moduleEsmSeen, moduleCjsSeen) are called from about twenty sites in strip.zena, each behind mode == 1 or a field only downlevel mode ever sets.

What downlevel mode changes #

Downlevel mode down-levels enums, instantiated namespaces, parameter properties, import x = require(), and export =, matching what ts.transpileModule emits for them. Every inserted construct is free of line breaks, so line numbers are preserved everywhere, and columns are preserved outside the converted construct itself.

  • An enum becomes var E;(function(E){...}).call(E||(E={}),E);, with the source's own braces as the IIFE body, so this inside is the enum object. Members with bindable names are also bound as locals so later initializers can reference them bare. Whether a member gets a reverse mapping follows TypeScript's own evaluator: a syntactic string gets none, a statically known number gets one, a value constant to TypeScript but runtime-typed is decided by typeof, and anything opaque to TypeScript gets one.
  • A namespace uses the same IIFE shape; dotted names nest, and exports are written after their statement (this.x = x; for const/function/class/import aliases, an enumerable accessor for let/var so the binding stays live in both directions). A namespace whose body has no values is restored from the retained input and re-blanked, which gives exactly erase mode's output for it.
  • A class with parameter properties anchors its body's {; field declarations are inserted at the constructor's body start (or after the first top-level super(...) in a derived class).
  • import x = require() becomes const x = require(); import x = A.B becomes a var alias; export = becomes module.exports = .

Because Zena compiles a string literal to a wasm:js-string import, and the engine has zero imports, every downlevel text fragment (var, the IIFE scaffolding, the per-member templates) is instead a single packed data segment, engine/downlevel-text.wat, read by offset. scripts/gen/downlevel-text.mjs regenerates both that .wat file and the generated span of downlevel.zena between its GENERATED markers from one set of templates; edit the templates there, not the generated Zena text.

ABI #

The ABI is the export contract between the wrapper and an engine module. Both build/erase.wasm and build/downlevel.wasm implement it identically; the wrapper cannot tell them apart except by the slot it loaded.

Export Signature Meaning
memory — The module's own linear memory, auto-exported.
init (mode: i32) -> void Starts a new input. mode 0 is erase, 1 is downlevel. The wrapper sends 1 only to build/downlevel.wasm.
inbuf (cap: i32) -> i32 Returns a pointer where the host may write up to cap bytes of the next input chunk, growing memory first if needed. Valid until the next push.
push (len: i32, fin: i32) -> i32 Consumes len bytes written at inbuf's last pointer. fin is 1 on the last chunk. Returns the number of output bytes now ready at outbuf(), or -1 if the input cannot be parsed (the stream is then dead until the next init).
outbuf () -> i32 Pointer to the bytes the last push made ready. Valid until the next push or init.
errorCount () -> i32 Number of (start, end) error pairs ready at errorsPtr().
errorsPtr () -> i32 Pointer to the error pairs, as consecutive i32s in UTF-16 units of the whole input.

Every export is an i32 function or the memory; nothing else crosses the boundary. This is also why --closed-world is a valid wasm-opt flag for this program (see "Build pipeline"): no caller outside the module can reach an internal function or type.

Output contract #

Positions are reported, and in erase mode preserved, in UTF-16 units, because that is what JavaScript source positions are measured in.

Erase mode. Output has exactly the same UTF-16 length as the input. Every kept unit is identical to the input; every changed unit is a space, or (to keep ASI or arrow- function semantics across a blanked range) ;, (, or ). Line terminators are never changed. So when a blanked range held multi-byte UTF-8, the output has fewer bytes than that range of input, even though the UTF-16 length matches. This is judged against ts-blank-space by test/erase.mjs: same length and kept units, a valid parse wherever the oracle's is valid, an equal JS AST, and equal error start positions.

Downlevel mode. Lines are preserved everywhere; columns are preserved everywhere outside a down-leveled construct, inside which only the rule "no inserted line breaks" holds. This is judged against ts.transpileModule by test/downlevel.mjs, by evaluating both outputs and comparing runtime behavior, not by text or AST equality (down-leveled code is new JavaScript ts-blank-space never emits).

UTF-8 and UTF-16 #

Both directions across the ABI are UTF-8 bytes. The wrapper never sends a buffer that splits a multi-byte code point across two push calls: TextEncoder.encodeInto only writes whole characters for a string chunk, and the raw-bytes path (src/core.js's pushBytes) holds back an incomplete trailing sequence for the next chunk itself. A lone surrogate in a string chunk is encoded by encodeInto as U+FFFD (3 bytes, still one UTF-16 unit), so positions still line up; erase mode additionally restores the original lone surrogate positionally in its output, since its contract requires exact unit identity for kept text, while downlevel mode's output is not positional and keeps U+FFFD as written.

Build pipeline #

./scripts/build.sh
  1. Compile. zena-cli -O s build engine/main-erase.zena (and main-downlevel.zena, debug.zena) produces a .wasm per entry point, with engine/hooks.zena copied in first from hooks-erase.zena or hooks-downlevel.zena as appropriate. zena -O 2 produces a byte-identical module to -O s for this program; every size and speed difference in this pipeline comes from wasm-opt, not Zena's own optimization level.
  2. Generate. scripts/gen/keywords.mjs, scripts/gen/chartable.mjs, and scripts/gen/downlevel-text.mjs are run ahead of time (not by build.sh itself) to produce the generated .zena sources and their matching .wat data-segment modules, all placed at fixed addresses from scripts/gen/layout.mjs.
  3. Merge. wasm-as assembles each .wat data module plus engine/rt.wat (the SIMD scan loops and memory.copy that Zena has no intrinsic for) to .wasm. wasm-merge links each into the compiled engine as an active data segment or direct call on the engine's own imported memory; this works because Zena cannot declare a data segment or call anything outside its own module except through @external imports, so merging the "rt" imports into real functions and the table modules in as data is exactly what is needed. rt.wat's own exports are then removed (--remove-exports) since they only existed to satisfy the import.
  4. Optimize. wasm-opt with:
    • --enable-simd --enable-gc --enable-reference-types --enable-bulk-memory --enable-sign-ext --enable-mutable-globals --enable-nontrapping-float-to-int: only the features Zena itself already emits. --all-features would let wasm-opt introduce ones no shipping engine implements yet (custom-descriptor exact types, which Node 24 rejects). SIMD is required for rt.wat's v128 loops; fixed-width SIMD ships in every engine that ships WasmGC.
    • --closed-world --gufa-optimizing: valid because both artifacts export only i32 functions plus memory (see "ABI"), so no external caller can reach an internal function or type; this unlocks GUFA's whole-program type and value inference.
    • --low-memory-unused: lets optimize-added-constants fold pointer + constant into a load or store offset. That fold is exact only when the sum cannot wrap past 4 GiB into the lowest 1 KiB of memory; the engine's addresses are always small positive sums, and the generated layout keeps all static data above 1 KiB (scripts/gen/layout.mjs), so the flag's precondition always holds here.
    • --one-caller-inline-max-function-size=150: caps single-caller inlining. Without the cap, -O2's default inlining builds one function too large for V8's Liftoff baseline compiler to keep; TurboFan then optimizes it once calls accumulate, which is most of the RSS a short-lived engine instance pays (11.16 MiB delta after 20,000 calls uncapped doubles to roughly 21 MiB; see "Size and speed decisions"). Capped at 150, raw size grows by about 150 B and gzip drops by about 40 B relative to no cap, for a flat RSS curve.
    • -O2 --converge --precompute-propagate --simplify-locals-nostructure -O2 --converge, run twice: -O2 rather than -Oz trades about 1% more gzip for 8-20% more throughput (see "Size and speed decisions"). The --precompute-propagate/--simplify-locals-nostructure pass between the two -O2 runs is a second constant-propagation and local-simplification sweep over code -O2's own inlining just produced, which finds a few more foldable values that a single pass misses.

scripts/build.sh prints raw and gzip sizes for both artifacts at the end, as a quick regression check; it does not assert a budget.

Size and speed decisions #

Current sizes: erase.wasm 30,306 B raw / 12,622 B gzip -9; downlevel.wasm 42,380 B / 17,838 B.

Where the bytes go (erase.wasm, after the architectural and -O2 passes) #

Area Share
strip.zena (statement/expression machine, output, anchors, errors) About 64% of code. Control flow: roughly 900 ifs, 1,000 calls, 2,100 reads of lexer/frame state.
types.zena (type recognizer) About 15% of code. One recursive-descent function per TypeScript type production; scanPrimary alone is about 1.1 KB.
lex.zena (lexer, scans) Smaller; includes the inlined keyword lookup.
keywords.zena code About 21 B; the lookup itself is now a data segment plus roughly 200 B of code inside scan.
Static data (keyword hash table, keyword text, punctuator table, byte-class table) About 1.06 KB.
ABI entry points About 600 B.
Globals, types, exports, header About 270-440 B; roughly 54 mutable globals at about 5 B each.

downlevel.wasm adds downlevel.zena's hook targets and helpers (enum/namespace emitters, the name table, about 9 KB) and its own text-template data segment (about 1.7 KB) on top of the same strip.zena/types.zena/lex.zena cost.

Kept #

  • Tables as data, not code. Zena has no data-segment literal: a string becomes a wasm:js-string import (which breaks the zero-import ABI), and an array literal becomes array.new_fixed constants built in the start function, both bigger and far less compressible than a raw byte table. Every table the engine needs (keyword hash table and text, punctuator map, byte-class table, downlevel text templates) is instead generated as a .wat module and wasm-merged in as an active data segment.
  • Perfect-hash keywords. Replacing an 84-entry if (n == N && bytes == X) chain with one i64 multiply into a 256-slot table cut keywords.zena's code from about 2.3 KB to about 21 B, at the cost of roughly 1.1 KB of data; net smaller and faster.
  • Frame addressing from fp. The frame stack address is fp (the top frame's own address) plus a small constant offset, not stackBase + (sp - 1) * 16 + k recomputed from two globals at every one of roughly 200 call sites.
  • --low-memory-unused plus data above 1 KiB. Lets constant-offset folding apply everywhere addresses are used, for free, because the precondition (no address can wrap) is kept true by construction.
  • Keyword and token sets as bitmasks. A KS_*/TS_* mask test replaces a chain of 4 or more || terms with one and; worthwhile once a chain is long enough or repeated, not for every short one-off chain (see "Tried and rejected").
  • --closed-world --gufa-optimizing. Both artifacts export only i32 functions and memory, so closed-world analysis is exactly correct, not an approximation; GUFA's whole-program inference then finds foldable values a plain -O2 pass cannot.
  • The 150-byte one-caller inline cap. See "Build pipeline"; keeps post-tier-up RSS flat (about 11 MiB at 20,000 calls) instead of doubling (about 21 MiB uncapped), for a cost of about 150 B raw / a 40 B gzip saving relative to no cap at all.
  • -O2 over -Oz for both wasm-opt passes. About +1,982 B raw / +118 B gzip (+1.0%) on erase.wasm, and about +2,369 B raw / +196 B gzip (+1.1%) on downlevel.wasm, for roughly +8-12% (lib.dom.d.ts) to +6-20% (a many-small-files OSS corpus) median throughput on erase.wasm, and roughly +7-16%/+10-19% on downlevel.wasm. RSS after tier-up is unaffected, because the inline cap (not the -O level) is what bounds it. A roughly 1% gzip cost for consistent double-digit throughput clears the bar for "keep it" on this codebase.
  • SIMD scan loops (rt.wat). find3 (next occurrence of one of three bytes, 16 bytes per step) speeds up line/block comments, strings, and template text scanning. blank16 (16-ASCII-byte-at-a-time space fill, stopping at a line terminator or a pending ;) speeds up the emitter's blank path. Together they cost about 629 B raw / 296 B gzip on erase.wasm and roughly double throughput on comment- and whitespace- heavy input (lib.dom.d.ts roughly +77% in one measured run). SIMD requires --enable-simd at every build step, which every WasmGC-capable engine already ships.
  • */ pair scan for block comments (findClose). Once a block comment has seen a line terminator, the lexer stops only at * immediately followed by / (two overlapping 16-byte loads, i8x16.eq each, v128.and, then a bitmask), instead of find3 stopping at every *, which in JSDoc is once per line. Before the first line terminator it still uses find3, because the comment's nl flag needs that stop. Costs +151 B raw / +45 B gzip (+0.36%) on erase.wasm and +153 B / +62 B (+0.35%) on downlevel.wasm. Medians of 6 alternating rounds: lib.dom.d.ts 158.0 to 165.9 MB/s (+5.0%) erase and 177.3 to 187.5 (+5.8%) downlevel, faster in every round; the corpus 119.6 to 122.1 (+2.1%) and 121.9 to 127.2 (+4.3%); the 20MB OSS sample flat (69.9 to 70.0, 69.2 to 69.5, 5 rounds), since runtime code has far fewer long comments. Line comments, strings, and template text already scan with find3, so they got no new helper.
  • Lexer token-replay cache (LEXC). A 256-entry direct-mapped cache of the last scan() result at each stream position, because lookahead ends in a reseek back to its mark at least as often as it advances past it (scan() was measured running about 1.54 times per distinct token). Costs a small amount of data and code for a double- digit throughput gain on both corpora.
  • A pool trimmed by a timer, not by a fixed cap. src/core.js used to drop every instance released past 4 idle per slot, so each round of N > 4 concurrent streams re-instantiated N - 4 engines. Instantiation itself is cheap (about 40 µs), but at 64 interleaved streams that churn cost 10-15% of the time: about 237 instantiations per 256 streams, plus the GC and RSS of the dropped memories (2,857 dead instances awaiting GC at the end of one burst, and peak RSS growth up to +46 MiB against +16 MiB). Keeping instances until a quiet timer tick removes all of it. Against the fixed cap, the time for 256 round-robin streams of 4 KiB chunks falls from 75-78 to 68-70 ms at 8 streams and from 83-85 to 69-72 ms at 64 streams (bench/profile/concurrency.mjs). After the burst settles, the pool holds the same external memory as before (3.0 MiB). One-stream throughput, cold load, and first call are unchanged within noise. Cost: about 164 B gzip of wrapper, and nothing in either engine.
  • Raw-bytes wrapper fast path. src/core.js's pushBytes copies a Uint8Array chunk straight into inbuf, skipping the decode-then-encode round trip a string chunk needs, as long as the chunk validates as UTF-8; it falls back to the decode path for the rest of a stream on the first invalid chunk. Costs wrapper size (bundled+minified+ gzip moved from about 1,958 B to about 2,402 B) for roughly 10-30% faster throughput on byte input, string input unaffected.

Tried and rejected #

Idea Result
-O3/-O4, or raising the inline cap to 300 instead of capping at 150 More raw and gzip size for no consistent extra throughput over -O2 at cap 150, and (for -O3/-O4/no-cap) the same doubled post-tier-up RSS the cap exists to prevent.
Dropping the one-caller inline cap entirely (nocap-O2) Faster (roughly +12-20% vs +6-12%) for about +216 B gzip, but doubles post-tier-up RSS (about 21 MiB vs about 11 MiB delta after 20,000 calls). Rejected for any long-lived engine instance; -O2 with the cap dominates it.
A table-driven (bytecode-interpreted) type recognizer Raw size dropped (27,825 to 27,396 B) but gzip and brotli both grew, because the straight-line if (!nx()) return false;-shaped code it replaced already compresses better than denser bytecode plus its interpreter loop (about 300 B fixed interpreter cost).
Moving downlevel.zena's tables from a GC array to linear memory Bounded win of about 20 B raw / 131 B gzip at best; a linear-memory version still needs its own growth logic, which would have to join strip.zena's region layout and cost erase.wasm too even though erase.wasm never uses these tables.
Single-site 3- and 4-term `
SIMD identifier-tail scan (idEnd) Measured flat throughput against the scalar byte-class loop once the lexer replay cache and -O2 were already in place; dropped for a smaller, simpler rt.wat.
SIMD whitespace-run scan (skip2) Slower: real indentation runs are short enough that the call and setup cost outweighs the vectorized loop.
Forcing mode == 1 to a compile-time false instead of swapping the hook file Only removed part of downlevel.zena's code from erase.wasm, because about 19 call sites read state only the real hooks ever set and aren't behind a foldable mode == 1 guard. Superseded by the hook-file swap, which removes the whole module from the import graph.
One engine instance per mode that parks and swaps each stream's state by handle (exp/shared-instance) 1-6% faster than the timer-trimmed pool at 8-64 interleaved streams (67-68 ms against 68-72 ms), but +6.6% gzip on erase.wasm and +7.4% on downlevel.wasm. It also holds more memory: about 10 MiB more RSS growth at one stream, and 14.8 MiB of external memory after a 64-stream burst against 3.0 MiB, because the one memory keeps the parked states' high-water.
Holding instances past the idle minimum through a WeakRef, so GC trims them The same speed as the timer, for about 45 B gzip instead of 164 B. But trimming then depends on GC timing. In an async run under allocation pressure, a major GC dropped the spares mid-load and 73 instances were created again in one measurement. An idle process that runs no major GC keeps a burst's memory indefinitely.
Pre-instantiating 4 instances per mode at compile No change at steady state, since the pool fills on the first burst anyway, and the idle memory is paid before any second stream needs it.
Starting linear memory at 4 pages instead of 1 (each instance grows to 4 on init) No measurable change at any concurrency: memory.grow on a fresh instance is a few microseconds.
--inline-functions-with-loops No consistent size or speed change over the flags already used; dropped as not worth the extra flag.