typestrip docs benchmarks.md
9.9 kB · 100 lines
Markdown
at jsx

Benchmarks #

typestrip compared with other TypeScript strippers and transformers, in two tables. The erase-mode table holds tools with a strip-only mode, which erase erasable syntax only. The downlevel-mode table holds tools that down-level enum, namespace, and parameter-property syntax. A tool appears in a table only if it has a documented, recommended fast path for that group.

Machine. Apple M1, macOS 27.0, Node v24.16.0.

Versions. typestrip (this build), ts-blank-space 0.10.1, amaro 1.3.0 (also what Node's built-in node:module strip mode runs), @swc/wasm-typescript 1.16.21, oxidase 0.0.3, sucrase 3.35.1, oxc-transform 0.156.0, typescript 6.0.3, esbuild 0.25.12.

Last run. 2026-11-12, at a load average of about 2. Absolute numbers move with machine load: every tool measured about 30% slower in this session than in an earlier one. Compare rows within a table, not against older copies of it.

Erase mode: erasable syntax only #

tool gzip brotli cold load first call MB/s (lib.dom.d.ts) MB/s (corpus) MB/s (20MB OSS sample) peak RSS growth (OSS sample) rejects (non-erasable syntax) / 2,485 OSS files
typestrip 15.1kB 13.7kB 1.3ms 3.9ms 160 120 69.6 +30.7MiB 251 (10.1%)
ts-blank-space 1.03MB 799.7kB 202.2ms 11.5ms 16 18 16.9 +116.9MiB 251 (10.1%)
amaro (strip-only) 1.12MB 789.9kB 49.5ms 13.1ms 45 38 25.5 +45.6MiB 251 (10.1%)
node:module stripTypeScriptTypes built in built in 0.1ms 49.8ms 46 35 22.4 +71.7MiB 251 (10.1%)
@swc/wasm-typescript (strip-only) 1.39MB 934.2kB 38.6ms 14.3ms 72 48 27.3 +55.0MiB 251 (10.1%)

All five tools reject the same 251 of 2,485 files (10.1%). Those files use enum, parameter properties, instantiated namespace, import x = require(), or <T>x casts, which no tool can erase without changing behavior. node:module runs amaro, and amaro is built on @swc/wasm-typescript, so this is three implementations agreeing on the scope, not five.

typestrip is the smallest download by about two orders of magnitude. The other tools ship a general TypeScript or JavaScript parser, and typestrip ships a purpose-built one. It also has the fastest cold load and first call, and the highest steady-state throughput on every corpus. node:module's first call includes Node's lazy load of amaro; once warm, its throughput tracks amaro's.

Downlevel mode: enums, namespaces, parameter properties down-leveled #

tool gzip/brotli or size note cold load first call MB/s (lib.dom.d.ts) MB/s (corpus) MB/s (20MB OSS sample) peak RSS growth (OSS sample) fails / 2,485 OSS files
typestrip (downlevel) 20.3kB gz / 18.4kB br 1.3ms 3.3ms 183 125 69.0 +31.0MiB 0
@swc/wasm-typescript (transform) 1.39MB / 934.2kB 23.2ms 28.6ms 77 37 15.2 +58.7MiB 0
sucrase 46.9kB / 39.7kB 45.1ms 9.2ms 39 24 23.8 +151.4MiB 59 (2.4%)
oxc-transform † 1.62MB / 1.14MB (native, NAPI) 16.0ms 12.0ms 144 95 50.7 +31.0MiB 0
ts.transpileModule 1.03MB / 797.1kB (ships all of typescript) 171.5ms 41.1ms 15 7 2.8 +398.0MiB 0
esbuild (transform API) † ~10MB native binary, n/a as transfer size 13.3ms 115.2ms 77 17 8.1 +123.5MiB 0
oxidase †† 228.9kB / 177.9kB 8.7ms 9.2ms 111 88 n/a n/a 2,485 (100%, see below)

† Native code: a NAPI addon (oxc-transform) or a Go binary driven over stdio (esbuild). Neither is something a bundler ships to a wasm or JS consumer, so the size column is install or binary size. oxc-transform's is its .node addon for this platform. esbuild's is its platform binary, @esbuild/darwin-arm64.

†† oxidase has no strip-only mode, because transpile() always down-levels, so it appears only in this table. On the OSS sample it panicked inside its Rust core on one file (already borrowed: BorrowMutError). That left its state poisoned for the rest of the process, so every later input failed too, and its MB/s and RSS are omitted. On the small corpus, which doesn't contain that file, it failed 1 of 362 inputs, and its numbers there are competitive.

typestrip in downlevel mode leads on size, cold load, and first call. It also has the highest steady-state throughput in the table, native tools included. ts.transpileModule is the slowest and heaviest, because it runs the whole compiler. esbuild's transform API pays a per-call IPC cost, which shows up as the highest first-call time in either table.

esbuild's transformSync doesn't read tsconfig.json, so by default it rejects parameter and property decorators, which are common in Angular and NestJS code. bench/tools.mjs passes tsconfigRaw: { compilerOptions: { experimentalDecorators: true } }. That took esbuild from 61 failures on the OSS sample to 0.

What the columns mean #

gzip and brotli. The tool's entry bundled with esbuild --bundle --minify --format=esm --platform=node, plus any wasm it loads, compressed with Node's zlib at gzip level 9 and brotli quality 11.

cold load and first call. Time to import the tool in a fresh process, then time for its first call on a small file.

MB/s. Steady-state throughput on lib.dom.d.ts (one large file), on the small corpus (test/corpus.mjs), and on a seeded 20MB sample of runtime .ts files drawn evenly from the OSS corpus (./scripts/oss.sh).

peak RSS growth. The process's peak resident memory over the OSS sample, minus its RSS before the run. V8 heap growth isn't deterministic, so this column is the noisiest.

rejects or fails. Inputs where the tool throws or emits output that doesn't parse as JavaScript. This is a cheap sanity check next to the speed numbers, not the correctness suite; see correctness.md for that. In the erase-mode table every tool runs with its default, throwing behavior.

Wasm, JS, and native code #

The same harness times every tool in Node, one subprocess per tool, but the tools don't all run the same kind of code:

  • wasm: typestrip, @swc/wasm-typescript, amaro and node:module (both built on @swc/wasm-typescript), and oxidase.
  • JS: ts-blank-space (which runs on the TypeScript compiler), sucrase, and ts.transpileModule.
  • native: oxc-transform and esbuild.

The wasm rows are the like-for-like comparison. When typestrip beats oxc-transform, its wasm isn't running faster than native code. It does less work for each byte of input:

  • No syntax tree. oxc-transform parses the file into a full AST, transforms the tree, and prints new JavaScript. typestrip makes one forward pass over the UTF-8 bytes with a token-level state machine and a type recognizer. It allocates nothing per syntax node and never regenerates code.
  • The output is the input. In erase mode each kept byte is copied as it is, and each erased byte becomes a space. Downlevel mode writes new text only around the constructs it converts.
  • Only the structure it needs. A full parser checks the whole file and reports every syntax error. typestrip tracks only enough structure to find where types start and end, and passes the rest through without checking it.

Both kinds of tool pay to cross into their engine. typestrip copies the source into wasm memory as UTF-8 and decodes the output back to a string, and a NAPI addon converts strings at its own boundary. No row is a pure engine measurement.

Running the benchmarks #

cd bench
pnpm install
pnpm bench

pnpm bench builds the OSS sample (scripts/oss-sample.mjs throughput, which needs ./scripts/oss.sh first), then runs metrics.mjs and throughput.mjs in both modes. ESBUILD overrides the esbuild binary used to bundle each tool for the size columns.

Script Measures
bench/metrics.mjs [tool] Size, cold load, first call, and small-corpus and lib.dom.d.ts throughput
bench/throughput.mjs <inputs.json> [tool] [--mode=downlevel] MB/s and peak RSS over a JSON array of sources
bench/tools.mjs The tool registry: how each tool is loaded, bundled, and called
bench/profile/boundary.mjs [file] One transform split into encode, wasm, and decode time
bench/profile/stream-memory.mjs [file | --concat=N] [chunkBytes] Peak wasm memory while streaming; it must stay flat
bench/profile/pathological.mjs [chunkChars] Time against input size for inputs built around one huge construct; it must scale linearly
bench/profile/concurrency.mjs [root ...] Time, peak RSS, and retained RSS for 1, 8, and 64 streams interleaved round-robin, A/B across checkouts
bench/profile/cpuprofile-top.cjs [n] The top self-time functions in a .cpuprofile in the current directory

Method for published tables #

Each published row is the median of 3 full-suite runs, made back to back in one session. Each script already runs one subprocess per tool, so one invocation measures every tool within a minute or two. Between rounds, rotate which measurement runs first (metrics.mjs, then erase-mode throughput.mjs, then downlevel-mode throughput.mjs), and alternate tool order with METRICS_REVERSE=1 and TP_REVERSE=1. No tool or mode then always lands in the same part of the session.

Measure everything in one session. Separate invocations minutes apart can differ by more than the real gap between two rows. One earlier draft showed downlevel mode faster than erase mode, which can't happen: downlevel mode does strictly more work on the same input. Run in one session, erase mode came out ahead every round.

Expect about ±10-15% noise run to run on the small-corpus numbers, about ±10% on the OSS sample, and up to ±30% on RSS.