typestrip
typescript type-stripping strip-types erasable-syntax transpiler wasm webassembly ts-blank-space browser zena
Star 8
Fork 0
Watch
typestrip CONTRIBUTING.md
6.8 kB · 123 lines
Markdown
at main

Contributing #

This guide gets you from a fresh clone to a passing test run, and shows where things live. Read docs/glossary.md before you write code or docs. It lists the words this repo uses for each concept.

Setup #

You need:

  • Node 24 or later.
  • The Zena compiler, zena-cli. Build it from elematic/zena at commit a7c1d2cd, following that repo's DEVELOPMENT.md. Then put zena-cli on your PATH, or set ZENA to its path. ZENA=$(./scripts/zena.sh) does all of this for you, and needs a Rust toolchain. A different Zena commit may produce a different binary.
pnpm install
./scripts/vendor.sh   # ts-blank-space (the erase-mode oracle), TypeScript 6's test suites, corpus sources
pnpm test             # build both engines, then run the default suite

pnpm install brings in binaryen (pinned, for wasm-opt and wasm-merge) and TypeScript. scripts/vendor.sh clones its sources into vendor/, which git ignores, and it is safe to re-run.

For the OSS sweeps and the benchmarks, also fetch the OSS corpus. It is several GB.

./scripts/oss.sh

Build #

pnpm build

scripts/build.sh compiles each engine with Zena, links in the generated data tables, and optimizes with wasm-opt. It writes:

File What it is
build/erase.wasm The erase engine (shipped)
build/downlevel.wasm The downlevel engine (shipped)
build/downlevel-debug.wasm The downlevel engine with a dbg() probe, for scripts/debug/

The build prints each engine's raw and gzip size. Check them against the size table in the README when you change the engine. docs/architecture.md explains each build step and its flags.

Some engine sources are generated. Edit the generator, run it, and commit both the generator and its output:

Generator Writes
node scripts/gen/keywords.mjs engine/keywords.zena, engine/keywords.wat
node scripts/gen/chartable.mjs engine/chartable.zena, engine/chartable.wat
node scripts/gen/downlevel-text.mjs The generated region of engine/downlevel.zena, and engine/downlevel-text.wat

scripts/gen/layout.mjs holds the memory addresses those tables share.

Test #

pnpm test builds, then runs the default suite. Each part also runs on its own:

pnpm test:erase       # erase mode against ts-blank-space
pnpm test:downlevel   # downlevel mode against ts.transpileModule
pnpm test:wrapper     # streaming splits, byte input, lazy loading, the browser entry
pnpm test:types       # the TypeScript declarations

The wider suites need the fetched inputs. Times are from an Apple M1:

pnpm test:erase:typescript      # 15 s   TypeScript 6's own sources and test suites
pnpm test:downlevel:typescript  # 1 min
pnpm test:erase:oss             # 2 min  needs ./scripts/oss.sh
pnpm test:downlevel:oss         # 16 min needs ./scripts/oss.sh
pnpm test:erase:bytediff        # 2 min  needs ./scripts/oss.sh

pnpm test:erase:all and pnpm test:downlevel:all run every suite for one mode, the default one included.

docs/correctness.md describes what each suite checks, the corpus files, the known-failure baselines, and the open bugs. docs/downlevel-oracle.md describes how the downlevel judge works.

Run the wider suite for a mode whenever you change how that mode parses or emits. Before you send a change, run at least pnpm test and pnpm test:erase:typescript.

Debug #

The tools in scripts/debug/ work on one input at a time. Each loads build/erase.wasm unless you set TYPESTRIP_ERASE_WASM. bisect.mjs and classify.mjs print engine state when TYPESTRIP_ERASE_WASM=build/downlevel-debug.wasm.

Tool Use it to
try.mjs '<snippet>' or try.mjs -f file.ts [chunkSize] Compare typestrip's output with ts-blank-space's for one input
minimize.mjs file [threw|chunked|ast|errors] Shrink a failing input while it still fails the same way
bisect.mjs file [maxChunk] Find the chunk sizes where streaming diverges, and where
judge.mjs <dir|file.jsonl|substring> [--list=path] Run the erase judge without stopping at the first throw, and tally failures by kind
classify.mjs fails.tsv [kind] [max] Group judge.mjs --list output by engine state and source context
semidiff.mjs file Find which ; changes the AST when output differs from the oracle only in ; and spaces
sweep.mjs [dir] Run whole and chunked transforms over a tree and list inputs that throw
stream-sweep.mjs <dir> [maxSmall] [shard/shards] Check several fixed chunk sizes against the whole-input result over a tree

Where things live #

Path Contents
engine/ The engine, in Zena. strip.zena is the parser and emitter both modes share.
src/ The wrapper: core.js, the Node entry index.js, the browser entry browser.js, and index.d.ts
scripts/ Build, vendoring, code generation (gen/), OSS sweeps, and debug tools (debug/)
test/ The judges, wrapper tests, fuzzers, and known-failure baselines (baselines/)
corpus/ Checked-in test inputs, regressions, and findings
bench/ Benchmarks against other tools, with their own package.json. See docs/benchmarks.md.
examples/ Usage examples
docs/ Reference docs
build/, vendor/ Build output and fetched sources, both ignored by git

Conventions #

  • Use the glossary. If you need a new term, add it to docs/glossary.md in the same change.
  • Every fixed bug gets a regression. Add a minimized input to corpus/regressions.jsonl or corpus/downlevel.jsonl. The steps are in docs/correctness.md.
  • Baselines only shrink. Don't add a new failure to a baseline. Record it as a finding.
  • Keep erase mode lean. Code that only downlevel mode needs goes in engine/downlevel.zena, behind a hook, so build/erase.wasm never contains it.
  • Measure size and speed changes. Report the raw and gzip size of both engines before and after, and benchmark changes that could affect throughput.
  • Comments explain why. Explain a constraint the code can't show. Don't narrate what the code does, and don't record history; that belongs in the commit message.

Release #

Add a changeset (pnpm changeset) with any change users will notice. .changeset/README.md explains how a version gets from a changeset to npm.

Zena compiler issues #

docs/zena-compiler-issues.md lists Zena compiler bugs and limitations this project has worked around. Each is waiting to be filed upstream.