# Zena compiler issues

Zena compiler bugs and limitations this project hit, kept here until each is filed upstream at [elematic/zena](https://github.com/elematic/zena). The early entries came from a port of sucrase's tokenizer that this repo no longer has, so their examples are standalone. Some entries cite `notes/size-log.tsv` and other notes that have since been removed. `git log -- notes/` has them.

## Assigning to a field of a namespace-import's member crashes the compiler

`import * as s from './mod.zena'` gives a read-only record `s`. Reading
`s.something` is fine, and so is reading-then-writing a *primitive* exported
`var` through it is fine for reads, but writing to a **field of an object
reached through that record** crashes the compiler (`-O 0` and `-O s` both):

```zena
// c.zena
export class State { var pos: i32 = 0; }
export let state: State = new State();
```
```zena
// main.zena
import * as s from './c.zena';
export function bump(): void {
  s.state.pos = s.state.pos + 1;   // crash
}
```

```
Exception caught in CLI compiler!
Error message: zir emit: ref.null type
```

The same assignment through a **named** import of the same binding compiles
fine:

```zena
import { state } from './c.zena';
export function bump(): void { state.pos = state.pos + 1; } // fine
```

An earlier, broader theory (assigning to any cross-module `var`, even a bare
`i32`, crashes -- see the original repro below) turned out to be a special
case of this: a bare exported `var` has no fields to assign *through* a
namespace record, so the record itself gets reassigned, which is exactly what
"read-only" forbids, and the compiler panics there instead of diagnosing it.

```zena
// a.zena
export var pos: i32 = 0;
```
```zena
// main.zena
import * as a from './a.zena';
export function bump(): void { a.pos = a.pos + 1; }   // crash, same error
```

**Workaround**: use named imports (`import { state } from './x.zena'`) for
any binding whose fields -- or itself, if a bare `var` -- get assigned from
outside its own module, instead of `import * as x`. Applied throughout
`zena/`: every module that needs to mutate `tokenizer/state.zena`'s shared
`State` singleton imports `state` (and the helper functions) by name rather
than as a namespace.

## Ternary `?:` is not valid expression syntax

Not a bug, just a syntax gap worth noting for anyone porting more JS/TS: Zena
has no C-style ternary. `cond ? a : b` is a parse error; use `if (cond) a else
b` as an expression. Hit repeatedly porting tokenizer/index.ts's inline
`? :` expressions (e.g. `code === charCodes.verticalBar ? tt.logicalOR :
tt.logicalAND`).

## No bitwise-NOT (`~`) unary operator

Only `!` and unary `-` are listed as unary operators; `~x` is a parse error.
Used `-8` in place of `~7` for the alignment mask in `state.zena`'s `align8`
(two's-complement `~7 == -8`, and masking with it has the same effect).

## A bare `return expr;` as a match case's only statement needs braces

`case _: return (-1, -1);` (no braces) is a parse error inside a `match`
used as a statement; wrapping it as `case _: { return (-1, -1); }` compiles.
Every other case in the same `match` already used a `{ ... }` block, so this
may just be a block-vs-bare-expression case-body ambiguity rather than a bug.

## Namespace-importing a module that exports a class crashes the compiler

Found porting the traverser. Even a pure *read* through the namespace
crashes, as long as the imported module exports a class:

```zena
// r1.zena
export let A: i32 = 5;
export class K {
  var pos: i32 = 0;
}
```
```zena
// r2.zena
import * as r from './r1.zena';
export function main(x: i32): i32 {
  return x + r.A;   // crash
}
```

```
Exception caught in CLI compiler!
Error message: zir emit: ref.null type
```

Dropping `export` from `class K`, or replacing the class with an exported
function, compiles and runs fine (`main(1) == 6`). This is likely the same
root cause as the entry above: the namespace record gets a field for the
exported class (or the instance-typed `let state`) and emitting its
`ref.null` initializer fails.

**Workaround**: namespace imports (`import * as tt`) only for modules that
export nothing but constants and functions (`types.zena`, `keywords.zena`,
`charcodes.zena`); everything from `tokenizer/state.zena` is imported by
name.

## A namespace import of a function module costs a trampoline per export

Found wiring the phase 2 emitter. `import * as erase from '../erase.zena'`
(a module that exports only functions) and calling `erase.statementBegin()`
compiles and runs correctly, but every exported function of `erase.zena`
gets a `$name$trampoline` wrapper plus globals for the namespace record,
whether or not it is called through the record. With about 60 exports that
was 24KB of `strip.wasm` (148,219 bytes against 124,156 with named imports).

**Workaround**: named imports, aliased so the call sites still read as hooks:
`import { statementBegin as eraseStatementBegin } from '../erase.zena'`.

## A namespace import of a constants-only module also pays per import site, not just per export

The entry above's workaround note said namespace imports were still fine
"for modules that export nothing but constants and functions
(`types.zena`, `keywords.zena`, `charcodes.zena`)". That turned out to be
wrong: a constants-only module gets the same treatment as a function
module, just with an extra layer. Each exported `let` becomes a
zero-arg getter function (`(structref) -> i32`, reading one field of a
backing data struct), the module gets a vtable struct of those getters,
and **every file that writes `import * as k from './consts.zena'` gets
its own fresh copy of the backing data struct and the `{structref,
vtable}` existential pair that wraps it** -- not a shared global reused
across importers. The cost scales with `(number of importing files) x
(number of exports)`, not just the export count.

Minimal repro, `consts.zena` exporting 5 `let`s, imported via namespace
by two tiny files vs. the same two files importing by name:

```zena
// consts.zena
export let a: i32 = 1;
export let b: i32 = 2;
export let c: i32 = 3;
export let d: i32 = 4;
export let e: i32 = 5;
```
```zena
// ns1.zena / ns2.zena (namespace form)
import * as k from './consts.zena';
export function f1(): i32 { return k.a + k.b; }
```
```zena
// named1.zena / named2.zena (named form)
import { a, b } from './consts.zena';
export function f1(): i32 { return a + b; }
```

`zena -O s build` on a `main.zena` that imports `f1`/`f2` from each pair:
**560 bytes** (namespace) vs. **75 bytes** (named). `wasm-tools print` on
the namespace build shows why: one getter function per export
(`struct.get 4 N` inside a `(structref) -> i32` function), a vtable
struct of those getters (type 1), a data struct holding the real values
(type 4), and an existential pair wrapping them (type 2, `{structref,
(ref 1)}`). `ns1.zena` and `ns2.zena` each get their own `struct.new 4`
+ `struct.new 2` in the global initializers -- the data struct is
rebuilt per importer instead of being one shared global.

Found while shrinking `strip.wasm`: `zena/tokenizer/types.zena` (113
`TokenType` constants) and `keywords.zena` (41 `ContextualKeyword`
constants) are each namespace-imported (`import * as tt` / `import *
as ck`) by around ten files across `erase.zena`, `plugins/`,
`traverser/` and `tokenizer/`. Converting just one small file
(`plugins/types.zena`, 4 constants used) from `import * as tt` to named
imports dropped `strip.wasm` by 1,432 bytes on its own; converting every
remaining namespace import of `types.zena`/`keywords.zena`/
`charcodes.zena` is the largest single lever found in this pass (see
`notes/size-log.tsv`).

**Workaround**: named imports for every module, including constants-only
ones. There is no case where `import * as x` is cheaper than `import {
the names actually used } from './x.zena'` once more than one file
imports it.

## A bare `case` label is always a type pattern, even for an imported or local `let`

Hit converting the namespace imports above to named imports.
`zena/tokenizer/index.zena`'s `getTokenFromCode` switched on a charcode
with `match (code) { case cc.numberSign: ...; case cc.dot: ...; ... }`.
Once `cc.numberSign` became the named import `numberSign`, the same
`case numberSign:` failed to compile:

```
'numberSign' names a type, but only classes can be matched by name.
Write 'let numberSign' to bind the matched value to a new variable.
```

Minimized, with no imports at all -- a bare top-level `let` fails the
same way as its own case label:

```zena
let foo: i32 = 5;
export function test(x: i32): i32 {
  match (x) {
    case foo: { return 1; }   // error: 'foo' names a type
    case _: { return 0; }
  }
}
```

So the rule isn't about imports: a bare identifier in `case` position is
always resolved as a type pattern (matching an instance of a class named
that), never as a value-equality check against a constant, no matter
where the constant is declared. `case (foo):` and `case foo + 0:` were
tried as workarounds; the first still errors the same way (parens don't
change the pattern grammar) and the second is a parse error (`case`
takes a pattern, not an arbitrary expression). Only a *qualified* name
(`cc.numberSign`, a member-access expression) is unambiguously a value
in that position, which is presumably why every pre-existing `match` in
this codebase already switched on namespace-qualified names rather than
local constants -- this limitation was apparently known and designed
around from the start, just never written down.

**Workaround**: don't put a bare constant in a `case` label. Either keep
it qualified behind a namespace import (defeats the point if the whole
reason to drop the namespace is size), or, as done in
`getTokenFromCode`, rewrite the `match` as an `if (code == x) { ...;
return; }` chain. The latter is what this pass did, since it was the one
spot this surfaced and losing the namespace import there was worth
~52KB of `strip.wasm` (see `notes/size-log.tsv`).

## `zena -O s` leaves most of its own size on the table; `wasm-opt -Oz` finds it

Measured on `engine/main.zena`'s strip-only build (`build/lite.zena.wasm`
before `wasm-opt`, 44,459 bytes; `build/lite.wasm` after, 29,204 bytes at
first with plain `-Oz --converge`, before the `--closed-world
--gufa-optimizing` and RSS-cap flags in `notes/size-log.tsv`'s later rows).
`wasm-opt -Oz --converge` alone removes 15,255 bytes (34.3%) that `zena -O
s` does not. Isolating each transformation by running it alone against the
pre-`wasm-opt` binary (so these numbers don't compound the way the real
`-Oz --converge` pipeline's iteration does, but do rank which single
transformation `zena`'s own `-O s` is most missing), ranked by bytes:

1. **Single-caller inlining, `--inlining-optimizing` alone: 8,300 bytes
   (18.7%).** The largest single lever by far. `zena -O s` compiles every
   function as written and never inlines across a function boundary, even
   when a function has exactly one call site -- which `strip.zena`'s
   per-state handlers (`hStmt`, `hExpr`, `hBinary`, ...) mostly are, called
   from the one `step()` dispatcher. Inlining collapses `wasm-opt
   --metrics`' function count from 143 to 80, removing call/return
   overhead and per-function prologue duplication, and opens up constant
   propagation across what used to be a call boundary.
2. **Branch and block folding, `--merge-blocks --remove-unused-brs
   --remove-unused-names --code-folding` together: 3,640 bytes (8.2%).**
   `zena`'s `match` on integers lowers to a compare chain (see the "integer
   `match` compiling to compare chains" entry above), and its lack of a
   ternary means every inline conditional is its own `if`/`else` block (see
   the "no ternary" entry). Both produce more block and branch nesting than
   the logic needs; these passes fold shared branch targets and remove
   blocks and labels nothing jumps to.
3. **Local variable reduction, `--simplify-locals --coalesce-locals
   --reorder-locals` together: 2,984 bytes (6.7%).** `wasm-opt --metrics`
   shows `[vars]` (locals beyond parameters) dropping from 1,119 to 173 over
   the full pipeline. `zena -O s` gives each `let`/`var` binding and each
   lookahead's saved position its own local slot and never reuses one once
   its last read is behind it; these passes merge locals whose live ranges
   don't overlap and renumber what's left for a smaller index encoding.
4. **Peephole simplification, `--optimize-instructions` alone: 2,690 bytes
   (6.1%).** Folds comparison and arithmetic idioms `zena` emits
   literally, including the `if (cond) a else b`-as-expression shape every
   inline conditional takes (no ternary) and the two's-complement `-8` for
   `~7` idiom (no bitwise-NOT) from the entries above.
5. **Dead code and module-element pruning, `--dce` /
   `--remove-unused-module-elements` / `--vacuum`, each alone: 1,081 bytes
   (2.4%).** `zena -O s` does not prune functions, globals or exports left
   unreferenced after whole-program analysis (as opposed to syntactically
   dead code in one function, which it does appear to skip already, since
   plain `--vacuum` alone finds the same 1,081 bytes as `--dce`).

Two further transformations only pay off with `--closed-world`, which
`wasm-opt` cannot infer on its own and `zena` has no equivalent flag for:
`--gufa-optimizing`'s whole-program value and type inference finds a few
more constant folds once it can see every call site to every internal
function (worth roughly 60-90 bytes gzip on this module, which is almost
entirely linear-memory `i32` logic rather than GC objects --
`--type-merging`/`--type-ssa` alone changed nothing, since there's barely
any GC type use for them to act on). `zena` itself could plausibly ship the
`--closed-world` inference internally for any module whose only exports are
scalar functions and memory, without waiting for a downstream `wasm-opt`
pass, since the compiler already knows its own export list.

One more finding is a *runtime* cost, not a size one, worth flagging
separately: `-Oz`'s default single-caller inlining has no cap, so it keeps
collapsing `strip.zena`'s mostly-linear call chain into one function until,
on this module, that function is about 14KB of wasm bytecode -- too big for
V8's Liftoff baseline tier, so V8 promotes it to TurboFan on first call.
That optimization is the majority of the RSS a short-lived process pays to
strip one small file: 7.9MiB over bare `node`, against 1.7-1.8MiB with
`--one-caller-inline-max-function-size=150` capping the inlining (see
`notes/size-log.tsv`'s `lite:one-caller-inline-cap-rss` row). `zena`'s own
largest function at `-O s` (no inlining at all) is about 3.2KB, so this is
entirely a `wasm-opt`-introduced cost, not one inherent to the source.

## Size findings from the architectural size pass (`notes/size-architecture.md`)

None of these is a correctness bug. Each one cost measurable bytes in `build/lite.wasm`.

- **No data segments.** User code has no way to put bytes in a data segment.
  With `--target js`, string literals become `wasm:js-string` imports.
  `FixedArray<T>` literals become `array.new_fixed` of constants in the start
  function, which costs about 1.3x to 1.4x the bytes for i64-packed text and
  compresses badly. `array.new_data` and `array.init_data` exist as
  intrinsics but are not reachable for a user-defined segment. The
  workaround is generated WAT modules that `wasm-merge` links in
  (`scripts/build.sh`).
- **No `memory.copy`, `memory.fill` or `i32.load16_u` intrinsics.**
  `Memory.getU16` is two `getU8` calls, and a memmove is a hand-written loop.
  The workaround is `engine/rt.wat`, an imported function that is merged in
  and inlined.
- **Hoisting `Memory.default` adds a null check.** `let mem = Memory.default;`
  emits `(drop (ref.as_non_null (global.get $Memory)))` in that function,
  which `wasm-opt` cannot remove, and it keeps the otherwise unused Memory
  struct global and its start-function `struct.new` alive. A direct
  `Memory.default.getX(p)` lowers to the bare load. The `hoist-memory-default-*`
  rows in `size-log.tsv`, measured on the older engine, no longer hold.
- **The inliner copies small straight-line bodies at every site, at `-O s`
  and under `-g`.** `codegen/ir/inline.zena` inlines any straight-line callee
  of 12 or fewer instructions (`straightBudget`) whatever the call count, and
  `-g`'s help text says it turns inlining off. `types.zena`'s `nx()` was
  copied into about 100 sites, about 500 B. There is no `@noinline`; the
  workaround is to return through a branch.
- **`if` chains and `match` on integers never become `br_table`.** Neither
  binary has one. `step`'s 35-state dispatch is a linear compare chain.
- **Constant module `let`s with a non-literal initializer are not folded.**
  `let M: i64 = (1 as i64) << (T_X as i64) | ...` becomes a mutable global
  set in the start function. A tiny pure helper such as
  `bit(t) = 1 << t` is neither inlined nor folded at constant call sites.
  The workaround is generated hex literals (`scripts/gen/keywords.mjs`).
- **Duplicate function types in the rec group.** `lite.wasm` has two `(func)`
  types (`$5`, `$10`) in one rec group, and closed-world type merging
  leaves both.

## Findings from the second size pass

- **`/` on two `i32`s is a float division.** `n / 10` with `n: i32` has type
  `f32` (a return of it from an `i32` function is a type error), and
  `(n / 10) as i32` lowers to `f32.convert_i32_s`, `f32.div` and
  `i32.trunc_sat_f32_s`, which is inexact above 2^24. `%` does lower to
  `i32.rem_s`. This pass found no integer division operator.
  `engine/downlevel.zena` had a repeated-subtraction `divBy`; `txDecimal` now
  divides by ten with the exact reciprocal multiply in i64,
  `n * 0xCCCCCCCD >> 35`.