we (web engine): Experimental web browser project to understand the limits of Claude

Test262 test harness and initial pass rate #105

open opened by pierrelf.com

Set up the Test262 test harness integration and measure initial pass rate.

Scope#

Test262 Submodule#

  • The Test262 suite is already a git submodule at tests/test262/
  • Contains thousands of tests for ECMAScript conformance
  • Tests use a standard harness ($262, assert, assert.sameValue, etc.)

Test Harness#

  • Implement the Test262 harness helpers that tests depend on:
    • assert(condition, message) — throw if falsy
    • assert.sameValue(actual, expected, message) — SameValue comparison
    • assert.notSameValue(actual, expected, message)
    • assert.throws(expectedError, fn, message) — verify fn throws expected error type
    • $262.createRealm() (basic — can return a fresh global)
    • $262.gc() — trigger GC
    • print() / $DONE() for async tests
  • Test runner that:
    • Discovers test files matching a pattern
    • Reads test metadata (frontmatter: features, flags, negative)
    • Skips tests requiring unimplemented features
    • Executes each test in a fresh realm
    • Reports pass/fail/skip/error
    • Handles negative tests (expected parse or runtime errors)
    • Handles async tests ($DONE callback)

Integration#

  • cargo test -p we-js should run a curated subset of Test262
  • Focus on core language tests first:
    • test/language/expressions/
    • test/language/statements/
    • test/language/types/
    • test/built-ins/Object/ (basic subset)
    • test/built-ins/Array/ (basic subset)
    • test/built-ins/String/ (basic subset)
  • Track pass rate (e.g., X/Y tests passing)
  • Tests that require unimplemented features should be skipped, not fail

Acceptance Criteria#

  • Test runner discovers and executes Test262 tests
  • Harness helpers (assert, assert.sameValue, assert.throws) work
  • Negative tests (expected errors) handled correctly
  • Test metadata parsing (frontmatter features and flags)
  • Feature-based test skipping for unimplemented features
  • Pass rate tracking with summary output
  • At least basic expression and statement tests pass
  • Integration with cargo test -p we-js

Phase 10 — JavaScript Engine (issue 15b of 15). Depends on: All previous Phase 10 issues.

sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:meotu43t6usg4qdwzenk4s2t/sh.tangled.repo.issue/3mhn3qbhr3c23