we (web engine): Experimental web browser project to understand the limits of Claude

WPT test harness and initial DOM pass rate #113

open opened by pierrelf.com

Set up the Web Platform Tests (WPT) harness and measure initial pass rate on DOM and event tests.

Scope#

WPT Submodule#

  • WPT is already configured as a git submodule at tests/wpt/
  • Contains thousands of browser conformance tests organized by specification
  • Tests use testharness.js (a JS test framework) or are ref tests (visual comparison)

Test Harness (testharness.js support)#

  • Implement enough of the WPT testharness.js API to run tests:
    • test(func, name) — synchronous test
    • async_test(name) — returns test object for async testing
    • t.step(func) — run a test step
    • t.done() — mark async test complete
    • promise_test(func, name) — Promise-based async test
    • assert_equals(actual, expected, description)
    • assert_not_equals(actual, expected, description)
    • assert_true(value, description) / assert_false(value, description)
    • assert_throws_js(constructor, func, description)
    • assert_throws_dom(name, func, description)

Test Runner#

  • cargo test -p we-browser --test wpt integration test
  • Discovers .html test files under tests/wpt/
  • For each test:
    • Parse the HTML
    • Execute inline <script> elements
    • Collect pass/fail results from testharness.js assertions
    • Handle timeouts for hanging tests
  • Report pass/fail/skip/error counts grouped by test directory

Test Scope (Initial)#

Focus on tests that exercise our Phase 11 implementations:

  • dom/nodes/ — basic DOM manipulation tests
  • dom/events/ — event dispatch and propagation tests
  • dom/collections/ — NodeList and HTMLCollection tests
  • html/dom/ — HTML DOM interface tests
  • console/ — Console API tests (if any)

Skip for now:

  • Visual/ref tests (require rendering comparison)
  • Tests requiring features we haven't implemented (CSS OM, Web Workers, etc.)
  • Tests requiring network access during test execution

Pass Rate Tracking#

  • Print summary statistics: X/Y tests passing (Z% pass rate)
  • Group results by test directory for visibility
  • Track known-failing tests separately from errors/panics
  • The test should pass cargo test even with failing WPT tests (report, don't assert)

Acceptance Criteria#

  • WPT test runner discovers and executes HTML test files
  • testharness.js assertions (assert_equals, assert_true, etc.) work
  • Synchronous and async tests are supported
  • Test results grouped by directory with pass/fail/skip counts
  • Tests that panic don't crash the runner
  • Timeout handling for hanging tests
  • Integration with cargo test -p we-browser --test wpt
  • Initial pass rate measured and reported for dom/ tests

Phase 11 — DOM-JS Bindings (issue 8 of 8). Depends on: all previous Phase 11 issues.

sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:meotu43t6usg4qdwzenk4s2t/sh.tangled.repo.issue/3mhyfpshyza2n