aite plan (draft 1, 2026-10-09) #

aite = AI Integrated Test Environment. An AI writes small readable scripts that drive a real browser; when one fails, a human can tell in under a minute whether the app or the script is wrong.

Status: plan only, nothing built. Draft 1 turns draft 0 into a build plan: decisions with defaults, a design with concrete interfaces, phases with checklists and acceptance tests. Anything marked (assumed) is a default I chose; change it before the phase that depends on it starts.

Contents

1. Principles
2. What BITE gives us, what does not carry over
3. Decisions (with defaults)
4. Design
5. Repo layout
6. Phases and checklist
7. How the AI writes and repairs scripts
8. How aite itself is tested
9. Security and accounts
10. Risks
11. Needs from Alex
12. Out of scope

The plan is split across files. This one is the overview; the details live beside it. Where this file and docs/design/*.md disagree, the design file wins (and the task list in TASKS.md wins over both for scope and acceptance):

File	What it holds
docs/decisions.md	D1 onward, each with status, the phase that needs it, and alternatives
docs/design/script-format.md	the grammar, every command and target, verify/wait forms, timing rules, error codes E001 to E007 (parser and runner), an example
docs/design/protocol.md	the runner and extension websocket protocol: handshake, messages, failure reasons, keep-alive, delivery rules, security
docs/design/data-formats.md	element descriptor and lock file, run.json, verdicts, repair requests, aite.config.json
docs/design/locator-repair.md	resolution order, scoring weights, outcomes (found, repaired, drifted, not-found, ambiguous), rules
docs/journeys.md	the mallbucks journeys the first scripts cover (send, gift, shop) with expected texts
docs/TASKS.md	the tasks (50 after the QA review) with acceptance criteria, test tiers, test cases, dependencies and labels; GitLab holds the live status
docs/test-plan.md	the QA test plan: risk tiers, cases with oracles, manual QA call-outs, gates, and the post-review task changes (section 6)
docs/adversarial-review.md	the independent review of this plan (20 findings, verified); where it conflicts with this file, the task list in TASKS.md and the changes in test-plan.md section 6 win

---

1. Principles #

1. A failure must be confirmable by a person fast. Every step carries its intent in words, so a failure reads as "wanted X, found Y", with a screenshot, not as a selector error.
2. Real session, real browser. Scripts run in a signed-in browser profile, so flows that need a genuine session (Bluesky sign-in, the posting permission, passkey wallets, popups) are tested for real, by the AI, without waiting for a human tester.
3. Scripts are plain text in git. Reviewable, diffable, written and repaired by the AI, readable by Qian.
4. Small command vocabulary. If a step cannot be said in a few words, the script is too clever.
5. Never fake a pass. A step that cannot find its element fails; waiting is explicit; no silent retries that hide flakiness (retries are counted and shown).
6. Reuse ideas, not the stack. BITE's design is worth keeping; its code (Closure, App Engine, Manifest V2) is not.

2. What BITE gives us, what does not carry over #

Read from source-archive-bite.zip (unpacked outside the repo; nothing was run; the archive is left out of the published repository, see README). License: Apache 2.0 (COPYING).

Keep (as ideas, rewritten in TypeScript):

• A tiny command language: click doubleClick drag enter type select change submit input find verify verifyNot wait sleep mouse move comparePosition changeUrl redirectTo run call validateBlock replaceHtml (tools/rpf/extension/src/libs/codegenerator.js).
• Element descriptors (XPath plus the element's attributes) instead of a single selector, and a locator-repair step (locatorsupdater.js, getactioninfo.js, common/ ElementDescriptor).
• Record then edit: recordmanager.js, eventsmanager.js turn live clicks into commands, which a person then tidies.
• Playback with waits and retries (playbackmanager.js, 1,700 lines) and an action layer built on Selenium's browser atoms (bothelper.js, actionshelper.js).

Drop:

• The extension shell: Manifest V2 (Chrome no longer loads it), uses removed APIs (chrome.extension.sendRequest, 17 uses of executeScript, 5 of eval; V3 forbids eval and remote code). Rewrite for Manifest V3.
• The server: Python 2.7 on Google App Engine. Replace with a small Node process.
• Closure Library, the compiler, GData and App Engine clients, the Ace editor: all of deps/ (89 MB).
• Bug tracking, crawlers and the Google-specific parts of the server.

3. Decisions (with defaults) #

#	Decision	Default (assumed)	Why / what would change it
D1	Script format	A small line-based format for steps (section 4.1), plus optional TypeScript for custom checks	Easiest for people to read and for the AI to write and diff. Cost: a parser (about 300 lines). Alternative: TypeScript functions only.
D2	Driver	MV3 Chrome extension, one runner interface so a Playwright backend can be added later	Alex's preference. An extension cannot answer an OS prompt (Touch ID) or run headless in CI; Playwright with a virtual passkey can.
D3	Language	TypeScript, no UI framework, esbuild for the bundle	Small, typed, no Closure.
D4	Where it runs	Headed Chrome with a persistent profile on balrog, started by a script; also runnable on a laptop	Real session needs a real profile; headless comes later with Playwright.
D5	Bridge	A local Node process, websocket on 127.0.0.1 only, a per-run token	Lets Claude start a run and read results; never exposed to the network.
D6	Results	JSON + screenshots on disk, rendered as one static HTML report; verdict buttons live in a small local page	Keeps v1 simple; a GitLab artifact can come later.
D7	Repo	This repo (root/aite). BITE stays as an archive/reference; new code lives beside it
D8	Large files	Decided 2026-10-09: leave the two zips in place; keep a short reference/ of files read	84 MB sits in history already.

4. Design #

 Claude (AI)  --writes-->  scripts/*.aite  (git)
      |                          |
      | start run, read result   v
      v                     +---------+   websocket (127.0.0.1)   +----------------------+
  bridge (Node) <---------> | runner  | <------------------------> | MV3 extension        |
      |                     +---------+                            |  service worker      |
      |                                                            |  content scripts     |
      v                                                            +----------+-----------+
  results/<run>/ (json, png, report.html)                                      | acts on
                                                                               v
                                                              real signed-in Chrome profile

4.1 Script format (sketch, final syntax is a Phase 2 task) #

One file per flow, scripts/<area>/<name>.aite. Plain text; # starts a comment.

test "send 1 MALLBUCKS to a handle"
  covers  J-09 J-13
  start   https://mallbucks.malldao.xyz/
  needs   session test-qian

  step "remember the starting balance"
    capture before  css:".balance"
  step "find the recipient"
    click  text:"Send"
    type   placeholder:"Search"  "alex"
    click  text~:"@distributed.camp"
  step "review shows the right amount"
    type   label:"Amount"  "1"
    click  role:button name:"Send"
    verify text:"1 MALLBUCKS"  within:"Review send"
  step "balance goes down by one within 5 seconds"
    verify css:".balance"  number:changed-by  from:before  equals:-1  timeout:5s

• Targets are descriptors, written compactly: text:, role:+name:, label:, placeholder:, testid:, css:, xpath:. The runner expands each to a full descriptor (section 4.2) when it first resolves it.
• Commands (v1): start goto click doubleClick type press select check uncheck verify verifyNot wait sleep capture set drag switchTab closeTab screenshot call run (the full table is in design/script-format.md, which wins). call runs a TypeScript custom check; run includes another script.
• Waits are explicit (wait, a timeout: modifier on a command, or a timeout line in the file header). No hidden global sleeps.
• Every step has a name (the intent). A script without step names does not parse.

4.2 Element descriptor #

When a target resolves, the runner stores a descriptor beside the script (scripts/<area>/<name>.aite.lock.json), so the next run can find the element even if the text changed:

{ "text": "Send", "role": "button", "tag": "button", "attrs": { "aria-label": "...", "data-testid": null },
  "xpath": "/html/body/.../button[2]", "css": "...", "near": { "heading": "Send money" }, "seen": "2026-10-09T13:00:00Z" }   # sketch; design/data-formats.md has the full shape and wins

Repair order when the first locator fails: other attributes of the descriptor, then text similarity, then position near; if a candidate is found with confidence below a threshold the step fails as "locator drifted" with the proposed element outlined, instead of passing.

4.3 Messages (runner <-> extension), v1 #

JSON over the websocket, request/response with an id:

{ "id": 7, "cmd": "resolve", "target": { "text": "Send" }, "timeoutMs": 5000 }          -> { "id": 7, "ok": true, "descriptor": {...} }
{ "id": 8, "cmd": "act", "action": "click", "descriptor": {...} }                      -> { "id": 8, "ok": true }
{ "id": 9, "cmd": "verify", "kind": "text", "value": "1 MALLBUCKS", "within": "..." }  -> { "id": 9, "ok": true, "found": "1 MALLBUCKS" }

Failures carry { "ok": false, "reason": <one of the reasons listed in design/protocol.md>, "detail": {...} } (the design file lists all twelve and wins).

4.4 Result schema #

results/<run-id>/run.json: the shape is defined in design/data-formats.md (which wins): tests contain steps, steps contain commands, each with its status, reason, expected, found, timing, retries and screenshot path. The static report is generated from it.

4.5 The confirm loop (the human part) #

A failed step renders as:

x  step "review shows the right amount"   (test: send 1 MALLBUCKS to a handle, covers J-13)
   wanted:  verify text "1 MALLBUCKS" within "Review send"
   found:   "1 MUCK"
   [screenshot with the "Review send" box outlined]
   ( App is broken )  ( Script is out of date )  ( Flaky, retry )

• App is broken files a GitLab issue in the right project with the screenshot, the step, and the run link.
• Script is out of date hands the failure to the AI, which proposes a script diff for the human to approve.
• Flaky reruns the whole test three times from a clean pre-state and records the pass rate (a step with side effects is never repeated alone).

5. Repo layout #

aite/
  README.md                      docs/PLAN.md (this file)
  NOTICE                         credit to Google's BITE (Apache 2.0) for derived ideas/code
  LICENSE                        Apache 2.0 (matches BITE)
  reference/                     the few BITE files worth re-reading (copied, not built)
  extension/                     MV3 extension: manifest.json, src/{background,content,locator,actions}.ts
  runner/                        script parser, step engine, result writer (Node/TypeScript)
  bridge/                        local websocket server + CLI (`aite run scripts/...`)
  report/                        static report generator, confirm page
  scripts/                       the test scripts (mallbucks/, fixtures/)
  fixtures/                      tiny static pages used to test aite itself (including a deliberately broken one)
  docs/

6. Phases and checklist #

Each phase ends with something running and a check a person can do. Do one phase at a time.

Phase 0: housekeeping (half a day) #

• D8 decided 2026-10-09: leave the zips in place (recorded in docs/decisions.md).
• Add LICENSE (Apache 2.0) and NOTICE crediting BITE.
• Copy only the files named in section 2 into reference/ (about 10 files, under 1 MB) with a note of their original path.
• Create the TypeScript project skeleton: package.json, tsconfig.json, esbuild script, a unit test runner (vitest).
• GitLab CI: type check + unit tests on every push (the stores repo lacked this and its tests rotted).
• Done when: npm test passes in CI on an empty skeleton, and a deliberately failing test turns the pipeline red.
• The checklists in this file are summaries. docs/TASKS.md has the full task list per phase (50 tasks, including the spikes, hardening and test tasks) and wins on scope and acceptance.

Phase 1: walking skeleton (1 to 2 days) #

• MV3 extension that loads in current Chrome (chrome://extensions, developer mode) and shows its status.
• Bridge + websocket on 127.0.0.1 with a per-run token; extension connects from its service worker (and reconnects: service workers sleep).
• Two commands end to end: click and verify text, against fixtures/hello.html.
• Screenshots, taken by the runner over a private debugging pipe (the extension cannot: see protocol.md), saved in results/<run>/shots/.
• Done when: aite run fixtures/hello.aite clicks a button on the fixture page and reports a pass with a screenshot.

Phase 2: the script language (2 days) #

• Parser for the format in 4.1 with good errors (line numbers, "step names are required").
• Step engine: sequential steps, explicit wait/timeout, step status, retry counting.
• Full v1 command set from 4.1 implemented in the extension's content script.
• Unit tests for every command against fixture pages; a parser test table.
• Fixture pages for every command and a CI self-check job that loads the extension and runs them (TASKS.md P2-4).
• Done when: the whole command set passes on fixtures, and a malformed script gives a one-line error pointing to its line.

Phase 3: element descriptors and repair (2 days) #

• Descriptor capture on first resolve, saved to the .lock.json file next to the script.
• Repair order from 4.2, with a confidence score and a drifted failure.
• Fixture fixtures/drift.html: a page whose button text changes between runs; the test proves repair works and that low confidence fails instead of passing.
• Done when: a renamed button is found by repair and reported as "repaired (text changed)", not silently.

Phase 4: reporting and the confirm loop (2 days) #

• run.json writer and the static report.html (steps, intents, expected/found, screenshots, outlines).
• The three verdicts: "App is broken" creates a GitLab issue (title, screenshot, step, repro command); "Script is out of date" writes a repair request file the AI picks up; "Flaky" reruns the whole test three times from a clean pre-state.
• fixtures/broken.html: a deliberately broken version of the fixture; a run against it fails with a readable report.
• Done when: a person, given only the report, correctly says "app" or "script" for the broken and the drifted fixtures in under a minute each.

Phase 5: real session and test accounts (1 to 2 days, needs Alex) #

• Persistent Chrome profile for the runner on balrog (--user-data-dir), loading the extension unpacked.
• Dedicated Bluesky test accounts and a wallet; the profile signs in once by hand (OAuth consent screens are web pages, fine), then the saved session is reused.
• A session-check script that fails loudly when the saved session has expired, with instructions.
• Documented procedure for renewing a session.
• Done when: a script runs signed in as the test account with no human present, and session-check correctly fails when the saved session is deliberately cleared.

Phase 6: first real milestone, mallbucks journeys (3 to 4 days) #

• scripts/mallbucks/send.aite (J-09, J-13): search, send 1 MALLBUCKS, check the review wording and the balance change.
• scripts/mallbucks/gift.aite (J-36, J-37, J-39, J-40): create a gift, Share to Bluesky, check the post exists with a tappable link, open the claim page.
• scripts/mallbucks/shop.aite (J-24 to J-28): open a shop, buy, pay in the wallet pop-up (a second window), download the file.
• A negative run: point the scripts at a deliberately broken build (a text change the app really makes, for example a different amount wording; see docs/journeys.md) and confirm the report catches it.
• Done when: all three pass on production, the negative run fails at the right step, and Qian confirms the report is clearer than a written test log.

Phase 7: authoring workflow and scheduling (2 days) #

• Record mode (BITE's idea): record clicks in the live page to a draft script, with names left for the AI/human to fill in.
• aite author: Claude writes a script from a journey test description, runs it, fixes it until it passes or reports why it cannot.
• Run on a schedule on balrog (after each deploy and nightly); failures open or update one GitLab issue per script, not a flood.
• Done when: a new journey from docs/qa-journey-test-plan.md (in the mallbucks repo) becomes a passing script with no hand-editing beyond approving it.

7. How the AI writes and repairs scripts #

1. Author: given a journey (steps in words), write the script with a named step for each sentence; use text: and role: targets first (they survive restyling), testid: where the app has one; run it; fix until green or report the blocker.
2. Ask the app for stable hooks: when a target is fragile, propose adding a data-testid in the app repo rather than a long selector.
3. Repair: on a "script is out of date" verdict, read the failing step, the screenshot and the current page; change the smallest thing (a text, a target); rerun that test only; show the diff for approval. Never weaken a verify to make it pass.
4. Never invent expected values. Expected text comes from the product spec or QA plan, not from whatever the page currently says.

8. How aite itself is tested #

• Unit tests: parser, step engine, descriptor matching, report generation (vitest).
• Fixture pages cover every command, plus drift.html and broken.html.
• A "self-check" CI job that loads the extension in the pinned Chrome and runs the fixture scripts through aite run itself (decisions.md D13: not Playwright, so the launch path under test is the real one), so a broken extension build is caught before it reaches balrog. Each fixture script carries a # expect: line; the job runs each 20 times, and a second run with the extension broken on purpose must fail.

9. Security and accounts #

• The bridge listens on 127.0.0.1 only, with a per-run token; the bridge closes any connection that lacks it.
• Test accounts are dedicated throwaways with little value (the wallet is on Monad testnet). Their sessions live in the Chrome profile on balrog, never in git; the profile directory is mode 0700 and excluded from backups that leave the machine.
• Scripts never contain secrets. A needs session <name> line names a profile; the runner fails if it is missing.
• Screenshots can show addresses and handles. Reports stay on the machine; sharing is an explicit step.
• The extension is loaded unpacked into one profile; it is not published.

10. Risks #

Risk	Mitigation
MV3 service worker sleeps and drops the websocket	Reconnect with backoff; keep the websocket alive with a 15 s ping (a chrome.alarms heartbeat every 30 s is only the backup, see protocol.md); prove it in Phase 1 before building more
Locator drift makes the suite noisy	Repair loop in Phase 3, built early; drift is a distinct failure kind
OS-level prompts (Touch ID) cannot be answered by an extension	Use a test wallet created with a passphrase backup for the extension runner; passkey flows go to a Playwright backend later (D2)
The pay pop-up is a second window	Phase 6 covers switchTab; the extension already sees all tabs
Real accounts expire or get rate limited	session-check fails loudly; runs are serial and few
AI "fixes" a failing test by loosening it	Rule 7.3 and a CI check that flags removed verify lines in script diffs

11. Needs from Alex #

1. Approve or change D1 to D7 (the defaults above stand if you say nothing). D8 is decided; D9 and D10 are in docs/decisions.md.
2. Create or name the Bluesky test accounts (Phase 5) and say where their sessions may live.
3. Confirm balrog can run a headed Chrome (a display, or a virtual one such as Xvfb) for the runner.

12. Out of scope (for now) #

Bug tracking and crawling (BITE server features); a hosted multi-user dashboard; mobile browsers; load or performance testing; visual diffing of whole pages; headless CI for the signed-in flows (comes with the Playwright backend).