From cceead913edb35db97fc7df8392b2fd2e2e6999e Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Tue, 10 Nov 2026 23:44:12 -0500 Subject: [PATCH] Run the terminal on bash and ghostty-web, and keep dev servers running in it The terminal panel runs bashkit (wasm bash) over the kernel's filesystem and renders with ghostty-web. node, npm and npx launch kernel processes, and a foreground process gets a pty for keys and resizes. - Lines run under bashkit's interactive profile with no timeout, so a dev server or watcher outlives the default 30-second limit. - A project with dependencies runs npm install in a terminal when it opens. - import() and import.meta.resolve run registerHooks() resolve hooks, and a file: URL or path with a ?query loads as its own module, as Vite's module runner needs. - Typeahead runs one line at a time, so each queued line is its own command and its own history entry. --- docs/VISION.md | 4 +- packages/app/package.json | 2 + packages/app/playground/editor.ts | 4 + packages/app/src/launch.ts | 38 +- packages/app/src/run-in-terminal.ts | 32 ++ packages/app/src/shell/dom/keys.ts | 20 +- packages/app/src/terminal-session.ts | 502 ++++++++++++++++---- packages/app/src/terminal/bins.ts | 50 ++ packages/app/src/terminal/host-fs.ts | 72 +++ packages/app/src/terminal/line-editor.ts | 139 ++++++ packages/app/src/views/processes.ts | 18 +- packages/app/src/views/terminal.ts | 290 ++++++----- packages/app/tests/bins.test.ts | 19 + packages/app/tests/compact.spec.ts | 17 +- packages/app/tests/fixtures/terminal.ts | 12 + packages/app/tests/floating.spec.ts | 8 +- packages/app/tests/keyboard.spec.ts | 18 +- packages/app/tests/launch.test.ts | 12 +- packages/app/tests/line-editor.test.ts | 31 ++ packages/app/tests/processes.spec.ts | 17 +- packages/app/tests/run-in-terminal.test.ts | 12 + packages/app/tests/shell.spec.ts | 13 +- packages/app/tests/terminal.spec.ts | 98 +++- packages/app/tests/views.spec.ts | 52 +- packages/kernel/src/kernel/main.ts | 11 + packages/kernel/src/kernel/npm/store.ts | 2 +- packages/kernel/src/runtime/main.ts | 42 +- packages/kernel/src/runtime/module.ts | 54 ++- packages/kernel/src/shared/package-files.ts | 24 +- packages/kernel/src/shared/package-store.ts | 7 +- packages/kernel/src/sw/sw.ts | 10 +- packages/kernel/tests/builtins.spec.ts | 38 +- packages/kernel/tests/install.spec.ts | 8 +- pnpm-lock.yaml | 16 + 34 files changed, 1281 insertions(+), 411 deletions(-) create mode 100644 packages/app/src/run-in-terminal.ts create mode 100644 packages/app/src/terminal/bins.ts create mode 100644 packages/app/src/terminal/host-fs.ts create mode 100644 packages/app/src/terminal/line-editor.ts create mode 100644 packages/app/tests/bins.test.ts create mode 100644 packages/app/tests/fixtures/terminal.ts create mode 100644 packages/app/tests/line-editor.test.ts create mode 100644 packages/app/tests/run-in-terminal.test.ts diff --git a/docs/VISION.md b/docs/VISION.md index 7cd6ef5..86fd954 100644 --- a/docs/VISION.md +++ b/docs/VISION.md @@ -384,7 +384,7 @@ Code: `packages/app/src/shell/` (`layout.ts` is the model and reducer, `workbenc Still stubbed: - The stock views read an in-memory `Files` interface. The kernel filesystem and Loro docs (D16) replace it. The editor is a textarea until the CodeMirror and Loro binding lands. -- The terminal is a line shell with `ls`, `cat`, `open`, and `preview`, and `node`, `npm`, and `npx` as kernel processes with pipes, not a pty. Its session is headless (`src/terminal-session.ts`), so it outlives the panel. bashkit from `examples/slopbox` replaces it. +- The terminal is bash (bashkit) on the kernel's filesystem, drawn by ghostty-web. `node`, `npm`, and `npx` run as kernel processes, on a pty when one of them is the whole line; a package's bin runs through `node` by name, because bashkit reads no `#!` line (`src/terminal/bins.ts`); `open` and `preview` open panels. Its session is headless (`src/terminal-session.ts`), so it outlives the panel. - The agent panel appends messages to `.agents/.json` and calls no model (D8). - Sandbox-origin panels render in-page. The preview uses an opaque-origin `srcdoc` iframe as a stand-in for a sandbox origin (D3). The browser panel frames the kernel's page-relative port URL, so the guest page is same-origin with the shell until sandboxes get origins of their own. - The processes panel lists the kernel's process table nested by parent, with each port as a button that opens a browser panel and a Stop button, and runs the project's npm scripts in a terminal. A headless watcher (`src/auto-browser.ts`) opens a browser panel when a port starts listening, unless the person closed one for that port this page session. @@ -528,7 +528,7 @@ Inspiration: vlt's client asks a registry for `/-/vlt/capabilities` and `POST /- | Persistence | In-memory store | Sync format before OPFS (D5), built on Loro (D16) | | Resolver | One scheme check for `node:` | Add `nib:` | | Previews | Guest HTTP through the Service Worker and kernel port table | Per-sandbox origins (D3), `nib:browser` (D7) | -| Shell | bashkit in `examples/slopbox` | Same permission checks as code mode (D7) | +| Shell | bashkit in the app's terminal (and `examples/slopbox`) | Same permission checks as code mode (D7) | | App shell | `` over the layout tree and commands in `packages/app/src/shell` (D17); a compact touch layout with a coding key bar; installable PWA playground; the terminal, processes, and browser panels run the project in the kernel | Editor and explorer bound to the kernel and Loro (the mirror is one way); sandbox-origin panels | | Packages | `web` has design tokens and WAI-ARIA elements (tree, split, tabs, toolbar, menu); `docs` and `www` are empty | | | Lexicons | `@nibling/lexicons`: `run.nibling.*` records and queries (D18, D21, D22), vendored `org.tangled.*`, types generated by `@atcute/lex-cli` | Not published (`_lexicon.nibling.run`) | diff --git a/packages/app/package.json b/packages/app/package.json index f4da8c9..71f17bd 100644 --- a/packages/app/package.json +++ b/packages/app/package.json @@ -30,12 +30,14 @@ "@codemirror/search": "^6.7.2", "@codemirror/state": "^6.7.6", "@codemirror/view": "^6.43.15", + "@everruns/bashkit-wasm": "^0.20.3", "@lezer/common": "^1.5.5", "@lezer/highlight": "^1.2.5", "@lezer/markdown": "^1.8.1", "@nibling/kernel": "workspace:*", "@nibling/sdk": "workspace:*", "@nibling/web": "workspace:*", + "ghostty-web": "^0.4.0", "loro-codemirror": "0.3.3", "loro-crdt": "1.17.1" }, diff --git a/packages/app/playground/editor.ts b/packages/app/playground/editor.ts index bdacafe..caa4fd7 100644 --- a/packages/app/playground/editor.ts +++ b/packages/app/playground/editor.ts @@ -7,6 +7,7 @@ import { bindKeys } from "../src/shell/dom/keys.ts"; import "../src/shell/dom/shell.ts"; import { mainStack, parseLayout, Workbench, type Unsubscribe } from "../src/shell/index.ts"; import { openListeningPorts } from "../src/auto-browser.ts"; +import { needsInstall, runInTerminal } from "../src/run-in-terminal.ts"; import { sandbox } from "../src/sandbox.ts"; import { stockViews } from "../src/views/index.ts"; @@ -69,6 +70,9 @@ export function startEditor({ platform, header, ask, importRepo, demo, route }: box.mirror(files); disconnect(); disconnect = shell.connect(workbench, stockViews(files, box)); + // The kernel starts empty on every load, so a project with dependencies installs them where + // its output shows, as a fresh clone's first step would. + if (needsInstall(files)) runInTerminal(box, workbench, "npm install"); }; showFiles(); platform.subscribe(showFiles); diff --git a/packages/app/src/launch.ts b/packages/app/src/launch.ts index a928f44..f723255 100644 --- a/packages/app/src/launch.ts +++ b/packages/app/src/launch.ts @@ -6,23 +6,24 @@ */ export type Launch = | { readonly entry: string; readonly argv: readonly string[]; readonly source?: string } - | { readonly error: string }; + | { readonly error: string; readonly code: number }; let evals = 0; -export function launch([command, ...args]: readonly string[], cwd: string): Launch | undefined { +/** `nextEval` numbers the files `-e`, `-p`, and `-v` write, so two sessions' never collide. */ +export function launch([command, ...args]: readonly string[], cwd: string, nextEval: () => number = () => ++evals): Launch | undefined { if (command === "npm" || command === "npx") return { entry: `/usr/local/bin/${command}`, argv: args }; if (command !== "node") return undefined; const [first, ...rest] = args; const run = (source: string, argv: readonly string[]) => { const ext = /^\s*(?:import|export)\b/m.test(source) ? "mjs" : "cjs"; - return { entry: `${cwd}/.nib-eval-${++evals}.${ext}`, argv, source }; + return { entry: `${cwd}/.nib-eval-${nextEval()}.${ext}`, argv, source }; }; - if (first === undefined) return { error: "node: the REPL is not available in this terminal; pass a file or -e" }; + if (first === undefined) return { error: "node: the REPL is not available in this terminal; pass a file or -e", code: 1 }; if (first === "-v" || first === "--version") return run("console.log(process.version)", []); if (first === "-e" || first === "--eval") return run(rest[0] ?? "", rest.slice(1)); if (first === "-p" || first === "--print") return run(`console.log((${rest[0] ?? ""}))`, rest.slice(1)); - if (first.startsWith("-")) return { error: `node: bad option: ${first}` }; + if (first.startsWith("-")) return { error: `node: bad option: ${first}`, code: 9 }; return { entry: resolve(cwd, first), argv: rest }; } @@ -34,30 +35,3 @@ function resolve(cwd: string, path: string): string { } return `/${parts.join("/")}`; } - -/** Splits a command line into words as sh quotes them: single quotes keep everything, double quotes all but `\"` and `\\`. */ -export function words(line: string): string[] { - const out: string[] = []; - let word: string | undefined; - let quote: "'" | '"' | undefined; - for (let i = 0; i < line.length; i++) { - const c = line[i]!; - if (quote) { - if (c === quote) quote = undefined; - else if (quote === '"' && c === "\\" && (line[i + 1] === '"' || line[i + 1] === "\\")) word += line[++i]!; - else word += c; - } else if (c === "'" || c === '"') { - quote = c; - word ??= ""; - } else if (c === "\\" && i + 1 < line.length) { - word = (word ?? "") + line[++i]!; - } else if (/\s/.test(c)) { - if (word !== undefined) out.push(word); - word = undefined; - } else { - word = (word ?? "") + c; - } - } - if (word !== undefined) out.push(word); - return out; -} diff --git a/packages/app/src/run-in-terminal.ts b/packages/app/src/run-in-terminal.ts new file mode 100644 index 0000000..0984ade --- /dev/null +++ b/packages/app/src/run-in-terminal.ts @@ -0,0 +1,32 @@ +import type { Files } from "./files.ts"; +import type { Sandbox } from "./sandbox.ts"; +import { stackOf, type PanelId } from "./shell/layout.ts"; +import type { Workbench } from "./shell/workbench.ts"; + +/** + * Runs `line` in a terminal that has nothing in its foreground, opening a new one when none is idle. + * `keep`, the panel that asked, stays showing; the new terminal opens in its stack. + */ +export function runInTerminal(sandbox: Sandbox, workbench: Workbench, line: string, keep?: PanelId): void { + const idle = workbench.panels("terminal").find((panel) => !sandbox.terminal(panel.input.session, panel.input.cwd).busy); + const input = idle?.input ?? { cwd: "/", session: workbench.env.id() }; + if (!idle) { + const stack = keep && stackOf(workbench.layout.root, keep); + workbench.open({ kind: "terminal", input }, stack ? { stack: stack.id } : undefined); + if (keep) workbench.dispatch({ type: "activate", panel: keep }); + } + sandbox.terminal(input.session, input.cwd).run(line, workbench); +} + +/** Whether a project's package.json names any dependencies to install. */ +export function needsInstall(files: Files): boolean { + try { + const pkg = JSON.parse(files.read("package.json") ?? "null") as Record | null; + return ["dependencies", "devDependencies", "optionalDependencies"].some((field) => { + const deps = pkg?.[field]; + return typeof deps === "object" && deps !== null && Object.keys(deps).length > 0; + }); + } catch { + return false; + } +} diff --git a/packages/app/src/shell/dom/keys.ts b/packages/app/src/shell/dom/keys.ts index 12480e1..3c0a89a 100644 --- a/packages/app/src/shell/dom/keys.ts +++ b/packages/app/src/shell/dom/keys.ts @@ -1,9 +1,14 @@ import { commands as defaultCommands, type ShellCommand } from "../commands.ts"; import type { Unsubscribe, Workbench } from "../workbench.ts"; +/** The command tables bound on this page, so a view that reads every key can leave theirs alone. */ +const bound = new Set(); + +const isMac = () => /Mac|iPhone|iPad/.test(navigator.platform); + /** Runs shell commands from their keybindings. */ export function bindKeys(target: Window, workbench: Workbench, commands: readonly ShellCommand[] = defaultCommands): Unsubscribe { - const mac = /Mac|iPhone|iPad/.test(navigator.platform); + const mac = isMac(); const onKey = (event: KeyboardEvent) => { const match = commands.find((c) => c.keys && matches(c.keys, event, mac)); if (!match) return; @@ -11,7 +16,18 @@ export function bindKeys(target: Window, workbench: Workbench, commands: readonl match.run(workbench); }; target.addEventListener("keydown", onKey); - return () => target.removeEventListener("keydown", onKey); + bound.add(commands); + return () => { + target.removeEventListener("keydown", onKey); + bound.delete(commands); + }; +} + +/** Whether `event` is a bound command's keybinding. */ +export function isBound(event: KeyboardEvent): boolean { + const mac = isMac(); + for (const commands of bound) if (commands.some((c) => c.keys && matches(c.keys, event, mac))) return true; + return false; } /** Matches `Mod+Shift+E` against a key event by physical key, so Alt and Shift don't change the letter. */ diff --git a/packages/app/src/terminal-session.ts b/packages/app/src/terminal-session.ts index 277e046..2254f49 100644 --- a/packages/app/src/terminal-session.ts +++ b/packages/app/src/terminal-session.ts @@ -1,135 +1,453 @@ -import type { RunningProcess } from "@nibling/kernel"; -import { launch, words } from "./launch.ts"; -import { PROCESS_ENV, kernelPath, type Sandbox } from "./sandbox.ts"; +import type { Bash, CommandContext } from "@everruns/bashkit-wasm"; +import type { Kernel, PtyProcess, RunningProcess } from "@nibling/kernel"; +import { launch, type Launch } from "./launch.ts"; +import { PROCESS_ENV, PROJECT_ROOT, kernelPath, type Sandbox } from "./sandbox.ts"; import type { Unsubscribe, Workbench } from "./shell/workbench.ts"; +import { binFunctions, nodeBins } from "./terminal/bins.ts"; +import { hostFs } from "./terminal/host-fs.ts"; +import { initialLine, step } from "./terminal/line-editor.ts"; -/** A command that went to the kernel: its process once it has started, and the lines typed for its stdin before then. */ -interface Job { - proc?: RunningProcess; - stdin?: WritableStreamDefaultWriter; - readonly early: string[]; - interrupted: boolean; +const encoder = new TextEncoder(); + +/** The most output a session keeps for a view that connects later. */ +const TRANSCRIPT_LIMIT = 1 << 20; + +let bashkit: Promise | undefined; +/** bashkit and its wasm, loaded once and only when a terminal starts. */ +const loadBashkit = () => + (bashkit ??= Promise.all([import("@everruns/bashkit-wasm"), import("@everruns/bashkit-wasm/bashkit_wasm_bg.wasm?url")]).then(async ([module, wasm]) => { + await module.initBashkit(wasm.default); + return module; + })); + +/** + * A `\n` becomes `\r\n` unless a `\r` precedes it. pty output already carries its own, so this is a + * no-op there. It works on bytes so piped binary output passes through, carrying the last byte + * across chunks. + */ +function crlf() { + let lastCR = false; + return (bytes: Uint8Array) => { + const out: number[] = []; + for (let i = 0; i < bytes.length; i++) { + const b = bytes[i]!; + if (b === 0x0a && !(i === 0 ? lastCR : bytes[i - 1] === 0x0d)) out.push(0x0d); + out.push(b); + lastCR = b === 0x0d; + } + return new Uint8Array(out); + }; } -const encoder = new TextEncoder(); -// npm and node color their output on a TTY only, but a script can print escapes anyway, and this -// terminal draws text, not a screen. -const ESCAPES = /\x1b\[[0-?]*[ -/]*[@-~]/g; +/** + * `analyze` throws for input bash knows is unfinished (an open quote, an `if` with no `fi`, a + * trailing `$(`). It accepts three other unfinished forms, which the session measures itself: a + * trailing unescaped `\`, a trailing `&&` or `||`, and an open heredoc. + */ +const analyzeSaysIncomplete = (message: string) => /unexpected EOF|unexpected end of file|unterminated/i.test(message); + +function trailingBackslash(text: string): boolean { + let n = 0; + for (let i = text.length - 1; i >= 0 && text[i] === "\\"; i--) n++; + return n % 2 === 1; +} + +const trailingAndOr = (text: string) => /(?:&&|\|\|)\s*$/.test(text); + +function openHeredoc(text: string): boolean { + const lines = text.split("\n"); + for (const m of text.matchAll(/<<(-?)\s*(['"]?)([A-Za-z_]\w*)\2/g)) { + const start = text.slice(0, m.index).split("\n").length - 1; + if (!lines.slice(start + 1).some((line) => (m[1] ? line.replace(/^\t+/, "") : line) === m[3])) return true; + } + return false; +} + +/** A trailing `\` continues the line without a break; anything else (a heredoc, a trailing `&&`) keeps it. */ +const joinContinuation = (pending: string, line: string) => (trailingBackslash(pending) ? pending.slice(0, -1) + line : `${pending}\n${line}`); + +const LAUNCHED = new Set(["node", "npm", "npx"]); + +type State = + /** Booting the kernel and bash; keys wait as typeahead. */ + | { readonly tag: "starting" } + /** The sandbox didn't start, so there is no shell. */ + | { readonly tag: "dead" } + /** The line editor has the keys. */ + | { readonly tag: "idle" } + /** A line is running: keys wait as typeahead and ^C interrupts it. */ + | { readonly tag: "running" } + /** A process on the pty has the keys. */ + | { readonly tag: "foreground"; readonly proc: PtyProcess }; /** - * One terminal's line shell, headless so it outlives its view (D12): `ls`, `cat`, `open`, and `preview` - * over the project's files and the workbench, and `node`, `npm`, and `npx` as kernel processes. A - * process in the foreground gets the lines typed while it runs as stdin. bashkit replaces it. + * One terminal's bash (bashkit) on the kernel's filesystem, headless so it outlives its view (D12). + * `node`, `npm`, and `npx` run as kernel processes, a package's bin runs through `node` (see + * terminal/bins.ts), and `open` and `preview` open panels on the workbench a view attached. It + * speaks bytes both ways: `input` takes what a terminal emulator sends and `onOutput` gives what to + * draw. A line made of one launched command runs on a pty, so it gets colors, raw keys, and ^C. */ export class TerminalSession { - readonly history: string[] = []; - #text = ""; - #listeners = new Set<(chunk: string) => void>(); - #job: Job | undefined; - readonly sandbox: Sandbox; + /** Where the shell started, in the project's namespace. */ readonly cwd: string; + #chunks: Uint8Array[] = []; + #size = 0; + /** The last byte printed, so a prompt after output with no final newline starts on its own line. */ + #last = 0x0a; + #listeners = new Set<(chunk: Uint8Array) => void>(); + #workbench: Workbench | undefined; + #cols = 80; + #rows = 24; + + #state: State = { tag: "starting" }; + #bash: Bash | undefined; + #kernel: Kernel | undefined; + #live = new Set(); + #line = initialLine(); + #pending: string | null = null; + #typeahead = ""; + #foregroundLine = false; + #lastExit = 0; + #pwd: string; + #evals = 0; + /** The cursor's row within the prompt and line being edited, counted from the prompt's row. */ + #row = 0; + constructor(sandbox: Sandbox, cwd: string) { this.sandbox = sandbox; this.cwd = cwd; + this.#pwd = kernelPath(cwd) || "/"; + void this.#start(); } - /** Everything printed so far. */ - get text(): string { - return this.#text; + /** Everything printed so far, up to the last megabyte, for a view to replay. */ + get transcript(): readonly Uint8Array[] { + return this.#chunks; } - /** Whether a process is in the foreground, so a line typed now goes to its stdin. */ + /** Whether a line is running or a process has the keys. */ get busy(): boolean { - return this.#job !== undefined; + return this.#state.tag === "running" || this.#state.tag === "foreground"; } /** Calls `listener` with each chunk printed from now on. */ - onOutput(listener: (chunk: string) => void): Unsubscribe { + onOutput(listener: (chunk: Uint8Array) => void): Unsubscribe { this.#listeners.add(listener); return () => this.#listeners.delete(listener); } - submit(line: string, workbench: Workbench): void { - const job = this.#job; - if (job) { - this.#print(`${line}\n`); - if (job.stdin) void job.stdin.write(encoder.encode(`${line}\n`)); - else job.early.push(line); - return; + /** Where `open` and `preview` open their panels, until the returned function detaches it. */ + attach(workbench: Workbench): Unsubscribe { + this.#workbench = workbench; + return () => { + if (this.#workbench === workbench) this.#workbench = undefined; + }; + } + + /** The size of the terminal drawing this session, which a foreground process is told about. */ + resize(cols: number, rows: number): void { + if (cols === this.#cols && rows === this.#rows) return; + this.#cols = cols; + this.#rows = rows; + if (this.#state.tag === "foreground") this.#state.proc.pty.resize(cols, rows); + } + + /** Keys and pastes, as a terminal emulator sends them. */ + input(data: string): void { + const state = this.#state; + switch (state.tag) { + case "foreground": + return state.proc.pty.write(data); + case "running": + // Keys typed while a line runs wait for the next prompt, as a tty holds them for a shell + // that isn't reading; ^C drops them, as ISIG does. + if (!data.includes("\x03")) return void (this.#typeahead += data); + this.#typeahead = ""; + for (const proc of this.#live) proc.kill("SIGINT"); + return this.#bash?.cancel(); + case "starting": + return void (this.#typeahead += data); + case "dead": + return; + case "idle": + return this.#edit(data); } - this.#print(`${this.cwd} $ ${line}\n`); - this.history.push(line); - this.#run(words(line), workbench); } - /** Ctrl+C: SIGINT to the foreground process. */ - interrupt(): void { - const job = this.#job; - if (!job) return; - this.#print("^C\n"); - job.interrupted = true; - job.proc?.kill("SIGINT"); + /** + * Runs `line` as if it were typed at the prompt, replacing what was typed so far. `workbench` is + * where `open` and `preview` go until a view attaches its own. + */ + run(line: string, workbench: Workbench): void { + this.#workbench ??= workbench; + if (this.#state.tag === "idle") { + this.#edit("\x05"); + this.#edit("\x15"); + } + this.input(`${line}\r`); } - #print(chunk: string): void { - this.#text += chunk; - for (const listener of this.#listeners) listener(chunk); + /** Ends every process the session started. */ + dispose(): void { + for (const proc of this.#live) proc.kill("SIGKILL"); } - #run(argv: string[], workbench: Workbench): void { - const files = this.sandbox.files; - const [name, path = ""] = argv; - switch (name) { - case undefined: - return; - case "ls": - return this.#print(`${files.list().join("\n")}\n`); - case "cat": - return this.#print(`${files.read(path) ?? `cat: ${path}: No such file`}\n`); - case "open": - case "preview": - if (files.read(path) === undefined) return this.#print(`${name}: ${path}: No such file\n`); - return void workbench.open({ kind: name === "open" ? "editor" : "preview", input: { path } }); - } - const cwd = kernelPath(this.cwd); - const launched = launch(argv, cwd); - if (!launched) return this.#print(`${name}: command not found (try ls, cat, open, preview, node, npm, npx)\n`); - if ("error" in launched) return this.#print(`${launched.error}\n`); - const job: Job = { early: [], interrupted: false }; - this.#job = job; - void this.#exec(job, launched, cwd) - .catch((error: unknown) => this.#print(`${name}: ${error instanceof Error ? error.message : String(error)}\n`)) - .finally(() => { - if (this.#job === job) this.#job = undefined; - }); + #write(chunk: Uint8Array | string): void { + const bytes = typeof chunk === "string" ? encoder.encode(chunk) : chunk; + if (!bytes.length) return; + this.#chunks.push(bytes); + this.#size += bytes.length; + this.#last = bytes[bytes.length - 1]!; + while (this.#size > TRANSCRIPT_LIMIT && this.#chunks.length > 1) this.#size -= this.#chunks.shift()!.length; + for (const listener of this.#listeners) listener(bytes); } - async #exec(job: Job, { entry, argv, source }: { entry: string; argv: readonly string[]; source?: string }, cwd: string): Promise { - if (!this.sandbox.ready) this.#print("Starting sandbox…\n"); - let kernel; + async #start(): Promise { + const announced = !this.sandbox.ready; + if (announced) this.#write("Starting sandbox…\r\n"); try { - kernel = await this.sandbox.kernel; + const [kernel, { Bash }] = await Promise.all([this.sandbox.kernel, loadBashkit()]); + this.#kernel = kernel; + this.#bash = new Bash({ + // A terminal line runs as long as it runs (a dev server, a watcher); bashkit's default + // profile ends any line after 30 seconds. + profile: "interactive", + timeoutMs: 2 ** 31 - 1, + fs: hostFs(kernel.fs), + cwd: this.#pwd, + env: { ...PROCESS_ENV, TERM: "xterm-256color", COLORTERM: "truecolor" }, + commands: { + node: (ctx) => this.#launch(ctx), + npm: (ctx) => this.#launch(ctx), + npx: (ctx) => this.#launch(ctx), + open: (ctx) => this.#open(ctx, "editor"), + preview: (ctx) => this.#open(ctx, "preview"), + }, + }); } catch (error) { - return this.#print(`The sandbox did not start: ${error instanceof Error ? error.message : String(error)}\n`); + this.#state = { tag: "dead" }; + return this.#write(`The sandbox did not start: ${error instanceof Error ? error.message : String(error)}\r\n`); + } + if (announced) { + this.#write("\x1b[1A\x1b[2K"); + this.#last = 0x0a; } - if (job.interrupted) return; - if (source !== undefined) await kernel.applyFiles({ [entry]: source }); - const proc = kernel.exec(entry, { argv: [...argv], cwd, env: PROCESS_ENV }); - const stdin = proc.stdin.getWriter(); - Object.assign(job, { proc, stdin }); - if (job.interrupted) proc.kill("SIGINT"); - for (const line of job.early) void stdin.write(encoder.encode(`${line}\n`)); - await Promise.all([this.#pipe(proc.stdout), this.#pipe(proc.stderr), proc.exited]); - if (source !== undefined) await kernel.fs.unlink(entry).catch(() => {}); + this.#idle(); } - async #pipe(stream: ReadableStream): Promise { - const decoder = new TextDecoder(); - const reader = stream.getReader(); - for (let read = await reader.read(); !read.done; read = await reader.read()) { - this.#print(decoder.decode(read.value, { stream: true }).replace(ESCAPES, "")); + #idle(): void { + this.#state = { tag: "idle" }; + this.#row = 0; + // `printf hi`, or a ^C the pty echoed, leaves the cursor mid-line; the prompt redraws its own row. + if (this.#last !== 0x0a) this.#write("\r\n"); + this.#redraw(this.#line.buffer, this.#line.cursor); + // A tty hands the shell typeahead one line per read, so each queued line runs, and enters + // history, on its own; what follows a line that starts running waits for the next prompt. + let keys = this.#typeahead; + this.#typeahead = ""; + while (keys && this.#state.tag === "idle") { + const end = keys.indexOf("\r") + 1 || keys.length; + this.#edit(keys.slice(0, end)); + keys = keys.slice(end); } + this.#typeahead = keys + this.#typeahead; + } + + #prompt(): string { + if (this.#pending !== null) return "> "; + const shown = this.#pwd.replace(new RegExp(`^${PROJECT_ROOT}(?=/|$)`), "~"); + return `\x1b[2m${shown}\x1b[0m \x1b[32m$\x1b[0m `; + } + + /** + * Repaints the prompt and `echo` from the prompt's row, clearing what a longer line left, then puts + * the cursor `cursor` characters into `echo`. The line wraps at the terminal's width, so the + * cursor's row and column are worked out from it. + */ + #redraw(echo: string, cursor: number): void { + const cols = this.#cols; + const prompt = this.#prompt(); + const width = prompt.replace(/\x1b\[[\d;]*m/g, "").length; + const end = width + echo.length; + const at = width + cursor; + let out = `${this.#row ? `\x1b[${this.#row}A` : ""}\r\x1b[J${prompt}${echo}`; + // A line that fills its last row exactly leaves the cursor waiting at that row's end. + const full = end > 0 && end % cols === 0; + let row = full ? end / cols - 1 : Math.floor(end / cols); + if (at === end && full) { + out += "\r\n"; + row++; + } else if (at !== end) { + const target = Math.floor(at / cols); + out += `\r${row > target ? `\x1b[${row - target}A` : ""}${at % cols ? `\x1b[${at % cols}C` : ""}`; + row = target; + } + this.#row = row; + this.#write(out); + } + + #edit(data: string): void { + const result = step(this.#line, data); + this.#line = result.state; + if (result.clear) { + this.#write("\x1b[2J\x1b[H"); + this.#row = 0; + return this.#redraw(result.echo, result.state.cursor); + } + if (result.interrupt) { + this.#pending = null; + this.#row = 0; + return this.#write(`^C\r\n${this.#prompt()}`); + } + if (result.submit === undefined) return this.#redraw(result.echo, result.state.cursor); + // A paste or replayed typeahead submits text that was never drawn key by key. + const [first = "", ...more] = result.submit.split("\n"); + this.#redraw(first, first.length); + this.#write(more.map((line) => `\r\n> ${line}`).join("") + "\r\n"); + this.#row = 0; + this.#submit(result.submit); + // Text typed after the last line break of a paste stays on the new prompt. + if (this.#state.tag === "idle" && result.echo) this.#redraw(result.echo, result.state.cursor); + } + + #submit(line: string): void { + const bash = this.#bash!; + const candidate = this.#pending === null ? line : joinContinuation(this.#pending, line); + let names: string[] = []; + let functions: string[] = []; + let error: string | undefined; + try { + const analysis = bash.analyze(candidate); + names = analysis.commandNames; + functions = analysis.functions; + this.#foregroundLine = analysis.commands.length === 1 && !analysis.redirects.length && !analysis.hasCommandSubstitution; + } catch (e) { + const message = e instanceof Error ? e.message : String(e); + if (analyzeSaysIncomplete(message)) { + this.#pending = candidate; + return this.#write(this.#prompt()); + } + error = message; + } + if (error === undefined && (trailingBackslash(candidate) || trailingAndOr(candidate) || openHeredoc(candidate))) { + this.#pending = candidate; + return this.#write(this.#prompt()); + } + this.#pending = null; + // Reported from the line as typed, since the line that runs starts with `(exit n);`. + if (error !== undefined) { + this.#lastExit = 2; + return this.#write(`bash: ${error}\r\n${this.#prompt()}`); + } + this.#state = { tag: "running" }; + void this.#run(candidate, names, functions).finally(() => this.#idle()); + } + + async #run(text: string, names: string[], functions: string[]): Promise { + const bash = this.#bash!; + const out = crlf(); + const err = crlf(); + const candidates = names.filter((name) => !LAUNCHED.has(name) && !functions.includes(name)); + const bins = candidates.length ? await nodeBins(this.#kernel!.fs, candidates, await this.#env("PATH"), this.#pwd) : new Map(); + // A lone command that is a package's bin runs on the pty, as node itself would. + if (this.#foregroundLine) this.#foregroundLine = LAUNCHED.has(names[0] ?? "") || bins.has(names[0] ?? ""); + bash.clearCancel(); + try { + if (bins.size) await bash.execute(binFunctions(bins), { capture: 0 }); + // Each execute starts `$?` at 0, so the line is seeded with the status the last one left. + const result = await bash.execute(`(exit ${this.#lastExit}); ${text}`, { + stdout: (bytes: Uint8Array) => this.#write(out(bytes)), + stderr: (bytes: Uint8Array) => this.#write(err(bytes)), + capture: 0, + }); + this.#lastExit = result.exitCode; + } catch (e) { + this.#write(err(encoder.encode(`bash: ${e instanceof Error ? e.message : String(e)}\n`))); + this.#lastExit = 2; + } finally { + if (bins.size) await bash.execute(`unset -f ${[...bins.keys()].join(" ")}`, { capture: 0 }).catch(() => {}); + } + this.#pwd = (await this.#env("PWD")) || this.#pwd; + } + + async #env(name: string): Promise { + const result = await this.#bash!.execute(`printf %s "$${name}"`, { capture: 1 << 16 }); + return result.stdout; + } + + async #launch(ctx: CommandContext): Promise { + const launched: Launch | undefined = launch([ctx.name, ...ctx.argv], ctx.cwd, () => ++this.#evals); + if (!launched) return 127; + if ("error" in launched) { + await ctx.stderr.write(encoder.encode(`${launched.error}\n`)); + return launched.code; + } + const kernel = this.#kernel!; + if (launched.source !== undefined) await kernel.fs.writeFile(launched.entry, launched.source); + try { + return await this.#exec(kernel, ctx, launched.entry, [...launched.argv]); + } finally { + if (launched.source !== undefined) await kernel.fs.unlink(launched.entry).catch(() => {}); + } + } + + async #exec(kernel: Kernel, ctx: CommandContext, entry: string, argv: string[]): Promise { + const options = { argv, cwd: ctx.cwd, env: { ...ctx.env } }; + if (this.#foregroundLine) { + // One process per line gets the pty; a second launch on the same line (a bin that runs node) pipes. + this.#foregroundLine = false; + const proc = kernel.exec(entry, { ...options, pty: { cols: this.#cols, rows: this.#rows } }); + this.#live.add(proc); + this.#state = { tag: "foreground", proc }; + try { + await pump(proc.stdout, ctx.stdout); + return await proc.exited; + } finally { + this.#live.delete(proc); + this.#state = { tag: "running" }; + } + } + const proc = kernel.exec(entry, options); + this.#live.add(proc); + const stdin = (async () => { + const writer = proc.stdin.getWriter(); + for await (const chunk of ctx.stdin) await writer.write(chunk); + await writer.close().catch(() => {}); + })(); + try { + await Promise.all([pump(proc.stdout, ctx.stdout), pump(proc.stderr, ctx.stderr)]); + return await proc.exited; + } finally { + stdin.catch(() => {}); + this.#live.delete(proc); + } + } + + async #open(ctx: CommandContext, kind: "editor" | "preview"): Promise { + const [target] = ctx.argv; + const fail = async (message: string) => (await ctx.stderr.write(encoder.encode(`${ctx.name}: ${message}\n`)), 1); + if (!target) return fail("usage: " + ctx.name + " "); + const absolute = resolve(ctx.cwd, target); + const path = absolute.startsWith(`${PROJECT_ROOT}/`) ? absolute.slice(PROJECT_ROOT.length + 1) : undefined; + if (path === undefined || this.sandbox.files.read(path) === undefined) return fail(`${target}: No such file in the project`); + if (!this.#workbench) return fail("no workbench to open it on"); + this.#workbench.open({ kind, input: { path } }); + return 0; + } +} + +async function pump(stream: ReadableStream, sink: { write(bytes: Uint8Array): Promise }): Promise { + const reader = stream.getReader(); + for (let read = await reader.read(); !read.done; read = await reader.read()) await sink.write(read.value); +} + +function resolve(cwd: string, path: string): string { + const parts: string[] = []; + for (const part of (path.startsWith("/") ? path : `${cwd}/${path}`).split("/")) { + if (part === "..") parts.pop(); + else if (part && part !== ".") parts.push(part); } + return `/${parts.join("/")}`; } diff --git a/packages/app/src/terminal/bins.ts b/packages/app/src/terminal/bins.ts new file mode 100644 index 0000000..2c8b8b6 --- /dev/null +++ b/packages/app/src/terminal/bins.ts @@ -0,0 +1,50 @@ +import type { HostFs } from "@nibling/kernel"; + +/** + * bashkit finds a command on PATH but runs every file it finds as a bash script; it reads no `#!` + * line. A package's bin is a node script, so for each command a line names that resolves on PATH + * to a file whose `#!` names node, the session defines a function that runs it through `node` + * first. `astro dev` then reaches the kernel the way `node node_modules/.bin/astro dev` does. + */ + +const decoder = new TextDecoder(); + +/** Whether a script's first line asks for node: `#!/usr/bin/env node`, `#!/usr/bin/env -S node --flag`, or a node binary's path. */ +export function wantsNode(head: string): boolean { + const shebang = /^#!\s*(\S+)(.*)$/m.exec(head.split("\n", 1)[0] ?? ""); + if (!shebang) return false; + const [, interpreter, rest] = shebang; + const program = interpreter!.endsWith("/env") ? rest!.trim().split(/\s+/).find((word) => !word.startsWith("-") && !word.includes("=")) : interpreter; + return program?.split("/").pop() === "node"; +} + +const BASH_NAME = /^[A-Za-z_][\w.+-]*$/; + +/** + * The node scripts among `names`, each by the path PATH resolves it to. A name with a `/` is not + * looked up, and neither is one that isn't a plain word bash accepts as a function name. + */ +export async function nodeBins(fs: HostFs, names: Iterable, path: string, cwd: string): Promise> { + const dirs = path.split(":").map((dir) => (dir === "" || dir === "." ? cwd : dir.startsWith("/") ? dir : `${cwd}/${dir}`)); + const found = new Map(); + await Promise.all( + [...new Set(names)].filter((name) => BASH_NAME.test(name)).map(async (name) => { + for (const dir of dirs) { + const file = `${dir}/${name}`; + const st = await fs.stat(file).catch(() => undefined); + if (st?.kind !== "file") continue; + const head = await fs.readFile(file).then((bytes) => decoder.decode(bytes.subarray(0, 256)), () => ""); + if (wantsNode(head)) found.set(name, file); + // The first file on PATH wins, as bash would run it, node script or not. + return; + } + }), + ); + return found; +} + +/** Shell-quotes `word` for a function body. */ +const quote = (word: string) => `'${word.replaceAll("'", `'\\''`)}'`; + +/** The function definitions for `bins`, one per line, ending in a newline when there are any. */ +export const binFunctions = (bins: ReadonlyMap) => [...bins].map(([name, file]) => `${name}() { node ${quote(file)} "$@"; }\n`).join(""); diff --git a/packages/app/src/terminal/host-fs.ts b/packages/app/src/terminal/host-fs.ts new file mode 100644 index 0000000..69d33f6 --- /dev/null +++ b/packages/app/src/terminal/host-fs.ts @@ -0,0 +1,72 @@ +import type { HostFileSystem } from "@everruns/bashkit-wasm"; +import type { HostFs } from "@nibling/kernel"; + +// readdir reports a link as itself; stat follows it, so it only ever sees a file or a directory. +const kind = (k: string): "file" | "directory" | "symlink" => (k === "dir" ? "directory" : k === "symlink" ? "symlink" : "file"); + +/** What bash prints for each errno the kernel rejects with, as strerror words it. */ +const STRERROR: Record = { + ENOENT: "No such file or directory", + EEXIST: "File exists", + ENOTDIR: "Not a directory", + EISDIR: "Is a directory", + ENOTEMPTY: "Directory not empty", + EACCES: "Permission denied", + EPERM: "Operation not permitted", + ELOOP: "Too many levels of symbolic links", + EINVAL: "Invalid argument", +}; + +/** + * The kernel's rejections carry a Node `code` but cross a worker boundary on the way here, and their + * message is Node's (`ENOENT: no such file or directory, open '/app/x'`). Rewrapped as a plain + * Error with bash's words, `cat x` prints `cat: x: No such file or directory`. + */ +function wrap(error: unknown): Error { + const e = error as { message?: string; code?: string }; + const code = e.code ?? "EIO"; + return Object.assign(new Error(STRERROR[code] ?? e.message ?? String(error)), { code }); +} + +const guard = + (fn: (...args: A) => Promise) => + async (...args: A): Promise => { + try { + return await fn(...args); + } catch (error) { + throw wrap(error); + } + }; + +/** The kernel's filesystem as bashkit's, so bash sees the files the project's processes do. */ +export function hostFs(fs: HostFs): HostFileSystem { + return { + read: guard((path) => fs.readFile(path)), + write: guard((path, content) => fs.writeFile(path, content)), + mkdir: guard((path, recursive) => fs.mkdir(path, { recursive })), + // A directory goes with rm (recursively, when asked); a file unlinks. + remove: guard(async (path, recursive) => { + const st = await fs.stat(path); + if (st.kind === "dir") await fs.rm(path, { recursive }); + else await fs.unlink(path); + }), + stat: guard(async (path) => { + const st = await fs.stat(path); + return { type: kind(st.kind), size: st.size, mode: st.mode, mtimeMs: st.mtimeMs }; + }), + readDir: guard(async (path) => { + const entries = await fs.readdir(path); + return Promise.all( + entries.map(async (d) => { + const st = await fs.stat(`${path === "/" ? "" : path}/${d.name}`).catch(() => undefined); + return { name: d.name, type: kind(d.kind), size: st?.size, mode: st?.mode, mtimeMs: st?.mtimeMs }; + }), + ); + }), + // A plain miss must not throw. + exists: (path) => fs.stat(path).then(() => true, () => false), + rename: guard((from, to) => fs.rename(from, to)), + symlink: guard((target, link) => fs.symlink(target, link)), + readLink: guard((path) => fs.readlink(path)), + }; +} diff --git a/packages/app/src/terminal/line-editor.ts b/packages/app/src/terminal/line-editor.ts new file mode 100644 index 0000000..675ad3e --- /dev/null +++ b/packages/app/src/terminal/line-editor.ts @@ -0,0 +1,139 @@ +/** + * A pure reducer over terminal input, with no DOM, bash, or I/O: the session drives each `onData` + * chunk through `step` and writes the result. + * + * Redraw contract: `echo` always equals the returned `state.buffer`, the full text after the prompt. + * The caller repaints with `"\r" + prompt + echo`, then moves the cursor left by + * `echo.length - state.cursor` columns. Redrawing the whole line on every key, rather than diffing, + * is simple and correct for a one-line prompt no wider than the terminal. + */ +export interface LineState { + readonly buffer: string; + readonly cursor: number; + readonly history: readonly string[]; + /** The history entry shown, or null while editing a fresh line. */ + readonly recall: number | null; + /** The fresh line, kept while walking the history. */ + readonly draft: string; +} + +export interface Step { + readonly state: LineState; + readonly echo: string; + readonly submit?: string; + readonly interrupt?: true; + readonly clear?: true; +} + +type Edit = { state: LineState; submit?: string; interrupt?: true }; + +export const initialLine = (history: readonly string[] = []): LineState => ({ buffer: "", cursor: 0, history, recall: null, draft: "" }); + +/** Non-blank, and different from the most recent entry. */ +const pushHistory = (history: readonly string[], line: string) => + line.trim() !== "" && history[history.length - 1] !== line ? [...history, line] : history; + +const submitLine = (state: LineState): Edit => ({ state: initialLine(pushHistory(state.history, state.buffer)), submit: state.buffer }); + +/** + * A pasted chunk carrying `\r` is one or more finished lines and then an unfinished one. Everything + * up to the last `\r` submits as one script, internal `\r`s becoming `\n` so a pasted heredoc keeps + * its line breaks, and the rest becomes the new buffer. + */ +function pasteSubmit(state: LineState, data: string): Edit { + const combined = state.buffer.slice(0, state.cursor) + data + state.buffer.slice(state.cursor); + const lastCr = combined.lastIndexOf("\r"); + const script = combined.slice(0, lastCr).replace(/\r/g, "\n"); + const remainder = combined.slice(lastCr + 1); + return { state: { ...initialLine(pushHistory(state.history, script)), buffer: remainder, cursor: remainder.length }, submit: script }; +} + +function insertText(state: LineState, text: string): Edit { + const buffer = state.buffer.slice(0, state.cursor) + text + state.buffer.slice(state.cursor); + return { state: { ...state, buffer, cursor: state.cursor + text.length } }; +} + +function backspace(state: LineState): Edit { + if (state.cursor === 0) return { state }; + return { state: { ...state, buffer: state.buffer.slice(0, state.cursor - 1) + state.buffer.slice(state.cursor), cursor: state.cursor - 1 } }; +} + +const deleteForward = (state: LineState): Edit => ({ state: { ...state, buffer: state.buffer.slice(0, state.cursor) + state.buffer.slice(state.cursor + 1) } }); +const moveLeft = (state: LineState): Edit => ({ state: { ...state, cursor: Math.max(0, state.cursor - 1) } }); +const moveRight = (state: LineState): Edit => ({ state: { ...state, cursor: Math.min(state.buffer.length, state.cursor + 1) } }); +const moveHome = (state: LineState): Edit => ({ state: { ...state, cursor: 0 } }); +const moveEnd = (state: LineState): Edit => ({ state: { ...state, cursor: state.buffer.length } }); +const killToStart = (state: LineState): Edit => ({ state: { ...state, buffer: state.buffer.slice(state.cursor), cursor: 0 } }); +const killToEnd = (state: LineState): Edit => ({ state: { ...state, buffer: state.buffer.slice(0, state.cursor) } }); +const interruptLine = (state: LineState): Edit => ({ state: initialLine(state.history), interrupt: true }); +// ^L changes nothing about the line; `step` turns it into `clear`. +const clearScreen = (state: LineState): Edit => ({ state }); + +/** Deletes the word behind the cursor: trailing whitespace, then the run of non-whitespace before it. */ +function killPrevWord(state: LineState): Edit { + const b = state.buffer; + let i = state.cursor; + while (i > 0 && /\s/.test(b[i - 1]!)) i--; + while (i > 0 && !/\s/.test(b[i - 1]!)) i--; + return { state: { ...state, buffer: b.slice(0, i) + b.slice(state.cursor), cursor: i } }; +} + +/** Up and Down walk the history, keeping the line being edited as `draft` on the way in and restoring it on the way out. */ +function historyUp(state: LineState): Edit { + if (state.history.length === 0 || state.recall === 0) return { state }; + const recall = state.recall === null ? state.history.length - 1 : state.recall - 1; + const buffer = state.history[recall]!; + return { state: { ...state, buffer, cursor: buffer.length, recall, draft: state.recall === null ? state.buffer : state.draft } }; +} + +function historyDown(state: LineState): Edit { + if (state.recall === null) return { state }; + if (state.recall < state.history.length - 1) { + const recall = state.recall + 1; + const buffer = state.history[recall]!; + return { state: { ...state, buffer, cursor: buffer.length, recall } }; + } + return { state: { ...state, buffer: state.draft, cursor: state.draft.length, recall: null, draft: "" } }; +} + +/** Exact single-key sequences. A paste (several characters with a `\r`) and printable text are handled in `step`. */ +const KEYS: Record Edit> = { + "\r": submitLine, + "\x7f": backspace, + "\x1b[3~": deleteForward, + "\x1b[D": moveLeft, + "\x1b[C": moveRight, + "\x1b[H": moveHome, + "\x1b[F": moveEnd, + "\x01": moveHome, + "\x05": moveEnd, + "\x1b[A": historyUp, + "\x1b[B": historyDown, + "\x03": interruptLine, + "\x15": killToStart, + "\x0b": killToEnd, + "\x17": killPrevWord, + "\x0c": clearScreen, +}; + +function isPrintable(data: string): boolean { + if (!data) return false; + for (const ch of data) { + const code = ch.codePointAt(0)!; + if (code < 0x20 || code === 0x7f) return false; + } + return true; +} + +/** One `onData` chunk applied to the line. An escape sequence this editor doesn't model is ignored. */ +export function step(state: LineState, data: string): Step { + const key = KEYS[data]; + const edit: Edit = data.length > 1 && data.includes("\r") ? pasteSubmit(state, data) : key ? key(state) : isPrintable(data) ? insertText(state, data) : { state }; + return { + state: edit.state, + echo: edit.state.buffer, + ...(edit.submit !== undefined ? { submit: edit.submit } : {}), + ...(edit.interrupt ? { interrupt: true as const } : {}), + ...(data === "\x0c" ? { clear: true as const } : {}), + }; +} diff --git a/packages/app/src/views/processes.ts b/packages/app/src/views/processes.ts index 1ebe6ce..6bba9a3 100644 --- a/packages/app/src/views/processes.ts +++ b/packages/app/src/views/processes.ts @@ -1,8 +1,9 @@ import { adopt, define, sheet } from "@nibling/web/internals"; import type { ProcessInfo } from "@nibling/kernel"; import { commandLine, localUrl, processTree, projectScripts, type ProcessNode } from "../process-table.ts"; +import { runInTerminal } from "../run-in-terminal.ts"; +import type { PanelId } from "../shell/layout.ts"; import type { Sandbox } from "../sandbox.ts"; -import { stackOf, type PanelId } from "../shell/layout.ts"; import type { Unsubscribe, Workbench } from "../shell/workbench.ts"; import { openBrowser } from "./browser.ts"; @@ -142,21 +143,6 @@ export class ProcessesElement extends HTMLElement { } } -/** - * Runs `line` in a terminal that has nothing in its foreground, opening a new one when none is idle, - * and keeps `keep` (the panel that asked) showing. - */ -function runInTerminal(sandbox: Sandbox, workbench: Workbench, line: string, keep: PanelId): void { - const idle = workbench.panels("terminal").find((panel) => !sandbox.terminal(panel.input.session, panel.input.cwd).busy); - const input = idle?.input ?? { cwd: "/", session: workbench.env.id() }; - if (!idle) { - const stack = stackOf(workbench.layout.root, keep); - workbench.open({ kind: "terminal", input }, stack ? { stack: stack.id } : undefined); - workbench.dispatch({ type: "activate", panel: keep }); - } - sandbox.terminal(input.session, input.cwd).submit(line, workbench); -} - declare global { interface HTMLElementTagNameMap { "nib-processes": ProcessesElement; diff --git a/packages/app/src/views/terminal.ts b/packages/app/src/views/terminal.ts index fdc75aa..6039e9f 100644 --- a/packages/app/src/views/terminal.ts +++ b/packages/app/src/views/terminal.ts @@ -1,160 +1,230 @@ import { adopt, define, sheet } from "@nibling/web/internals"; +import type { ITheme, Terminal } from "ghostty-web"; import type { TerminalSession } from "../terminal-session.ts"; -import { codeField, history, insertAtCaret, moveCaret, type CaretMove, type EditingTarget } from "./editing.ts"; +import { codeField, type CaretMove, type EditingTarget } from "./editing.ts"; import type { Language } from "./keys.ts"; +import { isBound } from "../shell/dom/keys.ts"; import type { Unsubscribe, Workbench } from "../shell/workbench.ts"; const terminalSheet = sheet(` - :host { display: flex; flex-direction: column; overflow: hidden; container: nib-terminal / size; } - [part="output"] { - flex: 1; - margin: 0; - padding: var(--nib-space-sm); - overflow: auto; - font: var(--nib-font-terminal); - background: var(--nib-color-canvas); - color: var(--nib-color-text); - cursor: text; - } - form { - display: flex; - gap: var(--nib-space-xs); - padding: var(--nib-space-xs); - background: var(--nib-color-surface); - border-top: var(--nib-size-border) solid var(--nib-color-border); - } - input { - flex: 1; - min-width: 0; - box-sizing: border-box; - font: var(--nib-font-terminal); - border: var(--nib-size-border) solid var(--nib-color-border); - background: var(--nib-color-canvas); - color: var(--nib-color-text); - } - input:focus-visible { outline: var(--nib-size-focus-ring) solid var(--nib-color-focus-ring); outline-offset: calc(-1 * var(--nib-size-focus-ring)); } - /* A narrow pane (the host's own size, not the viewport) drops a font step for more columns. */ - @container nib-terminal (max-width: 240px) { - [part="output"], input { font-size: var(--nib-font-size-compact); } - } + :host { display: flex; flex-direction: column; overflow: hidden; background: var(--nib-color-canvas); } + [part="screen"] { flex: 1; min-height: 0; padding: var(--nib-space-sm); box-sizing: border-box; cursor: text; } + /* What the screen shows, as text, for assistive technology; the canvas has none. */ + [role="log"] { position: absolute; width: 1px; height: 1px; overflow: hidden; clip-path: inset(50%); white-space: pre; } + [role="log"] > div { min-height: 1lh; } + /* Resolves the design tokens the emulator's theme and font are made from. */ + [part="probe"] { position: absolute; visibility: hidden; font: var(--nib-font-terminal); color: var(--nib-color-text); background: var(--nib-color-canvas); outline-color: var(--nib-color-accent); } `); +/** The rows the text mirror keeps, from the bottom of the scrollback. */ +const MIRROR_ROWS = 500; + +const SEQUENCES: Record = { left: "\x1b[D", right: "\x1b[C", up: "\x1b[A", down: "\x1b[B" }; + +let ghostty: Promise | undefined; +/** ghostty-web and its wasm, loaded once and only when a terminal is shown. */ +const loadGhostty = () => + (ghostty ??= import("ghostty-web").then(async (module) => { + await module.init(); + return module; + })); + /** - * The view of a terminal session (src/terminal-session.ts): its output, and a line to type into it. - * Ctrl+C interrupts the session's foreground process. + * The view of a terminal session (src/terminal-session.ts): a ghostty-web emulator drawing the + * session's output and sending it the keys. A mirror of the screen's text is a log for assistive + * technology, since the emulator draws to a canvas. */ export class TerminalElement extends HTMLElement implements EditingTarget { - readonly #output: HTMLPreElement; - readonly #input: HTMLInputElement; - #history: string[] = []; - #cursor = 0; - #submit: ((line: string) => void) | undefined; + readonly #screen: HTMLDivElement; + readonly #log: HTMLDivElement; + readonly #probe: HTMLSpanElement; + #term: Terminal | undefined; #session: TerminalSession | undefined; + #mirrorQueued = false; + /** Focus asked for before the emulator loaded, which the input takes once it has, if focus is still here. */ + #focusPending = false; constructor() { super(); const root = this.attachShadow({ mode: "open", delegatesFocus: true }); adopt(root, terminalSheet); - this.#output = document.createElement("pre"); - this.#output.part = "output"; - const form = document.createElement("form"); - this.#input = document.createElement("input"); - codeField(this.#input); - form.append(this.#input); - root.append(this.#output, form); - // A click anywhere in the output moves focus to the input, so reading output never strands you. - this.#output.addEventListener("click", () => this.#input.focus()); - form.addEventListener("submit", (event) => { - event.preventDefault(); - // An empty line still reaches a running process's stdin, as Enter does in a terminal. - if (!this.#input.value && !this.#session?.busy) return; - this.#submit?.(this.#input.value); - }); - this.#input.addEventListener("keydown", (event) => { - const input = this.#input; - // Ctrl+C copies on Windows and Linux, so it interrupts only with nothing selected to copy. - if (event.ctrlKey && !event.metaKey && !event.altKey && event.key.toLowerCase() === "c" && this.#session?.busy && input.selectionStart === input.selectionEnd) { - event.preventDefault(); - this.#session.interrupt(); - } else if (event.key === "ArrowUp") { - event.preventDefault(); - this.#recall(-1); - } else if (event.key === "ArrowDown") { - event.preventDefault(); - this.#recall(1); - } - }); + this.#screen = document.createElement("div"); + this.#screen.part = "screen"; + // Focusable but not in the tab order, so focus asked for before the emulator loads waits here. + this.#screen.tabIndex = -1; + this.#log = document.createElement("div"); + this.#probe = document.createElement("span"); + this.#probe.part = "probe"; + root.append(this.#screen, this.#log, this.#probe); } connectedCallback(): void { // role="log" implies aria-live="polite", so output narrates as it's printed. - this.#output.setAttribute("role", "log"); - this.#output.setAttribute("aria-label", "Terminal output"); - this.#input.setAttribute("aria-label", "Terminal input"); + this.#log.setAttribute("role", "log"); + this.#log.setAttribute("aria-label", "Terminal output"); } get language(): Language { return "shell"; } + override focus(options?: FocusOptions): void { + if (this.#term?.textarea) return this.#term.textarea.focus(options); + this.#focusPending = true; + super.focus(options); + } + override blur(): void { - this.#input.blur(); + this.#term?.textarea?.blur(); } + /** Types `text`, leaving the cursor `caret` characters into it (default: after it). */ insert(text: string, caret?: number): void { - insertAtCaret(this.#input, text, caret); + this.#session?.input(text + SEQUENCES.left.repeat(caret === undefined ? 0 : Math.max(0, text.length - caret))); } - /** Left and right move the caret; up and down walk the command history, as the arrow keys do. */ + /** The arrow keys, as the terminal sends them: left and right move the cursor, up and down walk the history. */ move(direction: CaretMove): void { - if (direction === "up" || direction === "down") { - this.#input.focus(); - return this.#recall(direction === "up" ? -1 : 1); - } - moveCaret(this.#input, direction); + this.#session?.input(SEQUENCES[direction]); } - // Tab completion belongs to the real shell (bashkit); until then Tab inserts spaces. indent(): void { - insertAtCaret(this.#input, " "); - } - - undo(): void { - history(this.#input, "undo"); - } - - redo(): void { - history(this.#input, "redo"); + this.#session?.input("\t"); } - #recall(direction: -1 | 1): void { - if (!this.#history.length) return; - this.#cursor = Math.min(Math.max(this.#cursor + direction, 0), this.#history.length); - this.#input.value = this.#history[this.#cursor] ?? ""; - } + // The shell's line editor has no undo. + undo(): void {} + redo(): void {} connect(session: TerminalSession, workbench: Workbench): Unsubscribe { this.#session = session; - this.#history = session.history; - this.#cursor = this.#history.length; - const print = (chunk: string) => { - this.#output.append(chunk); - this.#output.scrollTop = this.#output.scrollHeight; + const detach = session.attach(workbench); + let disposed = false; + let teardown = () => {}; + void loadGhostty().then((ghostty) => { + if (disposed) return; + let unmount = this.#mount(ghostty, session); + // Only if focus stayed here while the emulator loaded; it may have moved on since. + if (this.#focusPending && this.matches(":focus-within")) this.#term?.textarea?.focus(); + this.#focusPending = false; + // ghostty-web bakes its colors in when it opens, so a new color scheme gets a new emulator, + // drawn from the session's transcript. + const scheme = matchMedia("(prefers-color-scheme: dark)"); + const restyle = () => { + const focused = this.matches(":focus-within"); + unmount(); + this.#log.replaceChildren(); + unmount = this.#mount(ghostty, session); + if (focused) this.#term?.textarea?.focus(); + }; + scheme.addEventListener("change", restyle); + teardown = () => { + scheme.removeEventListener("change", restyle); + unmount(); + }; + }); + return () => { + disposed = true; + this.#focusPending = false; + teardown(); + detach(); + this.#session = undefined; }; - this.#output.replaceChildren(); - print(session.text); - const unsubscribe = session.onOutput(print); - this.#submit = (line) => { - session.submit(line, workbench); - this.#cursor = this.#history.length; - this.#input.value = ""; + } + + /** An emulator on `session`, drawing what it has printed so far and then what it prints. Returns its teardown. */ + #mount({ Terminal, FitAddon }: typeof import("ghostty-web"), session: TerminalSession): () => void { + const term = new Terminal({ ...this.#look(), cursorBlink: true, scrollback: 5000 }); + const fit = new FitAddon(); + term.loadAddon(fit); + // open() ends by focusing the terminal, which would take focus from wherever it is. + term.focus = () => {}; + term.open(this.#screen); + delete (term as { focus?: unknown }).focus; + // ghostty-web makes its container a labeled contenteditable textbox and adds a labeled textarea + // that pointer input focuses. The textarea alone is the input; keys bubble from it to the + // container's handler either way. + for (const name of ["contenteditable", "role", "aria-label", "aria-multiline"]) this.#screen.removeAttribute(name); + if (term.textarea) codeField(term.textarea); + // The workbench's keybindings (F6, Mod+B, Mod+J, ...) work from the terminal, as in VS Code; + // returning true leaves the key to bubble up to them. + term.attachCustomKeyEventHandler((event) => isBound(event)); + this.#term = term; + fit.fit(); + fit.observeResize(); + session.resize(term.cols, term.rows); + for (const chunk of session.transcript) term.write(chunk); + const output = session.onOutput((chunk) => term.write(chunk, () => this.#queueMirror())); + const data = term.onData((keys) => session.input(keys)); + // ghostty-web reads keydown and cancels every beforeinput, but an on-screen keyboard types + // through beforeinput (its keydown is the 229 placeholder), so its text would be lost. + const typed = (event: InputEvent) => { + const keys = event.inputType === "deleteContentBackward" ? "\x7f" : event.inputType === "insertLineBreak" || event.inputType === "insertParagraph" ? "\r" : event.data; + if (keys && !event.isComposing) session.input(keys); }; + this.#screen.addEventListener("beforeinput", typed, { capture: true }); + // A click lands focus on the container (ghostty-web's selection handling); the textarea is the input. + const refocus = () => term.textarea?.focus(); + this.#screen.addEventListener("focus", refocus); + const resize = term.onResize(({ cols, rows }) => session.resize(cols, rows)); + this.#queueMirror(); return () => { - unsubscribe(); - this.#submit = undefined; - this.#session = undefined; + output(); + data.dispose(); + resize.dispose(); + this.#screen.removeEventListener("beforeinput", typed, { capture: true }); + this.#screen.removeEventListener("focus", refocus); + fit.dispose(); + term.dispose(); + this.#term = undefined; + }; + } + + /** The emulator's font and colors, from the design tokens. */ + #look(): { fontSize: number; fontFamily: string; theme: ITheme } { + const style = getComputedStyle(this.#probe); + return { + fontSize: parseFloat(style.fontSize) || 13, + fontFamily: style.fontFamily, + theme: { foreground: style.color, background: style.backgroundColor, cursor: style.color, selectionBackground: style.outlineColor }, }; } + + #queueMirror(): void { + if (this.#mirrorQueued) return; + this.#mirrorQueued = true; + requestAnimationFrame(() => { + this.#mirrorQueued = false; + const buffer = this.#term?.buffer.active; + if (buffer) this.#mirror(buffer); + }); + } + + /** + * One element per line of the screen's text, a row the terminal soft-wrapped joined to the one + * before it. Only lines that changed are touched and new ones appended, so the log announces what + * was printed, not the whole screen again. + */ + #mirror(buffer: Terminal["buffer"]["active"]): void { + const lines: string[] = []; + for (let y = Math.max(0, buffer.length - MIRROR_ROWS); y < buffer.length; y++) { + const row = buffer.getLine(y); + const text = row?.translateToString(!buffer.getLine(y + 1)?.isWrapped) ?? ""; + if (row?.isWrapped && lines.length) lines[lines.length - 1] += text; + else lines.push(text); + } + while (lines.length && !lines[lines.length - 1]!.trim()) lines.pop(); + // Each line keeps its break, so the log's text reads as the screen does. + for (let i = 0; i < lines.length - 1; i++) lines[i] += "\n"; + const shown = this.#log.children; + // Scrollback that fell off the top, or a cleared screen, shifts every line: start over. + if (shown.length > lines.length || (shown.length && shown[0]!.textContent !== lines[0])) this.#log.replaceChildren(); + lines.forEach((line, i) => { + const el = shown[i]; + if (!el) this.#log.append(Object.assign(document.createElement("div"), { textContent: line })); + else if (el.textContent !== line) el.textContent = line; + }); + } } declare global { diff --git a/packages/app/tests/bins.test.ts b/packages/app/tests/bins.test.ts new file mode 100644 index 0000000..929341d --- /dev/null +++ b/packages/app/tests/bins.test.ts @@ -0,0 +1,19 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { binFunctions, wantsNode } from "../src/terminal/bins.ts"; + +test("a script wants node when its #! names node, directly or through env", () => { + assert.equal(wantsNode("#!/usr/bin/env node\nconsole.log(1)"), true); + assert.equal(wantsNode("#!/usr/bin/env -S node --no-warnings\n"), true); + assert.equal(wantsNode("#!/usr/local/bin/node"), true); + assert.equal(wantsNode("#!/usr/bin/env NODE_OPTIONS=x node"), true); + assert.equal(wantsNode("#!/bin/sh\nexec node x"), false); + assert.equal(wantsNode("#!/usr/bin/env bun"), false); + assert.equal(wantsNode("console.log(1)"), false); +}); + +test("each bin becomes a function that runs its file through node", () => { + assert.equal(binFunctions(new Map([["astro", "/app/node_modules/.bin/astro"]])), `astro() { node '/app/node_modules/.bin/astro' "$@"; }\n`); + assert.equal(binFunctions(new Map([["x", "/it's/x"]])), `x() { node '/it'\\''s/x' "$@"; }\n`); + assert.equal(binFunctions(new Map()), ""); +}); diff --git a/packages/app/tests/compact.spec.ts b/packages/app/tests/compact.spec.ts index 779d8e2..1a088c7 100644 --- a/packages/app/tests/compact.spec.ts +++ b/packages/app/tests/compact.spec.ts @@ -1,5 +1,6 @@ import { devices, expect, test, type Page } from "@playwright/test"; import { axNode } from "@nibling/web/testing/ax"; +import { run, terminalInput, terminalLog } from "./fixtures/terminal.ts"; // A phone: a coarse pointer and a narrow viewport, so the shell picks the compact presentation itself. test.use({ ...devices["Pixel 7"] }); @@ -93,21 +94,21 @@ test("Preview swaps the editor area to the preview without reloading its frame", test("the terminal is a bottom sheet that opens from its handle and closes with Escape", async ({ page }) => { const handle = button(page, "Terminal"); await expect(handle).toHaveAttribute("aria-expanded", "false"); - await expect(page.getByRole("textbox", { name: "Terminal input" })).toBeHidden(); + await expect(page.locator("nib-terminal")).toBeHidden(); const viewport = page.viewportSize()!; expect((await handle.boundingBox())!.y + 44).toBeGreaterThan(viewport.height - 2); await handle.tap(); await expect(handle).toHaveAttribute("aria-expanded", "true"); - const input = page.getByRole("textbox", { name: "Terminal input" }); - await expect(input).toBeVisible(); - await expect(input).toBeFocused(); - await input.fill("ls"); - await input.press("Enter"); - await expect(page.locator("nib-terminal [role=log]")).toContainText("README.md"); + const terminal = page.locator("nib-terminal"); + await expect(terminal).toBeVisible(); + await expect(terminalInput(page)).toBeFocused(); + await expect(terminalLog(page)).toContainText("~ $", { timeout: 20_000 }); + await run(page, "ls"); + await expect(terminalLog(page)).toContainText("README.md"); await page.keyboard.press("Escape"); await expect(handle).toHaveAttribute("aria-expanded", "false"); await expect(handle).toBeFocused(); - await expect(input).toBeHidden(); + await expect(terminal).toBeHidden(); }); test("turning the phone sideways past the breakpoint restores the desktop layout", async ({ page }) => { diff --git a/packages/app/tests/fixtures/terminal.ts b/packages/app/tests/fixtures/terminal.ts new file mode 100644 index 0000000..3821f06 --- /dev/null +++ b/packages/app/tests/fixtures/terminal.ts @@ -0,0 +1,12 @@ +import type { Page } from "@playwright/test"; + +/** The terminal's input (the emulator's textarea) and the text of its screen, as a log. */ +export const terminalInput = (page: Page) => page.getByRole("textbox", { name: "Terminal input" }); +export const terminalLog = (page: Page) => page.getByRole("log", { name: "Terminal output" }); + +/** Types `line` into the terminal key by key, as the emulator reads keys, and presses Enter. */ +export async function run(page: Page, line: string) { + await terminalInput(page).focus(); + await page.keyboard.type(line); + await page.keyboard.press("Enter"); +} diff --git a/packages/app/tests/floating.spec.ts b/packages/app/tests/floating.spec.ts index d68c09a..ae4a58b 100644 --- a/packages/app/tests/floating.spec.ts +++ b/packages/app/tests/floating.spec.ts @@ -1,4 +1,5 @@ import { expect, test, type Locator, type Page } from "@playwright/test"; +import { run, terminalLog } from "./fixtures/terminal.ts"; // `README.md`'s textbox role resolves to CodeMirror's contenteditable content element, which has // no `.value`; its shadow root's host (``) exposes the live doc text through `.view`. @@ -93,9 +94,8 @@ test("closing a floating window docks a panel whose stack is gone to its kind's await expect(stack(page, "panel")).toHaveCount(0); await popup.close(); await expect(stack(page, "panel").locator(tab(page, "Terminal"))).toBeVisible(); - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("open style.css"); - await input.press("Enter"); + await expect(terminalLog(page)).toContainText("~ $", { timeout: 20_000 }); + await run(page, "open style.css"); await expect(tab(page, "style.css")).toHaveJSProperty("selected", true); }); @@ -146,7 +146,7 @@ test("a tab dropped on a floating window joins its tabs instead of opening anoth await expect(popup.locator("nib-tablist > nib-tab").nth(1)).toHaveJSProperty("selected", true); await expect(popup).toHaveTitle("Terminal"); await expect(popup).toHaveURL(/\/new#terminal=default$/); - await expect(popup.getByRole("textbox", { name: "Terminal input" })).toBeVisible(); + await expect(popup.locator("nib-terminal")).toBeVisible(); await expect(popup.getByRole("textbox", { name: "README.md" })).toBeHidden(); await expect(stack(page, "panel")).toHaveCount(0); expect(opened).toBe(false); diff --git a/packages/app/tests/keyboard.spec.ts b/packages/app/tests/keyboard.spec.ts index 77bf866..843b39f 100644 --- a/packages/app/tests/keyboard.spec.ts +++ b/packages/app/tests/keyboard.spec.ts @@ -1,5 +1,6 @@ import { devices, expect, test, type Locator, type Page } from "@playwright/test"; import { axNode } from "@nibling/web/testing/ax"; +import { run, terminalInput, terminalLog } from "./fixtures/terminal.ts"; test.use({ ...devices["Pixel 7"] }); @@ -82,10 +83,8 @@ test("keys edit at the caret without taking focus, and join the undo history", a // The edit reached the file, not just the editor: the terminal reads it back. await key(page, "Hide keyboard").tap(); await page.getByRole("button", { name: "Terminal", exact: true }).tap(); - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("cat README.md"); - await input.press("Enter"); - await expect(page.locator("nib-terminal [role=log]")).toContainText(" (#)# Playground"); + await run(page, "cat README.md"); + await expect(terminalLog(page)).toContainText(" (#)# Playground", { timeout: 20_000 }); }); test("arrow keys move the caret by character and by line, keeping the column", async ({ page }) => { @@ -106,15 +105,14 @@ test("arrow keys move the caret by character and by line, keeping the column", a test("in the terminal, the arrow keys walk the command history", async ({ page }) => { await page.getByRole("button", { name: "Terminal", exact: true }).tap(); - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("ls"); - await input.press("Enter"); + await expect(terminalLog(page)).toContainText("~ $", { timeout: 20_000 }); + await run(page, "ls"); await expect(page.locator("[part=keys]")).toBeVisible(); // Editing in the sheet keeps the sheet; only the editor area makes it step aside. - await expect(input).toBeVisible(); + await expect(page.locator("nib-terminal")).toBeVisible(); await key(page, "Up").tap(); - await expect(input).toHaveValue("ls"); - await expect(input).toBeFocused(); + await expect(terminalLog(page)).toContainText(/~ \$ ls$/); + await expect(terminalInput(page)).toBeFocused(); }); test("an on-screen keyboard insets the layout, so the key bar sits right on top of it", async ({ page }) => { diff --git a/packages/app/tests/launch.test.ts b/packages/app/tests/launch.test.ts index e64c4f1..53be036 100644 --- a/packages/app/tests/launch.test.ts +++ b/packages/app/tests/launch.test.ts @@ -1,12 +1,6 @@ import assert from "node:assert/strict"; import { test } from "node:test"; -import { launch, words } from "../src/launch.ts"; - -test("words splits as sh quotes", () => { - assert.deepEqual(words(" npm run dev "), ["npm", "run", "dev"]); - assert.deepEqual(words(`node -e "console.log(\\"hi\\", 'x')" a`), ["node", "-e", `console.log("hi", 'x')`, "a"]); - assert.deepEqual(words(`echo 'a "b"' "" c\\ d`), ["echo", 'a "b"', "", "c d"]); -}); +import { launch } from "../src/launch.ts"; test("npm and npx run the guest npm with their arguments as they are", () => { assert.deepEqual(launch(["npm", "run", "dev"], "/app"), { entry: "/usr/local/bin/npm", argv: ["run", "dev"] }); @@ -34,6 +28,6 @@ test("node -e, -p, and -v run their source from a file beside the working direct test("other commands are not launched, and node refuses what it can't run", () => { assert.equal(launch(["ls"], "/app"), undefined); - assert.deepEqual(launch(["node"], "/app"), { error: "node: the REPL is not available in this terminal; pass a file or -e" }); - assert.deepEqual(launch(["node", "--inspect", "x.js"], "/app"), { error: "node: bad option: --inspect" }); + assert.deepEqual(launch(["node"], "/app"), { error: "node: the REPL is not available in this terminal; pass a file or -e", code: 1 }); + assert.deepEqual(launch(["node", "--inspect", "x.js"], "/app"), { error: "node: bad option: --inspect", code: 9 }); }); diff --git a/packages/app/tests/line-editor.test.ts b/packages/app/tests/line-editor.test.ts new file mode 100644 index 0000000..82ac512 --- /dev/null +++ b/packages/app/tests/line-editor.test.ts @@ -0,0 +1,31 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { initialLine, step, type LineState } from "../src/terminal/line-editor.ts"; + +const type = (keys: string[], state: LineState = initialLine()) => keys.reduce((s, key) => step(s, key).state, state); + +test("typing, moving, and deleting edit the line", () => { + const state = type(["e", "c", "o", "\x1b[D", "h", "\x1b[F", "!", "\x7f"]); + assert.deepEqual([state.buffer, state.cursor], ["echo", 4]); + assert.equal(type(["\x17"], state).buffer, ""); +}); + +test("Enter submits and records the line; Up and Down walk the history and restore the draft", () => { + const first = step(type(["l", "s"]), "\r"); + assert.equal(first.submit, "ls"); + const drafted = type(["p", "w"], first.state); + const up = step(drafted, "\x1b[A"); + assert.equal(up.echo, "ls"); + assert.equal(step(up.state, "\x1b[B").echo, "pw"); +}); + +test("a paste with line breaks submits all but its last line, which stays on the prompt", () => { + const result = step(initialLine(), "echo a\recho b\rec"); + assert.equal(result.submit, "echo a\necho b"); + assert.equal(result.echo, "ec"); +}); + +test("^C abandons the line and ^L asks for a clear screen", () => { + assert.equal(step(type(["x"]), "\x03").interrupt, true); + assert.equal(step(initialLine(), "\x0c").clear, true); +}); diff --git a/packages/app/tests/processes.spec.ts b/packages/app/tests/processes.spec.ts index e4634d9..183ec95 100644 --- a/packages/app/tests/processes.spec.ts +++ b/packages/app/tests/processes.spec.ts @@ -1,4 +1,5 @@ import { expect, test, type Page } from "@playwright/test"; +import { run, terminalInput, terminalLog } from "./fixtures/terminal.ts"; /** * The demo project's `npm run dev` (node:http on port 3000) from the Processes panel and from the @@ -38,8 +39,8 @@ test("Run dev starts the server, a browser panel opens on it, and Stop ends it", // The script ran in the terminal, where its output and Ctrl+C are. await tab(page, "Terminal").click(); - await expect(page.locator("nib-terminal [role=log]")).toContainText("/ $ npm run dev\n"); - await expect(page.locator("nib-terminal [role=log]")).toContainText("Serving on http://localhost:3000/"); + await expect(terminalLog(page)).toContainText("~ $ npm run dev\n"); + await expect(terminalLog(page)).toContainText("Serving on http://localhost:3000/"); await tab(page, "Processes").click(); await npm.getByRole("button", { name: /^Stop npm run dev/ }).first().click(); @@ -50,11 +51,9 @@ test("Run dev starts the server, a browser panel opens on it, and Stop ends it", test("npm run dev in the terminal does the same, and a closed auto-opened panel stays closed", async ({ page }) => { await tab(page, "Terminal").click(); - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("npm run dev"); - await input.press("Enter"); + await run(page, "npm run dev"); await expect(server(page).getByRole("heading", { level: 1 })).toHaveText("Hello from nibling", { timeout: 20_000 }); - await expect(page.locator("nib-terminal [role=log]")).toContainText("Serving on http://localhost:3000/"); + await expect(terminalLog(page)).toContainText("Serving on http://localhost:3000/"); await tab(page, "Processes").click(); await expect(list(page).getByRole("button", { name: "Open localhost:3000" })).toBeVisible(); @@ -63,12 +62,12 @@ test("npm run dev in the terminal does the same, and a closed auto-opened panel // Ctrl+C reaches npm, which passes it to the server; running it again leaves the closed panel closed. await tab(page, "Terminal").click(); - await input.press("Control+c"); + await terminalInput(page).focus(); + await page.keyboard.press("Control+c"); await tab(page, "Processes").click(); await expect(list(page).getByRole("listitem")).toHaveCount(0); await tab(page, "Terminal").click(); - await input.fill("npm run dev"); - await input.press("Enter"); + await run(page, "npm run dev"); await tab(page, "Processes").click(); await expect(list(page).getByRole("button", { name: "Open localhost:3000" })).toBeVisible({ timeout: 20_000 }); await expect(page.locator("nib-browser")).toHaveCount(0); diff --git a/packages/app/tests/run-in-terminal.test.ts b/packages/app/tests/run-in-terminal.test.ts new file mode 100644 index 0000000..fca7de3 --- /dev/null +++ b/packages/app/tests/run-in-terminal.test.ts @@ -0,0 +1,12 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { memoryFiles } from "../src/files.ts"; +import { needsInstall } from "../src/run-in-terminal.ts"; + +test("a project installs when its package.json names dependencies of any kind", () => { + assert.equal(needsInstall(memoryFiles({ "package.json": '{"dependencies":{"astro":"^7"}}' })), true); + assert.equal(needsInstall(memoryFiles({ "package.json": '{"devDependencies":{"vite":"^8"}}' })), true); + assert.equal(needsInstall(memoryFiles({ "package.json": '{"scripts":{"dev":"node server.js"},"dependencies":{}}' })), false); + assert.equal(needsInstall(memoryFiles({ "package.json": "not json" })), false); + assert.equal(needsInstall(memoryFiles({ "index.html": "" })), false); +}); diff --git a/packages/app/tests/shell.spec.ts b/packages/app/tests/shell.spec.ts index d8a75e1..531f5d3 100644 --- a/packages/app/tests/shell.spec.ts +++ b/packages/app/tests/shell.spec.ts @@ -1,5 +1,6 @@ import { expect, test, type Locator, type Page } from "@playwright/test"; import { axNode, axTree } from "@nibling/web/testing/ax"; +import { run, terminalLog } from "./fixtures/terminal.ts"; const escape = (text: string) => text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); const stack = (page: Page, slot: string) => page.locator(`nib-stack:state(slot-${slot})`); @@ -192,7 +193,7 @@ test("a closed terminal comes back from the activity bar", async ({ page }) => { await tab(page, "Terminal").locator("[part=close]").click(); await expect(page.locator("nib-terminal")).toHaveCount(0); await page.getByRole("button", { name: "Terminal", exact: true }).click(); - await expect(page.getByRole("textbox", { name: "Terminal input" })).toBeVisible(); + await expect(page.locator("nib-terminal")).toBeVisible(); }); test("the activity bar is a vertical toolbar with one tab stop", async ({ page }) => { @@ -335,13 +336,11 @@ test("a side bar's side edge joins it, and its bottom edge splits it no smaller }); test("the terminal drives the workbench with the same commands as the GUI", async ({ page }) => { - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("open style.css"); - await input.press("Enter"); + await expect(terminalLog(page)).toContainText("~ $", { timeout: 20_000 }); + await run(page, "open style.css"); await expect(tab(page, "style.css")).toHaveJSProperty("selected", true); - await input.fill("nope"); - await input.press("Enter"); - await expect(page.locator("nib-terminal [role=log]")).toContainText("nope: command not found"); + await run(page, "nope"); + await expect(terminalLog(page)).toContainText("nope: command not found"); }); test("code fields turn off autocomplete, autocorrect, and suggestions; the agent's prose field keeps them", async ({ page }) => { diff --git a/packages/app/tests/terminal.spec.ts b/packages/app/tests/terminal.spec.ts index 40758cc..92a5d55 100644 --- a/packages/app/tests/terminal.spec.ts +++ b/packages/app/tests/terminal.spec.ts @@ -1,33 +1,83 @@ import { expect, test } from "@playwright/test"; +import { run, terminalInput, terminalLog } from "./fixtures/terminal.ts"; test.beforeEach(async ({ page }) => { await page.goto("/new?reset"); + await expect(terminalLog(page)).toContainText("~ $", { timeout: 20_000 }); }); -const input = (page: import("@playwright/test").Page) => page.getByRole("textbox", { name: "Terminal input" }); -const log = (page: import("@playwright/test").Page) => page.getByRole("log", { name: "Terminal output" }); +test("bash runs the project's files through the kernel's filesystem, with pipes and bash's errors", async ({ page }) => { + await run(page, "ls"); + await expect(terminalLog(page)).toContainText("~ $ ls\nREADME.md"); + await run(page, "cat README.md | head -1"); + await expect(terminalLog(page)).toContainText("# Playground"); + await run(page, "cat nope.txt"); + await expect(terminalLog(page)).toContainText("cat: nope.txt: No such file or directory"); + // bashkit hands over stderr and stdout separately, so their lines can land in either order. + await run(page, "bogus; echo status=$?"); + await expect(terminalLog(page)).toContainText("bash: line 1: bogus: command not found"); + await expect(terminalLog(page)).toContainText("status=127"); +}); test("node runs the project's files in the kernel and prints what they print", async ({ page }) => { - await input(page).fill(`node -e "console.log(require('fs').readFileSync('style.css', 'utf8').trim(), process.cwd())"`); - await input(page).press("Enter"); - await expect(log(page)).toContainText("h1 { color: rebeccapurple; } /app", { timeout: 20_000 }); - await input(page).fill("node -p 6*7"); - await input(page).press("Enter"); - await expect(log(page)).toContainText("/ $ node -p 6*7\n42\n"); - await input(page).fill("node missing.js"); - await input(page).press("Enter"); - await expect(log(page)).toContainText("Cannot find module '/app/missing.js'"); -}); - -test("lines typed while a process runs are its stdin, and Ctrl+C interrupts it", async ({ page }) => { - await input(page).fill(`node -e "process.stdin.on('data', (d) => console.log('got ' + d.toString().trim()))"`); - await input(page).press("Enter"); - await input(page).fill("hello"); - await input(page).press("Enter"); - await expect(log(page)).toContainText("hello\ngot hello", { timeout: 20_000 }); - await input(page).press("Control+c"); - await expect(log(page)).toContainText("^C"); - await input(page).fill("ls"); - await input(page).press("Enter"); - await expect(log(page)).toContainText("/ $ ls\nREADME.md"); + await run(page, `node -e "console.log(require('fs').readFileSync('style.css', 'utf8').trim(), process.cwd())"`); + await expect(terminalLog(page)).toContainText("h1 { color: rebeccapurple; } /app", { timeout: 20_000 }); + await run(page, "node -p 6*7"); + await expect(terminalLog(page)).toContainText("~ $ node -p 6*7\n42\n"); + await run(page, "node missing.js"); + await expect(terminalLog(page)).toContainText("Cannot find module '/app/missing.js'"); +}); + +test("a package's bin runs by name through node, in a pipe too", async ({ page }) => { + await run(page, `mkdir -p node_modules/.bin node_modules/hi && printf '#!/usr/bin/env node\\nconsole.log("hi from", process.argv.slice(2).join(" "))\\n' > node_modules/hi/cli.js && ln -s ../hi/cli.js node_modules/.bin/hi`); + await run(page, "hi there"); + await expect(terminalLog(page)).toContainText("~ $ hi there\nhi from there", { timeout: 20_000 }); + await run(page, "hi piped | tr a-z A-Z"); + await expect(terminalLog(page)).toContainText("HI FROM PIPED"); + // The function that ran it was only for that line. + await run(page, "type hi"); + await expect(terminalLog(page)).toContainText("hi is /app/node_modules/.bin/hi"); +}); + +test("keys typed while a process runs are its stdin, and Ctrl+C interrupts it", async ({ page }) => { + await run(page, `node -e "process.stdin.on('data', (d) => console.log('got ' + d.toString().trim()))"`); + await page.keyboard.type("hello"); + await page.keyboard.press("Enter"); + await expect(terminalLog(page)).toContainText("hello\ngot hello", { timeout: 20_000 }); + await page.keyboard.press("Control+c"); + await expect(terminalLog(page)).toContainText("^C"); + await run(page, "ls"); + await expect(terminalLog(page)).toContainText("~ $ ls\nREADME.md"); +}); + +test("Up and Down recall prior lines, and a line wider than the terminal edits in place", async ({ page }) => { + await run(page, "echo one"); + await run(page, "echo two"); + await page.keyboard.press("ArrowUp"); + await expect(terminalLog(page)).toContainText(/~ \$ echo two$/); + await page.keyboard.press("ArrowUp"); + await expect(terminalLog(page)).toContainText(/~ \$ echo one$/); + await page.keyboard.press("ArrowDown"); + await expect(terminalLog(page)).toContainText(/~ \$ echo two$/); + await page.keyboard.press("Control+u"); + const long = `echo ${"x".repeat(200)}`; + await page.keyboard.type(long); + for (let i = 0; i < 100; i++) await page.keyboard.press("ArrowLeft"); + await page.keyboard.type("Y"); + await page.keyboard.press("Enter"); + await expect(terminalLog(page)).toContainText(`${"x".repeat(100)}Y${"x".repeat(100)}\n~ $`); +}); + +test("open and preview open panels on the workbench", async ({ page }) => { + await run(page, "open README.md && preview index.html"); + await expect.poll(() => page.evaluate(() => (globalThis as any).workbench.panels("preview").map((p: any) => p.input.path))).toContain("index.html"); + const editors = await page.evaluate(() => (globalThis as any).workbench.panels("editor").map((p: any) => p.input.path)); + expect(editors).toContain("README.md"); + await run(page, "open nope.md"); + await expect(terminalLog(page)).toContainText("open: nope.md: No such file in the project"); +}); + +test("clicking the screen focuses the input", async ({ page }) => { + await page.locator("nib-terminal canvas").click(); + await expect(terminalInput(page)).toBeFocused(); }); diff --git a/packages/app/tests/views.spec.ts b/packages/app/tests/views.spec.ts index d4dbeb4..bdcfecb 100644 --- a/packages/app/tests/views.spec.ts +++ b/packages/app/tests/views.spec.ts @@ -34,56 +34,8 @@ test("the preview iframe is titled and sandboxed", async ({ page }) => { await expect(iframe).toHaveAttribute("sandbox", "allow-scripts"); }); -test("the terminal runs ls, cat, and reports errors", async ({ page }) => { - const input = page.getByRole("textbox", { name: "Terminal input" }); - const output = page.getByRole("log", { name: "Terminal output" }); - await input.fill("ls"); - await input.press("Enter"); - await expect(output).toContainText("README.md"); - await input.fill("cat README.md"); - await input.press("Enter"); - await expect(output).toContainText("Views playground"); - await input.fill("cat nope.txt"); - await input.press("Enter"); - await expect(output).toContainText("cat: nope.txt: No such file"); - await input.fill("bogus"); - await input.press("Enter"); - await expect(output).toContainText("bogus: command not found (try ls, cat, open, preview, node, npm, npx)"); -}); - -test("terminal history recalls prior commands with Up and Down", async ({ page }) => { - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("ls"); - await input.press("Enter"); - await input.fill("cat README.md"); - await input.press("Enter"); - await input.press("ArrowUp"); - await expect(input).toHaveValue("cat README.md"); - await input.press("ArrowUp"); - await expect(input).toHaveValue("ls"); - await input.press("ArrowDown"); - await expect(input).toHaveValue("cat README.md"); -}); - -test("clicking the terminal output focuses the input", async ({ page }) => { - await page.getByRole("log", { name: "Terminal output" }).click(); - await expect(page.getByRole("textbox", { name: "Terminal input" })).toBeFocused(); -}); - -test("the terminal's open command opens an editor panel on the real workbench", async ({ page }) => { - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("open README.md"); - await input.press("Enter"); - const paths = await page.evaluate(() => (globalThis as any).workbench.panels("editor").map((p: any) => p.input.path)); - expect(paths).toContain("README.md"); -}); - -test("the terminal's preview command opens a preview panel", async ({ page }) => { - const input = page.getByRole("textbox", { name: "Terminal input" }); - await input.fill("preview index.html"); - await input.press("Enter"); - const paths = await page.evaluate(() => (globalThis as any).workbench.panels("preview").map((p: any) => p.input.path)); - expect(paths).toContain("index.html"); +test("the terminal says why it has no shell when the page can't boot the sandbox", async ({ page }) => { + await expect(page.getByRole("log", { name: "Terminal output" })).toContainText("The sandbox did not start"); }); test("Enter sends a message in the agent, Shift+Enter inserts a newline", async ({ page }) => { diff --git a/packages/kernel/src/kernel/main.ts b/packages/kernel/src/kernel/main.ts index 8a5c2d6..208e22b 100644 --- a/packages/kernel/src/kernel/main.ts +++ b/packages/kernel/src/kernel/main.ts @@ -717,6 +717,17 @@ function resolveSpecifier( } if (specifier.startsWith("#")) return importsResolve(specifier, parent, kind); + // import() takes a file: URL (require() does not); its ?query and #hash name no part of the path. + if (kind === "import" && specifier.startsWith("file:")) { + let path: string; + try { + path = decodeURIComponent(new URL(specifier).pathname); + } catch { + return { ok: true, val: { kind: "missing", code: "ERR_MODULE_NOT_FOUND" } }; + } + return loadAsFileOrDirectory(path, kind); + } + if (specifier.startsWith("./") || specifier.startsWith("../") || specifier.startsWith("/")) { const base = specifier.startsWith("/") ? specifier : joinPath(parent, specifier); return loadAsFileOrDirectory(base, kind); diff --git a/packages/kernel/src/kernel/npm/store.ts b/packages/kernel/src/kernel/npm/store.ts index f8b8552..62cfc51 100644 --- a/packages/kernel/src/kernel/npm/store.ts +++ b/packages/kernel/src/kernel/npm/store.ts @@ -15,7 +15,7 @@ // writer wins and nothing is lost. // // The package cache is held under a byte budget by an index (budget.ts), one JSON -// document in a third cache, nib-npm-index-v1: each stored package's size and when an +// document in a third cache, nib-npm-index-v2: each stored package's size and when an // install last used it. After its write pass an install records what it used and // deletes the least recently used packages it did not use until the store is under // budget, holding the Web Lock `nib-npm-store`. diff --git a/packages/kernel/src/runtime/main.ts b/packages/kernel/src/runtime/main.ts index fbd685d..5fb827f 100644 --- a/packages/kernel/src/runtime/main.ts +++ b/packages/kernel/src/runtime/main.ts @@ -6,6 +6,7 @@ import { awaitSyncReply, takeSignals, writeSyncRequest, SAB_BYTES } from "../sha import { builtinUrl, cjsUrl, vfsUrl, type HostToken } from "../shared/vurls"; import { toNodeError, loaderError, requireNotFound, webEntryNotEsm } from "./errors"; import { compileCjs, makeRequire, loadCjsByPath, newModule, type RequireContext, type ModuleCacheEntry } from "./modules"; +import { hasResolveHooks, resolveThroughHooks } from "./module"; import { esmBegin, esmEnd, importEsm } from "./esm-sync"; import { capture, runInFrame } from "./async_hooks"; import { loadBuiltin } from "./builtins"; @@ -296,10 +297,12 @@ self.onmessage = (ev: MessageEvent): void => { filename: path, dirname: ctx.dirnameOf(path), resolve: (specifier: string): string => - moduleUrlFor( - ctx.callSync("resolve", { specifier, parent: ctx.dirnameOf(path), kind: "import" }), - specifier, path, - ), + hasResolveHooks() + ? resolveThroughHooks(specifier, `file://${path}`, fileUrlFor) + : moduleUrlFor( + ctx.callSync("resolve", { specifier, parent: ctx.dirnameOf(path), kind: "import" }), + specifier, path, + ), }), // The guest npm command's way into the kernel's installer (src/npm-cli). Async, and // holding the loop while the install runs, as any pending syscall does. @@ -322,7 +325,18 @@ self.onmessage = (ev: MessageEvent): void => { importFrom: (from: string) => async (specifier: unknown): Promise => { loop.hold(); try { - const spec = String(specifier); + let spec = String(specifier); + // Resolve hooks (registerHooks) turn the specifier into a URL first. + if (hasResolveHooks()) spec = resolveThroughHooks(spec, `file://${from}`, fileUrlFor); + // A path or file: URL with a ?query or #hash names the file without them, and a module + // of its own per query, as Node's loader does: Vite imports `astro.config.mjs?t=` + // to load a config fresh. + const [file, suffix] = splitQuery(spec); + if (suffix) { + const resolved = await call("resolve", { specifier: file, parent: ctx.dirnameOf(from), kind: "import" }); + const url = moduleUrlFor(resolved, spec, from); + return await import(resolved.kind === "file" && resolved.format === "esm" ? `${url}&q=${encodeURIComponent(suffix)}` : url); + } return await importEsm( from, spec, () => call("resolve", { specifier: spec, parent: ctx.dirnameOf(from), kind: "import" }), @@ -362,6 +376,24 @@ const processRuntimes: Record = { web: { installGlobals: installWebGlobals, runEntry: runWebEntry }, }; +/** The kernel's own resolution of `specifier` from `parentURL`, as the URL Node's resolver + * returns: file:// for a file, node: for a builtin. The end of a resolve hook chain. */ +function fileUrlFor(specifier: string, parentURL: string | undefined): string { + const [file, suffix] = splitQuery(specifier); + const parent = parentURL?.startsWith("file:") ? decodeURIComponent(new URL(parentURL).pathname) : "/"; + const resolved = callSync("resolve", { specifier: file, parent: parent.slice(0, parent.lastIndexOf("/")) || "/", kind: "import" }); + if (resolved.kind === "builtin") return `node:${resolved.id}`; + if (resolved.kind === "missing") throw loaderError(resolved.code, specifier, parent); + return `file://${resolved.path}${suffix}`; +} + +/** A path-like specifier split at its `?query` or `#hash`; a bare or builtin one is left whole. */ +function splitQuery(spec: string): [file: string, suffix: string] { + if (!/^(?:file:|\.{0,2}\/)/.test(spec)) return [spec, ""]; + const at = spec.search(/[?#]/); + return at < 0 ? [spec, ""] : [spec.slice(0, at), spec.slice(at)]; +} + /** Map a resolved module record to its virtual-origin URL — the same mapping the * kernel's transformer applies when rewriting static specifiers (shared/vurls.ts). */ function moduleUrlFor(resolved: ResolvedModule, specifier: string, from: string): string { diff --git a/packages/kernel/src/runtime/module.ts b/packages/kernel/src/runtime/module.ts index f1211b9..6358fd0 100644 --- a/packages/kernel/src/runtime/module.ts +++ b/packages/kernel/src/runtime/module.ts @@ -37,6 +37,38 @@ const compileCacheStatus = { FAILED: 0, ENABLED: 1, ALREADY_ENABLED: 2, DISABLED // `parent` getter, matching what Node's own introspection sees. const parentOf = new WeakMap(); +/** What a registerHooks() resolve hook returns: the URL a specifier names. */ +export interface ResolveResult { + url: string; + format?: string | null; + shortCircuit?: boolean; +} +export interface ResolveContext { + conditions: string[]; + importAttributes: Record; + parentURL: string | undefined; +} +type NextResolve = (specifier: string, context?: Partial) => ResolveResult; +type ResolveHook = (specifier: string, context: ResolveContext, nextResolve: NextResolve) => ResolveResult; + +/** The resolve hooks registerHooks() added, in registration order. */ +const resolveHooks: ResolveHook[] = []; + +/** Whether any resolve hook is registered, so an unhooked import() keeps its fast path. */ +export const hasResolveHooks = (): boolean => resolveHooks.length > 0; + +/** + * Resolves `specifier` through the registered resolve hooks, the last registered first, as + * Node chains them; the end of the chain is `base`, the kernel's own resolution. + */ +export function resolveThroughHooks(specifier: string, parentURL: string | undefined, base: (specifier: string, parentURL: string | undefined) => string): string { + const chain = resolveHooks.reduce( + (next, hook) => (spec, context) => hook(spec, { conditions: ["node", "import"], importAttributes: {}, parentURL, ...context }, next), + (spec, context) => ({ url: base(spec, context?.parentURL ?? parentURL) }), + ); + return chain(specifier, { parentURL }).url; +} + export function makeModule(ctx: RequireContext) { const path = makePath(() => ctx.cwd, () => ctx.process.env); @@ -149,12 +181,24 @@ export function makeModule(ctx: RequireContext) { // register()/registerHooks() customize the ESM loader's resolve/load step. Guest // ESM runs through the browser's native linker — dynamic import() against the SW - // virtual origin — which has no JS-reachable hook for a loader chain to attach to, - // so there is nothing these could wire up. Refuse by name rather than silently - // accepting hooks that would never run. + // virtual origin — which links static imports with no JS-reachable hook. What the + // runtime does resolve itself, import() and import.meta.resolve, runs registerHooks()'s + // resolve hooks (Vite's module runner registers one). Load hooks, and register()'s + // off-thread hooks, would never run, so they refuse by name. const WHY_HOOKS = "ESM loads through the browser's native linker; there is no hook point to attach a resolve/load chain to"; + const WHY_LOAD = "ESM loads through the browser's native linker; only resolve hooks run, and only for import() and import.meta.resolve"; Module.register = markRefused(() => { throw unsupported("module", "register()", WHY_HOOKS); }, WHY_HOOKS); - Module.registerHooks = markRefused(() => { throw unsupported("module", "registerHooks()", WHY_HOOKS); }, WHY_HOOKS); + Module.registerHooks = (hooks: { resolve?: ResolveHook; load?: unknown }): { deregister(): void } => { + if (hooks?.load !== undefined) throw unsupported("module", "registerHooks() load hooks", WHY_LOAD); + const hook = hooks?.resolve; + if (typeof hook === "function") resolveHooks.push(hook); + return { + deregister() { + const at = hook ? resolveHooks.indexOf(hook) : -1; + if (at >= 0) resolveHooks.splice(at, 1); + }, + }; + }; // syncBuiltinESMExports re-copies a builtin's mutated CJS exports onto its live // ESM bindings. Every builtin facade here exports `const = m.` (a @@ -243,7 +287,7 @@ export function makeModule(ctx: RequireContext) { globalPaths: string[]; constants: { compileCacheStatus: typeof compileCacheStatus }; register(): never; - registerHooks(): never; + registerHooks(hooks: { resolve?: ResolveHook; load?: unknown }): { deregister(): void }; syncBuiltinESMExports(): void; enableCompileCache(): { status: number; message?: string }; getCompileCacheDir(): undefined; diff --git a/packages/kernel/src/shared/package-files.ts b/packages/kernel/src/shared/package-files.ts index e737767..005b54d 100644 --- a/packages/kernel/src/shared/package-files.ts +++ b/packages/kernel/src/shared/package-files.ts @@ -1,15 +1,15 @@ -// Which files of an installed package the runtime keeps: package.json files, runtime -// sources and WebAssembly modules, minus declarations and source maps, plus -// extensionless scripts with a shebang (a package's `bin` entries, which a child process -// may run). Native addons (.node) cannot load. The kernel installer applies it to every -// tarball file and src/node/snapshot.ts imports it under Node's type stripping, so an -// install and a snapshot keep the same files. Erasable TypeScript only. +// Which files of an installed package the runtime keeps: every file but docs, source maps, +// and native addons. Packages read more than their code at runtime (Astro reads .d.ts +// templates, CSS, and .astro components out of its own package), so the rule drops only +// what nothing loads: Markdown, .map files, and .node addons, which cannot load here, and +// whose absence a napi-rs loader takes as its cue to try the package's wasm build. The +// kernel installer applies it to every tarball file and src/node/snapshot.ts imports it +// under Node's type stripping, so an install and a snapshot keep the same files. Erasable +// TypeScript only. -const RUNTIME_EXT = /\.(?:js|mjs|cjs|json|ts|mts|cts|wasm)$/; -const DECLARATION = /\.d\.(?:ts|mts|cts)$/; +const DROPPED = /\.(?:md|markdown|map|node)$/i; -/** `name` is the file's basename. */ -export function keepInPackage(name: string, bytes: Uint8Array): boolean { - if (RUNTIME_EXT.test(name)) return !DECLARATION.test(name); - return !name.includes(".") && bytes[0] === 0x23 && bytes[1] === 0x21; +/** `name` is the file's basename. `bytes` are its contents. */ +export function keepInPackage(name: string, _bytes: Uint8Array): boolean { + return !DROPPED.test(name); } diff --git a/packages/kernel/src/shared/package-store.ts b/packages/kernel/src/shared/package-store.ts index 4fd6e21..270fb38 100644 --- a/packages/kernel/src/shared/package-store.ts +++ b/packages/kernel/src/shared/package-store.ts @@ -1,10 +1,11 @@ // The origin-wide package store's Cache Storage names: the kernel writes them, and the // host can measure or clear them without booting a runtime. -export const PACKAGE_STORES = { packages: "nib-npm-packages-v1", metadata: "nib-npm-meta-v1", index: "nib-npm-index-v1" } as const; +export const PACKAGE_STORES = { packages: "nib-npm-packages-v2", metadata: "nib-npm-meta-v1", index: "nib-npm-index-v2" } as const; /** Caches older store layouts wrote, which the store deletes: tarballs by URL - * (niw-npm-v1), then this layout under the engine's former niw prefix. */ -export const LEGACY_PACKAGE_STORES = ["niw-npm-v1", "niw-npm-packages-v1", "niw-npm-meta-v1", "niw-npm-index-v1"] as const; + * (niw-npm-v1), then this layout under the engine's former niw prefix, then packages + * unpacked under the old keep rule, which dropped declarations, CSS, and templates. */ +export const LEGACY_PACKAGE_STORES = ["niw-npm-v1", "niw-npm-packages-v1", "niw-npm-meta-v1", "niw-npm-index-v1", "nib-npm-packages-v1", "nib-npm-index-v1"] as const; /** Where kernel/kept-dirs.ts keeps directories across page loads, under their own budget. */ export const KEPT_DIRS_STORES = { dirs: "nib-kept-dirs-v1", index: "nib-kept-dirs-index-v1" } as const; diff --git a/packages/kernel/src/sw/sw.ts b/packages/kernel/src/sw/sw.ts index 55f147d..8e11feb 100644 --- a/packages/kernel/src/sw/sw.ts +++ b/packages/kernel/src/sw/sw.ts @@ -94,6 +94,10 @@ const clientApps = new Map(); /** A bridge round trip that never answers must not hang the module graph forever * (kernel terminated, host wedged). Bounded at 10 s → the fetch 404s/504s. */ const BRIDGE_TIMEOUT_MS = 10_000; +/** A guest server's answer is its own business: a dev server's first page compiles before it + * answers, which here can take longer than the bridge bound. A dead host still fails fast, + * since `check` answers for it; this bounds only a live server that never replies. */ +const HTTP_TIMEOUT_MS = 300_000; /** How long a fetch waits for its host to answer a rebridge before failing. */ const REVIVE_WAIT_MS = 3_000; @@ -267,13 +271,13 @@ function adopt(clientId: string, app: AppRef): void { } /** One bridge round trip; null when the host went away or never answered. */ -function roundTrip(host: HostToken, bridge: Bridge, build: (id: number) => BridgeRequest): Promise { +function roundTrip(host: HostToken, bridge: Bridge, build: (id: number) => BridgeRequest, timeoutMs = BRIDGE_TIMEOUT_MS): Promise { const id = nextId++; return new Promise((resolve) => { const timer = setTimeout(() => { pending.delete(id); resolve(null); - }, BRIDGE_TIMEOUT_MS); + }, timeoutMs); pending.set(id, { host, resolve: (reply) => { @@ -323,7 +327,7 @@ async function serveHttp(request: Request, app: AppRef, url: string): Promise ({ t: "http", id, port: app.port, req })); + const data = await roundTrip(app.host, bridge, (id) => ({ t: "http", id, port: app.port, req }), HTTP_TIMEOUT_MS); if (data?.t !== "httpResult") { return retired.has(app.host) ? textResponse(502, noHost(app.host)) diff --git a/packages/kernel/tests/builtins.spec.ts b/packages/kernel/tests/builtins.spec.ts index e53cf91..07e4879 100644 --- a/packages/kernel/tests/builtins.spec.ts +++ b/packages/kernel/tests/builtins.spec.ts @@ -183,16 +183,48 @@ test("node:trace_events refuses", async ({ page }) => { expect(lines).toEqual(["createTracing ERR_NIB_UNSUPPORTED"]); }); -test("node:module's loader hooks and native addons refuse", async ({ page }) => { +test("node:module's off-thread and load hooks and native addons refuse", async ({ page }) => { const lines = await guestLines(page, { "/app/main.cjs": [ 'const Module = require("module");', 'try { Module.register("./x.mjs"); } catch (e) { console.log("register", e.code); }', - 'try { Module.registerHooks({ resolve() {} }); } catch (e) { console.log("registerHooks", e.code); }', + 'try { Module.registerHooks({ load() {} }); } catch (e) { console.log("registerHooks load", e.code); }', 'try { Module._extensions[".node"]({ exports: {} }, "/app/x.node"); } catch (e) { console.log("node ext", e.code); }', ].join("\n"), }, "/app/main.cjs"); - expect(lines).toEqual(["register ERR_NIB_UNSUPPORTED", "registerHooks ERR_NIB_UNSUPPORTED", "node ext ERR_NIB_UNSUPPORTED"]); + expect(lines).toEqual(["register ERR_NIB_UNSUPPORTED", "registerHooks load ERR_NIB_UNSUPPORTED", "node ext ERR_NIB_UNSUPPORTED"]); +}); + +test("registerHooks resolve hooks run for import() and import.meta.resolve, last registered first, until deregistered", async ({ page }) => { + const lines = await guestLines(page, { + "/app/a.mjs": 'export default "a";', + "/app/b.mjs": 'export default "b";', + "/app/main.mjs": [ + 'import { registerHooks } from "node:module";', + 'const outer = registerHooks({ resolve: (s, c, next) => next(s === "virtual:x" ? "./a.mjs" : s, c) });', + 'const inner = registerHooks({ resolve: (s, c, next) => next(s === "virtual:y" ? "virtual:x" : s, c) });', + 'console.log((await import("virtual:y")).default, (await import("virtual:x")).default, (await import("./b.mjs")).default);', + 'console.log(import.meta.resolve("virtual:y"));', + 'inner.deregister();', + 'try { await import("virtual:y"); } catch (e) { console.log("after deregister", e.code); }', + 'outer.deregister();', + ].join("\n"), + }, "/app/main.mjs"); + expect(lines).toEqual(["a a b", "file:///app/a.mjs", "after deregister ERR_MODULE_NOT_FOUND"]); +}); + +test("import() takes a file: URL, and each ?query of one is a module of its own", async ({ page }) => { + const lines = await guestLines(page, { + "/app/config.mjs": 'export const loaded = Symbol("loaded");', + "/app/main.mjs": [ + 'const one = await import("file:///app/config.mjs?t=1");', + 'const again = await import("file:///app/config.mjs?t=1");', + 'const two = await import("./config.mjs?t=2");', + 'const plain = await import("file:///app/config.mjs");', + 'console.log(one === again, one.loaded === two.loaded, plain.loaded === one.loaded);', + ].join("\n"), + }, "/app/main.mjs"); + expect(lines).toEqual(["true false false"]); }); test("node:vm refuses every realm-creating entry point, naming the missing primitive", async ({ page }) => { diff --git a/packages/kernel/tests/install.spec.ts b/packages/kernel/tests/install.spec.ts index 1537ae7..b26ca29 100644 --- a/packages/kernel/tests/install.spec.ts +++ b/packages/kernel/tests/install.spec.ts @@ -534,10 +534,10 @@ test("a second install, and an install on another page of the origin, take every * and the index's document as it parses (null when absent or unparseable). */ async function storeState(page: Page): Promise<{ sizes: Record; index: { budget?: number; entries: Record } | null }> { return await page.evaluate(async () => { - const cache = await caches.open("nib-npm-packages-v1"); + const cache = await caches.open("nib-npm-packages-v2"); const sizes: Record = {}; for (const req of await cache.keys()) sizes[req.url] = (await (await cache.match(req))!.blob()).size; - const hit = await (await caches.open("nib-npm-index-v1")).match("https://nib.invalid/npm-store-index"); + const hit = await (await caches.open("nib-npm-index-v2")).match("https://nib.invalid/npm-store-index"); return { sizes, index: hit ? await hit.json().catch(() => null) : null }; }); } @@ -545,7 +545,7 @@ async function storeState(page: Page): Promise<{ sizes: Record; /** Write `text` as the index's document, or a budget into the one stored. */ async function writeIndex(page: Page, edit: { budget: number } | { text: string }): Promise { await page.evaluate(async (edit) => { - const cache = await caches.open("nib-npm-index-v1"); + const cache = await caches.open("nib-npm-index-v2"); const key = "https://nib.invalid/npm-store-index"; const text = "text" in edit ? edit.text : JSON.stringify({ ...await (await cache.match(key))!.json(), budget: edit.budget }); await cache.put(key, new Response(text)); @@ -640,7 +640,7 @@ test("a tarball that fails its integrity check rejects with EINTEGRITY, its pack (e: Error & { code?: string }) => ({ code: e.code, message: e.message }), ); const tree = (await runtime.fs.readdir("/app/node_modules")).map((d) => d.name).sort(); - return { result, stored: (await (await caches.open("nib-npm-packages-v1")).match(key)) !== undefined, tree }; + return { result, stored: (await (await caches.open("nib-npm-packages-v2")).match(key)) !== undefined, tree }; }; runtime.writeTree({ "/app/node_modules/before.txt": "kept" }); return { cold: await attempt(bad), good: await attempt(good), warm: await attempt(bad), again: await attempt(good) }; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index a2cf918..2e5e3d6 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -92,6 +92,9 @@ importers: '@codemirror/view': specifier: ^6.43.15 version: 6.43.15 + '@everruns/bashkit-wasm': + specifier: ^0.20.3 + version: 0.20.3 '@lezer/common': specifier: ^1.5.5 version: 1.5.5 @@ -110,6 +113,9 @@ importers: '@nibling/web': specifier: workspace:* version: link:../web + ghostty-web: + specifier: ^0.4.0 + version: 0.4.0 loro-codemirror: specifier: 0.3.3 version: 0.3.3(patch_hash=ce003e1e836ca18dde436474cb179c4e024680ded1c7c258cf491c8aac37b328)(@codemirror/state@6.7.6)(@codemirror/view@6.43.15)(loro-crdt@1.17.1) @@ -847,6 +853,9 @@ packages: '@everruns/bashkit-wasm@0.20.1': resolution: {integrity: sha512-IcaEMVAKhDpjghr3MUWQcqGrLSat7ppBsglnn81CGZ83TXzkY6ynvQ8wd8FTb/h5GAiV7ZJnbjJrYkKVaye2vA==} + '@everruns/bashkit-wasm@0.20.3': + resolution: {integrity: sha512-Yau4chwoMe/vHsuCZU/TopP6KAKaQiAJRYG5eP3hTzlRBhkwv6AnzArPt04pxidACbZ7reo1US8AJU99+/eQCA==} + '@img/colour@1.1.0': resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} engines: {node: '>=18'} @@ -1408,6 +1417,9 @@ packages: resolution: {integrity: sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==} engines: {node: '>= 0.4'} + ghostty-web@0.4.0: + resolution: {integrity: sha512-0puDBik2qapbD/QQBW9o5ZHfXnZBqZWx/ctBiVtKZ6ZLds4NYb+wZuw1cRLXZk9zYovIQ908z3rvFhexAvc5Hg==} + gopd@1.2.0: resolution: {integrity: sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==} engines: {node: '>= 0.4'} @@ -2356,6 +2368,8 @@ snapshots: '@everruns/bashkit-wasm@0.20.1': {} + '@everruns/bashkit-wasm@0.20.3': {} + '@img/colour@1.1.0': {} '@img/sharp-darwin-arm64@0.35.5': @@ -2861,6 +2875,8 @@ snapshots: dunder-proto: 1.0.1 es-object-atoms: 1.1.2 + ghostty-web@0.4.0: {} + gopd@1.2.0: {} has-property-descriptors@1.0.2: -- knotmirror 0.0.1