shard
Star 2
Fork 0
Watch
shard src web PROFILING.md
37 kB · 583 lines
Markdown
at main

Web UI Profiling — CPU & Memory Playbook #

How to profile the web client, what to look for, and what we already know is expensive. The reported symptoms: the browser tab's memory can reach 1–2 GB and CPU spikes during agent activity. This document exists so the next investigation starts from evidence instead of guesses.

  • Overlay: src/web/lib/perf/overlay.ts + PerfOverlay.svelte (opt-in, zero cost when off)
  • Enable: append ?perf=1 to the web UI URL (?perf=0/?perf=false force-off)

Profile the built app, not vite dev. Dev-mode Svelte does far more work per update than production. Use bun run web:build + shard web (the daemon serves build/), open the printed URL, then append ?perf=1. See DEVELOPMENT.md §2.

Quickstart — the whole loop in ten steps #

bun run web:build && shard web --web-port 8787        # 1. built app (never vite dev)
# 2. open the printed URL with ?perf=1&perf-record=1  # overlay + recording from load
  1. In the DevTools console: __shardPerf.mark('baseline')
  2. Run the §2 repro, marking every action: __shardPerf.mark('paged to 5k'), ('big doc'), ('scrolled up'), ('opened Recall'), ('opened tabs'), ('idle start') — end with ≥30s of idle (the leak detectors need it).
  3. (CPU) While streaming, record a Performance-panel trace at 4–6× CPU throttle (§3.1) — alongside the recording, so timestamps line up.
  4. __shardPerf.download() → shard-perf-<timestamp>.json.
  5. bun run perf:analyze shard-perf-*.json → attribution + ranked findings (§1.2).
  6. Follow each finding's → line (decision tree, §1.2) to its §3 capture and §4 fix.
  7. Fix, rebuild, re-record the same repro, re-run the analyzer, compare numbers.
  8. Memory findings: repeat with the §3.2 snapshot workflow instead of step 5.

Console surface: __shardPerf.{snapshot,startRecording,stopRecording,mark,export,download}.

1. The overlay #

With ?perf=1, +page.svelte mounts a fixed panel (bottom-right) fed by every connection publication:

perf · ?perf=1
fps 58 · pub/s 42
md/s 38 · avg 6.2ms · max 180ms · in 41.2k
heap 684MB / 4096MB
dom 96.4k nodes
longtasks 12 · 2.4s
transcript 5.2k entries · 3 tabs

(When a recording is active, a rec hh:mm:ss · Ns line leads the panel — see §1.1.)

Metric Meaning Healthy Suspicious
fps rAF frames in the last second ~display refresh <30 during interaction
pub/s connection publications (state applications) 0 idle; ≤60 streaming high while idle (event storm)
md/s renderMarkdown calls per second ~pub/s at most ≫ pub/s (duplicate re-renders)
avg/max markdown render cost (marked + DOMPurify + highlight.js over the whole text) <2ms growing max as a message streams (O(n²) signature)
in largest markdown input seen (chars) small ~full message size — every delta re-parses it
heap JS heap (Chrome performance.memory; n/a elsewhere) <200MB ratcheting up and not coming back down
dom live DOM nodes <20k >50k (the transcript mounts every entry)
longtasks >50ms main-thread blocks (count · total ms, 5s window) 0 idle; few while streaming dozens; seconds of blocking
transcript summed transcript entries across open session tabs — grows with every opened session (see §5 M3)

Rules of thumb for reading it:

  • CPU high while streaming, tab otherwise responsive → look at md/s/max first (per-delta markdown re-parse, §4 F1), then longtasks.
  • Memory grows while paging back / opening sessions and never shrinks → dom + heap ratchet → retention problem (§4 M1–M3). Note heap does not include DOM node storage — a tab can show 400MB heap and still hold 1GB+ of DOM/strings.
  • CPU high while the tab is idle → ambient animations (§4 F6) or an event storm (pub/s > 0 while nothing is happening).

The overlay is deliberately self-bounding (sample rings pruned at 60s/8192) — it cannot become the leak it is hunting. dispose() removes the markdown telemetry hook.

1.1 Recording & export — turning the overlay into profiling data #

The live panel answers "what is happening now". To get data out for real analysis, run a recording. Recordings capture a per-second time series (publications, markdown renders + cost, long tasks, FPS, heap, DOM nodes) plus labeled annotations, and export as a self-contained JSON report.

Starting a recording

?perf=1&perf-record=1          # record from page load

or live, from the DevTools console (installed by start()):

__shardPerf.startRecording()   // begin (fresh series)
__shardPerf.mark('opened Recall')  // annotate the timeline
__shardPerf.stopRecording()    // end (data stays exportable)

Getting the data out

__shardPerf.download()   // saves shard-perf-<ISO timestamp>.json
__shardPerf.export()     // returns the report object — pipe through DevTools `copy()`
copy(__shardPerf.export())          // → clipboard → paste anywhere
__shardPerf.snapshot()              // live numbers, same as the overlay shows

Report format (shard-perf-report-v1): meta (URL, UA, epoch timestamps, recording window), summary (totals/peaks: pubs, markdown ms + max input chars, long tasks, heap peak MB, DOM peak, FPS min), series (one row per second — {t, pubs, md, mdMs, mdMaxMs, mdMaxChars, longTasks, longTaskMs, fps, heapMb, dom}, zero-filled so gaps plot as zeros), and marks (labeled annotations, epoch ms).

Plotting one series without ceremony:

jq -r '.series[] | [.t, .pubs, .mdMs, .heapMb, .dom] | @tsv' shard-perf-*.json > perf.tsv
# → spreadsheet / gnuplot / matplotlib
python3 -c "import json,matplotlib.pyplot as plt;d=json.load(open('shard-perf-1.json'));s=d['series'];\
plt.plot([r['t'] for r in s],[r['heapMb'] for r in s],label='heapMB');\
plt.plot([r['t'] for r in s],[r['dom'] for r in s],label='dom nodes');plt.legend();plt.show()"

Overhead & bounds. Recording aggregates per second (events still flow through the live rings untouched), so capture cost is negligible; markdown renders additionally emit a User-Timing measure only while recording (see below). Series are capped at 24h of seconds; marks at 1000 (oldest dropped). Restarting a recording discards the previous one — download before restarting. exportReport() zero-fills the tail up to the export moment, so a main-thread-starved sampler cannot make the recording look shorter than it was. Related: when the page is saturated you may see the overlay's rec clock (a 500ms interval) freeze — that is the starvation making itself visible, not lost data; the collector aggregates on the publication events themselves.

DevTools correlation. While recording, every renderMarkdown emits performance.measure('shard:markdown', {start, duration}) — open the Performance panel, record, and the per-render markdown cost appears in the Timings track next to the flame chart, interleaved with long tasks and script samples. Both the report and DevTools timestamps are wall-clock ms, so the JSON series and the captured trace correlate directly: run a recording and a Performance-panel capture over the same repro (§2), then use the JSON to find when it degraded and the trace to see why.

1.2 Analyzing a recording — from JSON to ranked bottlenecks #

The raw export is deliberately boring (per-second counters). The triage step is scripted, so analysis is two commands:

bun run perf:analyze shard-perf-*.json    # or: bun scripts/analyze-perf.ts <file.json>…

It prints wall-time attribution, a ranked findings list whose codes map 1:1 onto the §4 catalog, and ±5s deltas around every mark. Full pipeline:

  1. Record a §2 repro with marks for each action (__shardPerf.mark('opened Recall')).
  2. __shardPerf.download() → bun run perf:analyze on the file.
  3. Read the ranked findings; each one names the §4 code, the measured evidence, and the matching fix direction.
  4. For every non-OK finding, run the paired DevTools capture from §3 to get function-level attribution (the recording tells you when/what; the trace/snapshots tell you why), fix per §4, then re-record and compare the same numbers — the report is the before/after benchmark.

What the analyzer measures

  • Attribution — % of wall time in markdown renders, in ≥50ms long tasks, and the remainder ("unattributed CPU" — long-task time markdown doesn't explain). Also pub/s, markdown renders per publication (≈1 healthy, ≫1 = amplification), min fps.
  • F1 probe — per-char cost of the largest single render per second, compared across a median split of input sizes (×≥2 flags) plus Pearson r against input size. This is the O(n²) re-parse signature. It deliberately uses the single-render cost, not per-second totals: once renders saturate the main thread, per-second totals pin at ~1000ms/s and per-char totals fall while inputs keep growing — totals hide the signature exactly when F1 is worst; a single render's cost stays measurable (see the saturation test in tests/web-perf-analysis.vitest.ts).
  • Idle detectors — fps<45 seconds while nothing renders: ≥80% of sampled idle seconds → sustained (F6, ambient animations); scattered → sporadic (F4, scroll-path scans). Longest idle run ≥30s gives heap MB/s and DOM nodes/s slopes (M1/M3).
  • Memory release — after a ≥10s idle tail: how much of the activity-time heap rise was returned (low % = retention, M2/M3).
  • Marks — each annotation with its ±5s metric deltas, so recorded actions line up with the degradation in the series.

Decision tree (analyzer output → meaning → next capture)

Finding Meaning Confirms Do next
[F1] … — dominant render cost grows with streamed size F1 §4 F1 fix; confirm via §3.1 trace filtered to renderMarkdown
[F2] unattributed CPU (+ high pub/s) ≥50ms tasks beyond markdown F2/F3 §3.1 trace during streaming; weigh mergeLiveSnapshot/enrichTranscriptTools vs Svelte keyed-each diff
[F2/F3] publication storm burst pubs + jank in those bursts, no long tasks, cheap markdown F2/F3 amplified by pub rate §3.1 trace of one burst, long-task filter off; msgs/s top: names the wire types; P1 client coalescing is in — if it still storms, batch live-event application per frame (P2) or daemon-side (§6.5)
[F7] frames bound by DOM size frame time tracks DOM size (r≥0.55) at a steady pub rate F3 rendering (style/layout/paint over the whole transcript), invisible to the long-task API P3a is in (content-visibility rows); re-record — if still DOM-bound, virtualize; §3.1 trace in Chrome for the scripting/rendering split
[F6] sustained idle jank frames slow while nothing renders F6 §3.1 Paint Flashing; pause the animation loops
[F4] sporadic jank dips tied to scrolling F4 §3.1 trace while scrolling
[M1] heap idle slope heap grows while idle M1/M3 §3.2 three-snapshot diff on the growth
[M2] low released % activity memory not returned M2/M3 §3.2 active-vs-idle snapshots; close-tab test
[M2-dom] dom idle slope detached nodes / retained tabs M3 §3.2 snapshot diff → detached node list
[OK] nothing dominant in this window — re-run §2 with the missing actions (F5/Recall, scrolling) during the recording

Sample output (synthetic 90s recording — streaming a large doc, then idle):

shard perf recording — triage
  url: http://localhost:8787/?perf=1&perf-record=1  ·  1m 30s · 90s samples · 3 marks

attribution (share of wall time)
  markdown renders     61.7%   (180 renders · avg 308.5ms · max input 96,400 chars)
  long tasks ≥50ms     65.3%   (240 tasks · avg 245ms)
  unattributed CPU      3.6%   (long-task time beyond markdown)
  pub/s 4.0 avg · 100 peak · md/pub 0.50 · fps min 38
  F1 probe: per-char cost growth ×2.90 · r(maxRenderMs, chars)=0.97

findings (ranked)
  [F1] Markdown re-parses the full text per delta (O(n²) per message) — dominant
      · per-char cost of the largest single render grew ×2.90 from small→large inputs
      · max streamed input 96,400 chars · avg render 308.5ms over 180 renders
      · markdown = 61.7% of wall time · 2.0/s renders
      → PROFILING.md §4 F1 — …
  [F2/F3] Publication storm: frame budget starved by many sub-50ms per-publication tasks — notable
      · pub/s peaked at 100 (avg 68.6); janky (fps<45) in 86% of 120 burst seconds
      · fps min 11 ≈ 91ms frames vs a 16.7ms budget
      · markdown 0.3% of wall · long tasks 0.0% — LongTask API silent
      → PROFILING.md §3.1 — record one burst with the long-task filter OFF …
  [M1] Heap grows while idle — notable
      · 2.20 MB/s over a 30s idle run (210→546 MB)
      → PROFILING.md §3.2 — …
  [M2] Heap largely not released after activity stops — notable
      · peak 546MB → last 546MB after a ≥10s idle tail; only 0.0% of the rise released
      → PROFILING.md §3.2 — …

marks
  04:45:52  asked for big doc            ±5s: pubs 66 · md 33 · longTaskMs 10890 · heap 323→368MB

Caveats. The series triages — only the trace/snapshots attribute to functions. Thresholds are heuristics (constants at the top of src/web/lib/perf/analysis.ts); tune them as real recordings accumulate. fps=0 seconds (hidden tab, rAF stopped) are excluded from jank stats. The LongTask API only counts tasks ≥50ms — publication storms chop work into 1–40ms tasks (invisible), and style/layout/paint from DOM diffing never produces long tasks at all, so a "0 long tasks" line is not evidence of an idle main thread; the fps series is the witness (that is what the [F2/F3] storm finding keys on). mdMaxMs/mdMaxChars are independent per-second maxima, so the F1 pairing is a proxy — the trend across a median split is the signal, not any single second.

2. Reproducing the worst case #

  1. Build + serve: bun run web:build && shard web --web-port 8787; open the URL with ?perf=1&perf-record=1 (§1.1) so everything below lands in an exportable series.
  2. Open a long-running session; repeatedly trigger load older until the overlay shows transcript ≥ 5k entries (each page prepends entries; nothing is virtualized, so dom climbs with it). __shardPerf.mark('paged to 5k') as you go.
  3. Start a task that streams a large artifact — e.g. ask the agent to write a long markdown document with several large fenced code blocks.
  4. While it streams: watch md max climb (F1), longtasks accumulate, fps sag.
  5. Scroll up (leaves the at-bottom state) and scroll around: the scroll handler does a full-DOM scan per event (F4) — jank here is F4/F3, not F1.
  6. Open Recall (transcript search) and type: per-keystroke indexing of the whole transcript (F5).
  7. Open 3–4 more sessions as tabs, then re-check heap/dom: per-tab transcript data is all retained (M3 — heap grows with tab count); dom stays at the active pane's size (single mounted viewport, §5 M3).
  8. For memory specifically: capture heap snapshots at steps 2, 3, and 7 (§3.2).
  9. __shardPerf.download() then bun run perf:analyze shard-perf-*.json (§1.2) — the ranked triage tells you which §3 capture to run for function-level attribution.

3. DevTools workflows #

3.1 CPU — Performance panel #

  1. DevTools → Performance → CPU throttle 4×–6× (emulates the slow machines users actually report from) → record 10–20s during step 3 above. Keep the ?perf-record recording running concurrently — its JSON series (§1.1) marks when degradation happened, and this trace shows why; both use wall-clock ms.
  2. Read the Bottom-Up view; the stack that matters (Streaming tab in a long session):
    • renderMarkdown → marked.parse / sanitize / hljs.highlight — F1
    • mergeLiveSnapshot → collectTranscriptToolCalls / enrichTranscriptTools — F2
    • Svelte flush/update keyed-each diff over every row — F3
    • visibleAnchor under scroll events — F4
    • rows() / JSON.stringify with Recall open — F5
  3. While recording, per-render markdown cost also appears in the Timings track as shard:markdown measures (§1.1) — scan it for the growth-over-time F1 signature.
  4. Long tasks <50ms filter: everything the overlay counts appears here. But remember the blind spots: (a) long tasks are only tasks ≥50ms — during publication storms (burst pub/s with low fps and zero long tasks in the overlay) the main thread is saturated by sub-50ms tasks the filter hides. In that case record a burst and read the Summary tab per frame instead: Scripting share → F2 (state merge per publication); Rendering share (style/layout/paint from diffing thousands of mounted rows) → F3. Then fix the rate: coalesce/batch publications before onUpdate (render-on-quiet) so per-frame work runs once per frame, not per message. (b) Firefox emits no long-task entries at all — the overlay's long-task counts and the analyzer's CPU attribution (F2/unattributed %) are structurally blind there; zero long tasks in a Firefox recording means unmeasured, not none. Record the same repro once in Chrome when you need the scripting-vs-rendering split, and rely on the F7 DOM-correlation detector for Firefox-only captures.
  5. Rendering tab → Paint Flashing: any region flashing while idle is an ambient animation (F6 — reactor spins on session cards, ActivitySignal dashes). Also check Frame Rendering Stats while the SessionRack has many running sessions.
  6. Svelte DevTools: confirm whether a publication invalidates the whole SessionDrawer subtree or just the streaming MessageText (prop identity churn in paneState() is a candidate — new object literals per publication).

3.2 Memory — three-snapshot + allocation timeline #

Chrome (or a Chromium build) is the only browser with the full memory toolkit: performance.memory exists there, so the ?perf-record harness itself gains heapMb columns — the cheapest first pass is simply re-running the same §2 recording in Chrome and running the analyzer: the M1 (idle heap slope) and M2 (released %) detectors fire automatically on thresholds. Firefox gives null heap columns; its snapshots and the LongTask API are structurally absent (§3.1 4b).

Chrome gotchas: use a clean profile / guest window (extensions inflate heap + DOM and will pollute every diff); performance.memory numbers are quantized (bucketed, ~±5%) — fine for slopes and before/after deltas, not for exact sizes (launch with --enable-precise-memory-info if you need exact); DOM and compositor memory live outside the JS heap, so DevTools heap numbers can look small next to Task Manager — cross-check there and via chrome://tracing (§3.3). During snapshots, annotate phases in the recording with window.__shardPerf.mark('label') so the JSON series correlates with what you did.

(The ?perf-record JSON series gives you the when for free: correlate heapMb/dom trends in the export with the actions you annotated via __shardPerf.mark().)

  1. Memory panel → heap snapshot at rest (t₀).
  2. Do the suspected action (page back N times / open K sessions / stream one big message).
  3. Snapshot again (t₁), optionally a third after forcing GC (🗑 button).
  4. Comparison view t₁ vs t₀, sorted by Size Delta, filtered to # added:
    • TranscriptEntry / ToolEvent objects + their content strings → M1 (full tool output is retained by design for the dedicated viewer)
    • huge string counts → markdown intermediate HTML + retained message/tool text
    • HTMLSpanElement / Text nodes → DOM growth from highlight trees, F1's output
  5. Allocation instrumentation on timeline: record while streaming; the per-delta churn (new transcript arrays from immutable merges, concatenated strings, HTML strings) shows as short-lived blue bars — churn itself is GC pressure, sustained growth of retained (grey) objects is the leak.
  6. Detached check: filter snapshot for detached HTMLLIElement/HTMLSpanElement. If large detached trees exist, something holds references to removed DOM (not currently suspected — FlightPlan keyed-each reuses nodes — but verify after any rendering change).
  7. Retention chain drill-down: click a fat ToolEvent string → Retainers pane → expect tab.transcript → sessionTabs → WebConnectionState. That confirms which retention path to fix (M1/M3).

3.3 Cross-checks #

  • chrome://tracing / about:tracing for renderer-process memory when DevTools heap numbers seem too small (DOM lives outside the JS heap).
  • Task Manager (Shift+Esc) → per-tab "Memory" + "GPU memory": the 1–2GB number users report is usually here, i.e. including DOM/compositor — consistent with the no-virtualization hypothesis (M2), not necessarily with JS leaks.
  • performance.measureUserAgentSpecificMemory is not available cross-origin-isolated here; performance.memory (Chrome) is what the overlay uses.

4. Known hot paths (ranked, with expected profile signatures) #

These are static-analysis findings from the current code, each with the signature that confirms or clears it in a profile. Fixing them is deliberately not part of this doc — measure first, then pick off what the profile actually shows.

CPU #

F1 — Full markdown re-parse per streaming delta (prime suspect). MessageText.svelte ($: rendered = renderMarkdown(text, { streaming })) re-runs marked + DOMPurify + highlight.js over the entire accumulated block text on every publication in which that text changed. Streamed text appends per delta (mergeLiveEntry: previous.text + next.text), so per-message cost is O(n²) in the final text length — a 100KB message streamed in ~500 deltas re-parses ~50MB of markdown cumulatively, each pass also re-highlighting the full (growing) code fence. Signature: md/s ≈ pub/s, md max grows monotonically while one message streams; flame chart dominated by marked.parse/sanitize/hljs.highlight. Directions: parse-on-paint with coalescing keyed to visible tail; incremental re-parse of only the trailing paragraph/fence; cache the rendered prefix before the last blank line / open fence; pause re-parse when the block is offscreen.

F2 — O(transcript) state work per delta. Every live delta runs mergeLiveSnapshot (session-history.ts): findLiveMatch per existing entry, then enrichTranscriptTools → collectTranscriptToolCalls walks all entries and content blocks rebuilding a Map, then maps all entries again. Also sequenceUnkeyedSystems builds a Set of all system entries. All O(n) with allocations per delta. Signature: mergeLiveSnapshot/collectTranscriptToolCalls visible in bottom-up at streaming time; worse the longer the session. Directions: maintain the tool-call index incrementally instead of rebuilding; short-circuit enrichment for pure text deltas.

F3 — No virtualization: keyed-each diff over every row per publication. FlightPlan.svelte {#each transcript as entry, i (transcriptKey(entry))} — Svelte computes keys for all N entries on every publication and diffs them; every row's DOM is mounted (also the memory driver, M2). Signature: Svelte update cost proportional to N in the flame chart even though only the last row changed. Directions: windowed rendering (render only entries near the viewport — the viewport already tracks anchors), or at minimum a capped live tail + virtualized history.

Amplifier — publication rate (storm). The F2/F3 per-publication cost is multiplied by the publication rate. A real capture recorded bursts of 72–100 pubs/s (session open hydration, live streaming, split-view sync) with fps collapsing to 11 while long tasks stayed at zero — per-publication work runs in 1–40ms slices, so the LongTask API never fires but the 16.7ms frame budget is gone. Overlay signature: pub/s avg tens, peaks ~100, fps dips tracking the bursts, longtasks 0, cheap markdown. The first-order fix is rate, not per-call cost: coalesce/batch publications before onUpdate (e.g. rAF-batched or 30–60ms trailing-edge throttle — render-on-quiet), so F2/F3 run once per frame instead of once per message. The [F2/F3] storm finding in §1.2 detects this from a recording.

F4 — Scroll handler does full-DOM scans. SessionTranscriptViewport.svelte: when not at bottom, every scroll event runs visibleAnchor() (querySelectorAll('[data-entry-id]') + offsetTop reads → forced layout) and emits state; handleTranscriptChange does the same walk after each transcript change off-bottom. With 10k+ rows this is jank on every wheel tick. Signature: visibleAnchor/layout under scroll handlers. Directions: rAF-throttle scroll handling; track the anchor incrementally from the last known index.

F5 — Recall rebuilds a full-text index of the transcript. Recall.svelte: $: all = rows(transcript) stringifies every tool result into a Row (re-runs on every publication while open), and every keystroke lowercases + scans all row texts. With a long transcript and Recall open during streaming this dominates. Signature: rows/JSON.stringify per publication; per-keystroke freeze. Directions: build the index lazily/memoized per transcript identity; search without full lowercase copies (e.g. scan with a case-insensitive regex on demand, or debounce).

F6 — Ambient animations. Per-card ShardReactor spin (SessionRack cards), ActivitySignal SVG dash/cursor loops, wp-pulse on running tool rows, FlightPlan's 1s ticker while tools are in-flight (elapsedByTool recompute). Small each, but they burn CPU continuously and scale with visible cards/rows. Signature: paint flashing while idle; composited frame cost. Directions: content-visibility, pause when document.hidden or the pane is offscreen, respect reduced-motion globally (partially done).

F7 — Frames bound by DOM size (rendering, long-task-invisible). Every coalesced flush mutates text inside a transcript whose whole document is laid out: style recalculation + layout + paint scale with total mounted DOM, not with what changed. Recorded (2026-08-31): steady 20fps plateaus on an ~18.7k-node transcript vs 60fps on ~3.7k at the identical message rate — r(domNodes, frameMs) = 0.73, ~38ms/frame ≥10k nodes vs ~17ms below. Firefox emits no long-task entries, so task-based attribution shows 0% — the analyzer flags this via the DOM correlation instead (F7). Signature: fps floor ∝ transcript size at constant pub rate; §3.1 trace shows Rendering (purple/green) dominating, near-zero Scripting. Directions: done — P3a (content-visibility: auto rows + measure pass in FlightPlan/SessionTranscriptViewport). If still DOM-bound after re-recording: full virtualization (window the keyed-each to entries near the viewport).

Memory #

M1 — Full tool output retained per entry (by design, unbounded in aggregate). ToolEvent.content keeps the complete result text ("the full data stays on the entry for the dedicated viewer / copy" — transcript.ts). A session that read many large files/outputs retains all of it for the session's lifetime in the tab. Inline display is bounded (24 lines/8k chars), so this is invisible in the UI while dominating heap. Signature: fat string retention under ToolEvent in snapshots. Directions: keep full output behind a session-scoped LRU/refetch (session.entry replay already exists daemon-side), or cap retained bytes per tab.

M2 — Every transcript entry is live DOM. No virtualization (F3): markdown HTML with highlight.js span trees for every code block, a <details> per tool row, etc. Ballpark: hundreds of nodes per rich entry → a 5k-entry session is easily 10⁵–10⁶ nodes; at ~0.5–1.5KB per node that alone explains the 1–2GB reports. Fixing F3 fixes this; the overlay's dom number is the direct measurement.

M3 — Every opened session tab retains its full transcript (data, not DOM). Tabs are keyed by session in WebConnectionState.sessionTabs; switching sessions does not evict (only explicit close does, closeSessionTab). Open tabs persist to localStorage and are re-opened (and re-replayed) on reload. Ten opened long sessions = ten full transcripts (entries + full tool-output strings). DOM is per visible pane, not per tab: SessionDrawer mounts one SessionTranscriptViewport for the active session (+1 in split mode) under {#key activeId} — background tabs hold data only. Signature: heap grows with the number of tabs opened; DOM does not (it scales with visible panes — see M2). Verified (2026-08-31 Chrome trace): closing the 19.7k-node session dropped dom 19.7k→4.8k and heap 86→29MB within ~2s; teardown is clean. Remount cost: {#key activeId} rebuilds the full mounted DOM on every tab switch — the cold-open jank (fps 17–25 opening the big session) is this remount, softened by the markdown memo + measure pass. Directions: cap retained tool output per entry (the M1 lever), evict inactive tabs' transcripts (keep the newest page + viewport anchor; re-replay on activation — the activation buffer machinery already supports this), or cap total retained entries. Windowing the viewport (P2/F3) cuts both the visible-pane DOM and the remount cost.

M4 — Per-delta allocation churn (GC pressure, not a leak). Immutable merges build a new transcript array + entry objects + accumulated strings per delta; markdown produces full HTML strings per render. Transient, but at 30–60 publications/s over long sessions it forces frequent major GCs (visible as frame spikes in the Performance panel's Memory track). Fixing F1/F2 reduces it as a side effect; no independent action needed now.

Already guarded (do not re-blame these) #

  • Publications are frame-coalesced for streaming fragments (setStreaming, one markdown pass per paint) — connection.ts. Immediate publication happens only on completion/tool/permission/lifecycle boundaries.
  • Inline tool output and diffs are bounded previews (24 lines / 8k chars; diffs capped at 8 dense lines); expansion is explicit.
  • Activation buffer is capped (256 records / 1MiB) — connection.ts.
  • Viewport persistence is capped (MAX_VIEWPORTS = 50) — +page.svelte.
  • argsText accumulation is cleared when a tool call completes — mergeToolEvents.
  • highlight.js uses a curated language set (bounded bundle), unknown languages degrade to escaped plain text.

5. Adding future instrumentation #

  • The overlay subscribes to two signals today: publications (onPublication in +page.svelte#onUpdate) and markdown renders (setMarkdownHook in src/web/lib/markdown.ts). To watch another transform (e.g. mergeLiveSnapshot cost), add a sibling setXHook + onX(input, ms) pair and a snapshot field — keep the same rules: opt-in, guarded, self-bounding. Feed it into the recording buckets the same way (if (recording) bucketFor(t).x++) so it lands in exported reports.
  • For one-off measurements use User Timing directly in a scratch change: performance.mark('x-start') / performance.measure('x', 'x-start') — marks show up in the Performance panel's Timings track and are trivial to delete. The markdown hook already does this while recording (shard:markdown).
  • The window.__shardPerf console surface (§1.1) is installed by start() and removed by dispose() — extend ShardPerfConsole (overlay.ts) if a new control is needed.
  • jsdom-safe unit tests for collector behavior (including recording/export) live in tests/web-perf-overlay.vitest.ts (included in bun run test:web).

6. Order of attack (from the first recorded capture) #

The first real recording (session-open → stream → split-view, 3 min) re-ranked the catalog: fps min 11 during publication bursts of 72–100/s, zero long tasks, 0.3% markdown — the LongTask API is blind to sub-50ms per-publication work (§1.2 caveats), so the storm is invisible to task-based detectors. Frame load ≈ C × rate, where C = per-publication work (O(transcript) merge F2 + O(rows) diff/layout F3) and rate is the burst multiplier. Fix the multiplier first, then C — and treat memory (mounted DOM) as its own track, since rate fixes don't shrink it.

  1. Trace one burst before coding (§3.1 step 4): Performance panel, long-task filter off, CPU 4×, record a session-open burst. Summary per frame: Scripting-heavy → F2 is the first per-call fix; Rendering-heavy (style/layout/paint) → F3 is. Also check how much time is JSON parsing — if message decode itself is significant, daemon-side batching becomes relevant (§6.6).
  2. P1 — Coalesce publications (render-on-quiet). Implemented, verified. WebConnection now coalesces every steady-state subscriber notification to the next frame (rAF, opt-in via deps.requestFrame, which +page.svelte provides); phase transitions (error/offline/enrollment/connecting/connected) still publish immediately, and protocol state stays synchronous + lossless — only the notification defers. Wire-type instrumentation (deps.onMessage → __shardPerf recording, msgs/s top: in the analyzer) shows what traffic is made of. Measured (2026-08-31 re-record, same repro): pub/s peak 121 → 26, avg 45.8 → 10.6; the [F2/F3] storm finding is gone. The residual dips are a different bottleneck (F7, below). Still open (P2): mergeLiveSnapshot + enrichTranscriptTools still run per message — batch the live-event application per frame if a Chrome trace shows scripting-long-tasks.
  3. P3a — content-visibility on transcript rows. Implemented. The second recording isolated the residual dips: at a constant publication rate (~20 pubs/s, ~130 msgs/s), frame time tracked DOM size — r(dom, frameMs) = 0.73, ~38ms/frame at ≥10k nodes vs ~17ms below, rock-steady 20fps plateaus on an 18.7k-node transcript, 60fps on a 3.7k one (F7, now auto-detected by the analyzer). Firefox emits no long-task entries, so this rendering cost was invisible to task-based attribution. Fix: FlightPlan rows carry content-visibility: auto + contain-intrinsic-size: auto 64px — offscreen rows skip style/layout/paint entirely. Because placeholder heights would corrupt the viewport's offset-based scroll restores (the prepend no-jump guarantee, saved-anchor restore), the viewport arms a measure pass around head changes (initial load, older-page prepend, session switch): one full-layout frame with skipping disabled, after which contain-intrinsic-size: auto remembers every row's real size and the skipping re-arms. Expect: streaming frame time decoupled from transcript size.
  4. P3b — markdown render memo. Implemented. The same recording showed 150–374 parse calls/s in burst seconds where only 1–2 blocks had changed (row remounts on message_complete + sibling block churn after a live merge re-render unchanged MessageTexts; a single 34k-char re-parse costs up to ~18ms). renderMarkdown is now memoized in a bounded LRU (512 entries / 4M input chars, keyed on text+streaming) — unchanged blocks become ~0ms cache hits; the hook still fires per call so counts separate real parse work from churn.
  5. P2 — per-publication scripting cost, only if a Chrome §3.1 trace (Firefox has no long-task API) still shows scripting long-tasks during bursts: incremental tool-call index + skip enrichment for pure-text deltas (F2), and/or transcript windowing — render only entries near the viewport (F3/memory: visible-pane DOM stops scaling with transcript length — the M2 driver. Per-tab data retention is M3: tool-output caps / page eviction).
  6. Daemon-side batching (only if the trace shows decode cost): batch stream deltas into fewer WebSocket frames (e.g. 30–50ms flush) so the client parses once per batch. Client coalescing cannot remove per-message parse cost; this can. The recording's msgs/s top: line (~21/s each for session_runtime_changed, agent_activity, message_delta in the 2026-08-31 capture) says the daemon sends ~130 msgs/s during streaming — if P2 scripting cost matters, this is the lever.
  7. Verify each step with the harness: same §2 repro, ?perf=1&perf-record=1, bun run perf:analyze before/after. Acceptance: no storm/F7 finding; fps min ≥ 45 during bursts (≥ 30 at 4× throttle); after windowing, dom nodes ≈ viewport-sized and heap no longer scales with transcript length × open tabs.

Re-rank after every trace — this order reflects one capture, not a law.