Video Reframe Issue - Compositing Stack Investigation#

Date: 2025-10-19
Updated: 2025-10-20 (Refocused on actual issue)
Problem: Screen buffer freezes between reframes when rec.present() is enabled
Status: � IN PROGRESS - Compositing stack issue in bios.mjs, NOT button repositioning

🎯 ACTUAL ISSUE (2025-10-20)#

Corrected Problem Statement#

The issue is NOT about button repositioning - buttons work fine and reposition correctly.

The real problem: During tape playback (rec.present() enabled), the screen buffer temporarily freezes when the window is resized. This causes:

Brief animation pause during dimension sync
Dimension mismatch between canvas and imageData
Multiple rapid reframe cycles before stabilizing

Root cause location: The bios.mjs compositing stack, specifically around lines 11920-12520 where imageData is created and composited to canvas during underlayFrame (tape playback) mode.

🔍 Cleaned Up Logging (2025-10-20)#

Logs Removed#

video.mjs:

❌ Removed: "✅ Buttons created at screen: X x Y"
❌ Removed: "🎨 VIDEO PAINT: returning true"
❌ Removed: "🎨 VIDEO PAINT: returning true (no buttons yet)"

bios.mjs:

❌ Removed: "🔄 REFRAME PATH (fallback)"
❌ Removed: "📸 VIDEO: Created imageData in normal path"
❌ Removed: "🚫 BLOCKED: imageData creation blocked"
❌ Removed: "🔄 REFRAME PATH (main)"
❌ Removed: "🎬 VIDEO: pixelsDidChange = ..."

disk.mjs:

❌ Removed: "🖌️ WIPE: Using screen dimensions X x Y" (fired every frame)

Logs Kept/Updated#

bios.mjs:

✅ "⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: X x Y | ImageData: X x Y"
- Only logs during dimension mismatch (the freeze condition)
- Shows exactly what dimensions don't match

disk.mjs:

✅ "📐 REFRAME: Worker dimensions updated X x Y → X x Y"
- Only logs when dimensions actually change
- Shows the transition clearly

Expected Log Pattern#

Normal reframe (working correctly):

📐 REFRAME: Worker dimensions updated 300 x 197 → 300 x 215
[smooth continuation - no freeze]

Problematic reframe (freeze condition):

⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: 300 x 197 | ImageData: 300 x 211
⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: 300 x 197 | ImageData: 300 x 211
[repeats multiple times - THIS IS THE FREEZE]
📐 REFRAME: Worker dimensions updated 300 x 197 → 300 x 215
[freeze ends - animation resumes]

📊 Pipeline Flow During Reframe#

What Should Happen#

User resizes window
Bios detects resize → sends "reframed" message to worker
Worker updates dimensions → screen.width/height updated immediately
Worker paints → uses new dimensions in wipe/paint
Worker sends pixels → buffer matches new canvas size
Bios receives pixels → imageData dimensions match canvas
Bios composites → putImageData succeeds, overlays paint
✅ Animation continues smoothly

What Actually Happens (Freeze Condition)#

User resizes window
Bios detects resize → canvas resizes immediately
Bios sends "reframed" → but worker hasn't processed yet
Worker still painting → using OLD dimensions
Worker sends pixels → buffer has OLD dimensions
Bios receives pixels → imageData.width !== canvas.width ❌
Bios BLOCKS rendering → ⏸️ REFRAME: Dimension mismatch logged
Bios requests repaint → setTimeout needs-paint
[Steps 4-8 repeat] → THIS IS THE FREEZE LOOP
Worker finally processes reframe → 📐 REFRAME: Worker dimensions updated
Worker paints with new dimensions → buffer matches canvas
Bios receives matching pixels → compositing succeeds
✅ Animation resumes

The Freeze Duration#

The freeze lasts for the number of frames it takes for:

The worker to receive the "reframed" message
The worker to process it and update dimensions
The worker to complete a paint cycle with new dimensions
The bios to receive the matching buffer

Typically 10-30 frames (shown in your logs as ~30 repetitions of the mismatch message).

🔬 Investigation Findings (2025-10-20)#

Worker Message Queue Analysis#

Architecture discovered:

// disk.mjs line 7207
onmessage = makeFrame;

async function makeFrame({ data: { type, content } }) {
  // Sequential if-else chain, no queue
  if (type === "init-from-bios") { /* ... */ return; }
  if (type === "needs-paint") { noPaint = false; return; }
  if (type === "reframed") {
    // Lines 8234-8260: Updates dimensions immediately
    screen.width = content.width;
    screen.height = content.height;
    screen.pixels = new Uint8ClampedArray(content.width * content.height * 4);
    reframed = true;
    return; // Early return - doesn't paint
  }
  if (type === "frame") { /* Main paint loop */ }
  // ...
}

Key findings:

No message priority system - Messages processed in arrival order
Reframe returns early - Updates dimensions but doesn't trigger paint
Next "frame" message - Uses new dimensions for paint
The delay: The time between:
- Bios sends "reframed" message
- Worker receives and processes it (updates dimensions)
- Bios sends "needs-paint"
- Worker receives and processes it
- Worker completes paint with new dimensions
- Worker sends buffer back to bios
- Bios receives matching buffer

Message Flow During Reframe#

Timeline:

T+0ms:   User resizes window
T+0ms:   Bios canvas resizes (synchronous DOM)
T+0ms:   Bios sends "reframed" message
T+0ms:   Bios sends "needs-paint" message
T+0ms:   Worker painting frame N with OLD dimensions
T+16ms:  Worker completes frame N, sends OLD buffer
T+16ms:  Bios receives OLD buffer → MISMATCH #1
T+16ms:  Bios setTimeout → sends "needs-paint"
T+16ms:  Worker processes "reframed" → updates dimensions
T+16ms:  Worker processes "needs-paint" → noPaint = false
T+32ms:  Worker paints frame N+1 with NEW dimensions
T+32ms:  Worker sends NEW buffer
T+32ms:  Bios receives NEW buffer → MATCH! ✅

Actual freeze duration: ~1-2 frames (16-32ms)

But because each mismatch triggers another paint request, and if the worker is busy, multiple frames can accumulate before sync, resulting in the 10-30 frame freeze seen in logs.

ROOT CAUSE IDENTIFIED (2025-10-20) 🎯#

The freeze was caused by a buffer recycling bug in disk.mjs:

The Bug:

// disk.mjs line 8298 (BEFORE FIX)
if (type === "frame") {
  let pixels;
  if (content.pixels) {
    pixels = new Uint8ClampedArray(content.pixels);
    if (screen) screen.pixels = pixels;  // ← BLINDLY OVERWRITES!
  }
}

What happens:

User resizes → Bios sends "reframed" with new dimensions (300x145)
Worker processes "reframed" → Creates NEW buffer: screen.pixels = new Uint8ClampedArray(300 * 145 * 4)
Bios sends "frame" with OLD buffer (from previous size 300x154)
Worker processes "frame" → OVERWRITES new buffer with old buffer!
Worker paints → Sends buffer with 184,800 bytes but claims 300x145 (180,000 bytes expected)
Bios receives → Dimension mismatch! → Freeze loop begins

The Fix:

// disk.mjs line 8298 (AFTER FIX)
if (content.pixels) {
  pixels = new Uint8ClampedArray(content.pixels);
  const expectedLength = screen.width * screen.height * 4;
  if (screen && pixels.length === expectedLength) {
    screen.pixels = pixels;  // OK - sizes match
  } else if (screen && pixels.length !== expectedLength) {
    // REJECT mismatched buffer - keep the reframed buffer
    console.log('⚠️ FRAME: Ignoring mismatched buffer from bios');
  }
}

Expected result: Freeze should drop from 20-170 frames to 0-1 frames (instantaneous resize).

Diagnostic Logging Added#

Bios (bios.mjs):

📤 REFRAME: Sending reframe message to worker. New dimensions: X x Y
⏸️ REFRAME: Dimension mismatch #N. Canvas: X x Y | ImageData: X x Y
- Logs first mismatch, then every 10th to avoid spam
✅ REFRAME: Dimension sync restored after N mismatched frames. Canvas: X x Y

Worker (disk.mjs):

📐 REFRAME: Worker dimensions updated X x Y → X x Y
- Only logs when dimensions actually change

🔬 Investigation: Compositing Stack Freeze#

The Core Issue#

Location: bios.mjs lines ~12500-12520 - the dimension mismatch handler

Current behavior:

if (underlayFrame) {
  // During tape playback, keep the canvas at correct size and wait for matching data
  console.log('⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: ...
  skipImmediateOverlays = true; // Don't paint overlays
  // Keep requesting paint so we get fresh data with correct dimensions
  setTimeout(() => send({ type: "needs-paint" }), 0);
}

This creates a busy-wait loop that causes the freeze:

Canvas is resized → new dimensions (e.g., 300 x 215)
ImageData arrives → old dimensions (e.g., 300 x 197)
Mismatch detected → skip rendering, request repaint
Worker sends another frame → still old dimensions
Loop continues until worker processes reframe message

Why It Happens#

Timing issue in the reframe message pipeline:

Bios is too fast:
- Canvas resizes instantly (synchronous DOM operation)
- "reframed" message sent to worker (async postMessage)
Worker is delayed:
- Worker event loop processes messages between paint cycles
- Multiple paint frames may complete before reframe message is processed
- Each paint uses old screen dimensions
Result:
- Bios has new canvas size (215)
- Worker keeps sending old buffer size (197)
- Dimension check fails → freeze loop

Potential Solutions#

Option 1: Immediate Worker Notification ⭐ BEST#

Process "reframed" messages with highest priority in worker:

Move reframe handling to top of message queue
Process before paint, before any other messages
Update dimensions synchronously before next paint cycle

UPDATE: Investigation shows no message queue exists - messages are processed sequentially via onmessage. The issue is that worker sends one more frame with old dimensions before processing reframe message.

Option 2: Render Mismatched Data with Scaling ⭐ PRACTICAL#

Instead of blocking, scale the imageData to fit canvas:

Use ctx.drawImage() with source and dest rects
Temporary visual distortion vs complete freeze
Less jarring user experience

Option 3: Cache Last Valid Frame#

During dimension mismatch:

Keep rendering the last valid frame
Don't request repaint until dimensions match
Avoids busy-wait loop

Option 4: Predictive Dimension Update#

Worker preemptively checks for pending reframes:

Before each paint, check if canvas dimensions changed
Update screen dimensions proactively
Reduces lag between reframe message and dimension update

📋 Completion Summary (2025-10-20)#

✅ Issue RESOLVED#

The video button reframe issue has been successfully fixed! The screen buffer no longer freezes during window resize when playing back tapes with rec.present().

Final Results:

✅ Buffer freeze duration: 20-170 frames → 0-1 frames (instantaneous)
✅ Dimension mismatch: Eliminated - buffer sizes match immediately
✅ Button repositioning: Working correctly - buttons stay in corner positions
✅ Animation continuity: Perfect - no visible freeze or stutter
✅ Rapid resizing: Handles smoothly without artifacts

Root Cause: Worker's frame message handler (disk.mjs line 8298) was blindly overwriting the correctly-sized reframed buffer with an old buffer transferred from bios.

Solution: Added buffer size validation before accepting transferred buffers. Worker now rejects mismatched buffers and keeps the correctly-sized reframed buffer.

Files Modified:

disk.mjs (line 8300-8315): Added buffer size validation
disk.mjs (line 8306): Fixed TypeError with screen existence check

Cleanup Status:

✅ Feature-rich video.mjs confirmed as production version (1576 lines)
✅ No oldvideo.mjs file exists (file search showed duplicates)
✅ Button repositioning works correctly (reposition() called every frame)
✅ All export features intact (POST, MP4, GIF, ZIP)

Additional improvements:

✅ Removed black bar under buttons in video.mjs - cleaner transparent overlay
✅ Fixed button tap behavior - ZIP/GIF/MP4 buttons no longer pause video when tapped
- Issue: Play/pause toggle only checked postBtn.down, causing other buttons to trigger pause
- Fix: Now checks if ANY button is down before toggling play/pause (lines 1147-1163)
✅ Created shared tape-player library (disks/common/tape-player.mjs)
- Extracted common progress bar rendering from replay.mjs
- Functions: deriveProgressState(), renderLoadingProgressBar(), renderStreamingBadge(), formatMegabytes()
- Ready for both video.mjs and replay.mjs to share code
✅ Global progress bar already working via rec.tapeProgress system
- Renders VHS-style red progress bar underneath video during exports/playback
- No additional code needed - bios.mjs handles the rendering
✅ Refactored replay.mjs to use shared library
- Removed ~210 lines of duplicate code
- Now imports and uses shared functions
- Consistent UI rendering with video.mjs

Files Created/Modified:

disks/common/tape-player.mjs: New shared library for tape playback UI (260 lines)
video.mjs (lines 203-207): Removed black button background bar
video.mjs (lines 1147-1163): Fixed play/pause toggle to check all buttons
replay.mjs: Removed duplicate functions, added imports, updated to use shared library
plans/tape-player-refactor.md: Detailed refactoring summary document

Next Steps: Both video.mjs and replay.mjs now share common UI code for consistent tape operation feedback. See tape-player-refactor.md for complete refactoring analysis and future opportunities.

📋 Action Items#

Phase 1: Confirm the Freeze Pattern ✅ DONE#

Clean up verbose logging
Add focused reframe tracking
Test window resize during tape playback
Observe freeze duration and frequency

Phase 2: Investigate Worker Message Priority ✅ COMPLETE#

Check worker message queue implementation in disk.mjs
- Finding: No message priority queue exists
- Finding: Messages processed sequentially via onmessage = makeFrame
- Finding: "reframed" is processed as early-return (lines 8234-8260)
- Finding: Updates dimensions immediately but paint still uses old buffer
Add diagnostic timing logs
- Bios logs when sending reframe message
- Worker logs when processing reframe message
- Bios counts dimension mismatch iterations
- Bios logs when sync is restored
Measure actual delay between reframe sent and reframe processed
- Result: Worker updates dimensions immediately!
- Problem: Worker continues sending OLD buffer size for 20-170 frames
- Added: More detailed worker send logs to trace buffer size
Investigate why worker sends old buffer despite updating dimensions

Phase 3: Implement Solution ✅ COMPLETE#

Identify root cause
- ROOT CAUSE FOUND: Worker's "frame" handler overwrites correctly-sized buffer
- Line 8298: screen.pixels = pixels blindly overwrites with OLD buffer from bios
- After reframe creates new buffer, next frame message destroys it
Implement fix
- SOLUTION: Validate buffer size before accepting it
- Only use transferred buffer if pixels.length === screen.width * screen.height * 4
- Keep reframed buffer if bios sends mismatched size
Test fix
- Verify dimension mismatch count drops to 0-1 frames ✅ CONFIRMED
- Verify no visual freeze during resize ✅ CONFIRMED
- Test rapid resizing ✅ WORKS PERFECTLY
- Test slow dragging ✅ WORKS PERFECTLY

Phase 4: Performance Optimization#

Measure frame drops during reframe
Optimize dimension update pipeline
Consider batching rapid reframes
Add hysteresis to prevent resize thrashing

🧪 Testing Checklist#

Test during tape playback (rec.present() active):

Slow window resize (drag corner smoothly)
Fast window resize (rapid dragging)
Multiple rapid resizes in succession
Maximize/restore window
Fullscreen toggle
Different aspect ratios

Expected results after fix:

✅ No visible freeze during resize
✅ Animation continues smoothly
✅ No frame drops or stuttering
✅ Buttons remain responsive
✅ No visual artifacts or stretching

Problem Statement#

Despite multiple optimization attempts, the export buttons in video.mjs still do not move when the window is resized. The current implementation:

✅ Creates buttons once (persistent instances)
✅ Returns true from paint() for continuous rendering
✅ Detects screen dimension changes via screenChanged flag
✅ Calls reposition() only when screen changes
❌ BUTTONS STILL DON'T MOVE AFTER REFRAME

Current Implementation (video.mjs)#

// Module-level variables
let postBtn, mp4Btn, gifBtn, zipBtn;
let lastScreenWidth = 0;
let lastScreenHeight = 0;

function paint({ screen, api, wipe, ink, ui, /* ... */ }) {
  // Detect screen changes
  const screenChanged = screen.width !== lastScreenWidth || screen.height !== lastScreenHeight;
  if (screenChanged) {
    lastScreenWidth = screen.width;
    lastScreenHeight = screen.height;
  }

  // Transparent wipe during video playback
  if (rec.presenting || rec.playing) {
    wipe(0, 0, 0, 0); // DOM video overlay shows through
  } else {
    wipe(255);
  }

  // Draw buttons
  if (exportAvailable) {
    if (!postBtn) {
      postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
    } else if (screenChanged) {
      postBtn.reposition({ right: 6, bottom: 6, screen });
    }
    postBtn.paint(api);
    // ... similar for mp4Btn, gifBtn, zipBtn
  }

  return true; // Always repaint
}

Observations#

Screen change detection works - screenChanged flag correctly identifies when dimensions change
reposition() is called - Only when screen dimensions change (optimization working)
Buttons paint every frame - Return true keeps paint loop running
Transparent wipe used - wipe(0,0,0,0) for DOM video passthrough
Buttons never move visually - They stay at their original screen corner positions

Comparison: Working Examples from Other Pieces#

1. gameboy.mjs - Simple Recreate Pattern#

export function paint({ ink, wipe, screen, paste, sound, num, hud, ui }) {
  // Create buttons on first paint or when screen size changes
  if (!uiButtons.up || uiButtons.up.box.w !== 30) {
    createGameBoyButtons({ screen, ui });
  }
  // ... paint logic
}

function createGameBoyButtons({ screen, ui }) {
  const buttonSize = 24;
  const dpadY = screen.height - buttonSize * 3; // Calculate from screen
  
  // Recreate all buttons with new positions
  uiButtons.up = new ui.Button(dpadX + buttonSize, dpadY, buttonSize, buttonSize);
  uiButtons.down = new ui.Button(dpadX + buttonSize, dpadY + buttonSize * 2, buttonSize, buttonSize);
  // ... etc
}

Key differences:

❌ No reposition() method used
✅ Recreates buttons when screen changes
✅ Simple condition: check if button exists or size changed
🤔 Works reliably without complexity

2. stample.mjs - Direct Box Manipulation on Reframe#

function act({ event: e, screen, ui, /* ... */ }) {
  if (e.is("reframed")) {
    genPats({ screen, ui }); // Regenerate button grid
    
    // Direct box manipulation
    micRecordButton.box.y = screen.height - 32;
    patsButton.box.x = screen.width - patsButton.box.w;
  }
}

function genPats({ screen, ui }) {
  btns.length = 0; // Clear array
  for (let i = 0; i < pats; i += 1) {
    // Recreate all buttons
    const button = new ui.Button(x, y, width, height);
    btns.push(button);
  }
}

Key differences:

✅ Listens for "reframed" event in act()
✅ Directly modifies button.box properties
✅ Recreates button arrays completely
🎯 No reposition() method needed

3. notepat.mjs - Geometry Rebuild Pattern#

function setupButtons({ ui, screen, geo }) {
  // Recalculate layout metrics from screen
  const layout = getButtonLayoutMetrics(screen, { /* ... */ });
  
  buttonNotes.forEach((label, i) => {
    const geometry = [x, y, buttonWidth, buttonHeight];
    
    if (!buttons[label]) {
      buttons[label] = new ui.Button(...geometry);
    } else {
      // Replace box with new geo.Box instance
      buttons[label].box = new geo.Box(...geometry);
    }
  });
}

Key differences:

✅ Replaces entire button.box with new geo.Box
✅ Recalculates complete layout from screen dimensions
✅ Called on reframe/resize events
🎯 Direct box replacement, not reposition()

4. prutti.mjs - Hybrid Pattern#

function act({ event: e, screen, ui, /* ... */ }) {
  if (e.is("reframed")) {
    if (scrubButton) {
      // Update existing button's box properties
      scrubButton.box.x = buttonStartX;
      scrubButton.box.y = 0;
      scrubButton.box.w = buttonWidth;
      scrubButton.box.h = barHeight;
    } else {
      // Create new button
      scrubButton = new ui.Button(buttonStartX, 0, buttonWidth, barHeight);
    }
  }
}

Key differences:

✅ Responds to "reframed" event
✅ Directly mutates box properties (x, y, w, h)
✅ Recreates if missing
🎯 Manual property assignment

5. painting.mjs - Simple Creation, No Reposition#

function boot({ screen, ui, /* ... */ }) {
  if (!showMode) {
    printBtn = new ui.TextButton(`Print`, {
      bottom: butBottom,
      right: butSide,
      screen,
    });
  }
}

function paint({ screen, /* ... */ }) {
  if (printBtn) {
    printBtn.paint(api);
  }
}

Key differences:

✅ Creates once in boot
❌ No resize/reframe handling at all
🤔 Buttons may not work correctly after resize
⚠️ Not a good pattern for reframeable pieces

Pattern Analysis Summary#

Pattern	Used By	Reposition Method	Direct Box Access	Recreate	Event
Recreate on detect	gameboy	❌	❌	✅	paint check
Direct box mutation	stample, prutti	❌	✅	Sometimes	`"reframed"`
Box replacement	notepat	❌	✅ (replace)	❌	setup call
Reposition method	video (current)	✅	❌	❌	screen change
No handling	painting	❌	❌	❌	❌

Key Finding: NO OTHER PIECE USES `reposition()` METHOD#

All working examples use one of:

Recreate buttons (gameboy)
Direct button.box.x/y/w/h mutation (stample, prutti)
Replace button.box with new geo.Box (notepat)

NONE use the button.reposition() method that video.mjs is trying to use!

ROOT CAUSE IDENTIFIED: `reposition()` Implementation#

Investigation Result: ✅ `reposition()` EXISTS AND SHOULD WORK#

From lib/ui.mjs lines 839-842:

reposition(pos, txt) {
  if (txt) this.txt = txt;
  this.btn.box = Box.from(this.#computePosition(this.txt, pos));
}

How it works:

Takes pos object with {right, bottom, screen} or {left, top, screen}
Calls internal #computePosition() method to calculate x, y, w, h
Replaces this.btn.box with new Box instance

The #computePosition() method (lines 817-835):

#computePosition(txt, pos) {
  const m = TYPEFACE_UI.metrics(txt);
  const w = m.box.w + 8;
  const h = m.box.h + 8;
  
  let x = 0;
  let y = 0;
  
  if (pos.bottom !== undefined) {
    y += pos.screen.height - pos.bottom - h;
  } else {
    y += pos.top || 0;
  }
  
  if (pos.right !== undefined) {
    x += pos.screen.width - pos.right - w;
  } else {
    x += pos.left || 0;
  }
  
  return { x, y, w, h };
}

⚠️ CRITICAL FINDING: `reposition()` SHOULD WORK!#

The implementation looks correct:

✅ Handles corner anchoring (right/bottom)
✅ Uses current screen dimensions
✅ Replaces box entirely (not mutating)
✅ Recalculates position from scratch

🔍 Why Doesn't It Work Then?#

Hypothesis: The issue might be that reposition() is being called but:

Paint order issue? - Are buttons painted before reposition happens?
Screen object stale? - Is the screen object in paint() up-to-date?
Transparent wipe issue? - Old button graphics not clearing?
Box reference issue? - Internal this.btn reference not updating?
Event timing? - Need to respond to "reframed" event instead of detecting in paint?

Why video.mjs Uses `reposition()` When Others Don't#

Looking at the patterns:

TextButton has reposition() because it needs to recalculate text metrics
Regular Button doesn't have reposition() - pieces just recreate or mutate box
video.mjs is the ONLY piece using TextButton.reposition()
This is either cutting-edge API usage or... it's broken in practice

Does the transparent wipe affect button rendering?
- wipe(0,0,0,0) doesn't clear canvas
- Are old button graphics persisting?
- Do buttons need an opaque clear/redraw after reframe?
Is screen change detection working correctly?
- Add debug logging to verify screenChanged is true after resize
- Verify reposition() is actually being called
- Check if screen object has updated dimensions

Proposed Solutions (Priority Order)#

Solution 1: Use Direct Box Mutation (Like stample.mjs) ⭐ RECOMMENDED#

Switch from reposition() to direct box property mutation in act():

function act({ event: e, screen, /* ... */ }) {
  if (e.is("reframed")) {
    // Direct box manipulation for corner positioning
    if (postBtn) {
      postBtn.box.x = screen.width - 6 - postBtn.box.w;
      postBtn.box.y = screen.height - 6 - postBtn.box.h;
    }
    if (mp4Btn) {
      mp4Btn.box.x = screen.width - 44 - mp4Btn.box.w;
      mp4Btn.box.y = screen.height - 6 - mp4Btn.box.h;
    }
    // ... etc for gifBtn, zipBtn
  }
}

Pros:

✅ Proven pattern from working pieces
✅ Direct control over position
✅ Responds to reframe event (standard AC pattern)
✅ No dependency on potentially broken reposition() method

Cons:

❌ Need to know button dimensions (width/height)
⚠️ More manual calculation

Solution 2: Replace Box with geo.Box (Like notepat.mjs)#

function paint({ screen, geo, ui, /* ... */ }) {
  const screenChanged = /* ... */;
  
  if (!postBtn) {
    postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
  } else if (screenChanged) {
    const x = screen.width - 6 - postBtn.box.w;
    const y = screen.height - 6 - postBtn.box.h;
    postBtn.box = new geo.Box(x, y, postBtn.box.w, postBtn.box.h);
  }
}

Pros:

✅ Proven pattern from notepat
✅ Clean box replacement
✅ Stays in paint() (no act() needed)

Cons:

❌ Still need manual position calculation
❌ Requires geo.Box import

Solution 3: Recreate Buttons (Like gameboy.mjs)#

function paint({ screen, ui, /* ... */ }) {
  const screenChanged = /* ... */;
  
  if (!postBtn || screenChanged) {
    postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
    mp4Btn = new ui.TextButton("MP4", { right: 44, bottom: 6, screen });
    gifBtn = new ui.TextButton("GIF", { right: 76, bottom: 6, screen });
    zipBtn = new ui.TextButton("ZIP", { right: 108, bottom: 6, screen });
  }
}

Pros:

✅ Simplest solution
✅ Proven pattern
✅ TextButton constructor handles positioning

Cons:

⚠️ Recreating objects (may lose internal state?)
⚠️ Slightly less efficient
❓ Will this work with TextButton's corner positioning API?

Solution 4: Create New Piece (newvideo.mjs) for Testing#

Copy video.mjs → newvideo.mjs and test different patterns without breaking existing video playback:

cp system/public/aesthetic.computer/disks/video.mjs system/public/aesthetic.computer/disks/newvideo.mjs

Then test each solution in isolation.

Investigation Tasks#

[✅] Task 1: Check lib/ui.mjs - Does TextButton.reposition() exist and work?
- Result: YES - It exists and implementation looks correct!
- See analysis above for details
Task 2: Add debug logging to verify:
- screenChanged is true after resize
- reposition() is being called
- Screen dimensions in paint vs actual window size
Task 3: Try Solution 1 (direct box mutation in act with reframed event)
Task 4: Try Solution 3 (recreate buttons on screen change)
Task 5: Create newvideo.mjs test piece to isolate changes
Task 6: Document which pattern works and why

Debug Changes Added#

The following debug logging has been added to video.mjs:

Screen change detection (line ~186):

if (screenChanged) {
  console.log("🔄 SCREEN CHANGED:", {
    old: { w: lastScreenWidth, h: lastScreenHeight },
    new: { w: screen.width, h: screen.height }
  });
}

Button creation (line ~217):

if (!postBtn) {
  postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
  console.log("✅ POST button created at:", postBtn.btn.box);
}

Button reposition (line ~219):

else if (screenChanged) {
  const oldBox = { ...postBtn.btn.box };
  postBtn.reposition({ right: 6, bottom: 6, screen });
  console.log("🔄 POST button repositioned:", {
    old: oldBox,
    new: postBtn.btn.box,
    screen: { w: screen.width, h: screen.height }
  });
}

Reframed event (line ~497):

if (e.is("reframed")) {
  console.log("📐 REFRAMED event detected in act():", {
    screen: { w: screen.width, h: screen.height },
    postBtn: postBtn?.btn.box,
    mp4Btn: mp4Btn?.btn.box
  });
}

What to Look For#

When you resize the window, check the console for:

✅ Does "🔄 SCREEN CHANGED" appear?
✅ Does "🔄 POST button repositioned" appear?
✅ Does the new box position match the new screen dimensions?
✅ Does "📐 REFRAMED event" appear in act()?
❓ Is there a timing difference between reframed event and screen change detection?

Next Steps#

Test with debug logging - Resize window and observe console output
Analyze findings - Determine if:
- Screen change is detected ✓
- Reposition is called ✓
- New box coordinates are correct ✓
- But buttons still don't move visually ✗
If reposition() is working but not visible:
- Try adding opaque wipe after reframe
- Try listening to reframed event in paint()
- Try direct box mutation pattern from stample.mjs
If reposition() is NOT being called:
- Screen change detection logic may be wrong
- Screen object may not be updating
Create newvideo.mjs - Test alternative patterns without breaking existing functionality

⚠️ PARTIAL RESOLUTION (2025-10-20) - ISSUE STILL OPEN#

Progress: Fixed Canvas Freeze, Button Repositioning Still Broken#

The button repositioning issue revealed a deeper architectural problem in the worker-bios rendering pipeline during tape playback with rec.present(). The underlying freeze has been fixed, but buttons still do not reposition after window resize.

🔍 LOG ANALYSIS (2025-10-20)#

Current Implementation Issues Found#

Problem 1: Misleading Debug Logs The current video.mjs has both log statements firing every frame:

if (exportAvailable) {
  console.log('🎨 VIDEO PAINT: returning true'); // Inside if
  // ... paint buttons
}
console.log('🎨 VIDEO PAINT: returning true (no buttons yet)'); // Outside if - ALWAYS FIRES!
return true;

This makes debugging impossible because we see both messages even when buttons exist.

Problem 2: Buttons Reposition Every Frame The current code repositions buttons on EVERY paint() call:

postBtn.reposition({ right: 6, bottom: 6, screen });
mp4Btn.reposition({ right: 44, bottom: 6, screen });
gifBtn.reposition({ right: 76, bottom: 6, screen });
zipBtn.reposition({ right: 108, bottom: 6, screen });

While this should work, it's inefficient and the logs show no evidence of:

Button creation (no "✅ Buttons created" message)
Button position changes
Screen dimension tracking

Problem 3: No Screen Change Detection Unlike the documented plan, the current code has NO screen change detection:

// MISSING:
// let lastScreenWidth = 0;
// let lastScreenHeight = 0;
// const screenChanged = screen.width !== lastScreenWidth || screen.height !== lastScreenHeight;

Log Evidence#

From recent console output:

🖌️ WIPE: Using screen dimensions 300 x 197
⏭️ VIDEO: Dimension mismatch - waiting for worker. Canvas: 300 x 197 ImageData: 300 x 211
🎨 VIDEO PAINT: returning true
🎨 VIDEO PAINT: returning true (no buttons yet)
[Repeats ~30 times during dimension sync]

📐 WORKER: Updated screen dimensions from 300 x 197 to 300 x 215
🔄 REFRAME PATH (fallback): Created fresh imageData with dimensions: 300 x 215

🖌️ WIPE: Using screen dimensions 300 x 215
📸 VIDEO: Created imageData in normal path
🎨 VIDEO PAINT: returning true
🎨 VIDEO PAINT: returning true (no buttons yet)
[Continues indefinitely]

Key observations:

✅ Worker dimension updates work correctly
✅ Dimension mismatch fallback prevents canvas freeze
❌ NO button creation logs appear
❌ BOTH log messages fire every frame (logic error)
❓ Are buttons even being created? (exportAvailable might be false)

Hypothesis#

Buttons may not be created at all because:

rec.presenting may be false during initial playback
rec.recorded may not be set yet
exportAvailable evaluates to false, so buttons never instantiate

OR buttons ARE created but:

reposition() method doesn't actually update button positions
Button rendering happens at stale coordinates
The UI system doesn't pick up box changes from reposition()

Required Diagnostic Changes#

To properly debug, video.mjs needs:

// Track screen changes
let lastScreenWidth = 0;
let lastScreenHeight = 0;

function paint({ wipe, ink, screen, rec, ui, api, needsPaint }) {
  const screenChanged = screen.width !== lastScreenWidth || screen.height !== lastScreenHeight;
  
  if (screenChanged) {
    console.log('📐 VIDEO: Screen changed from', lastScreenWidth, 'x', lastScreenHeight, 
                'to', screen.width, 'x', screen.height);
    lastScreenWidth = screen.width;
    lastScreenHeight = screen.height;
  }
  
  const presenting = rec?.presenting ?? false;
  const exportAvailable = presenting || (rec?.recorded ?? false);
  
  console.log('📊 VIDEO: exportAvailable =', exportAvailable, 
              'presenting =', presenting, 
              'recorded =', rec?.recorded);
  
  if (exportAvailable) {
    if (!postBtn) {
      postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
      console.log('✅ POST button CREATED at:', postBtn.btn.box);
    } else if (screenChanged) {
      const oldBox = { ...postBtn.btn.box };
      postBtn.reposition({ right: 6, bottom: 6, screen });
      console.log('🔄 POST button REPOSITIONED from', oldBox, 'to', postBtn.btn.box);
    }
    
    console.log('🖼️ Painting buttons at positions:', {
      post: postBtn.btn.box,
      mp4: mp4Btn?.btn.box,
      gif: gifBtn?.btn.box,
      zip: zipBtn?.btn.box
    });
    
    postBtn.paint(api);
    mp4Btn?.paint(api);
    gifBtn?.paint(api);
    zipBtn?.paint(api);
    
    return true; // WITH buttons
  }
  
  return true; // WITHOUT buttons
}

This will reveal:

Whether buttons are ever created
When screen dimensions change
What rec.presenting and rec.recorded values are
Whether reposition() actually changes box coordinates

Initial Problem#

After window resize during tape playback, the canvas buffer would freeze - paint() continued running in the worker, but no new frames were displayed to the user. Buttons wouldn't reposition because the entire canvas was frozen.

Root Cause Discovery#

The worker's screen.width/height was only being updated from content.width/height, which came from the PREVIOUS frame. This created a one-frame lag that became permanent during tape playback when wipe() used the old screen dimensions to create transparent buffers.

The freeze sequence:

Window resizes → canvas resizes in bios
Worker still has old screen dimensions
wipe() creates buffer with old dimensions
Worker paints and sends buffer to bios
Bios receives buffer with wrong dimensions → dimension mismatch
Bios blocks rendering to prevent stretched/distorted canvas
Worker never receives updated dimensions → permanent freeze

Solution Implemented#

Three key changes to the rendering pipeline:

Immediate dimension update in worker (disk.mjs lines 8235-8260):

if (msg.type === "reframed") {
  const oldWidth = screen.width;
  const oldHeight = screen.height;

  screen.width = content.width;
  screen.height = content.height;
  screen.pixels = new Uint8ClampedArray(content.width * content.height * 4);

  console.log(`📐 WORKER: Updated screen dimensions from ${oldWidth} x ${oldHeight} to ${screen.width} x ${screen.height}`);
}

Worker now receives and applies new dimensions immediately via "reframed" message.

Use current dimensions for paint API (disk.mjs lines 9001-9007):
```
$api.screen = {
  width: screen.width,   // Changed from content.width
  height: screen.height, // Changed from content.height
  // ...
};
```
The screen object passed to piece's paint() now uses worker's current dimensions, not lagged content dimensions.

Message reordering (bios.mjs lines 1053-1062):

send({ 
  type: "reframed", 
  content: { 
    width: screen.width, 
    height: screen.height 
  } 
});
send({ type: "needs-paint" }); // Sent AFTER reframed

Ensure worker updates dimensions before starting paint.

Dimension mismatch handler for tape playback (bios.mjs lines 12502-12524):
```
if (underlayFrame) {
  console.log("⏭️ VIDEO: Dimension mismatch - waiting for worker to catch up");
  skipImmediateOverlays = true;
  setTimeout(() => send({ type: "needs-paint" }), 0);
  return;
}
```
If dimensions don't match during tape playback, skip the frame but keep requesting paint until dimensions sync. This prevents stretching while maintaining animation.

Result#

✅ Window resize during tape playback works smoothly
✅ Worker receives correct dimensions immediately
✅ Canvas doesn't freeze or stretch
✅ Animation continues without interruption
✅ Buttons will now reposition correctly (with proper implementation)

Logging Added for Debugging#

Worker dimension updates:

📐 WORKER: Updated screen dimensions from 300 x 162 to 300 x 192

Worker wipe operations:

🖌️ WIPE: Using screen dimensions 300 x 192

Bios dimension mismatch handling:

⏭️ VIDEO: Dimension mismatch - waiting for worker to catch up

Bios reframe fallback:

🔄 REFRAME PATH (fallback): Created fresh imageData with dimensions: 300 x 192

Test Results#

Confirmed working - resize during tape playback shows:

Worker updates dimensions: 📐 WORKER: Updated screen dimensions from 300 x 192 to 300 x 192
Worker uses new dimensions: 🖌️ WIPE: Using screen dimensions 300 x 192
Bios creates fresh imageData: 🔄 REFRAME PATH (fallback): Created fresh imageData
Animation continues smoothly without freezing or stretching

✅ Canvas freeze is FIXED
❌ Button repositioning is STILL BROKEN

Remaining Task: Button Repositioning Implementation#

Status: 🔴 CRITICAL - STILL BROKEN

The underlying freeze issue is now FIXED - the canvas correctly updates during resize and animation continues. However, the export buttons (POST/MP4/GIF/ZIP) still do not move to their new corner positions after window resize.

What's working:

✅ Canvas resizes correctly
✅ Animation continues without freezing
✅ Worker receives updated dimensions
✅ Screen buffer updates properly

What's NOT working:

❌ Buttons remain at their original pixel positions
❌ Buttons do not move to maintain corner alignment
❌ Button repositioning logic is ineffective

Next Steps: The button repositioning itself needs proper implementation using one of the proven patterns from other pieces (Solution 1: direct box mutation, or Solution 3: recreate buttons). Now that the canvas correctly updates during resize, buttons should be able to reposition using standard patterns.

Questions for Further Investigation#

Why was reposition() method chosen when no other piece uses it?
Does TextButton handle corner positioning differently than Button?
Is there documentation about proper button repositioning in AC framework?
Should the transparent wipe be changed to opaque after reframe to clear old graphics?
Does the DOM video overlay affect canvas button rendering after reframe?

Video Reframe Issue - Compositing Stack Investigation#

🎯 ACTUAL ISSUE (2025-10-20)#

Corrected Problem Statement#

🔍 Cleaned Up Logging (2025-10-20)#

Logs Removed#

Logs Kept/Updated#

Expected Log Pattern#

📊 Pipeline Flow During Reframe#

What Should Happen#

What Actually Happens (Freeze Condition)#

The Freeze Duration#

🔬 Investigation Findings (2025-10-20)#

Worker Message Queue Analysis#

Message Flow During Reframe#

ROOT CAUSE IDENTIFIED (2025-10-20) 🎯#

Diagnostic Logging Added#

🔬 Investigation: Compositing Stack Freeze#

The Core Issue#

Why It Happens#

Potential Solutions#

Option 1: Immediate Worker Notification ⭐ BEST#

Option 2: Render Mismatched Data with Scaling ⭐ PRACTICAL#

Option 3: Cache Last Valid Frame#

Option 4: Predictive Dimension Update#

📋 Completion Summary (2025-10-20)#

✅ Issue RESOLVED#

✅ UI Polish & Code Sharing (2025-10-20)#

📋 Action Items#

Phase 1: Confirm the Freeze Pattern ✅ DONE#

Phase 2: Investigate Worker Message Priority ✅ COMPLETE#

Phase 3: Implement Solution ✅ COMPLETE#

Phase 4: Performance Optimization#

🧪 Testing Checklist#

Problem Statement#

Current Implementation (video.mjs)#

Observations#

Comparison: Working Examples from Other Pieces#

1. gameboy.mjs - Simple Recreate Pattern#

2. stample.mjs - Direct Box Manipulation on Reframe#

3. notepat.mjs - Geometry Rebuild Pattern#

4. prutti.mjs - Hybrid Pattern#

5. painting.mjs - Simple Creation, No Reposition#

Pattern Analysis Summary#

Key Finding: NO OTHER PIECE USES reposition() METHOD#

ROOT CAUSE IDENTIFIED: reposition() Implementation#

Investigation Result: ✅ reposition() EXISTS AND SHOULD WORK#

⚠️ CRITICAL FINDING: reposition() SHOULD WORK!#

🔍 Why Doesn't It Work Then?#

Why video.mjs Uses reposition() When Others Don't#

Proposed Solutions (Priority Order)#

Solution 1: Use Direct Box Mutation (Like stample.mjs) ⭐ RECOMMENDED#

Solution 2: Replace Box with geo.Box (Like notepat.mjs)#

Solution 3: Recreate Buttons (Like gameboy.mjs)#

Solution 4: Create New Piece (newvideo.mjs) for Testing#

Investigation Tasks#

Debug Changes Added#

What to Look For#

Next Steps#

⚠️ PARTIAL RESOLUTION (2025-10-20) - ISSUE STILL OPEN#

Progress: Fixed Canvas Freeze, Button Repositioning Still Broken#

🔍 LOG ANALYSIS (2025-10-20)#

Current Implementation Issues Found#

Log Evidence#

Hypothesis#

Required Diagnostic Changes#

Initial Problem#

Root Cause Discovery#

Solution Implemented#

Result#

Logging Added for Debugging#

Test Results#

Remaining Task: Button Repositioning Implementation#

Questions for Further Investigation#

Key Finding: NO OTHER PIECE USES `reposition()` METHOD#

ROOT CAUSE IDENTIFIED: `reposition()` Implementation#

Investigation Result: ✅ `reposition()` EXISTS AND SHOULD WORK#

⚠️ CRITICAL FINDING: `reposition()` SHOULD WORK!#

Why video.mjs Uses `reposition()` When Others Don't#