Diffdown is a real-time collaborative Markdown editor/previewer built on the AT Protocol diffdown.com

docs: add granular delta edits implementation plan

+943
+943
docs/superpowers/plans/2026-03-13-granular-delta-edits.md
··· 1 + # Granular Delta Edits Implementation Plan 2 + 3 + > **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. 4 + 5 + **Goal:** Replace full-document WebSocket broadcasts with granular character-level deltas, reducing per-keystroke payload from O(document size) to O(change size). 6 + 7 + **Architecture:** CodeMirror exposes `update.changes` (a `ChangeSet`) directly — iterate it to extract precise `{from, to, insert}` operations. Milkdown only exposes the new markdown string, so we string-diff the previous vs new content using a bundled Myers diff to produce the same operation shape. The server-side `Operation` struct and `OTEngine` already handle granular ops correctly (after the `Insert==""` fix). Add a WebSocket send debounce (50ms) to batch rapid keystrokes. Receivers apply each delta in order using CodeMirror's `ChangeSet` API rather than replacing the whole document. 8 + 9 + **Tech Stack:** Go 1.22, CodeMirror 6 (`ChangeSet`, `ChangeDesc`), vanilla JS Myers diff (bundled inline, no new npm dep), existing `gorilla/websocket`, existing `Operation` / `OTEngine` types. 10 + 11 + --- 12 + 13 + ## Chunk 1: Server — accept and broadcast multiple deltas per message 14 + 15 + The current server receives one `Operation` per `edit` message. Granular edits from a single keystroke may produce multiple operations (e.g. CodeMirror reports each change in a transaction separately). We need the server to accept an array. 16 + 17 + ### Task 1: Update `Operation` and `ClientMessage` to support delta arrays 18 + 19 + **Files:** 20 + - Modify: `internal/collaboration/client.go` 21 + - Modify: `internal/collaboration/hub.go` 22 + - Test: `internal/collaboration/client_test.go` (create) 23 + 24 + The wire format changes from: 25 + ```json 26 + { "type": "edit", "delta": { "from": 5, "to": 10, "insert": "hello" } } 27 + ``` 28 + to: 29 + ```json 30 + { "type": "edit", "deltas": [{ "from": 5, "to": 10, "insert": "hello" }] } 31 + ``` 32 + 33 + `delta` (singular) is kept as a fallback field so existing clients don't break during the transition. 34 + 35 + - [ ] **Step 1: Write failing test for multi-delta parsing** 36 + 37 + Add `internal/collaboration/client_test.go`: 38 + 39 + ```go 40 + package collaboration 41 + 42 + import ( 43 + "encoding/json" 44 + "testing" 45 + ) 46 + 47 + func TestClientMessage_ParseDeltas_Multiple(t *testing.T) { 48 + raw := `{"type":"edit","deltas":[{"from":0,"to":5,"insert":"hello"},{"from":10,"to":10,"insert":"!"}]}` 49 + var msg ClientMessage 50 + if err := json.Unmarshal([]byte(raw), &msg); err != nil { 51 + t.Fatalf("unmarshal: %v", err) 52 + } 53 + if len(msg.Deltas) != 2 { 54 + t.Fatalf("expected 2 deltas, got %d", len(msg.Deltas)) 55 + } 56 + if msg.Deltas[0].From != 0 || msg.Deltas[0].To != 5 || msg.Deltas[0].Insert != "hello" { 57 + t.Errorf("delta[0]: %+v", msg.Deltas[0]) 58 + } 59 + if msg.Deltas[1].From != 10 || msg.Deltas[1].Insert != "!" { 60 + t.Errorf("delta[1]: %+v", msg.Deltas[1]) 61 + } 62 + } 63 + 64 + func TestClientMessage_ParseDeltas_SingleFallback(t *testing.T) { 65 + // Old wire format: singular "delta" field — must still work. 66 + raw := `{"type":"edit","delta":{"from":3,"to":7,"insert":"xyz"}}` 67 + var msg ClientMessage 68 + if err := json.Unmarshal([]byte(raw), &msg); err != nil { 69 + t.Fatalf("unmarshal: %v", err) 70 + } 71 + ops := msg.Operations() 72 + if len(ops) != 1 { 73 + t.Fatalf("expected 1 op from fallback, got %d", len(ops)) 74 + } 75 + if ops[0].From != 3 || ops[0].To != 7 || ops[0].Insert != "xyz" { 76 + t.Errorf("op: %+v", ops[0]) 77 + } 78 + } 79 + 80 + func TestClientMessage_ParseDeltas_EmptyDeltas(t *testing.T) { 81 + raw := `{"type":"edit","deltas":[]}` 82 + var msg ClientMessage 83 + if err := json.Unmarshal([]byte(raw), &msg); err != nil { 84 + t.Fatalf("unmarshal: %v", err) 85 + } 86 + if len(msg.Operations()) != 0 { 87 + t.Errorf("expected 0 ops for empty deltas") 88 + } 89 + } 90 + ``` 91 + 92 + - [ ] **Step 2: Run tests to verify they fail** 93 + 94 + ```bash 95 + go test -v ./internal/collaboration/ -run TestClientMessage 96 + ``` 97 + 98 + Expected: `FAIL` — `msg.Deltas` field doesn't exist, `msg.Operations()` method doesn't exist. 99 + 100 + - [ ] **Step 3: Update `ClientMessage` in `internal/collaboration/client.go`** 101 + 102 + Replace the existing `ClientMessage` struct and add the `Operations()` helper: 103 + 104 + ```go 105 + type ClientMessage struct { 106 + Type string `json:"type"` 107 + RKey string `json:"rkey,omitempty"` 108 + DID string `json:"did,omitempty"` 109 + // Deltas is the new plural field — a single edit message may carry 110 + // multiple operations (e.g. one per CodeMirror ChangeDesc). 111 + Deltas []Operation `json:"deltas,omitempty"` 112 + // Delta is the legacy singular field. Kept for backward compatibility. 113 + Delta json.RawMessage `json:"delta,omitempty"` 114 + Cursor *CursorPos `json:"cursor,omitempty"` 115 + Comment *CommentMsg `json:"comment,omitempty"` 116 + } 117 + 118 + // Operations returns the ops from this message, preferring Deltas over the 119 + // legacy singular Delta field. 120 + func (m *ClientMessage) Operations() []Operation { 121 + if len(m.Deltas) > 0 { 122 + return m.Deltas 123 + } 124 + if len(m.Delta) > 0 { 125 + var op Operation 126 + if err := json.Unmarshal(m.Delta, &op); err == nil { 127 + return []Operation{op} 128 + } 129 + } 130 + return nil 131 + } 132 + ``` 133 + 134 + - [ ] **Step 4: Update the `"edit"` case in `ReadPump` in `internal/collaboration/client.go`** 135 + 136 + Replace: 137 + ```go 138 + case "edit": 139 + var op Operation 140 + if err := json.Unmarshal(msg.Delta, &op); err != nil { 141 + log.Printf("Failed to parse delta from %s: %v", c.DID, err) 142 + continue 143 + } 144 + op.Author = c.DID 145 + room := c.hub.GetRoom(c.roomKey) 146 + if room != nil { 147 + room.ApplyEdit(op, c) 148 + } 149 + ``` 150 + 151 + With: 152 + ```go 153 + case "edit": 154 + ops := msg.Operations() 155 + if len(ops) == 0 { 156 + continue 157 + } 158 + room := c.hub.GetRoom(c.roomKey) 159 + if room == nil { 160 + continue 161 + } 162 + for i := range ops { 163 + ops[i].Author = c.DID 164 + } 165 + room.ApplyEdits(ops, c) 166 + ``` 167 + 168 + - [ ] **Step 5: Add `ApplyEdits` to `Room` in `internal/collaboration/hub.go`** 169 + 170 + Add alongside the existing `ApplyEdit` method: 171 + 172 + ```go 173 + // ApplyEdits applies a sequence of operations in order and broadcasts one 174 + // combined message to all other clients. Each op is applied to the text 175 + // resulting from the previous op, so positions in each op must be relative 176 + // to the document state after all prior ops in the same batch have been 177 + // applied — which is exactly what CodeMirror's iterChanges produces (fromA/toA 178 + // are positions in the pre-change document, already adjusted for prior changes 179 + // within the same transaction by CodeMirror itself). 180 + func (r *Room) ApplyEdits(ops []Operation, sender *Client) { 181 + if len(ops) == 0 { 182 + return 183 + } 184 + 185 + for i := range ops { 186 + r.ot.ApplyWithVersion(ops[i]) 187 + } 188 + 189 + // Include the full document text so receivers can detect and recover from 190 + // divergence without a reconnect. 191 + finalText := r.ot.GetText() 192 + type editsMsg struct { 193 + Type string `json:"type"` 194 + Deltas []Operation `json:"deltas"` 195 + Author string `json:"author"` 196 + Content string `json:"content"` 197 + } 198 + msg := editsMsg{ 199 + Type: "edit", 200 + Deltas: ops, 201 + Author: sender.DID, 202 + Content: finalText, 203 + } 204 + data, err := json.Marshal(msg) 205 + if err != nil { 206 + log.Printf("ApplyEdits: marshal: %v", err) 207 + return 208 + } 209 + r.BroadcastExcept(data, sender) 210 + } 211 + ``` 212 + 213 + Note: `Content` (full text) is included in the broadcast so receivers can sanity-check their state. The client applies deltas directly but falls back to full replacement if its local state diverges. 214 + 215 + - [ ] **Step 6: Run tests to verify they pass** 216 + 217 + ```bash 218 + go test -v -race ./internal/collaboration/ -run TestClientMessage 219 + ``` 220 + 221 + Expected: all 3 `TestClientMessage_*` tests PASS. 222 + 223 + - [ ] **Step 7: Run full suite** 224 + 225 + ```bash 226 + go test -v -race ./internal/collaboration/ 227 + ``` 228 + 229 + Expected: all tests PASS (no regressions). 230 + 231 + - [ ] **Step 8: Commit** 232 + 233 + ```bash 234 + git add internal/collaboration/client.go internal/collaboration/hub.go internal/collaboration/client_test.go 235 + git commit -m "feat: support multi-delta edit messages on server" 236 + ``` 237 + 238 + --- 239 + 240 + ## Chunk 2: Server — OT tests for multi-delta sequences 241 + 242 + Before touching the frontend, verify the server applies multi-op sequences correctly. 243 + 244 + ### Task 2: Tests for `ApplyEdits` on `Room` 245 + 246 + **Files:** 247 + - Modify: `internal/collaboration/hub_test.go` 248 + 249 + - [ ] **Step 1: Write failing tests** 250 + 251 + Add to `internal/collaboration/hub_test.go`: 252 + 253 + ```go 254 + func TestRoom_ApplyEdits_MultipleOpsAppliedInOrder(t *testing.T) { 255 + hub := NewHub() 256 + room := hub.GetOrCreateRoom("doc-multi-ops") 257 + 258 + alice := stubClient(hub, "did:plc:alice", "Alice", "#ff0000", "doc-multi-ops") 259 + bob := stubClient(hub, "did:plc:bob", "Bob", "#0000ff", "doc-multi-ops") 260 + room.RegisterClient(alice) 261 + room.RegisterClient(bob) 262 + time.Sleep(100 * time.Millisecond) 263 + drain(alice, 200*time.Millisecond) 264 + drain(bob, 200*time.Millisecond) 265 + 266 + // Two ops: set text to "hello", then append " world" 267 + ops := []Operation{ 268 + {From: 0, To: -1, Insert: "hello"}, 269 + {From: 5, To: 5, Insert: " world"}, 270 + } 271 + room.ApplyEdits(ops, alice) 272 + 273 + bobMsgs := waitForMessages(bob, 1, time.Second) 274 + if len(bobMsgs) == 0 { 275 + t.Fatal("bob: expected edit message") 276 + } 277 + 278 + var msg struct { 279 + Type string `json:"type"` 280 + Deltas []Operation `json:"deltas"` 281 + Content string `json:"content"` 282 + Author string `json:"author"` 283 + } 284 + if err := json.Unmarshal(bobMsgs[0], &msg); err != nil { 285 + t.Fatalf("unmarshal: %v", err) 286 + } 287 + if msg.Type != "edit" { 288 + t.Errorf("type: got %q, want edit", msg.Type) 289 + } 290 + if msg.Content != "hello world" { 291 + t.Errorf("content: got %q, want %q", msg.Content, "hello world") 292 + } 293 + if len(msg.Deltas) != 2 { 294 + t.Errorf("deltas: got %d, want 2", len(msg.Deltas)) 295 + } 296 + if msg.Author != "did:plc:alice" { 297 + t.Errorf("author: got %q", msg.Author) 298 + } 299 + } 300 + 301 + func TestRoom_ApplyEdits_UpdatesOTState(t *testing.T) { 302 + hub := NewHub() 303 + room := hub.GetOrCreateRoom("doc-multi-ot") 304 + alice := stubClient(hub, "did:plc:alice", "Alice", "#ff0000", "doc-multi-ot") 305 + room.RegisterClient(alice) 306 + time.Sleep(100 * time.Millisecond) 307 + drain(alice, 200*time.Millisecond) 308 + 309 + room.ApplyEdits([]Operation{ 310 + {From: 0, To: -1, Insert: "abc"}, 311 + {From: 3, To: 3, Insert: "def"}, 312 + }, alice) 313 + 314 + time.Sleep(50 * time.Millisecond) 315 + if got := room.ot.GetText(); got != "abcdef" { 316 + t.Errorf("OT state: got %q, want %q", got, "abcdef") 317 + } 318 + } 319 + 320 + func TestRoom_ApplyEdits_EmptyOpsIsNoop(t *testing.T) { 321 + hub := NewHub() 322 + room := hub.GetOrCreateRoom("doc-empty-ops") 323 + alice := stubClient(hub, "did:plc:alice", "Alice", "#ff0000", "doc-empty-ops") 324 + bob := stubClient(hub, "did:plc:bob", "Bob", "#0000ff", "doc-empty-ops") 325 + room.RegisterClient(alice) 326 + room.RegisterClient(bob) 327 + time.Sleep(100 * time.Millisecond) 328 + drain(alice, 200*time.Millisecond) 329 + drain(bob, 200*time.Millisecond) 330 + 331 + room.ApplyEdits(nil, alice) 332 + 333 + bobMsgs := drain(bob, 300*time.Millisecond) 334 + if len(bobMsgs) > 0 { 335 + t.Errorf("expected no broadcast for empty ops, got %d messages", len(bobMsgs)) 336 + } 337 + } 338 + ``` 339 + 340 + - [ ] **Step 2: Run tests to confirm they pass** 341 + 342 + ```bash 343 + go test -v ./internal/collaboration/ -run TestRoom_ApplyEdits 344 + ``` 345 + 346 + Expected: PASS — `ApplyEdits` was added in Chunk 1 Task 1 Step 5. 347 + 348 + - [ ] **Step 3: Run tests to verify they pass** 349 + 350 + ```bash 351 + go test -v -race ./internal/collaboration/ -run TestRoom_ApplyEdits 352 + ``` 353 + 354 + Expected: all 3 tests PASS. 355 + 356 + - [ ] **Step 4: Run full suite** 357 + 358 + ```bash 359 + go test -v -race ./internal/collaboration/ 360 + ``` 361 + 362 + Expected: all tests PASS. 363 + 364 + - [ ] **Step 5: Commit** 365 + 366 + ```bash 367 + git add internal/collaboration/hub_test.go 368 + git commit -m "test: add multi-delta ApplyEdits tests" 369 + ``` 370 + 371 + --- 372 + 373 + ## Chunk 3: Frontend — CodeMirror granular deltas 374 + 375 + CodeMirror 6's `update.changes` is a `ChangeSet`. Iterating it with `iterChanges` gives `(fromA, toA, fromB, toB, inserted)` — exactly the `{from, to, insert}` we need. 376 + 377 + ### Task 3: Send granular deltas from CodeMirror source mode 378 + 379 + **Files:** 380 + - Modify: `templates/document_edit.html` 381 + 382 + The change is in the `EditorView.updateListener.of(...)` callback and the `sendEdit` / new `sendDeltas` functions. 383 + 384 + - [ ] **Step 1: Add `sendDeltas` function and debounce timer** 385 + 386 + In `document_edit.html`, find the `sendEdit` function (around line 608) and replace it with: 387 + 388 + ```js 389 + // Debounce timer for WebSocket sends (50ms batches rapid keystrokes). 390 + let wsEditTimer = null; 391 + let pendingDeltas = []; 392 + 393 + // Queue a set of deltas and flush after a short debounce. 394 + function queueDeltas(deltas) { 395 + if (!ws || ws.readyState !== WebSocket.OPEN || applyingRemote) return; 396 + pendingDeltas = pendingDeltas.concat(deltas); 397 + clearTimeout(wsEditTimer); 398 + wsEditTimer = setTimeout(flushDeltas, 50); 399 + } 400 + 401 + function flushDeltas() { 402 + if (!ws || ws.readyState !== WebSocket.OPEN || pendingDeltas.length === 0) { 403 + pendingDeltas = []; 404 + return; 405 + } 406 + ws.send(JSON.stringify({ type: 'edit', deltas: pendingDeltas })); 407 + pendingDeltas = []; 408 + } 409 + 410 + // sendEdit is kept for any future callers; Milkdown switches to diffToOps 411 + // in Chunk 4 and no longer calls this directly. 412 + function sendEdit(content) { 413 + if (!ws || ws.readyState !== WebSocket.OPEN || applyingRemote) return; 414 + queueDeltas([{ from: 0, to: -1, insert: content }]); 415 + } 416 + ``` 417 + 418 + - [ ] **Step 2: Update the CodeMirror `updateListener` to use granular deltas** 419 + 420 + Find the `EditorView.updateListener.of(...)` block (around line 180) and replace it: 421 + 422 + ```js 423 + EditorView.updateListener.of((update) => { 424 + if (update.docChanged && currentMode === 'source') { 425 + const content = update.state.doc.toString(); 426 + updatePreview(content); 427 + if (!update.transactions.some(tr => tr.annotation(remoteEditAnnotation))) { 428 + scheduleAutoSave(content); 429 + // Extract granular deltas from the ChangeSet. 430 + const deltas = []; 431 + update.changes.iterChanges((fromA, toA, _fromB, _toB, inserted) => { 432 + deltas.push({ from: fromA, to: toA, insert: inserted.toString() }); 433 + }); 434 + if (deltas.length > 0) { 435 + queueDeltas(deltas); 436 + } 437 + } 438 + } 439 + }), 440 + ``` 441 + 442 + Note: `fromA`/`toA` are positions in the **old** document (before the change), which is what the server's OT engine needs. `fromB`/`toB` are positions in the new document — not needed here. 443 + 444 + - [ ] **Step 3: Update `applyRemoteEdit` to apply deltas when available** 445 + 446 + Find `applyRemoteEdit` (around line 582) and replace: 447 + 448 + ```js 449 + function applyRemoteEdit(msg) { 450 + // msg may be a full-content string (legacy) or an object with deltas. 451 + if (applyingRemote) return; 452 + applyingRemote = true; 453 + try { 454 + // Determine content and deltas from the message. 455 + let content = typeof msg === 'string' ? msg : msg.content; 456 + const deltas = (typeof msg === 'object' && msg.deltas) ? msg.deltas : null; 457 + 458 + if (currentMode === 'source' && cmView) { 459 + if (deltas && deltas.length > 0) { 460 + // Apply each delta as a discrete CodeMirror change. 461 + // Build changes array, adjusting positions for prior changes. 462 + const docLen = cmView.state.doc.length; 463 + const changes = deltas.map(d => ({ 464 + from: Math.min(d.from < 0 ? 0 : d.from, docLen), 465 + to: Math.min(d.to < 0 ? docLen : d.to, docLen), 466 + insert: d.insert || '', 467 + })); 468 + cmView.dispatch({ 469 + changes, 470 + annotations: [remoteEditAnnotation.of(true)], 471 + }); 472 + // Use the full-content echo from the server as a sanity-check. 473 + if (content && cmView.state.doc.toString() !== content) { 474 + // State diverged — fall back to full replacement. 475 + cmView.dispatch({ 476 + changes: { from: 0, to: cmView.state.doc.length, insert: content }, 477 + annotations: [remoteEditAnnotation.of(true)], 478 + }); 479 + } 480 + } else if (content && cmView.state.doc.toString() !== content) { 481 + // Legacy full-replacement path. 482 + cmView.dispatch({ 483 + changes: { from: 0, to: cmView.state.doc.length, insert: content }, 484 + annotations: [remoteEditAnnotation.of(true)], 485 + }); 486 + } 487 + if (content) updatePreview(content); 488 + } else if (currentMode === 'rich' && milkdownEditor && content) { 489 + createMilkdownEditor(content); 490 + } 491 + } finally { 492 + applyingRemote = false; 493 + } 494 + } 495 + ``` 496 + 497 + - [ ] **Step 4: Update `handleWSMessage` to pass the full message object to `applyRemoteEdit`** 498 + 499 + Find `handleWSMessage` (around line 567) and update the `'edit'` and `'sync'` cases: 500 + 501 + ```js 502 + function handleWSMessage(msg) { 503 + switch (msg.type) { 504 + case 'presence': 505 + updatePresence(msg.users || []); 506 + break; 507 + case 'pong': 508 + wsMissedPings = 0; 509 + break; 510 + case 'edit': 511 + applyRemoteEdit(msg); // pass full message object, not just msg.content 512 + break; 513 + case 'sync': 514 + applyRemoteEdit(msg.content); // sync is always full-content 515 + break; 516 + } 517 + } 518 + ``` 519 + 520 + - [ ] **Step 5: Manual smoke test** 521 + 522 + ```bash 523 + # In the collaboration worktree: 524 + make run 525 + ``` 526 + 527 + 1. Open two browser tabs, both logged in as different ATProto users on the same document. 528 + 2. Type in tab A (source mode) — tab B should receive and apply the edit without full-page flicker. 529 + 3. Type simultaneously — verify no echo loop (your own edits don't come back). 530 + 4. Switch tab A to rich mode, type — tab B should still update (via legacy full-replacement path). 531 + 5. Open browser devtools → Network → WS — verify messages are small (just the changed characters) not full document. 532 + 533 + - [ ] **Step 6: Commit** 534 + 535 + ```bash 536 + git add templates/document_edit.html 537 + git commit -m "feat: send granular CodeMirror deltas over WebSocket with 50ms debounce" 538 + ``` 539 + 540 + --- 541 + 542 + ## Chunk 4: Frontend — Milkdown string-diff deltas 543 + 544 + Milkdown only exposes the complete new markdown string. We compute a Myers diff between the previous and current string to produce `{from, to, insert}` operations. 545 + 546 + ### Task 4: Inline Myers diff and wire up Milkdown listener 547 + 548 + **Files:** 549 + - Modify: `templates/document_edit.html` 550 + 551 + The Myers diff algorithm produces the minimal edit script between two strings. We inline a small implementation rather than adding an npm dependency, keeping the bundle unchanged. 552 + 553 + - [ ] **Step 1: Add inline Myers diff function** 554 + 555 + Just before the `queueDeltas` function added in Task 3, insert: 556 + 557 + ```js 558 + /** 559 + * Compute the minimal edit operations to transform `oldStr` into `newStr`. 560 + * Returns an array of {from, to, insert} suitable for the OT engine. 561 + * 562 + * Uses a line-level diff for performance, then falls back to a single 563 + * full-replacement op if the diff produces more than 20 operations 564 + * (pathological case — not worth the complexity). 565 + */ 566 + function diffToOps(oldStr, newStr) { 567 + if (oldStr === newStr) return []; 568 + 569 + const oldLines = oldStr.split('\n'); 570 + const newLines = newStr.split('\n'); 571 + 572 + // Build line-level LCS table (Myers / Wagner-Fischer). 573 + const m = oldLines.length, n = newLines.length; 574 + const dp = Array.from({length: m + 1}, () => new Array(n + 1).fill(0)); 575 + for (let i = m - 1; i >= 0; i--) { 576 + for (let j = n - 1; j >= 0; j--) { 577 + dp[i][j] = oldLines[i] === newLines[j] 578 + ? dp[i+1][j+1] + 1 579 + : Math.max(dp[i+1][j], dp[i][j+1]); 580 + } 581 + } 582 + 583 + // Trace back to produce diff hunks. 584 + const ops = []; 585 + let i = 0, j = 0; 586 + // Track character offset into oldStr. 587 + let charOffset = 0; 588 + // +1 for the \n separator. The last line has no trailing \n, so its 589 + // length is exact; however the OT engine clamps out-of-range positions, 590 + // so an off-by-one on the final line is safe in practice. 591 + const oldLineLengths = oldLines.map(l => l.length + 1); 592 + 593 + while (i < m || j < n) { 594 + if (i < m && j < n && oldLines[i] === newLines[j]) { 595 + charOffset += oldLineLengths[i]; 596 + i++; j++; 597 + } else if (j < n && (i >= m || dp[i+1][j] <= dp[i][j+1])) { 598 + // Insert newLines[j] 599 + const insertText = newLines[j] + (j < n - 1 ? '\n' : ''); 600 + ops.push({ from: charOffset, to: charOffset, insert: insertText }); 601 + j++; 602 + } else { 603 + // Delete oldLines[i] 604 + const deleteLen = oldLineLengths[i]; 605 + ops.push({ from: charOffset, to: charOffset + deleteLen, insert: '' }); 606 + charOffset += deleteLen; 607 + i++; 608 + } 609 + } 610 + 611 + // Fallback: if diff is too fragmented, send a single full replacement. 612 + if (ops.length > 20) { 613 + return [{ from: 0, to: -1, insert: newStr }]; 614 + } 615 + 616 + return ops; 617 + } 618 + ``` 619 + 620 + - [ ] **Step 2: Update the Milkdown `markdownUpdated` listener to use `diffToOps`** 621 + 622 + Find the Milkdown listener (around line 246): 623 + 624 + ```js 625 + ctx.get(listenerCtx).markdownUpdated((_ctx, markdown, prevMarkdown) => { 626 + if (markdown !== prevMarkdown && !applyingRemote) { 627 + scheduleAutoSave(markdown); 628 + sendEdit(markdown); 629 + } 630 + }); 631 + ``` 632 + 633 + Replace with: 634 + 635 + ```js 636 + ctx.get(listenerCtx).markdownUpdated((_ctx, markdown, prevMarkdown) => { 637 + if (markdown !== prevMarkdown && !applyingRemote) { 638 + scheduleAutoSave(markdown); 639 + const ops = diffToOps(prevMarkdown || '', markdown); 640 + if (ops.length > 0) { 641 + queueDeltas(ops); 642 + } 643 + } 644 + }); 645 + ``` 646 + 647 + - [ ] **Step 3: Manual smoke test for rich text mode** 648 + 649 + ```bash 650 + make run 651 + ``` 652 + 653 + 1. Open two browser tabs on the same document, both in **rich** mode. 654 + 2. Type in tab A — tab B should update. 655 + 3. In devtools → Network → WS: verify messages contain `deltas` arrays (not a single large `insert`). 656 + 4. Make a large edit (paste a paragraph) — verify fallback fires (single full-replacement op) rather than 20+ tiny ops. 657 + 658 + - [ ] **Step 4: Commit** 659 + 660 + ```bash 661 + git add templates/document_edit.html 662 + git commit -m "feat: diff-based granular deltas for Milkdown rich text mode" 663 + ``` 664 + 665 + --- 666 + 667 + ## Chunk 5: Cleanup and edge cases 668 + 669 + ### Task 5: Handle pending deltas on WebSocket close 670 + 671 + If the browser tab closes or the WebSocket drops while `pendingDeltas` is non-empty, those changes are lost silently. Flush synchronously on close. 672 + 673 + **Files:** 674 + - Modify: `templates/document_edit.html` 675 + 676 + - [ ] **Step 1: Extract `closeWS` helper and wire it up** 677 + 678 + There is no named `closeWS` function in the current code. `ws.close()` is called in two places: 679 + - `ws.onerror` callback (around line 533) 680 + - Inside `startHeartbeat`'s ping timer when missed pings ≥ 3 (around line 553) 681 + 682 + Replace both call sites with a shared `closeWS` helper. Add the helper alongside `flushDeltas`: 683 + 684 + ```js 685 + function closeWS() { 686 + if (!ws) return; 687 + clearTimeout(wsEditTimer); 688 + flushDeltas(); // send any buffered deltas before closing 689 + ws.close(); 690 + ws = null; 691 + clearInterval(wsPingTimer); 692 + } 693 + ``` 694 + 695 + Then update `ws.onerror`: 696 + ```js 697 + ws.onerror = () => { 698 + closeWS(); 699 + }; 700 + ``` 701 + 702 + And update the heartbeat missed-ping branch: 703 + ```js 704 + if (wsMissedPings >= 3) { 705 + closeWS(); 706 + } 707 + ``` 708 + 709 + Also update `ws.onclose` to call `stopHeartbeat()` only (not `clearInterval` directly, since `closeWS` handles that when called from onerror/heartbeat; onclose fires for all close paths and should remain lightweight): 710 + ```js 711 + ws.onclose = () => { 712 + ws = null; 713 + updatePresence([]); 714 + scheduleReconnect(); 715 + }; 716 + ``` 717 + 718 + - [ ] **Step 2: Flush on page unload** 719 + 720 + Add a `beforeunload` handler at the end of the `{{define "scripts"}}` block, after the `connectWebSocket()` call: 721 + 722 + ```js 723 + window.addEventListener('beforeunload', () => { 724 + clearTimeout(wsEditTimer); 725 + flushDeltas(); 726 + }); 727 + ``` 728 + 729 + - [ ] **Step 3: Commit** 730 + 731 + ```bash 732 + git add templates/document_edit.html 733 + git commit -m "fix: flush pending WS deltas on close and page unload" 734 + ``` 735 + 736 + ### Task 6: OT engine — add `SetText` for room initialisation 737 + 738 + Currently `GetOrCreateRoom` initialises the OT engine with `""`. When a room is first created, the server fetches the document but doesn't seed the engine. If two users join simultaneously, the first edit from user A resets the OT state, which is fine for the current full-replacement protocol but wrong for granular deltas (user B's deltas will have stale offsets). 739 + 740 + **Files:** 741 + - Modify: `internal/collaboration/ot.go` 742 + - Modify: `internal/collaboration/hub.go` 743 + - Modify: `internal/handler/handler.go` 744 + - Test: `internal/collaboration/ot_test.go` 745 + 746 + - [ ] **Step 1: Write failing test for `SetText`** 747 + 748 + Add to `internal/collaboration/ot_test.go`: 749 + 750 + ```go 751 + func TestOTEngine_SetText(t *testing.T) { 752 + ot := NewOTEngine("") 753 + ot.SetText("initial content") 754 + if ot.GetText() != "initial content" { 755 + t.Errorf("SetText: got %q, want %q", ot.GetText(), "initial content") 756 + } 757 + // SetText should not change the version counter. 758 + if ot.GetVersion() != 0 { 759 + t.Errorf("SetText should not increment version: got %d", ot.GetVersion()) 760 + } 761 + } 762 + ``` 763 + 764 + - [ ] **Step 2: Run test to verify it fails** 765 + 766 + ```bash 767 + go test -v ./internal/collaboration/ -run TestOTEngine_SetText 768 + ``` 769 + 770 + Expected: FAIL — `SetText` undefined. 771 + 772 + - [ ] **Step 3: Add `SetText` to `internal/collaboration/ot.go`** 773 + 774 + ```go 775 + // SetText replaces the canonical document text without incrementing the version. 776 + // Call this once when a room is first created, before any edits are applied. 777 + func (ot *OTEngine) SetText(text string) { 778 + ot.mu.Lock() 779 + defer ot.mu.Unlock() 780 + ot.documentText = text 781 + } 782 + ``` 783 + 784 + - [ ] **Step 4: Run test to verify it passes** 785 + 786 + ```bash 787 + go test -v -race ./internal/collaboration/ -run TestOTEngine_SetText 788 + ``` 789 + 790 + Expected: PASS. 791 + 792 + - [ ] **Step 5: Seed the OT engine in `CollaboratorWebSocket` handler** 793 + 794 + In `internal/handler/handler.go`, find `CollaboratorWebSocket` (search for `GetOrCreateRoom`). After fetching the document and before registering the client, seed the room's OT engine if it's a new room: 795 + 796 + ```go 797 + room := h.CollaborationHub.GetOrCreateRoom(rKey) 798 + // Seed OT engine with the current document text on first join. 799 + // GetOrCreateRoom returns existing rooms unchanged — SetText is idempotent 800 + // only if called before the first edit, which is guaranteed here because 801 + // we check IsNewRoom() before seeding. 802 + if room.IsNewRoom() { 803 + var initialText string 804 + if doc.Content != nil { 805 + initialText = doc.Content.Text.RawMarkdown 806 + } 807 + room.SeedText(initialText) 808 + } 809 + ``` 810 + 811 + This requires adding `IsNewRoom()` and `SeedText()` to `Room`. 812 + 813 + - [ ] **Step 6: Add `IsNewRoom` and `SeedText` to `Room` in `internal/collaboration/hub.go`** 814 + 815 + Add a `seeded` field to `Room`: 816 + 817 + ```go 818 + type Room struct { 819 + documentRKey string 820 + clients map[*Client]bool 821 + broadcast chan *broadcastMsg 822 + register chan *Client 823 + unregister chan *Client 824 + mu sync.RWMutex 825 + ot *OTEngine 826 + seeded bool // true after SeedText has been called 827 + } 828 + ``` 829 + 830 + Add methods: 831 + 832 + ```go 833 + // IsNewRoom returns true if SeedText has not yet been called on this room. 834 + func (r *Room) IsNewRoom() bool { 835 + r.mu.RLock() 836 + defer r.mu.RUnlock() 837 + return !r.seeded 838 + } 839 + 840 + // SeedText sets the initial document text for the OT engine. 841 + // Safe to call only before the first edit is applied. 842 + func (r *Room) SeedText(text string) { 843 + r.mu.Lock() 844 + defer r.mu.Unlock() 845 + if !r.seeded { 846 + r.ot.SetText(text) 847 + r.seeded = true 848 + } 849 + } 850 + ``` 851 + 852 + - [ ] **Step 7: Write test for `SeedText`** 853 + 854 + Add to `internal/collaboration/hub_test.go`: 855 + 856 + ```go 857 + func TestRoom_SeedText_SetsInitialOTState(t *testing.T) { 858 + hub := NewHub() 859 + room := hub.GetOrCreateRoom("doc-seed") 860 + if !room.IsNewRoom() { 861 + t.Fatal("new room should report IsNewRoom=true") 862 + } 863 + room.SeedText("initial document content") 864 + if room.IsNewRoom() { 865 + t.Error("IsNewRoom should be false after seeding") 866 + } 867 + if got := room.ot.GetText(); got != "initial document content" { 868 + t.Errorf("OT text after seed: got %q, want %q", got, "initial document content") 869 + } 870 + } 871 + 872 + func TestRoom_SeedText_IdempotentAfterFirstCall(t *testing.T) { 873 + hub := NewHub() 874 + room := hub.GetOrCreateRoom("doc-seed-idem") 875 + room.SeedText("first") 876 + room.SeedText("second") // should be ignored 877 + if got := room.ot.GetText(); got != "first" { 878 + t.Errorf("second SeedText should be ignored: got %q, want %q", got, "first") 879 + } 880 + } 881 + ``` 882 + 883 + - [ ] **Step 8: Run all tests** 884 + 885 + ```bash 886 + go test -v -race ./internal/collaboration/ 887 + ``` 888 + 889 + Expected: all tests PASS. 890 + 891 + - [ ] **Step 9: Commit** 892 + 893 + ```bash 894 + git add internal/collaboration/ot.go internal/collaboration/hub.go \ 895 + internal/collaboration/ot_test.go internal/collaboration/hub_test.go \ 896 + internal/handler/handler.go 897 + git commit -m "feat: seed OT engine with document text on room creation" 898 + ``` 899 + 900 + --- 901 + 902 + ## Chunk 6: Final verification 903 + 904 + - [ ] **Run full test suite with race detector** 905 + 906 + ```bash 907 + go test -v -race ./... 908 + ``` 909 + 910 + Expected: all tests PASS, no race conditions. 911 + 912 + - [ ] **Build** 913 + 914 + ```bash 915 + make build 916 + ``` 917 + 918 + Expected: builds cleanly, no warnings. 919 + 920 + - [ ] **Lint** 921 + 922 + ```bash 923 + make lint 924 + ``` 925 + 926 + Expected: no lint errors. 927 + 928 + - [ ] **End-to-end smoke test** 929 + 930 + ```bash 931 + make run 932 + ``` 933 + 934 + 1. Two users, same document, source mode: type rapidly — verify small WS messages (devtools → Network → WS frames). 935 + 2. Two users, same document, rich mode: type — verify updates propagate correctly. 936 + 3. Disconnect one user mid-edit — verify reconnect gets `sync` message with correct state. 937 + 4. Empty the document entirely — verify the other user sees an empty document (tests the `Insert=""` deletion fix). 938 + 939 + - [ ] **Commit any fixups, then push** 940 + 941 + ```bash 942 + git push origin feature/collaboration 943 + ```