···11+# PE Frame-Based Redesign — Phase 2: Emulator Core — PE Rewrite
22+33+**Goal:** Rewrite `ProcessingElement` with frame-based matching, reversed pipeline order (IFETCH before MATCH), mode-driven output routing via `OutputStyle`, and PE-level EXTRACT_TAG handling.
44+55+**Architecture:** Replace the `matching_store` 2D array + `gen_counters` with `frames` (list of frame slot arrays), `tag_store` (act_id → frame_id mapping), `presence` (per-frame dense match metadata indexed by match slot), `port_store` (per-frame port metadata), and `free_frames` (available frame IDs). Match operands are stored in the frame's low slots (0 to `matchable_offsets - 1`), indexed by the token's IRAM offset (`token.offset`). Constants and destinations are in higher frame slots, indexed by the instruction's `fref` field per the mode table. Pipeline becomes IFETCH → act_id resolution → MATCH/FRAME → EXECUTE → EMIT. Output routing is driven by `Instruction.output` (OutputStyle enum) rather than `_output_mode()` reading dest_l/dest_r from ALUInst. The PE only handles `Instruction` objects — legacy ALUInst/SMInst are gone.
66+77+**Tech Stack:** Python 3.12, SimPy 4.1
88+99+**Scope:** Phase 2 of 8 from the PE frame-based redesign design plan.
1010+1111+**Codebase verified:** 2026-03-07
1212+1313+---
1414+1515+## Acceptance Criteria Coverage
1616+1717+This phase implements and tests:
1818+1919+### pe-frame-redesign.AC3: Frame-based PE matching and output routing
2020+- **pe-frame-redesign.AC3.1 Success:** PE constructor accepts `frame_count`, `frame_slots`, `matchable_offsets` as configurable parameters
2121+- **pe-frame-redesign.AC3.2 Success:** Pipeline order is IFETCH → act_id resolution → MATCH/FRAME → EXECUTE → EMIT
2222+- **pe-frame-redesign.AC3.3 Success:** Dyadic matching uses tag store + presence bits + frame SRAM (not matching_store array)
2323+- **pe-frame-redesign.AC3.4 Success:** INHERIT output reads `FrameDest` from frame slot and constructs token directly
2424+- **pe-frame-redesign.AC3.5 Success:** CHANGE_TAG output calls `unpack_flit1()` on left operand to get `FrameDest`
2525+- **pe-frame-redesign.AC3.6 Success:** SINK output writes result to `frames[frame_id][fref]`, emits no token
2626+- **pe-frame-redesign.AC3.7 Success:** EXTRACT_TAG handled PE-level (not ALU), produces packed flit 1 via `pack_flit1()`
2727+- **pe-frame-redesign.AC3.8 Success:** Frame allocation (ALLOC) and deallocation (FREE) via FrameControlToken work correctly
2828+- **pe-frame-redesign.AC3.9 Success:** PELocalWriteToken with `is_dest=True` decodes data to FrameDest on write
2929+- **pe-frame-redesign.AC3.10 Success:** Pipeline timing preserved: 5 cycles dyadic, 4 cycles monadic, 1 cycle side paths
3030+3131+### pe-frame-redesign.AC1: Token hierarchy correctly models frame-based architecture
3232+- **pe-frame-redesign.AC1.5 Success:** Network routing uses `isinstance(token, PEToken)` for all PE-bound tokens
3333+- **pe-frame-redesign.AC1.6 Failure:** Token with invalid `act_id` (no tag store mapping) is dropped with `TokenRejected` event
3434+3535+---
3636+3737+<!-- START_SUBCOMPONENT_A (tasks 1-2) -->
3838+<!-- START_TASK_1 -->
3939+### Task 1: Update emu/alu.py for opcode renames
4040+4141+**Verifies:** None (infrastructure — prerequisite for PE rewrite)
4242+4343+**Files:**
4444+- Modify: `emu/alu.py` (rename shift ops and FREE_CTX, add EXTRACT_TAG and ALLOC_REMOTE)
4545+4646+**Implementation:**
4747+4848+Phase 1 renumbers all opcodes and renames shifts (`SHIFT_L` → `SHL`, `SHIFT_R` → `SHR`, `ASHFT_R` → `ASR`). Shifts stay in `ArithOp`, so no structural change to ALU dispatch. Update `emu/alu.py`:
4949+5050+1. In `_execute_arith()`: rename match cases:
5151+ ```python
5252+ case ArithOp.SHL: # was SHIFT_L
5353+ result = (left << const) & UINT16_MASK
5454+ case ArithOp.SHR: # was SHIFT_R
5555+ result = (left >> const) & UINT16_MASK
5656+ case ArithOp.ASR: # was ASHFT_R
5757+ signed = to_signed(left)
5858+ result = (signed >> const) & UINT16_MASK
5959+ ```
6060+6161+2. In `_execute_routing()`: rename `FREE_CTX` → `FREE_FRAME`, add `EXTRACT_TAG` and `ALLOC_REMOTE` as no-op fallbacks (both handled at PE level):
6262+ ```python
6363+ case RoutingOp.FREE_FRAME:
6464+ return 0, False
6565+ case RoutingOp.EXTRACT_TAG:
6666+ return 0, False
6767+ case RoutingOp.ALLOC_REMOTE:
6868+ return 0, False
6969+ ```
7070+7171+**Verification:**
7272+Run: `python -m pytest tests/test_alu.py -v`
7373+Expected: All ALU tests pass
7474+7575+**Commit:** `refactor: rename shift ops (SHL/SHR/ASR), FREE_CTX→FREE_FRAME, add EXTRACT_TAG and ALLOC_REMOTE`
7676+7777+<!-- END_TASK_1 -->
7878+7979+<!-- START_TASK_2 -->
8080+### Task 2: Rewrite ProcessingElement in emu/pe.py
8181+8282+**Verifies:** pe-frame-redesign.AC3.1, pe-frame-redesign.AC3.2, pe-frame-redesign.AC3.3, pe-frame-redesign.AC3.4, pe-frame-redesign.AC3.5, pe-frame-redesign.AC3.6, pe-frame-redesign.AC3.7, pe-frame-redesign.AC3.8, pe-frame-redesign.AC3.9, pe-frame-redesign.AC3.10, pe-frame-redesign.AC1.6
8383+8484+**Files:**
8585+- Modify: `emu/pe.py` (full rewrite of ProcessingElement class)
8686+8787+**Implementation:**
8888+8989+This is a full rewrite. The new `ProcessingElement` replaces the matching_store/gen_counters model with frames/tag_store/presence. There is no legacy path — the PE only handles `Instruction` objects.
9090+9191+**Constructor changes:**
9292+9393+Replace the old constructor parameters and instance variables. The PE now accepts:
9494+- `frame_count` (from PEConfig, default 8)
9595+- `frame_slots` (from PEConfig, default 64)
9696+- `matchable_offsets` (from PEConfig, default 8)
9797+- `initial_frames` and `initial_tag_store` (optional, for pre-loaded state)
9898+9999+New instance variables:
100100+```python
101101+self.frames: list[list[FrameSlotValue]] # [frame_count][frame_slots] — all per-activation data
102102+self.tag_store: dict[int, int] # act_id → frame_id
103103+self.presence: list[list[bool]] # [frame_count][matchable_offsets] — match pending
104104+self.port_store: list[list[Port | None]] # [frame_count][matchable_offsets] — port of pending operand
105105+self.free_frames: list[int] # available frame IDs
106106+```
107107+108108+Match metadata (`presence`, `port_store`) is a dense, small structure — one entry per matchable offset per frame, NOT one per frame slot. The match slot index is derived from the low bits of the token's IRAM offset: `match_slot = token.offset % matchable_offsets`. This allows dyadic instructions to be placed at ANY IRAM offset — the assembler just ensures no two simultaneously-live dyadic instructions in the same activation collide in their low-bit hash. Match operands are stored in `frames[frame_id][match_slot]`. `fref` is separate — it indexes into the frame for const/dest reads by the output formatter.
109109+110110+Remove: `matching_store`, `gen_counters`, `_ctx_slots`, `_offsets`
111111+112112+The `iram` type is `dict[int, Instruction]`.
113113+114114+**Updated imports:**
115115+116116+```python
117117+import logging
118118+from typing import Optional
119119+120120+import simpy
121121+122122+from cm_inst import (
123123+ ALUOp, ArithOp, FrameDest, FrameOp, FrameSlotValue,
124124+ Instruction, LogicOp, MemOp, OutputStyle, Port, RoutingOp,
125125+ TokenKind, is_monadic_alu,
126126+)
127127+from encoding import pack_flit1, unpack_flit1, unpack_instruction
128128+from emu.alu import execute
129129+from emu.events import (
130130+ Emitted, EventCallback, Executed, FrameAllocated, FrameFreed,
131131+ FrameSlotWritten, IRAMWritten, Matched, TokenReceived, TokenRejected,
132132+)
133133+from tokens import (
134134+ CMToken, DyadToken, FrameControlToken,
135135+ MonadToken, PELocalWriteToken, PEToken, SMToken,
136136+)
137137+```
138138+139139+**Pipeline rewrite — `_process_token()`:**
140140+141141+The new pipeline order is:
142142+1. Side path handling (PELocalWriteToken, FrameControlToken) — 1 cycle each
143143+2. For CMTokens (DyadToken, MonadToken):
144144+ a. IFETCH — look up IRAM by `token.offset` (1 cycle)
145145+ b. Act_id resolution — validate `token.act_id` against tag_store
146146+ c. MATCH — for dyadic: presence-based matching on `frames[frame_id][token.offset % matchable_offsets]`
147147+ d. EXECUTE — ALU execute or SM dispatch (1 cycle)
148148+ e. EMIT — mode-driven output routing (1 cycle)
149149+150150+**Side path: FrameControlToken handling:**
151151+152152+```python
153153+if isinstance(token, FrameControlToken):
154154+ self._handle_frame_control(token)
155155+ yield self.env.timeout(1)
156156+ return
157157+```
158158+159159+`_handle_frame_control`:
160160+- ALLOC: pop frame_id from `free_frames`, set `tag_store[token.act_id] = frame_id`, initialise frame slots to None, emit `FrameAllocated` event
161161+- FREE: look up frame_id from tag_store, remove tag_store entry, push frame_id back to `free_frames`, emit `FrameFreed` event. Frame slot data is NOT cleared (stale data overwritten on next ALLOC, matching hardware)
162162+163163+**Side path: PELocalWriteToken handling:**
164164+165165+```python
166166+if isinstance(token, PELocalWriteToken):
167167+ self._handle_local_write(token)
168168+ yield self.env.timeout(1)
169169+ return
170170+```
171171+172172+`_handle_local_write`:
173173+- `region=0`: IRAM write — `self.iram[token.slot] = unpack_instruction(token.data)`, emit `IRAMWritten` event
174174+- `region=1`: Frame write — look up `frame_id = self.tag_store[token.act_id]`, write to `self.frames[frame_id][token.slot]`. If `token.is_dest` is True, decode `token.data` via `unpack_flit1()` and store the resulting `FrameDest`. Otherwise store `token.data` as int. Emit `FrameSlotWritten` event.
175175+176176+**Main pipeline for CMTokens:**
177177+178178+```python
179179+# IFETCH (1 cycle)
180180+inst = self.iram.get(token.offset)
181181+yield self.env.timeout(1)
182182+if inst is None:
183183+ logger.warning("PE %d: no instruction at offset %d", self.pe_id, token.offset)
184184+ return
185185+186186+# Act_id resolution
187187+if token.act_id not in self.tag_store:
188188+ self._on_event(TokenRejected(
189189+ time=self.env.now, component=self._component,
190190+ token=token, reason=f"act_id {token.act_id} not in tag store",
191191+ ))
192192+ return
193193+frame_id = self.tag_store[token.act_id]
194194+```
195195+196196+**Matching:**
197197+198198+Determine monadic/dyadic from the token type:
199199+200200+```python
201201+if isinstance(token, MonadToken):
202202+ left, right = token.data, None
203203+elif isinstance(token, DyadToken):
204204+ if is_monadic_alu(inst.opcode) if isinstance(inst.opcode, ALUOp) else True: # all SM ops are monadic
205205+ left, right = token.data, None
206206+ else:
207207+ # Dyadic matching via presence bits
208208+ operands = self._match_frame(token, inst, frame_id)
209209+ yield self.env.timeout(1) # match cycle
210210+ if operands is None:
211211+ return # waiting for partner
212212+ left, right = operands
213213+else:
214214+ return
215215+216216+# EXECUTE
217217+if isinstance(inst.opcode, MemOp):
218218+ yield self.env.timeout(1)
219219+ self._build_and_emit_sm_new(inst, left, right, token.act_id, frame_id)
220220+ yield self.env.timeout(1)
221221+elif inst.opcode == RoutingOp.EXTRACT_TAG:
222222+ # PE-level: pack current PE/act_id/offset into flit 1
223223+ yield self.env.timeout(1)
224224+ result = pack_flit1(FrameDest(
225225+ target_pe=self.pe_id,
226226+ offset=token.offset,
227227+ act_id=token.act_id,
228228+ port=Port.L,
229229+ token_kind=TokenKind.DYADIC,
230230+ ))
231231+ self._on_event(Executed(
232232+ time=self.env.now, component=self._component,
233233+ op=inst.opcode, result=result, bool_out=False,
234234+ ))
235235+ self._do_emit_new(inst, result, False, token.act_id, frame_id)
236236+ yield self.env.timeout(1)
237237+elif inst.opcode == RoutingOp.ALLOC_REMOTE:
238238+ # PE-level: read target PE and act_id from frame constants, construct
239239+ # and deliver a FrameControlToken(ALLOC) to the target PE.
240240+ # fref points to const/dest slots (mode table). ALLOC_REMOTE reads
241241+ # target PE and act_id from frame constants at fref, fref+1.
242242+ yield self.env.timeout(1)
243243+ target_pe = self.frames[frame_id][inst.fref] # target PE id from frame constant
244244+ target_act = self.frames[frame_id][inst.fref + 1] # target act_id from frame constant
245245+ fct = FrameControlToken(
246246+ target=target_pe,
247247+ act_id=target_act,
248248+ op=FrameOp.ALLOC,
249249+ payload=0,
250250+ )
251251+ self._on_event(Executed(
252252+ time=self.env.now, component=self._component,
253253+ op=inst.opcode, result=0, bool_out=False,
254254+ ))
255255+ self.env.process(self._deliver(self.route_table[target_pe], fct))
256256+ yield self.env.timeout(1)
257257+elif inst.opcode == RoutingOp.FREE_FRAME:
258258+ # Deallocate frame: clear tag_store entry, return frame to free pool.
259259+ # Frame slot data is NOT cleared — stale data is harmless (overwritten
260260+ # on next ALLOC). This matches hardware behavior.
261261+ yield self.env.timeout(1)
262262+ result, bool_out = execute(inst.opcode, left, right, None)
263263+ self._on_event(Executed(...))
264264+ if token.act_id in self.tag_store:
265265+ freed_frame = self.tag_store.pop(token.act_id)
266266+ self.free_frames.append(freed_frame)
267267+ self._on_event(FrameFreed(
268268+ time=self.env.now, component=self._component,
269269+ act_id=token.act_id, frame_id=freed_frame,
270270+ ))
271271+ # No emit — FREE_FRAME suppresses output
272272+ yield self.env.timeout(1)
273273+else:
274274+ # Normal ALU execute
275275+ # fref points to const/dest per mode table. For modes 1/3/5/7: const at fref.
276276+ const_val = self.frames[frame_id][inst.fref] if inst.has_const else None
277277+ result, bool_out = execute(inst.opcode, left, right, const_val)
278278+ self._on_event(Executed(
279279+ time=self.env.now, component=self._component,
280280+ op=inst.opcode, result=result, bool_out=bool_out,
281281+ ))
282282+ yield self.env.timeout(1)
283283+ self._do_emit_new(inst, result, bool_out, token.act_id, frame_id, left=left)
284284+ yield self.env.timeout(1)
285285+```
286286+287287+**Frame-based matching — `_match_frame()`:**
288288+289289+Matching derives the match slot from the low bits of the token's IRAM offset: `match_slot = token.offset % matchable_offsets`. Dyadic instructions can be at any IRAM offset — the assembler ensures no two simultaneously-live dyadic instructions in the same activation collide in their match slot. `fref` is separate — it points to const/dest slots for the output formatter.
290290+291291+Both L and R tokens for the same instruction write to `frames[frame_id][match_slot]`. The stored port metadata determines left/right ordering when the second token arrives.
292292+293293+```python
294294+def _match_frame(self, token: DyadToken, inst: Instruction, frame_id: int) -> tuple[int, int] | None:
295295+ match_slot = token.offset % self._matchable_offsets # low bits of IRAM offset
296296+297297+ if self.presence[frame_id][match_slot]:
298298+ # Partner already waiting — pair them
299299+ partner_data = self.frames[frame_id][match_slot]
300300+ partner_port = self.port_store[frame_id][match_slot]
301301+ self.presence[frame_id][match_slot] = False
302302+ self.frames[frame_id][match_slot] = None
303303+304304+ # Use port metadata to determine left/right ordering
305305+ if partner_port == Port.L:
306306+ left, right = partner_data, token.data
307307+ else:
308308+ left, right = token.data, partner_data
309309+310310+ self._on_event(Matched(
311311+ time=self.env.now, component=self._component,
312312+ left=left, right=right, act_id=token.act_id,
313313+ offset=token.offset, frame_id=frame_id,
314314+ ))
315315+ return left, right
316316+ else:
317317+ # Store and wait for partner
318318+ self.frames[frame_id][match_slot] = token.data
319319+ self.port_store[frame_id][match_slot] = token.port
320320+ self.presence[frame_id][match_slot] = True
321321+ return None
322322+```
323323+324324+**Mode-driven output routing — `_do_emit_new()`:**
325325+326326+Note: `left` operand must be preserved through the execute stage and passed here for CHANGE_TAG routing. The PE's execute path must thread `left` alongside `result`.
327327+328328+```python
329329+def _do_emit_new(self, inst: Instruction, result: int, bool_out: bool, act_id: int, frame_id: int, left: int = 0):
330330+ if isinstance(inst.opcode, RoutingOp) and inst.opcode == RoutingOp.GATE and not bool_out:
331331+ return # GATE suppressed
332332+333333+ match inst.output:
334334+ case OutputStyle.INHERIT:
335335+ self._emit_inherit(inst, result, bool_out, frame_id)
336336+ case OutputStyle.CHANGE_TAG:
337337+ self._emit_change_tag(inst, result, left)
338338+ case OutputStyle.SINK:
339339+ self._emit_sink(inst, result, frame_id)
340340+```
341341+342342+**INHERIT output — `_emit_inherit()`:**
343343+344344+Read `FrameDest` from frame slots starting at `inst.fref`. The mode table defines the layout starting from fref: mode 0 = `[dest]`, mode 1 = `[const, dest]`, mode 2 = `[dest1, dest2]`, mode 3 = `[const, dest1, dest2]`. Match operands are in a separate region (indexed by token.offset, slots 0–7); fref points to const/dest slots only.
345345+346346+```python
347347+def _emit_inherit(self, inst: Instruction, result: int, bool_out: bool, frame_id: int):
348348+ # fref is the base of [const?, dest1, dest2?] — see mode table.
349349+ # If has_const, const is at fref and dests start at fref+1.
350350+ # If no const, dests start at fref.
351351+ dest_base = inst.fref + (1 if inst.has_const else 0)
352352+353353+ if inst.dest_count >= 1:
354354+ dest_l: FrameDest = self.frames[frame_id][dest_base]
355355+ out_token = self._make_token_from_dest(dest_l, result)
356356+ self.output_log.append(out_token)
357357+ self._on_event(Emitted(time=self.env.now, component=self._component, token=out_token))
358358+ self.env.process(self._deliver(self.route_table[dest_l.target_pe], out_token))
359359+360360+ if inst.dest_count >= 2:
361361+ dest_r: FrameDest = self.frames[frame_id][dest_base + 1]
362362+ # For switch ops, route based on bool_out
363363+ if isinstance(inst.opcode, RoutingOp) and inst.opcode in (
364364+ RoutingOp.SWEQ, RoutingOp.SWGT, RoutingOp.SWGE, RoutingOp.SWOF,
365365+ ):
366366+ # Undo the dest_l append above — switch re-routes both outputs
367367+ self.output_log.pop()
368368+ if bool_out:
369369+ taken, not_taken = dest_l, dest_r
370370+ else:
371371+ taken, not_taken = dest_r, dest_l
372372+ data_tok = self._make_token_from_dest(taken, result)
373373+ trig_tok = self._make_token_from_dest(not_taken, 0)
374374+ self.output_log.append(data_tok)
375375+ self.output_log.append(trig_tok)
376376+ # ... events and delivery for both tokens
377377+ else:
378378+ out_r = self._make_token_from_dest(dest_r, result)
379379+ self.output_log.append(out_r)
380380+ self._on_event(Emitted(time=self.env.now, component=self._component, token=out_r))
381381+ self.env.process(self._deliver(self.route_table[dest_r.target_pe], out_r))
382382+```
383383+384384+**CHANGE_TAG output — `_emit_change_tag()`:**
385385+386386+The left operand carries the packed flit 1 (destination descriptor from the caller's EXTRACT_TAG). The ALU result is the data payload to send. These are distinct values — `left` determines WHERE to send, `result` determines WHAT to send.
387387+388388+```python
389389+def _emit_change_tag(self, inst: Instruction, result: int, left: int):
390390+ dest = unpack_flit1(left)
391391+ out_token = self._make_token_from_dest(dest, result)
392392+ self.output_log.append(out_token)
393393+ self._on_event(Emitted(time=self.env.now, component=self._component, token=out_token))
394394+ self.env.process(self._deliver(self.route_table[dest.target_pe], out_token))
395395+```
396396+397397+**SM dispatch — `_build_and_emit_sm_new()`:**
398398+399399+For SM instructions (where `isinstance(inst.opcode, MemOp)`), the PE constructs an SMToken. The return route is a FrameDest stored in the frame slot following the mode table layout at `inst.fref`. The `FrameDest.token_kind` determines whether the SM return token will be a DyadToken or MonadToken — this replaces the old `SMInst.ret_dyadic` field.
400400+401401+```python
402402+def _build_and_emit_sm_new(self, inst: Instruction, left: int, right: int, act_id: int, frame_id: int):
403403+ # fref layout per mode table: [const?, dest]. For SM ops with return routes,
404404+ # the dest slot holds the return FrameDest.
405405+ ret_slot = inst.fref + (1 if inst.has_const else 0)
406406+ ret_dest: FrameDest | None = self.frames[frame_id][ret_slot] if inst.dest_count > 0 else None
407407+408408+ # Build return CMToken from FrameDest if return route exists
409409+ ret_token: CMToken | None = None
410410+ if ret_dest is not None:
411411+ ret_token = self._make_token_from_dest(ret_dest, 0) # data will be filled by SM
412412+413413+ # SM target (SM_id + address) can come from frame[fref] OR from
414414+ # operand data, depending on the operation. The exact source mapping
415415+ # is determined by the instruction's mode and opcode — the assembler's
416416+ # codegen sets up frame slots and operand routing accordingly.
417417+ # Flit 2 source (ALU out / R operand / frame slot) also varies by
418418+ # operation — see pe-design.md flit 2 source mux table.
419419+ #
420420+ # For the emulator, we dispatch based on has_const: when the instruction
421421+ # has a constant (frame[fref] holds SM params), target comes from frame.
422422+ # Otherwise, target comes from the left operand.
423423+ if inst.has_const:
424424+ target_packed = self.frames[frame_id][inst.fref]
425425+ else:
426426+ target_packed = left
427427+ sm_token = SMToken(
428428+ target=(target_packed >> 8) & 0xFF,
429429+ addr=target_packed & 0xFF,
430430+ op=inst.opcode,
431431+ flags=right if right is not None else None,
432432+ data=right if inst.has_const else left,
433433+ ret=ret_token,
434434+ )
435435+ self.output_log.append(sm_token)
436436+ self._on_event(Emitted(time=self.env.now, component=self._component, token=sm_token))
437437+ self.env.process(self._deliver(self.sm_routes[sm_token.target], sm_token))
438438+```
439439+440440+Note: SM target source (frame slot vs operand) depends on the operation and mode. The `has_const` dispatch above is a simplified emulator heuristic — the hardware uses the instruction decoder EEPROM to select the source mux. The return route's token kind comes from `FrameDest.token_kind` in the frame slot, not from a separate instruction field.
441441+442442+**SINK output — `_emit_sink()`:**
443443+444444+```python
445445+def _emit_sink(self, inst: Instruction, result: int, frame_id: int):
446446+ self.frames[frame_id][inst.fref] = result
447447+ self._on_event(FrameSlotWritten(
448448+ time=self.env.now, component=self._component,
449449+ frame_id=frame_id, slot=inst.fref, value=result,
450450+ ))
451451+ # No output token emitted
452452+```
453453+454454+**Token construction from FrameDest — `_make_token_from_dest()`:**
455455+456456+```python
457457+def _make_token_from_dest(self, dest: FrameDest, data: int) -> CMToken:
458458+ match dest.token_kind:
459459+ case TokenKind.DYADIC:
460460+ return DyadToken(
461461+ target=dest.target_pe, offset=dest.offset,
462462+ act_id=dest.act_id, data=data,
463463+ port=dest.port, wide=False,
464464+ )
465465+ case TokenKind.MONADIC:
466466+ return MonadToken(
467467+ target=dest.target_pe, offset=dest.offset,
468468+ act_id=dest.act_id, data=data, inline=False,
469469+ )
470470+ case TokenKind.INLINE:
471471+ return MonadToken(
472472+ target=dest.target_pe, offset=dest.offset,
473473+ act_id=dest.act_id, data=data, inline=True,
474474+ )
475475+```
476476+477477+**Timing invariants:**
478478+- Dyadic CMToken: dequeue(1) + IFETCH(1) + MATCH(1) + EXECUTE(1) + EMIT(1) = 5 cycles
479479+- Monadic CMToken: dequeue(1) + IFETCH(1) + EXECUTE(1) + EMIT(1) = 4 cycles
480480+- FrameControlToken / PELocalWriteToken: dequeue(1) + handle(1) = 2 cycles (1 cycle side path after dequeue)
481481+482482+**Testing:**
483483+484484+Tests must verify each AC:
485485+- pe-frame-redesign.AC3.1: Construct PE with `frame_count=4`, `frame_slots=32`, `matchable_offsets=4` and verify they're stored
486486+- pe-frame-redesign.AC3.2: Inject a DyadToken pair, capture events, verify event order is TokenReceived → Matched → Executed → Emitted (IFETCH is implicit before match)
487487+- pe-frame-redesign.AC3.3: Set up tag_store and frame with presence bits, inject two DyadTokens at same offset/act_id, verify they match via frame SRAM (not matching_store)
488488+- pe-frame-redesign.AC3.4: Set up frame with FrameDest in destination slot, inject tokens, verify output token has target/offset/act_id from the FrameDest
489489+- pe-frame-redesign.AC3.5: Set up CHANGE_TAG instruction, inject token whose left operand is a packed flit 1 value, verify output token destination matches unpacked FrameDest
490490+- pe-frame-redesign.AC3.6: Set up SINK instruction, inject tokens, verify result written to frame slot and no output token emitted
491491+- pe-frame-redesign.AC3.7: Set up EXTRACT_TAG instruction, inject MonadToken, verify result is packed flit 1 of (pe_id, offset, act_id, port, kind)
492492+- pe-frame-redesign.AC3.8: Inject FrameControlToken(ALLOC), verify tag_store updated and FrameAllocated event. Inject FrameControlToken(FREE), verify tag_store cleared and FrameFreed event. Also test FREE_FRAME opcode path: set up a FREE_FRAME instruction in IRAM, inject a MonadToken, verify tag_store cleared, frame returned to free_frames, FrameFreed event emitted, and no output token produced. Also test ALLOC_REMOTE: set up ALLOC_REMOTE instruction with target PE/act_id in frame slots, inject MonadToken, verify a FrameControlToken(ALLOC) is delivered to the target PE's input store.
493493+- pe-frame-redesign.AC3.9: Inject PELocalWriteToken(region=1, is_dest=True, data=pack_flit1(some_dest)), verify frame slot contains FrameDest object (not raw int)
494494+- pe-frame-redesign.AC3.10: Time dyadic pair processing, verify 5 cycles. Time monadic, verify 4 cycles. Time side paths, verify 1 cycle after dequeue.
495495+- pe-frame-redesign.AC1.6: Inject DyadToken with act_id not in tag_store, verify TokenRejected event and no crash
496496+497497+Test file: `tests/test_pe_frames.py` (new file)
498498+499499+**Verification:**
500500+Run: `python -m pytest tests/test_pe_frames.py -v`
501501+Expected: All new frame-based tests pass. Legacy PE tests (test_pe_events.py, test_cycle_timing.py) will break — they will be fixed in the cleanup/E2E phase.
502502+503503+**Commit:** `feat: rewrite ProcessingElement with frame-based matching and output routing`
504504+505505+<!-- END_TASK_2 -->
506506+<!-- END_SUBCOMPONENT_A -->
···11+# PE Frame-Based Redesign — Phase 3: Emulator Supporting — SM, Network, ALU
22+33+**Goal:** Update SM for T0 raw int storage, network for PEToken routing, and wire everything together with new PEConfig fields.
44+55+**Architecture:** SM's `t0_store` changes from `list[Token]` to `list[int]`. EXEC handler uses `flit_count()` + `unpack_token()` to reconstitute tokens from raw int sequences. Network routing changes from `isinstance(token, CMToken)` to `isinstance(token, PEToken)` to route all PE-bound tokens (CMToken, PELocalWriteToken, FrameControlToken). `build_topology()` passes new PEConfig frame parameters to ProcessingElement constructor.
66+77+**Tech Stack:** Python 3.12, SimPy 4.1
88+99+**Scope:** Phase 3 of 8 from the PE frame-based redesign design plan.
1010+1111+**Codebase verified:** 2026-03-07
1212+1313+---
1414+1515+## Acceptance Criteria Coverage
1616+1717+This phase implements and tests:
1818+1919+### pe-frame-redesign.AC4: T0 raw storage and EXEC
2020+- **pe-frame-redesign.AC4.1 Success:** SM T0 stores `list[int]` (16-bit words), not Token objects
2121+- **pe-frame-redesign.AC4.2 Success:** EXEC reads consecutive ints, uses `flit_count()` for packet boundaries, reconstitutes tokens via `unpack_token()`
2222+- **pe-frame-redesign.AC4.3 Success:** `pack_token()` / `unpack_token()` round-trip for all token types
2323+- **pe-frame-redesign.AC4.4 Failure:** Malformed flit sequence in T0 (invalid prefix bits) is handled gracefully
2424+2525+### pe-frame-redesign.AC1: Token hierarchy correctly models frame-based architecture
2626+- **pe-frame-redesign.AC1.5 Success:** Network routing uses `isinstance(token, PEToken)` for all PE-bound tokens
2727+2828+---
2929+3030+<!-- START_SUBCOMPONENT_A (tasks 1-2) -->
3131+<!-- START_TASK_1 -->
3232+### Task 1: Update SM for T0 raw int storage and EXEC
3333+3434+**Verifies:** pe-frame-redesign.AC4.1, pe-frame-redesign.AC4.2, pe-frame-redesign.AC4.4
3535+3636+**Files:**
3737+- Modify: `emu/sm.py` (change t0_store type, rewrite EXEC handler, update T0 WRITE)
3838+3939+**Implementation:**
4040+4141+Changes to `emu/sm.py`:
4242+4343+1. Change `t0_store` type from `list[Token]` to `list[int]`:
4444+```python
4545+self.t0_store: list[int] = []
4646+```
4747+4848+2. Add import for encoding functions:
4949+```python
5050+from encoding import flit_count, unpack_token
5151+```
5252+5353+3. Rewrite `_handle_exec()` to parse raw ints using flit boundaries:
5454+5555+```python
5656+def _handle_exec(self, addr: int):
5757+ """EXEC: read raw int sequences from T0, reconstitute tokens, inject into network."""
5858+ if self.system is None:
5959+ logger.warning("SM%d: EXEC but no system reference", self.sm_id)
6060+ return
6161+ t0_idx = addr - self.tier_boundary
6262+ if t0_idx >= len(self.t0_store):
6363+ return
6464+ yield self.env.timeout(1) # process cycle
6565+6666+ pos = t0_idx
6767+ while pos < len(self.t0_store):
6868+ header = self.t0_store[pos]
6969+ if header is None:
7070+ break
7171+ try:
7272+ count = flit_count(header)
7373+ except (ValueError, KeyError):
7474+ logger.warning("SM%d: malformed flit at T0[%d], stopping EXEC", self.sm_id, pos)
7575+ break
7676+ if pos + count > len(self.t0_store):
7777+ logger.warning("SM%d: truncated packet at T0[%d], stopping EXEC", self.sm_id, pos)
7878+ break
7979+ flits = self.t0_store[pos:pos + count]
8080+ try:
8181+ token = unpack_token(flits)
8282+ except (ValueError, KeyError):
8383+ logger.warning("SM%d: failed to unpack token at T0[%d], stopping EXEC", self.sm_id, pos)
8484+ break
8585+ yield from self.system.send(token)
8686+ yield self.env.timeout(1) # per-token injection cycle
8787+ pos += count
8888+```
8989+9090+4. `_handle_t0_write()` already stores `token.data` (an int) — no change needed for the write path. The t0_store type annotation changes but the actual stored values were already ints.
9191+9292+5. `_handle_t0_read()` — verify it returns int values (it does: `self.t0_store[t0_idx]`). No change needed.
9393+9494+6. Remove any isinstance(entry, Token) checks in the EXEC path — the new implementation processes raw ints via flit_count/unpack_token instead.
9595+9696+**Testing:**
9797+9898+Tests must verify:
9999+- pe-frame-redesign.AC4.1: After construction, `sm.t0_store` is `list[int]` type. After T0 WRITE, stored values are ints.
100100+- pe-frame-redesign.AC4.2: Pre-load t0_store with packed token flits (via `pack_token()`), trigger EXEC, verify reconstituted tokens arrive at target PE with correct field values.
101101+- pe-frame-redesign.AC4.4: Pre-load t0_store with invalid prefix bits (e.g., `[0xFFFF]`), trigger EXEC, verify it stops gracefully without crash and logs warning.
102102+- Test multiple consecutive packets in T0 — EXEC should parse each one by flit_count boundary.
103103+- Test truncated packet (flit_count says 3 but only 2 ints remain) — should stop gracefully.
104104+105105+Test file: `tests/test_sm_t0_raw.py` (new file)
106106+107107+**Verification:**
108108+Run: `python -m pytest tests/test_sm_t0_raw.py -v`
109109+Expected: All tests pass
110110+111111+**Commit:** `feat: change SM T0 to raw int storage with flit-based EXEC parsing`
112112+113113+<!-- END_TASK_1 -->
114114+115115+<!-- START_TASK_2 -->
116116+### Task 2: Update SM T0 tests for raw int storage
117117+118118+**Verifies:** pe-frame-redesign.AC4.3
119119+120120+**Files:**
121121+- Modify: `tests/test_sm_tiers.py` (update T0 tests for int storage instead of Token storage)
122122+- Modify: `tests/test_exec_bootstrap.py` (update EXEC tests to pre-load packed flits)
123123+124124+**Implementation:**
125125+126126+Existing T0 tests that pre-load `t0_store` with Token objects need updating to pre-load with packed int flits instead.
127127+128128+In `tests/test_sm_tiers.py`:
129129+- Any test that does `sm.t0_store.append(some_token)` or `t0_store.append(some_token)` must change to `t0_store.extend(pack_token(some_token))`
130130+- Import `pack_token` from `encoding`
131131+- T0 READ tests: stored values are now ints, verify int retrieval
132132+133133+In `tests/test_exec_bootstrap.py`:
134134+- Tests that pre-load T0 with Token objects for EXEC to read must change to `t0_store.extend(pack_token(token))` for each token
135135+- Verify EXEC reconstitutes the same tokens (round-trip via pack_token -> t0_store -> flit_count -> unpack_token)
136136+- pe-frame-redesign.AC4.3: pack_token/unpack_token round-trip is implicitly tested through EXEC end-to-end
137137+138138+**Verification:**
139139+Run: `python -m pytest tests/test_sm_tiers.py tests/test_exec_bootstrap.py tests/test_sm_t0_raw.py -v`
140140+Expected: `test_sm_t0_raw.py` and the updated T0 tests in `test_sm_tiers.py` and `test_exec_bootstrap.py` pass. Other tests in these files that depend on types removed in Phase 1 (e.g. `IRAMWriteToken`, `DyadToken.gen`, `MatchEntry`) may fail — that is expected and will be addressed in Phase 8.
141141+142142+**Commit:** `refactor: update T0 tests for raw int storage with packed flits`
143143+144144+<!-- END_TASK_2 -->
145145+<!-- END_SUBCOMPONENT_A -->
146146+147147+<!-- START_SUBCOMPONENT_B (tasks 3-4) -->
148148+<!-- START_TASK_3 -->
149149+### Task 3: Update network routing for PEToken
150150+151151+**Verifies:** pe-frame-redesign.AC1.5
152152+153153+**Files:**
154154+- Modify: `emu/network.py` (update _target_store routing, update build_topology for new PEConfig fields)
155155+156156+**Implementation:**
157157+158158+1. Update `_target_store()` in System class to route on `PEToken` instead of `CMToken`:
159159+160160+```python
161161+from tokens import PEToken, SMToken
162162+163163+def _target_store(self, token: Token) -> simpy.Store:
164164+ if isinstance(token, SMToken):
165165+ return self.sms[token.target].input_store
166166+ if isinstance(token, PEToken):
167167+ return self.pes[token.target].input_store
168168+ raise TypeError(f"Unknown token type: {type(token).__name__}")
169169+```
170170+171171+This ensures `CMToken`, `PELocalWriteToken`, and `FrameControlToken` all route to the target PE's input store.
172172+173173+2. Update `build_topology()` PE construction to pass new PEConfig fields. The PE constructor only accepts frame-based parameters — do not pass `ctx_slots`, `offsets`, or `gen_counters`, which no longer exist:
174174+175175+```python
176176+for cfg in pe_configs:
177177+ pe = ProcessingElement(
178178+ env=env,
179179+ pe_id=cfg.pe_id,
180180+ iram=cfg.iram,
181181+ frame_count=cfg.frame_count,
182182+ frame_slots=cfg.frame_slots,
183183+ matchable_offsets=cfg.matchable_offsets,
184184+ fifo_capacity=fifo_capacity,
185185+ on_event=cfg.on_event,
186186+ )
187187+ # Load initial frames if provided
188188+ if cfg.initial_frames is not None:
189189+ for frame_id, slot_values in cfg.initial_frames.items():
190190+ pe.frames[frame_id] = list(slot_values)
191191+ if cfg.initial_tag_store is not None:
192192+ pe.tag_store.update(cfg.initial_tag_store)
193193+ # Mark these frames as allocated (remove from free_frames)
194194+ for frame_id in cfg.initial_tag_store.values():
195195+ if frame_id in pe.free_frames:
196196+ pe.free_frames.remove(frame_id)
197197+ pes[cfg.pe_id] = pe
198198+```
199199+200200+3. Update `t0_store` type annotation:
201201+```python
202202+t0_store: list[int] = []
203203+```
204204+205205+4. Update `inject()` method — check `PEToken` instead of `CMToken`:
206206+```python
207207+def inject(self, token: Token):
208208+ if isinstance(token, SMToken):
209209+ store = self.sms[token.target].input_store
210210+ elif isinstance(token, PEToken):
211211+ store = self.pes[token.target].input_store
212212+ else:
213213+ raise TypeError(f"Unknown token type: {type(token).__name__}")
214214+ store.items.append(token)
215215+```
216216+217217+**Testing:**
218218+219219+Tests must verify:
220220+- pe-frame-redesign.AC1.5: Inject `PELocalWriteToken(target=0, ...)`, verify it arrives at PE 0's input_store. Inject `FrameControlToken(target=0, ...)`, verify it arrives at PE 0's input_store. Inject `DyadToken(target=0, ...)`, verify it arrives at PE 0's input_store. Inject `SMToken(target=0, ...)`, verify it arrives at SM 0's input_store.
221221+- `build_topology` constructs PE with frame_count, frame_slots, matchable_offsets from config
222222+- `build_topology` loads initial_frames and initial_tag_store into PE
223223+224224+Test file: `tests/test_network_routing.py` (new file, or append to test_network_events.py)
225225+226226+**Verification:**
227227+Run: `python -m pytest tests/test_network_routing.py -v`
228228+Expected: All tests pass
229229+230230+**Commit:** `feat: update network routing to PEToken and pass frame config to PE`
231231+232232+<!-- END_TASK_3 -->
233233+234234+<!-- START_TASK_4 -->
235235+### Task 4: Update emu/__init__.py exports
236236+237237+**Verifies:** None (infrastructure)
238238+239239+**Files:**
240240+- Modify: `emu/__init__.py` (add new event types to exports)
241241+242242+**Implementation:**
243243+244244+Add the new event types to the exports:
245245+246246+```python
247247+from emu.events import (
248248+ CellWritten,
249249+ DeferredRead,
250250+ DeferredSatisfied,
251251+ Emitted,
252252+ EventCallback,
253253+ Executed,
254254+ FrameAllocated,
255255+ FrameFreed,
256256+ FrameSlotWritten,
257257+ IRAMWritten,
258258+ Matched,
259259+ ResultSent,
260260+ SimEvent,
261261+ TokenReceived,
262262+ TokenRejected,
263263+)
264264+from emu.network import System, build_topology
265265+from emu.types import PEConfig, SMConfig
266266+```
267267+268268+**Verification:**
269269+Run: `python -c "from emu import FrameAllocated, FrameFreed, FrameSlotWritten, TokenRejected; print('OK')"`
270270+Expected: Prints "OK"
271271+272272+**Commit:** `feat: export new frame-based event types from emu package`
273273+274274+<!-- END_TASK_4 -->
275275+<!-- END_SUBCOMPONENT_B -->
276276+277277+<!-- START_TASK_5 -->
278278+### Task 5: Run phase 2-3 emulator tests
279279+280280+**Verifies:** None (regression check for new work only)
281281+282282+**Files:**
283283+- None (test-only verification step)
284284+285285+**Implementation:**
286286+287287+Run only the test files that were created or updated in phases 2 and 3:
288288+289289+```
290290+python -m pytest tests/test_sm_t0_raw.py tests/test_network_routing.py tests/test_pe_frames.py -v
291291+```
292292+293293+Legacy tests (`test_pe_events.py`, `test_sm_events.py`, `test_sm_tiers.py`, `test_exec_bootstrap.py`, `test_cycle_timing.py`, `test_network_events.py`, etc.) reference removed types — `ALUInst`, `SMInst`, `MatchEntry`, `IRAMWriteToken`, `DyadToken.gen`, and similar — and will fail. That is expected. These tests will be addressed in Phase 8 when the old test suite is updated to match the new architecture.
294294+295295+**Verification:**
296296+Run: `python -m pytest tests/test_sm_t0_raw.py tests/test_network_routing.py tests/test_pe_frames.py -v`
297297+Expected: All tests in these three files pass. Failures in any other test file are out of scope for this phase.
298298+299299+**Commit:** None (verification step only)
300300+301301+<!-- END_TASK_5 -->
···11+# PE Frame-Based Redesign — Phase 5: Assembler Core — Allocate Rewrite
22+33+**Goal:** Rewrite the allocate pass for frame layout allocation, IRAM offset assignment with instruction deduplication, activation ID assignment, and mode computation.
44+55+**Architecture:** The allocate pass gains four new sub-functions: `_assign_iram_offsets()` (rewritten for deduplication), `_assign_act_ids()` (replaces `_assign_context_slots()`), `_compute_frame_layouts()` (new), and `_compute_modes()` (new). Destination resolution produces `FrameDest` objects instead of `Addr`. Frame layouts are canonical per function body — all activations of the same function share the layout, enabling instruction deduplication at the IRAM level.
66+77+**Tech Stack:** Python 3.12
88+99+**Scope:** Phase 5 of 8 from the PE frame-based redesign design plan.
1010+1111+**Codebase verified:** 2026-03-07
1212+1313+---
1414+1515+## Acceptance Criteria Coverage
1616+1717+This phase implements and tests:
1818+1919+### pe-frame-redesign.AC5: Assembler allocate produces frame layouts
2020+- **pe-frame-redesign.AC5.1 Success:** IRAM offsets deduplicated — identical Instruction templates on same PE share entries
2121+- **pe-frame-redesign.AC5.2 Success:** Frame layouts canonical per function body — all activations of same function share layout
2222+- **pe-frame-redesign.AC5.3 Success:** Activation IDs assigned sequentially (0-7), at most `frame_count` concurrent per PE
2323+- **pe-frame-redesign.AC5.4 Success:** Frame slot assignment: match operands at 0 to matchable_offsets-1, then constants (deduped), then destinations (deduped), then sinks/SM params
2424+- **pe-frame-redesign.AC5.5 Success:** Mode (OutputStyle + has_const + dest_count) computed from edge topology and opcode
2525+- **pe-frame-redesign.AC5.6 Failure:** Frame slot overflow (> frame_slots) reports error
2626+- **pe-frame-redesign.AC5.7 Failure:** Act_id exhaustion (>8 needed) reports error
2727+2828+---
2929+3030+<!-- START_SUBCOMPONENT_A (tasks 1-3) -->
3131+<!-- START_TASK_1 -->
3232+### Task 1: Rewrite _assign_iram_offsets with deduplication
3333+3434+**Verifies:** pe-frame-redesign.AC5.1
3535+3636+**Files:**
3737+- Modify: `asm/allocate.py` (rewrite `_assign_iram_offsets()`)
3838+3939+**Implementation:**
4040+4141+The current `_assign_iram_offsets()` assigns sequential offsets with dyadic first, monadic after. The new version adds instruction template deduplication.
4242+4343+Two instructions on the same PE share an IRAM entry when they produce identical `Instruction` objects — same opcode, output style, has_const, dest_count, wide, and fref. Since `fref` and mode aren't computed until `_compute_frame_layouts()` and `_compute_modes()`, deduplication at IRAM assignment time is based on the subset of fields available: opcode and arity (dyadic vs monadic).
4444+4545+Actually, per the design, deduplication happens at the **template** level — after all fields are computed. This means IRAM offset assignment should be a two-phase process:
4646+4747+**Phase A: Assign provisional offsets** (same algorithm as current — dyadic at low offsets, monadic above).
4848+4949+**Phase B: Deduplicate** (after frame layouts and modes are computed — called later in the allocate pipeline, or combined into a single function that does both).
5050+5151+For this task, restructure `_assign_iram_offsets()` to:
5252+1. Still partition dyadic/monadic and assign sequential offsets
5353+2. Return the offset assignments AND enough metadata for later deduplication
5454+3. All instructions return IRAM cost of 1 (per Phase 4 change)
5555+5656+The deduplication step proper happens after `_compute_modes()` and `_compute_frame_layouts()` produce the `fref` and mode fields. Add `_deduplicate_iram()` as a post-processing step:
5757+5858+```python
5959+def _deduplicate_iram(
6060+ nodes_on_pe: dict[str, IRNode],
6161+ pe_id: int,
6262+) -> dict[str, IRNode]:
6363+ """Deduplicate IRAM entries for nodes that produce identical Instruction templates.
6464+6565+ Two nodes share an IRAM offset when they have identical:
6666+ opcode, output (OutputStyle), has_const, dest_count, wide, fref.
6767+ """
6868+ template_to_offset: dict[tuple, int] = {}
6969+ updated = {}
7070+7171+ for name, node in nodes_on_pe.items():
7272+ if node.seed or node.iram_offset is None:
7373+ updated[name] = node
7474+ continue
7575+7676+ # Build template key from the fields that make an Instruction
7777+ mode = node.mode # (OutputStyle, has_const, dest_count)
7878+ if mode is None:
7979+ updated[name] = node
8080+ continue
8181+8282+ template_key = (
8383+ node.opcode,
8484+ mode[0], # OutputStyle
8585+ mode[1], # has_const
8686+ mode[2], # dest_count
8787+ node.wide,
8888+ node.fref,
8989+ )
9090+9191+ if template_key in template_to_offset:
9292+ # Reuse existing offset
9393+ updated[name] = replace(node, iram_offset=template_to_offset[template_key])
9494+ else:
9595+ template_to_offset[template_key] = node.iram_offset
9696+ updated[name] = node
9797+9898+ return updated
9999+```
100100+101101+This is called near the end of allocate(), after modes and frame layouts are assigned.
102102+103103+**Testing:**
104104+105105+Tests must verify:
106106+- pe-frame-redesign.AC5.1: Two nodes on the same PE with identical opcode, mode, and fref share the same IRAM offset after deduplication
107107+- Nodes with different opcodes get different offsets
108108+- Nodes with same opcode but different modes get different offsets
109109+- Seed nodes are excluded from IRAM assignment
110110+111111+Test file: `tests/test_allocate_frames.py` (new file)
112112+113113+**Verification:**
114114+Run: `python -m pytest tests/test_allocate_frames.py -v`
115115+Expected: All tests pass
116116+117117+**Commit:** `feat: rewrite IRAM offset assignment with template deduplication`
118118+119119+<!-- END_TASK_1 -->
120120+121121+<!-- START_TASK_2 -->
122122+### Task 2: Add _assign_act_ids
123123+124124+**Verifies:** pe-frame-redesign.AC5.3, pe-frame-redesign.AC5.7
125125+126126+**Files:**
127127+- Modify: `asm/allocate.py` (add `_assign_act_ids()`, remove `_assign_context_slots()`)
128128+129129+**Implementation:**
130130+131131+Add `_assign_act_ids()` and remove `_assign_context_slots()`. The logic is similar but:
132132+- Uses 3-bit act_ids (0-7) instead of ctx_slots
133133+- Limit is `system.frame_count` (default 8)
134134+- Field set on IRNode is `act_id`
135135+- Uses the same scope extraction logic (`_extract_function_scope()`)
136136+- CallSite field reference: `free_frame_nodes`
137137+138138+```python
139139+def _assign_act_ids(
140140+ nodes_on_pe: list[IRNode],
141141+ all_nodes: dict[str, IRNode],
142142+ frame_count: int,
143143+ pe_id: int,
144144+ call_sites: list[CallSite] | None = None,
145145+) -> tuple[dict[str, IRNode], list[AssemblyError]]:
146146+ """Assign activation IDs (0 to frame_count-1) per function scope per PE."""
147147+```
148148+149149+The algorithm is identical to the removed `_assign_context_slots()` except:
150150+- Replace `frame_count` in validation checks (was `ctx_slots`)
151151+- Replace `node.act_id` in field assignments (was `node.ctx`)
152152+- Replace `free_frame_nodes` in CallSite field access (was `free_ctx_nodes`)
153153+- Error message: "activation ID exhaustion"
154154+- Error category: `ErrorCategory.FRAME`
155155+156156+**Testing:**
157157+158158+Tests must verify:
159159+- pe-frame-redesign.AC5.3: Act_ids assigned 0, 1, 2, ... sequentially per function scope
160160+- pe-frame-redesign.AC5.7: When >frame_count scopes needed, error with `ErrorCategory.FRAME`
161161+- Single function scope gets act_id 0
162162+- Multiple functions on same PE get different act_ids
163163+- Same function across different PEs can reuse act_id 0
164164+165165+Test file: `tests/test_allocate_frames.py` (append)
166166+167167+**Verification:**
168168+Run: `python -m pytest tests/test_allocate_frames.py -v`
169169+Expected: All tests pass
170170+171171+**Commit:** `feat: add _assign_act_ids, remove _assign_context_slots`
172172+173173+<!-- END_TASK_2 -->
174174+175175+<!-- START_TASK_3 -->
176176+### Task 3: Add _compute_frame_layouts and _compute_modes
177177+178178+**Verifies:** pe-frame-redesign.AC5.2, pe-frame-redesign.AC5.4, pe-frame-redesign.AC5.5, pe-frame-redesign.AC5.6
179179+180180+**Files:**
181181+- Modify: `asm/allocate.py` (add two new functions)
182182+183183+**Implementation:**
184184+185185+**`_compute_modes()`** — derives OutputStyle, has_const, dest_count from edge topology and opcode:
186186+187187+```python
188188+def _compute_modes(
189189+ nodes_on_pe: dict[str, IRNode],
190190+ edges_by_source: dict[str, list[IREdge]],
191191+) -> dict[str, IRNode]:
192192+ """Compute (OutputStyle, has_const, dest_count) for each node from edge topology."""
193193+ updated = {}
194194+ for name, node in nodes_on_pe.items():
195195+ if node.seed:
196196+ updated[name] = node
197197+ continue
198198+199199+ out_edges = edges_by_source.get(name, [])
200200+ dest_count = len(out_edges)
201201+ has_const = node.const is not None
202202+203203+ # Determine OutputStyle from opcode and edge topology
204204+ if isinstance(node.opcode, MemOp):
205205+ # SM instructions: mode depends on MemOp semantics
206206+ # WRITE, CLEAR, FREE, SET_PAGE, WRITE_IMM → no return value → SINK
207207+ # READ, RD_INC, RD_DEC, CMP_SW, RAW_READ, EXT → return value → INHERIT
208208+ _sink_ops = {MemOp.WRITE, MemOp.CLEAR, MemOp.FREE, MemOp.SET_PAGE, MemOp.WRITE_IMM}
209209+ if node.opcode in _sink_ops:
210210+ output = OutputStyle.SINK
211211+ dest_count = 0
212212+ else:
213213+ output = OutputStyle.INHERIT
214214+ elif node.opcode == RoutingOp.FREE_FRAME:
215215+ output = OutputStyle.SINK
216216+ dest_count = 0
217217+ elif node.opcode == RoutingOp.EXTRACT_TAG:
218218+ output = OutputStyle.INHERIT
219219+ else:
220220+ # Check if any outgoing edge has ctx_override=True
221221+ # ctx_override=True on an edge means the source node is a
222222+ # cross-function return — its left operand carries a packed
223223+ # flit 1 (from EXTRACT_TAG) that determines the destination.
224224+ # In the frame model, this maps to OutputStyle.CHANGE_TAG.
225225+ has_ctx_override = any(e.ctx_override for e in out_edges)
226226+ output = OutputStyle.CHANGE_TAG if has_ctx_override else OutputStyle.INHERIT
227227+228228+ mode = (output, has_const, dest_count)
229229+ updated[name] = replace(node, mode=mode)
230230+231231+ return updated
232232+```
233233+234234+Note on `ctx_override` → `CHANGE_TAG` mapping: In the current codebase, `expand.py` sets `ctx_override=True` on edges that cross function boundaries (return paths from callee to caller). The old model packed the target context into the instruction's const field via `ctx_mode`. The new model uses `OutputStyle.CHANGE_TAG` instead — the PE reads a packed flit 1 from the left operand (produced by the caller's EXTRACT_TAG instruction) and uses it as the destination. The `IREdge.ctx_override` field should be renamed to `change_tag` in a later cleanup, but functionally it serves the same purpose: marking edges that need dynamic destination routing.
235235+236236+**`_compute_frame_layouts()`** — per-activation frame slot assignment:
237237+238238+```python
239239+def _compute_frame_layouts(
240240+ nodes_on_pe: dict[str, IRNode],
241241+ edges_by_source: dict[str, list[IREdge]],
242242+ edges_by_dest: dict[str, list[IREdge]],
243243+ all_nodes: dict[str, IRNode],
244244+ frame_slots: int,
245245+ pe_id: int,
246246+) -> tuple[dict[str, IRNode], list[AssemblyError]]:
247247+ """Compute frame slot layouts per activation.
248248+249249+ Slot assignment order:
250250+ 0 to matchable_offsets-1: match operands (one pair per dyadic instruction)
251251+ then: constants (deduplicated by value)
252252+ then: destinations (deduplicated by FrameDest identity)
253253+ then: sinks and SM parameters
254254+255255+ All activations of the same function share the canonical layout.
256256+ """
257257+```
258258+259259+Frame layout algorithm — match operands and const/dest are in **separate regions** of the frame:
260260+261261+**Match region** (slots 0 to `matchable_offsets - 1`):
262262+- Dyadic instructions are placed at low IRAM offsets (0 to `matchable_offsets - 1`) by the assembler's IRAM offset assignment (Phase 4 / Task 1).
263263+- Match operands are stored at `frames[frame_id][token.offset]`, where `token.offset` = IRAM offset.
264264+- The match region is indexed by IRAM offset, NOT by `fref`.
265265+- Multiple dyadic instructions can share a match slot if the compiler proves their operands are never pending simultaneously (liveness-based allocation).
266266+- If the activation has more dyadic instructions than `matchable_offsets`, emit a warning (per AC5.8).
267267+268268+**Const/dest region** (slots `matchable_offsets` and above):
269269+- Each instruction gets a contiguous `fref` group in this region, laid out per the mode table:
270270+ - Mode 0: `[dest]` — 1 slot
271271+ - Mode 1: `[const, dest]` — 2 slots
272272+ - Mode 2: `[dest1, dest2]` — 2 slots
273273+ - Mode 3: `[const, dest1, dest2]` — 3 slots
274274+ - Mode 4: no frame slots
275275+ - Mode 5: `[const]` — 1 slot
276276+ - Mode 6: `[sink_target]` — 1 slot (write-back)
277277+ - Mode 7: `[RMW_target]` — 1 slot (read-modify-write)
278278+- Constants are deduplicated by value — instructions sharing the same constant value can point to the same slot.
279279+- Destinations are deduplicated by FrameDest identity — instructions with the same target can share dest slots.
280280+281281+Algorithm:
282282+1. Group nodes by act_id (same act_id = same activation)
283283+2. For each activation group:
284284+ a. Count dyadic instructions → these occupy match slots 0 to N-1 (their IRAM offsets). Warn if N > `matchable_offsets`.
285285+ b. Collect all constants → deduplicate by value → assign const/dest slots starting at `matchable_offsets`.
286286+ c. Collect all destinations (from edges) → deduplicate by FrameDest identity → assign dest slots after constants.
287287+ d. Collect sinks and SM params → assign remaining slots.
288288+ e. Set `fref` on each node to point to the base of its const/dest group.
289289+3. Build `FrameLayout` with `FrameSlotMap`
290290+4. If total slots exceed `frame_slots`: report error with `ErrorCategory.FRAME`
291291+292292+Each node gets:
293293+- `fref` = base of its const/dest group in the frame (NOT the match slot)
294294+- `frame_layout` = the shared `FrameLayout` for its activation
295295+296296+**Testing:**
297297+298298+Tests must verify:
299299+- pe-frame-redesign.AC5.2: Two activations of the same function (same set of instructions) produce identical FrameLayout objects
300300+- pe-frame-redesign.AC5.4: Match operands at slots 0 to matchable_offsets-1 (indexed by IRAM offset), const/dest groups at matchable_offsets and above (indexed by fref)
301301+- pe-frame-redesign.AC5.5: OutputStyle derived from edge count (0 edges → SINK, 1-2 edges → INHERIT), has_const from const field, dest_count from edge count
302302+- pe-frame-redesign.AC5.6: When total frame slots > frame_slots limit, error with `ErrorCategory.FRAME`
303303+- Constants with same value share a slot
304304+- Destinations with same FrameDest share a slot
305305+306306+Test file: `tests/test_allocate_frames.py` (append)
307307+308308+**Verification:**
309309+Run: `python -m pytest tests/test_allocate_frames.py -v`
310310+Expected: All tests pass
311311+312312+**Commit:** `feat: add frame layout computation and mode derivation to allocate`
313313+314314+<!-- END_TASK_3 -->
315315+<!-- END_SUBCOMPONENT_A -->
316316+317317+<!-- START_TASK_4 -->
318318+### Task 4: Rework _resolve_destinations for FrameDest
319319+320320+**Verifies:** None directly (infrastructure for Phase 6 codegen)
321321+322322+**Files:**
323323+- Modify: `asm/allocate.py` (update `_resolve_destinations()` to produce FrameDest)
324324+325325+**Implementation:**
326326+327327+The current `_resolve_destinations()` produces `Addr` objects. The new version produces `FrameDest` objects that include `target_pe`, `offset`, `act_id`, `port`, and `token_kind`.
328328+329329+Token kind determination:
330330+- If destination node is dyadic: `TokenKind.DYADIC`
331331+- If destination node is monadic and inline: `TokenKind.INLINE`
332332+- Otherwise: `TokenKind.MONADIC`
333333+334334+For now, determine token kind from the destination node's opcode and edge properties:
335335+- If the destination node's IRAM instruction expects a DyadToken (dyadic opcode): `TokenKind.DYADIC`
336336+- Otherwise: `TokenKind.MONADIC`
337337+338338+Updated resolution for each edge:
339339+```python
340340+dest_node = all_nodes[edge.dest]
341341+frame_dest = FrameDest(
342342+ target_pe=dest_node.pe,
343343+ offset=dest_node.iram_offset,
344344+ act_id=dest_node.act_id,
345345+ port=edge.port,
346346+ token_kind=_determine_token_kind(dest_node),
347347+)
348348+resolved = ResolvedDest(name=edge.dest, addr=None, frame_dest=frame_dest)
349349+```
350350+351351+Note: `ResolvedDest` may need a `frame_dest` field added, or replaced entirely. Check `asm/ir.py` for the ResolvedDest definition and add `frame_dest: Optional[FrameDest] = None`.
352352+353353+Also assign FrameDest objects to frame slots during this step — each destination gets stored in the activation's frame layout at the appropriate dest slot offset.
354354+355355+**Verification:**
356356+Run: `python -m pytest tests/test_allocate_frames.py -v`
357357+Expected: All tests pass
358358+359359+**Commit:** `feat: resolve destinations as FrameDest objects with token kind`
360360+361361+<!-- END_TASK_4 -->
362362+363363+<!-- START_TASK_5 -->
364364+### Task 5: Wire new functions into allocate() main flow
365365+366366+**Verifies:** All AC5 criteria (integration)
367367+368368+**Files:**
369369+- Modify: `asm/allocate.py` (update `allocate()` to call new functions in order)
370370+371371+**Implementation:**
372372+373373+Update the `allocate()` main function to use the new sub-functions:
374374+375375+```python
376376+def allocate(graph: IRGraph) -> IRGraph:
377377+ errors = list(graph.errors)
378378+ system = graph.system
379379+380380+ if system is None:
381381+ # ... existing error handling ...
382382+383383+ all_nodes, all_edges = collect_all_nodes_and_edges(graph)
384384+ edges_by_source = _build_edge_index(all_edges)
385385+ edges_by_dest = _build_edge_index_by_dest(all_edges)
386386+387387+ # Validate non-commutative ops (unchanged)
388388+ errors.extend(_validate_noncommutative_const(all_nodes, edges_by_dest))
389389+390390+ # Assign SM IDs (unchanged)
391391+ sm_updated, sm_errors = _assign_sm_ids(all_nodes, system.sm_count)
392392+ errors.extend(sm_errors)
393393+ all_nodes.update(sm_updated)
394394+395395+ # Group nodes by PE
396396+ nodes_by_pe = _group_nodes_by_pe(all_nodes)
397397+398398+ intermediate_nodes = {}
399399+ for pe_id, nodes_on_pe in sorted(nodes_by_pe.items()):
400400+ # 1. Assign IRAM offsets (provisional)
401401+ iram_updated, iram_errors = _assign_iram_offsets(
402402+ nodes_on_pe, all_nodes, system.iram_capacity, pe_id
403403+ )
404404+ errors.extend(iram_errors)
405405+ if iram_errors: continue
406406+407407+ # 2. Assign activation IDs
408408+ act_updated, act_errors = _assign_act_ids(
409409+ list(iram_updated.values()), all_nodes,
410410+ system.frame_count, pe_id,
411411+ call_sites=graph.call_sites,
412412+ )
413413+ errors.extend(act_errors)
414414+ if act_errors: continue
415415+416416+ # 3. Compute modes (OutputStyle, has_const, dest_count)
417417+ mode_updated = _compute_modes(act_updated, edges_by_source)
418418+419419+ # 4. Compute frame layouts (assigns fref, frame_layout)
420420+ frame_updated, frame_errors = _compute_frame_layouts(
421421+ mode_updated, edges_by_source, edges_by_dest,
422422+ all_nodes, system.frame_slots, pe_id,
423423+ )
424424+ errors.extend(frame_errors)
425425+ if frame_errors: continue
426426+427427+ # 5. Deduplicate IRAM entries
428428+ deduped = _deduplicate_iram(frame_updated, pe_id)
429429+430430+ intermediate_nodes.update(deduped)
431431+432432+ # Resolve destinations (produces FrameDest)
433433+ for pe_id in sorted(nodes_by_pe.keys()):
434434+ nodes_on_this_pe = {
435435+ name: node for name, node in intermediate_nodes.items()
436436+ if node.pe == pe_id
437437+ }
438438+ if not nodes_on_this_pe: continue
439439+440440+ resolved, resolve_errors = _resolve_destinations(
441441+ nodes_on_this_pe, intermediate_nodes, edges_by_source
442442+ )
443443+ errors.extend(resolve_errors)
444444+ intermediate_nodes.update(resolved)
445445+446446+ # Reconstruct graph
447447+ result_graph = update_graph_nodes(graph, intermediate_nodes)
448448+ return replace(result_graph, errors=errors)
449449+```
450450+451451+**Verification:**
452452+Run: `python -m pytest tests/test_allocate_frames.py -v`
453453+Expected: All tests pass. `tests/test_allocate.py` may fail — that is expected and addressed in Task 6.
454454+455455+**Commit:** `feat: wire frame-based allocation into allocate() main flow`
456456+457457+<!-- END_TASK_5 -->
458458+459459+<!-- START_TASK_6 -->
460460+### Task 6: Update existing allocate tests
461461+462462+**Verifies:** None (regression)
463463+464464+**Files:**
465465+- Modify: `tests/test_allocate.py` (update for act_id, frame_count, new fields)
466466+467467+**Implementation:**
468468+469469+Update existing allocate tests:
470470+1. `ctx` field access → `act_id`
471471+2. `ctx_slots` in SystemConfig → `frame_count`
472472+3. Tests that check context overflow → check act_id exhaustion
473473+4. Tests that check IRAM cost of 2 for dyadic → expect 1
474474+5. Add assertions for new fields: `mode`, `fref`, `frame_layout`
475475+6. Update `_assign_context_slots` test calls → `_assign_act_ids`
476476+477477+**Verification:**
478478+Run: `python -m pytest tests/test_allocate_frames.py -v`
479479+Expected: All tests pass. `tests/test_allocate.py` tests that have been updated should also pass; stale tests in that file may still fail until fully converted.
480480+481481+**Commit:** `refactor: update allocate tests for frame-based allocation`
482482+483483+<!-- END_TASK_6 -->
···11+# PE Frame-Based Redesign — Phase 8: Fix Remaining Broken Tests and End-to-End Verification
22+33+**Goal:** Fix all tests that still reference removed or renamed types from earlier phases, verify the full pipeline works end-to-end, and get the entire test suite green.
44+55+**Architecture:** Earlier phases made targeted changes without maintaining backward compatibility. Many legacy test files will be broken by the time this phase begins. Phase 8 is where those are found and fixed. The codebase itself should already be correct — this phase is about aligning tests with what the code now does.
66+77+**Tech Stack:** Python 3.12
88+99+**Scope:** Phase 8 of 8 from the PE frame-based redesign design plan.
1010+1111+**Codebase verified:** 2026-03-07
1212+1313+---
1414+1515+## Acceptance Criteria Coverage
1616+1717+This phase implements and tests:
1818+1919+### pe-frame-redesign.AC7: End-to-end pipeline
2020+- **pe-frame-redesign.AC7.1 Success:** Existing test programs produce correct results through full pipeline (parse → ... → emulate)
2121+- **pe-frame-redesign.AC7.3 Success:** No backward compatibility shims remain (ALUInst, SMInst, IRAMWriteToken, MatchEntry removed)
2222+- **pe-frame-redesign.AC7.4 Success:** `python -m pytest tests/ -v` passes clean
2323+2424+---
2525+2626+<!-- START_TASK_1 -->
2727+### Task 1: Search for and remove remaining stale references
2828+2929+**Verifies:** pe-frame-redesign.AC7.3 (partial)
3030+3131+**Files:**
3232+- Modify: any source or test file containing stale references
3333+3434+**Implementation:**
3535+3636+Search the codebase for any remaining references to removed types and names. Fix or remove each one.
3737+3838+Types and names to search for:
3939+- `ALUInst` — removed from cm_inst.py in Phase 1
4040+- `SMInst` — removed from cm_inst.py in Phase 1
4141+- `Addr` — removed from cm_inst.py in Phase 1
4242+- `IRAMWriteToken` — removed from tokens.py in Phase 1
4343+- `MatchEntry` — removed from emu/types.py in Phase 1
4444+- `FREE_CTX` — replaced by `FREE_FRAME`
4545+- `ctx_slots` — replaced by `frame_count`
4646+- `gen_counters` — removed from PEConfig
4747+- `CtxSlotRef` — renamed to `ActSlotRef`
4848+- `CtxSlotRange` — renamed to `ActSlotRange`
4949+- `free_ctx_nodes` — renamed to `free_frame_nodes`
5050+- `.ctx` on IRNode — renamed to `.act_id`
5151+- `.ctx_slot` on IRNode — renamed to `.act_slot`
5252+- `node.ctx` in codegen — renamed to `node.act_id`
5353+- `gen=` in DyadToken construction — field removed
5454+- `ArithOp.SHIFT_L` — renamed to `ArithOp.SHL`
5555+- `ArithOp.SHIFT_R` — renamed to `ArithOp.SHR`
5656+- `ArithOp.ASHFT_R` — renamed to `ArithOp.ASR`
5757+- `shiftl`/`shiftr`/`ashiftr` in dfasm grammar — renamed to `shl`/`shr`/`asr`
5858+- `ctx_override` on IREdge — rename to `change_tag` (maps to OutputStyle.CHANGE_TAG in allocate)
5959+6060+For each reference found:
6161+- If in a test file: update the test to use the new name/type
6262+- If in a source file: this indicates an incomplete migration from an earlier phase — fix it
6363+6464+Run grep or similar across the entire repo before and after to confirm nothing remains.
6565+6666+**Verification:**
6767+Run: `python -c "import cm_inst, tokens, emu.types, asm.ir; print('imports OK')"`
6868+Expected: Prints "imports OK" (no import errors from stale references)
6969+7070+**Commit:** `chore: remove all remaining stale references to legacy types`
7171+7272+<!-- END_TASK_1 -->
7373+7474+<!-- START_TASK_2 -->
7575+### Task 2: Fix remaining broken test files
7676+7777+**Verifies:** pe-frame-redesign.AC7.4 (partial)
7878+7979+**Files:**
8080+- Modify: all test files that fail due to stale type/field references
8181+8282+**Implementation:**
8383+8484+Run `python -m pytest tests/ -v` and collect all failures. For each failing test:
8585+8686+1. Identify the cause — stale import, stale field access, stale constructor argument, or test helper that constructs old token types
8787+2. Update the test to use the new types and field names
8888+3. Re-run to confirm that specific test now passes
8989+9090+Common patterns to fix:
9191+- `from cm_inst import ALUInst` → remove or replace with `from cm_inst import Instruction`
9292+- `from tokens import IRAMWriteToken` → remove or replace with `from tokens import PELocalWriteToken`
9393+- `DyadToken(..., ctx=x, gen=y, ...)` → `DyadToken(..., act_id=x, ...)`
9494+- `MonadToken(..., ctx=x, ...)` → `MonadToken(..., act_id=x, ...)`
9595+- `PEConfig(ctx_slots=N, offsets=M, ...)` → `PEConfig(frame_count=N, ...)`
9696+- `pe_snap.matching_store` → `pe_snap.frames`
9797+- `pe_snap.gen_counters` → removed (no replacement needed in assertions)
9898+- `event.ctx` on Matched events → `event.act_id`
9999+- `node.ctx` → `node.act_id`
100100+- `system.ctx_slots` → `system.frame_count`
101101+102102+Work through the full test suite until no failures remain from stale references.
103103+104104+**Verification:**
105105+Run: `python -m pytest tests/ -v --tb=no -q`
106106+Expected: Failure count decreasing with each fix
107107+108108+**Commit:** `fix: update test files for frame-based types and field names`
109109+110110+<!-- END_TASK_2 -->
111111+112112+<!-- START_TASK_3 -->
113113+### Task 3: Fix remaining broken test files (continued)
114114+115115+**Verifies:** pe-frame-redesign.AC7.4 (partial)
116116+117117+**Files:**
118118+- Modify: test helper files and conftest.py if they construct old types
119119+120120+**Implementation:**
121121+122122+Check `tests/conftest.py` for Hypothesis strategies that generate old token types:
123123+- Strategies that produce `DyadToken` with `gen=` or `ctx=` fields need updating
124124+- Strategies that produce `IRAMWriteToken` need replacement with `PELocalWriteToken`
125125+- Strategies that use `ALUInst`/`SMInst` need replacement with `Instruction`
126126+127127+Also check any shared test utility functions (helper builders used across multiple test files) for stale type usage.
128128+129129+After updating conftest.py and helpers, re-run the full suite to catch any newly exposed failures.
130130+131131+**Verification:**
132132+Run: `python -m pytest tests/ -v --tb=short -q`
133133+Expected: Only genuine test logic failures remain (if any), no import or AttributeError failures
134134+135135+**Commit:** `fix: update conftest and test helpers for frame-based types`
136136+137137+<!-- END_TASK_3 -->
138138+139139+<!-- START_TASK_4 -->
140140+### Task 4: Update test_migration_cleanup.py
141141+142142+**Verifies:** pe-frame-redesign.AC7.3
143143+144144+**Files:**
145145+- Modify: `tests/test_migration_cleanup.py` (add checks for removed frame-migration types)
146146+147147+**Implementation:**
148148+149149+Add test cases verifying removed types are absent:
150150+151151+```python
152152+def test_aluinst_removed():
153153+ """ALUInst should not be importable from cm_inst."""
154154+ with pytest.raises(ImportError):
155155+ from cm_inst import ALUInst
156156+157157+def test_sminst_removed():
158158+ """SMInst should not be importable from cm_inst."""
159159+ with pytest.raises(ImportError):
160160+ from cm_inst import SMInst
161161+162162+def test_addr_removed():
163163+ """Addr should not be importable from cm_inst."""
164164+ with pytest.raises(ImportError):
165165+ from cm_inst import Addr
166166+167167+def test_iram_write_token_removed():
168168+ """IRAMWriteToken should not be importable from tokens."""
169169+ with pytest.raises(ImportError):
170170+ from tokens import IRAMWriteToken
171171+172172+def test_match_entry_removed():
173173+ """MatchEntry should not be importable from emu.types."""
174174+ with pytest.raises(ImportError):
175175+ from emu.types import MatchEntry
176176+177177+def test_free_ctx_removed():
178178+ """FREE_CTX should not exist as a separate value in RoutingOp."""
179179+ from cm_inst import RoutingOp
180180+ assert not hasattr(RoutingOp, 'FREE_CTX') or RoutingOp.FREE_CTX == RoutingOp.FREE_FRAME
181181+```
182182+183183+Also add positive tests that new types ARE importable:
184184+```python
185185+def test_instruction_exists():
186186+ from cm_inst import Instruction, OutputStyle, FrameDest, FrameOp, TokenKind
187187+ assert Instruction is not None
188188+189189+def test_pe_token_exists():
190190+ from tokens import PEToken, PELocalWriteToken, FrameControlToken
191191+ assert issubclass(PELocalWriteToken, PEToken)
192192+```
193193+194194+**Verification:**
195195+Run: `python -m pytest tests/test_migration_cleanup.py -v`
196196+Expected: All tests pass
197197+198198+**Commit:** `test: verify removed legacy types are absent from codebase`
199199+200200+<!-- END_TASK_4 -->
201201+202202+<!-- START_TASK_5 -->
203203+### Task 5: Update E2E tests for full pipeline
204204+205205+**Verifies:** pe-frame-redesign.AC7.1, pe-frame-redesign.AC7.4
206206+207207+**Files:**
208208+- Modify: `tests/test_e2e.py` (update run_program_direct and run_program_tokens helpers)
209209+- Modify: `tests/test_integration.py` (update for new types)
210210+211211+**Implementation:**
212212+213213+1. Update `run_program_direct()` helper:
214214+ - Use `assemble()` which returns `AssemblyResult` with `setup_tokens` and `seed_tokens`
215215+ - `PEConfig` now has `Instruction` IRAM and frame fields
216216+ - `build_topology()` takes new PEConfig format
217217+ - Inject `setup_tokens` before `seed_tokens`
218218+219219+2. Update `run_program_tokens()` helper:
220220+ - Use `assemble_to_tokens()` which returns new token types
221221+ - Token sequence includes `PELocalWriteToken`, `FrameControlToken`
222222+ - Load tokens via `system.load()`
223223+224224+3. Update all token constructor calls: `ctx=` → `act_id=`, remove `gen=`
225225+226226+4. Update all PEConfig constructor calls: remove `ctx_slots`, `offsets`, `gen_counters`
227227+228228+5. Verify existing test programs produce correct results:
229229+ - Simple add program: two values → correct sum
230230+ - Const program: constant injection → correct output
231231+ - Any branch/switch programs: correct routing
232232+ - SM programs: correct I-structure semantics
233233+234234+**Testing:**
235235+236236+The E2E tests themselves ARE the AC7.1 verification. They run the full pipeline and check results.
237237+238238+**Verification:**
239239+Run: `python -m pytest tests/test_e2e.py tests/test_integration.py -v`
240240+Expected: All tests pass
241241+242242+**Commit:** `test: update E2E tests for frame-based pipeline`
243243+244244+<!-- END_TASK_5 -->
245245+246246+<!-- START_TASK_6 -->
247247+### Task 6: Full test suite clean pass
248248+249249+**Verifies:** pe-frame-redesign.AC7.4
250250+251251+**Files:**
252252+- Modify: any remaining test files with stale references
253253+254254+**Implementation:**
255255+256256+This is the final task. Run the full test suite and fix every remaining failure:
257257+258258+```
259259+python -m pytest tests/ -v
260260+```
261261+262262+Work through failures one by one. At this point all failures should be:
263263+- Stale type references not caught in Tasks 1-3
264264+- Test logic that needs updating to match new behaviour
265265+- Any test helpers that still construct old token types
266266+267267+After each fix, re-run the suite to confirm the fix did not introduce new failures.
268268+269269+The suite is done when `python -m pytest tests/ -v` exits with zero failures and zero errors.
270270+271271+**Verification:**
272272+Run: `python -m pytest tests/ -v`
273273+Expected: All tests pass with zero failures
274274+275275+**Commit:** `chore: final cleanup — all tests pass clean`
276276+277277+<!-- END_TASK_6 -->
278278+279279+<!-- START_TASK_7 -->
280280+### Task 7: Update CLAUDE.md files
281281+282282+**Verifies:** None (documentation)
283283+284284+**Files:**
285285+- Modify: `CLAUDE.md` (root project instructions)
286286+- Modify: `asm/CLAUDE.md` (assembler package docs)
287287+288288+**Implementation:**
289289+290290+Both CLAUDE.md files extensively reference removed types and old field names. Update:
291291+292292+1. Root `CLAUDE.md`:
293293+ - Token hierarchy: remove `IRAMWriteToken`, add `PELocalWriteToken` and `FrameControlToken`, rename `ctx` → `act_id`, remove `gen` from DyadToken, add PEToken base class
294294+ - Instruction set: replace `ALUInst`/`SMInst`/`Addr` docs with `Instruction`/`FrameDest`/`OutputStyle`/`TokenKind`
295295+ - PE section: replace matching_store/gen_counters with frames/tag_store/presence/port_store/free_frames, update pipeline order to IFETCH→act_id resolution→MATCH→EXECUTE→EMIT
296296+ - SM section: t0_store is `list[int]` not `list[Token]`
297297+ - Event types: add FrameAllocated, FrameFreed, FrameSlotWritten, TokenRejected; update Matched.ctx→act_id
298298+ - PEConfig: remove ctx_slots, offsets, gen_counters; add frame_count, frame_slots, matchable_offsets, initial_frames, initial_tag_store
299299+ - Module dependency graph: add encoding.py
300300+301301+2. `asm/CLAUDE.md`:
302302+ - IR types: ctx→act_id, ctx_slot→act_slot, add mode/fref/wide/frame_layout fields, FrameSlotMap, FrameLayout
303303+ - SystemConfig: ctx_slots→frame_count, add frame_slots, matchable_offsets
304304+ - Allocate: describe frame layout computation, IRAM dedup, act_id assignment
305305+ - Codegen: describe setup_tokens, PELocalWriteToken/FrameControlToken generation
306306+ - ErrorCategory: add FRAME
307307+308308+Update the `<!-- freshness: ... -->` date on both files.
309309+310310+**Verification:**
311311+Manual review — ensure no references to removed types remain in either file.
312312+313313+**Commit:** `docs: update CLAUDE.md files for frame-based architecture`
314314+315315+<!-- END_TASK_7 -->
···11+# Test Requirements -- PE Frame-Based Redesign
22+33+This document maps each acceptance criterion from the PE frame-based redesign
44+design plan to either an automated test or a documented human verification step.
55+Every AC is accounted for. Test file paths reflect both new files introduced by
66+the implementation plan and updates to existing test files.
77+88+## Automated Test Coverage
99+1010+### AC1: Token hierarchy correctly models frame-based architecture
1111+1212+| AC | Test Type | Test File | Description |
1313+|---|---|---|---|
1414+| AC1.1 | unit | tests/test_foundation_types.py | Verify `issubclass(CMToken, PEToken)`, `issubclass(PELocalWriteToken, PEToken)`, `issubclass(FrameControlToken, PEToken)`, `issubclass(PEToken, Token)`. Verify transitive: `isinstance(DyadToken(...), PEToken)` is True. Verify `isinstance(SMToken(...), PEToken)` is False. Verify `IRAMWriteToken` does not exist in `tokens` module. |
1515+| AC1.2 | unit | tests/test_foundation_types.py | Verify `CMToken` constructor accepts `act_id` keyword (not `ctx`). Verify `DyadToken` constructor has no `gen` parameter (raises `TypeError` if `gen=` is passed). Verify all token types are frozen dataclasses. |
1616+| AC1.3 | unit | tests/test_foundation_types.py | Verify `PELocalWriteToken` has `region`, `slot`, `data`, `is_dest` fields. Construct instances with various field values and confirm frozen dataclass semantics. |
1717+| AC1.4 | unit | tests/test_foundation_types.py | Verify `FrameControlToken` has `act_id`, `op`, `payload` fields. Verify `op` accepts `FrameOp.ALLOC` and `FrameOp.FREE` values. |
1818+| AC1.5 | integration | tests/test_network_routing.py | Inject `PELocalWriteToken(target=0)`, `FrameControlToken(target=0)`, `DyadToken(target=0)`, and `SMToken(target=0)` via `System.inject()`. Verify PE-bound tokens arrive at PE 0's `input_store` and SM-bound tokens arrive at SM 0's `input_store`. Verify `build_topology()` routes all three PE-bound token types via the single `isinstance(token, PEToken)` check. |
1919+| AC1.6 | unit | tests/test_pe_frames.py | Inject a `DyadToken` with an `act_id` not present in the PE's `tag_store`. Verify the token is dropped (no crash, no output token). Verify a `TokenRejected` event is emitted with `reason` containing the invalid `act_id`. |
2020+2121+### AC2: Semantic Instruction model with pack/unpack
2222+2323+| AC | Test Type | Test File | Description |
2424+|---|---|---|---|
2525+| AC2.1 | unit | tests/test_foundation_types.py | Construct `Instruction` with all field types (`opcode: ALUOp`, `output: OutputStyle`, `has_const: bool`, `dest_count: int`, `wide: bool`, `fref: int`). Verify frozen dataclass. Verify `OutputStyle`, `TokenKind`, `FrameOp` enum values. Verify `RoutingOp.EXTRACT_TAG` (value 31), `RoutingOp.FREE_FRAME` (value 29), `RoutingOp.ALLOC_REMOTE` (value 30) exist. Verify `ArithOp.SHL` (8), `ArithOp.SHR` (9), `ArithOp.ASR` (10) exist and old names (`SHIFT_L`, `SHIFT_R`, `ASHFT_R`) do not. Verify all ALUOp values fit in 5 bits. |
2626+| AC2.2 | unit | tests/test_encoding.py | Property-based round-trip: `unpack_instruction(pack_instruction(inst)) == inst` for all valid mode combinations (modes 0-7) x wide x opcode classes (ArithOp, LogicOp, RoutingOp, MemOp) x fref 0-63. Verify mode encoding matches design-notes bit decode table. Verify opcode encoding is identity (Python IntEnum values match hardware). `pack_flit1`/`unpack_flit1` round-trip for all token kinds (DYADIC, MONADIC, INLINE) with various target_pe, offset, act_id, port values. Edge cases: fref=0, fref=63, target_pe=0, target_pe=3, act_id=0, act_id=7. |
2727+| AC2.3 | unit | tests/test_foundation_types.py | Verify `isinstance(Instruction(opcode=MemOp.READ, ...).opcode, MemOp)` returns `True`. Verify `isinstance(Instruction(opcode=ArithOp.ADD, ...).opcode, MemOp)` returns `False`. No stored `type` field on `Instruction`. |
2828+| AC2.4 | integration | tests/test_pe_frames.py | Verify the PE processes `Instruction` objects using only semantic fields (`output`, `has_const`, `dest_count`, `wide`, `fref`). Verify no raw mode bits or pack/unpack calls in the PE's main pipeline (IFETCH/MATCH/EXECUTE/EMIT). Only CHANGE_TAG, EXTRACT_TAG, and PELocalWriteToken(region=0) use encoding boundary functions. |
2929+3030+### AC3: Frame-based PE matching and output routing
3131+3232+| AC | Test Type | Test File | Description |
3333+|---|---|---|---|
3434+| AC3.1 | unit | tests/test_pe_frames.py | Construct PE with `frame_count=4`, `frame_slots=32`, `matchable_offsets=4`. Verify values stored on PE instance. Verify defaults (8, 64, 8) when not specified. |
3535+| AC3.2 | integration | tests/test_pe_frames.py | Inject a DyadToken pair targeting the same offset and act_id. Capture events via `on_event`. Verify event ordering: `TokenReceived` then `Matched` then `Executed` then `Emitted`. Verify IFETCH occurs before MATCH (instruction fetched from IRAM before matching attempted). |
3636+| AC3.3 | integration | tests/test_pe_frames.py | Pre-configure `tag_store` and frame with presence bits. Inject two `DyadToken`s at the same offset and act_id (one Port.L, one Port.R). Verify they match via frame SRAM and presence bits (not via `matching_store` array). Verify `Matched` event contains correct `left`, `right`, `act_id`, `offset`, `frame_id`. Verify port metadata determines left/right ordering. |
3737+| AC3.4 | integration | tests/test_pe_frames.py | Pre-load a `FrameDest` into a frame destination slot. Set up an `Instruction` with `OutputStyle.INHERIT`. Inject tokens to trigger execution. Verify the output token has `target`, `offset`, `act_id`, `port` matching the `FrameDest` values. |
3838+| AC3.5 | integration | tests/test_pe_frames.py | Set up an `Instruction` with `OutputStyle.CHANGE_TAG`. Inject a token whose left operand is a packed flit 1 value (from `pack_flit1(some_dest)`). Verify the output token's destination matches the unpacked `FrameDest` (target_pe, offset, act_id, port from the left operand, not from a frame slot). |
3939+| AC3.6 | integration | tests/test_pe_frames.py | Set up an `Instruction` with `OutputStyle.SINK`. Inject tokens to trigger execution. Verify the ALU result is written to `frames[frame_id][inst.fref]`. Verify no output token is emitted (`output_log` unchanged). Verify `FrameSlotWritten` event is emitted. |
4040+| AC3.7 | integration | tests/test_pe_frames.py | Set up an `Instruction` with `opcode=RoutingOp.EXTRACT_TAG` and `OutputStyle.INHERIT`. Inject a `MonadToken`. Verify the result is `pack_flit1(FrameDest(target_pe=pe_id, offset=token.offset, act_id=token.act_id, port=Port.L, token_kind=TokenKind.DYADIC))`. Verify the result is emitted as the data field of the output token. Verify `Executed` event shows the EXTRACT_TAG opcode. |
4141+| AC3.8 | integration | tests/test_pe_frames.py | Inject `FrameControlToken(op=FrameOp.ALLOC, act_id=X)`. Verify `tag_store[X]` is set to a valid `frame_id`, frame_id removed from `free_frames`, `FrameAllocated` event emitted. Inject `FrameControlToken(op=FrameOp.FREE, act_id=X)`. Verify `tag_store[X]` is removed, frame_id returned to `free_frames`, `FrameFreed` event emitted. Also test `FREE_FRAME` opcode path: set up a `FREE_FRAME` instruction in IRAM, inject a `MonadToken`, verify tag_store cleared, frame returned, `FrameFreed` event, no output token. Also test `ALLOC_REMOTE`: set up `ALLOC_REMOTE` instruction with target PE/act_id in frame constants, inject `MonadToken`, verify a `FrameControlToken(ALLOC)` is delivered to the target PE. |
4242+| AC3.9 | integration | tests/test_pe_frames.py | Inject `PELocalWriteToken(region=1, is_dest=True, data=pack_flit1(some_dest))`. Verify the frame slot at `token.slot` contains a `FrameDest` object (not a raw int). Verify `FrameSlotWritten` event. Also test `is_dest=False`: verify frame slot contains the raw int. |
4343+| AC3.10 | integration | tests/test_pe_frames.py | Inject a dyadic token pair at time T. Verify output token emitted at T+5 (dequeue 1 + IFETCH 1 + MATCH 1 + EXECUTE 1 + EMIT 1). Inject a monadic token at time T. Verify output at T+4 (dequeue 1 + IFETCH 1 + EXECUTE 1 + EMIT 1). Inject `FrameControlToken`/`PELocalWriteToken` at time T. Verify completion at T+2 (dequeue 1 + handle 1). Network delivery adds 1 additional cycle (separate process). |
4444+4545+### AC4: T0 raw storage and EXEC
4646+4747+| AC | Test Type | Test File | Description |
4848+|---|---|---|---|
4949+| AC4.1 | unit | tests/test_sm_t0_raw.py | After SM construction, verify `t0_store` is `list[int]`. Perform a T0 WRITE operation, verify stored values are ints (not Token objects). |
5050+| AC4.2 | integration | tests/test_sm_t0_raw.py | Pre-load `t0_store` with packed token flits via `pack_token()` for multiple tokens (DyadToken, MonadToken, SMToken). Trigger EXEC at the starting address. Verify reconstituted tokens arrive at target PEs/SMs with correct field values. Verify `flit_count()` correctly determines packet boundaries for consecutive multi-flit packets. |
5151+| AC4.3 | unit | tests/test_encoding.py | Property-based round-trip: `unpack_token(pack_token(token))` produces a token with matching fields for `DyadToken`, `MonadToken`, and `SMToken`. Verify `SMToken.ret` is not preserved (round-trips as `None`). Also tested implicitly in `tests/test_sm_t0_raw.py` through EXEC end-to-end. Update `tests/test_sm_tiers.py` and `tests/test_exec_bootstrap.py` to use `pack_token()` for T0 pre-loading. |
5252+| AC4.4 | unit | tests/test_sm_t0_raw.py | Pre-load `t0_store` with invalid prefix bits (e.g., `[0xFFFF]`). Trigger EXEC. Verify SM stops gracefully without crash and logs a warning. Also test truncated packet: set `flit_count` to indicate 3 flits but provide only 2 ints -- verify graceful stop with warning. |
5353+5454+### AC5: Assembler allocate produces frame layouts
5555+5656+| AC | Test Type | Test File | Description |
5757+|---|---|---|---|
5858+| AC5.1 | unit | tests/test_allocate_frames.py | Create two IR nodes on the same PE with identical opcode, mode, wide, and fref. Run `_deduplicate_iram()`. Verify both nodes share the same IRAM offset. Create nodes with different opcodes or modes and verify they get different offsets. Verify seed nodes are excluded. |
5959+| AC5.2 | unit | tests/test_allocate_frames.py | Create two activations of the same function body (same set of instructions). Run `_compute_frame_layouts()`. Verify both activations produce identical `FrameLayout` objects (same `FrameSlotMap`). |
6060+| AC5.3 | unit | tests/test_allocate_frames.py | Run `_assign_act_ids()` with multiple function scopes on a single PE. Verify act_ids assigned 0, 1, 2, ... sequentially. Verify at most `frame_count` concurrent act_ids per PE. Verify same function on different PEs can reuse act_id 0. |
6161+| AC5.4 | unit | tests/test_allocate_frames.py | Run `_compute_frame_layouts()` on an activation with dyadic and monadic nodes, constants, and multiple output edges. Verify match operand slots at 0 to `matchable_offsets-1` (indexed by IRAM offset). Verify constants (deduplicated by value) at `matchable_offsets` and above. Verify destinations (deduplicated by FrameDest identity) after constants. Verify sinks/SM params after destinations. |
6262+| AC5.5 | unit | tests/test_allocate_frames.py | Run `_compute_modes()` on nodes with varying edge topologies. Verify: 0 outgoing edges + FREE_FRAME opcode produces `(OutputStyle.SINK, *, 0)`. 1 outgoing edge produces `(OutputStyle.INHERIT, *, 1)`. 2 outgoing edges produces `(OutputStyle.INHERIT, *, 2)`. Edge with `ctx_override=True` produces `OutputStyle.CHANGE_TAG`. Verify `has_const` derived from `node.const is not None`. Verify SM sink ops (WRITE, CLEAR, etc.) produce `OutputStyle.SINK`. |
6363+| AC5.6 | unit | tests/test_allocate_frames.py | Create an activation requiring more frame slots than `frame_slots` limit. Verify `_compute_frame_layouts()` reports an error with `ErrorCategory.FRAME`. |
6464+| AC5.7 | unit | tests/test_allocate_frames.py | Create a PE requiring more than `frame_count` concurrent activations. Verify `_assign_act_ids()` reports an error with `ErrorCategory.FRAME` and message containing "activation ID exhaustion". |
6565+| AC5.8 | unit | tests/test_allocate_frames.py | Create an activation with more dyadic IRAM offsets than `matchable_offsets`. Verify a warning is emitted with `ErrorCategory.FRAME` and `severity="warning"`. Verify placement still succeeds (not an error). Also tested in `tests/test_place.py` (update existing `_count_iram_cost` tests to expect cost=1 for all node types). |
6666+6767+### AC6: Assembler codegen produces frame setup tokens
6868+6969+| AC | Test Type | Test File | Description |
7070+|---|---|---|---|
7171+| AC6.1 | unit | tests/test_codegen_frames.py | Verify `AssemblyResult` dataclass has a `setup_tokens` field of type `list[Token]`. |
7272+| AC6.2 | integration | tests/test_codegen_frames.py | Run `generate_tokens()` on a simple program. Inspect the output token list. Verify ordering: all `SMToken` writes first, then `PELocalWriteToken(region=0)` IRAM writes, then `FrameControlToken(op=ALLOC)`, then `PELocalWriteToken(region=1)` frame slot writes, then seed `DyadToken`/`MonadToken` last. |
7373+| AC6.3 | unit | tests/test_codegen_frames.py | Run `_build_iram_for_pe()` on allocated IR nodes. Verify IRAM entries are `Instruction` objects. Verify IRAM write `PELocalWriteToken` data field equals `pack_instruction(inst)` for each entry. Verify CM vs SM type derived from `isinstance(opcode, MemOp)`. |
7474+| AC6.4 | unit | tests/test_codegen_frames.py | Inspect destination frame slot write tokens in setup sequence. Verify each has `is_dest=True` and `data == pack_flit1(expected_frame_dest)`. |
7575+| AC6.5 | integration | tests/test_codegen_frames.py | For a program with T0 bootstrap data, verify T0 writes use `pack_token()` to produce packed flit sequences. Verify the written ints can be round-tripped back via `unpack_token()`. |
7676+| AC6.6 | unit | tests/test_codegen_frames.py | Verify seed tokens use `act_id` field. Verify `DyadToken` seeds have no `gen` field. Verify seed `MonadToken`s use `act_id`. Update `tests/test_codegen.py` for `act_id` rename and `gen` removal. |
7777+7878+### AC7: End-to-end pipeline
7979+8080+| AC | Test Type | Test File | Description |
8181+|---|---|---|---|
8282+| AC7.1 | e2e | tests/test_e2e.py | Run existing test programs through the full pipeline (parse, lower, expand, resolve, place, allocate, codegen, build_topology, inject setup_tokens + seed_tokens, run SimPy). Verify correct results for: simple add, constant injection, branch/switch routing, SM I-structure read/write. Update `run_program_direct()` and `run_program_tokens()` helpers for new types. |
8383+| AC7.1 | e2e | tests/test_integration.py | Update integration tests for new token types, field names, and PEConfig format. Verify programs produce correct outputs through the new frame-based pipeline. |
8484+| AC7.2 | integration | tests/test_backend.py | Verify `_handle_load()` injects `setup_tokens` before `seed_tokens`. Verify monitor loads programs and returns `GraphLoaded` with valid snapshot containing frame state. |
8585+| AC7.2 | integration | tests/test_snapshot.py | Verify `PESnapshot` has `frames`, `tag_store`, `presence`, `port_store`, `free_frames`. Verify `PESnapshot` does NOT have `matching_store` or `gen_counters`. Verify `capture()` reads frame data from live PE. Verify `SMSnapshot.t0_store` contains ints. |
8686+| AC7.2 | integration | tests/test_monitor_graph_json.py | Verify node JSON has `act_id` key (not `ctx`). Verify `Matched` event JSON has `act_id` and `frame_id`. Verify new event types (`FrameAllocated`, `FrameFreed`, `FrameSlotWritten`, `TokenRejected`) serialize correctly. Verify PE state JSON has `frames`, `tag_store`, `free_frames`. |
8787+| AC7.2 | integration | tests/test_repl.py | Verify `pe` command displays frame state (not matching store). Verify `send` command accepts `act_id` parameter. |
8888+| AC7.3 | unit | tests/test_migration_cleanup.py | Verify `ALUInst`, `SMInst`, `Addr` not importable from `cm_inst`. Verify `IRAMWriteToken` not importable from `tokens`. Verify `MatchEntry` not importable from `emu.types`. Verify `RoutingOp.FREE_CTX` does not exist (or is the same as `FREE_FRAME`). Verify positive: `Instruction`, `OutputStyle`, `FrameDest`, `PEToken`, `PELocalWriteToken`, `FrameControlToken` all importable. |
8989+| AC7.4 | e2e | tests/ (full suite) | `python -m pytest tests/ -v` passes with zero failures and zero errors. This is the final gate criterion verified in Phase 8 Task 6. |
9090+9191+## Additional Test Files Updated (not new)
9292+9393+These existing test files require updates for renamed fields and removed types.
9494+They do not map to specific ACs but are necessary for AC7.4 (full suite green):
9595+9696+| Test File | Changes Required |
9797+|---|---|
9898+| tests/conftest.py | Update Hypothesis strategies: `ctx` to `act_id`, remove `gen`, add `PELocalWriteToken` and `FrameControlToken` strategies |
9999+| tests/test_alu.py | Rename shift op references (`SHIFT_L` to `SHL`, etc.), `FREE_CTX` to `FREE_FRAME`, add `EXTRACT_TAG`/`ALLOC_REMOTE` as no-op fallback cases |
100100+| tests/test_pe.py | Rename to use new PE constructor params, `act_id`, frame-based matching. May be superseded by `test_pe_frames.py`. |
101101+| tests/test_pe_events.py | Update for `act_id`, `Instruction` IRAM, frame-based PE constructor, new event fields |
102102+| tests/test_cycle_timing.py | Update PE construction and token fields; timing values unchanged (5/4/1 cycles) |
103103+| tests/test_sm_events.py | Update token constructor calls (`act_id`, no `gen`) |
104104+| tests/test_sm_tiers.py | Update T0 tests to use `pack_token()` for pre-loading, int retrieval |
105105+| tests/test_exec_bootstrap.py | Pre-load T0 with packed flits via `pack_token()` instead of Token objects |
106106+| tests/test_network_events.py | Update token types and field names |
107107+| tests/test_network.py | Update `PEConfig` construction, token fields |
108108+| tests/test_allocate.py | `ctx` to `act_id`, `ctx_slots` to `frame_count`, IRAM cost=1 for all nodes |
109109+| tests/test_place.py | `ctx_slots` to `frame_count`, IRAM cost=1, matchable offset warning |
110110+| tests/test_autoplacement.py | `ctx_slots` to `frame_count` |
111111+| tests/test_codegen.py | `ctx` to `act_id`, `gen` removal, `Instruction` IRAM, `PEConfig` new fields |
112112+| tests/test_lower.py | `ctx_slot` to `act_slot` |
113113+| tests/test_expand.py | `CtxSlotRef` to `ActSlotRef`, `FREE_CTX` to `FREE_FRAME` |
114114+| tests/test_call_wiring.py | `free_ctx_nodes` to `free_frame_nodes`, `FREE_CTX` to `FREE_FRAME` |
115115+| tests/test_serialize.py | `ctx` to `act_id`, `ctx_slot` to `act_slot` |
116116+| tests/test_seed_const.py | `ctx` to `act_id`, `gen` removal |
117117+| tests/test_sm_graph_nodes.py | Token field renames |
118118+| tests/test_macro_ir.py | `ctx_slot` to `act_slot` |
119119+| tests/test_macro_ret_wiring.py | `ctx_slot` to `act_slot`, `FREE_CTX` to `FREE_FRAME` |
120120+| tests/test_opcodes.py | Shift mnemonic renames, `free_ctx` to `free_frame`, add `extract_tag`/`alloc_remote` |
121121+| tests/test_dfgraph_categories.py | `FREE_CTX` to `FREE_FRAME`, add `EXTRACT_TAG` |
122122+| tests/test_dfgraph_json.py | `ctx` to `act_id` |
123123+| tests/test_monitor_server.py | Token field renames, new event types in WebSocket protocol |
124124+125125+## Human Verification
126126+127127+| AC | Justification | Verification Approach |
128128+|---|---|---|
129129+| AC2.4 | "Emulator PE never touches raw mode bits -- only semantic fields" is an architectural invariant about code structure, not runtime behavior. An automated test can verify that the PE does not import pack/unpack for its main pipeline, but confirming the *absence* of bit-level manipulation throughout the PE is a code review concern. | Code review of `emu/pe.py` after Phase 2. Verify: (1) no raw integer bit masking/shifting on instruction fields in `_process_token()`, `_match_frame()`, `_do_emit_new()`, `_emit_inherit()`, or `_emit_sink()`; (2) `pack_flit1`/`unpack_flit1` calls appear only in `_emit_change_tag()` (CHANGE_TAG path), EXTRACT_TAG handler, and `_handle_local_write()` (PELocalWriteToken region=0 decode); (3) `Instruction` fields accessed by name (`inst.output`, `inst.has_const`, `inst.fref`) rather than by bit extraction. A supplementary automated test in `tests/test_pe_frames.py` can use `inspect.getsource()` to grep for bit operations as a heuristic check. |
130130+| AC7.2 (REPL display) | The REPL `pe` command display of frame state is a human-facing text formatting concern. Automated tests verify the data is present, but the visual layout (indentation, column alignment, readability) requires human judgment. | Run `python -m monitor` with a loaded program, execute `pe 0`, visually confirm: tag store mapping displayed, free frames listed, frame slot contents shown per frame, no references to "matching store" or "gen counters". |
131131+| AC7.2 (WebSocket events) | While JSON serialization of new event types is tested automatically, verifying the monitor web UI correctly renders frame state and new events in the browser requires a running frontend. | Start `python -m monitor --web`, load a program, step through execution in the browser. Verify: frame state panel shows frames/tag_store/free_frames, event log shows FrameAllocated/FrameFreed/FrameSlotWritten/TokenRejected events with correct fields, no rendering errors in the console. |
132132+| Phase 8 CLAUDE.md updates | Documentation accuracy cannot be verified by automated tests. The root `CLAUDE.md` and `asm/CLAUDE.md` must be manually reviewed to ensure all references to removed types, old field names, and old pipeline descriptions have been updated. | Review both files against the final codebase state. Check: (1) token hierarchy section matches new PEToken-based hierarchy; (2) instruction set section describes Instruction/FrameDest/OutputStyle not ALUInst/SMInst/Addr; (3) PE section describes frames/tag_store not matching_store/gen_counters; (4) PEConfig section lists frame_count/frame_slots/matchable_offsets not ctx_slots/offsets; (5) event types section includes all four new events; (6) freshness date updated. |
133133+134134+## Rationalization Against Implementation Decisions
135135+136136+The following implementation decisions from the phase plans affect test strategy:
137137+138138+1. **Phase 1 removes old types immediately** (no backward-compat shims). This means
139139+ many existing tests break at Phase 1 and are not fixed until Phase 8. Test files
140140+ created in Phases 1-7 (`test_foundation_types.py`, `test_encoding.py`,
141141+ `test_pe_frames.py`, `test_sm_t0_raw.py`, `test_network_routing.py`,
142142+ `test_allocate_frames.py`, `test_codegen_frames.py`) are self-contained and pass
143143+ within their phase. The "full suite green" criterion (AC7.4) is only achievable
144144+ after Phase 8.
145145+146146+2. **Enum value renumbering** (Task 1 of Phase 1) to match 5-bit hardware encoding
147147+ changes integer values of existing enums. Tests using `frozenset` membership on
148148+ enum objects are unaffected (membership is by identity). Tests comparing raw
149149+ integer values of enum members need updating.
150150+151151+3. **IRAM deduplication is a post-processing step** (Phase 5 Task 1). The
152152+ `_deduplicate_iram()` function runs after `_compute_modes()` and
153153+ `_compute_frame_layouts()` produce `fref` and mode fields. AC5.1 tests must
154154+ set up fully-computed nodes (with mode and fref) before calling dedup.
155155+156156+4. **Match slot indexing** uses `token.offset % matchable_offsets` (Phase 2 Task 2),
157157+ not a direct frame slot index. Tests for AC3.3 must set up IRAM offsets and
158158+ inject tokens at those offsets, not at arbitrary slot indices.
159159+160160+5. **CHANGE_TAG uses the left operand** (not the ALU result) as the destination
161161+ descriptor (Phase 2 Task 2). Tests for AC3.5 must inject tokens where the left
162162+ data field contains a packed flit 1, and verify the destination is derived from
163163+ that left operand.
164164+165165+6. **SMToken.ret is NOT preserved** through pack/unpack round-trip (Phase 1 Task 4).
166166+ Tests for AC4.3 must verify SMToken round-trips with `ret=None`. This is by
167167+ design -- SM return routes are stored as FrameDest in frame slots in the new model.
168168+169169+7. **ctx_override on IREdge maps to OutputStyle.CHANGE_TAG** (Phase 5 Task 3). The
170170+ field is not renamed in Phase 5 but is scheduled for rename to `change_tag` in
171171+ Phase 8 cleanup. Tests for AC5.5 must use the `ctx_override` field name when
172172+ testing CHANGE_TAG mode derivation.
+31-10
tests/conftest.py
···44from hypothesis import strategies as st
55from lark import Lark
6677-from cm_inst import ArithOp, LogicOp, MemOp, Port, RoutingOp
88-from tokens import CMToken, DyadToken, MonadToken, SMToken
77+from cm_inst import ArithOp, FrameOp, LogicOp, MemOp, Port, RoutingOp
88+from tokens import CMToken, DyadToken, FrameControlToken, MonadToken, PELocalWriteToken, SMToken
991010GRAMMAR_PATH = Path(__file__).parent.parent / "dfasm.lark"
1111···15151616arith_dyadic_ops = st.sampled_from([ArithOp.ADD, ArithOp.SUB])
1717arith_monadic_ops = st.sampled_from([ArithOp.INC, ArithOp.DEC])
1818-shift_ops = st.sampled_from([ArithOp.SHIFT_L, ArithOp.SHIFT_R, ArithOp.ASHFT_R])
1818+shift_ops = st.sampled_from([ArithOp.SHL, ArithOp.SHR, ArithOp.ASR])
1919logic_dyadic_ops = st.sampled_from([LogicOp.AND, LogicOp.OR, LogicOp.XOR])
2020comparison_ops = st.sampled_from([LogicOp.EQ, LogicOp.LT, LogicOp.LTE, LogicOp.GT, LogicOp.GTE])
2121branch_ops = st.sampled_from([RoutingOp.BREQ, RoutingOp.BRGT, RoutingOp.BRGE])
···252526262727@st.composite
2828-def dyad_token(draw, target: int = 0, offset: int | None = None, ctx: int | None = None, gen: int | None = None) -> DyadToken:
2828+def dyad_token(draw, target: int = 0, offset: int | None = None, act_id: int | None = None) -> DyadToken:
2929 return DyadToken(
3030 target=target,
3131 offset=draw(st.integers(min_value=0, max_value=63)) if offset is None else offset,
3232- ctx=draw(st.integers(min_value=0, max_value=3)) if ctx is None else ctx,
3232+ act_id=draw(st.integers(min_value=0, max_value=7)) if act_id is None else act_id,
3333 data=draw(uint16),
3434 port=draw(st.sampled_from(list(Port))),
3535- gen=draw(st.integers(min_value=0, max_value=3)) if gen is None else gen,
3635 wide=False,
3736 )
383739384039@st.composite
4141-def monad_token(draw, target: int = 0, offset: int | None = None, ctx: int | None = None) -> MonadToken:
4040+def monad_token(draw, target: int = 0, offset: int | None = None, act_id: int | None = None) -> MonadToken:
4241 return MonadToken(
4342 target=target,
4443 offset=draw(st.integers(min_value=0, max_value=63)) if offset is None else offset,
4545- ctx=draw(st.integers(min_value=0, max_value=3)) if ctx is None else ctx,
4444+ act_id=draw(st.integers(min_value=0, max_value=7)) if act_id is None else act_id,
4645 data=draw(uint16),
4746 inline=False,
4847 )
···6463 _addr = draw(st.integers(min_value=0, max_value=255)) if addr is None else addr
6564 _op = draw(sm_all_ops) if op is None else op
6665 _data = draw(uint16) if data is None else data
6767- ret = CMToken(target=0, offset=0, ctx=0, data=0)
6666+ ret = CMToken(target=0, offset=0, act_id=0, data=0)
6867 return SMToken(
6968 target=0,
7069 addr=_addr,
···8079 return CMToken(
8180 target=target,
8281 offset=draw(st.integers(min_value=0, max_value=63)),
8383- ctx=draw(st.integers(min_value=0, max_value=3)),
8282+ act_id=draw(st.integers(min_value=0, max_value=7)),
8483 data=0,
8484+ )
8585+8686+8787+@st.composite
8888+def pe_local_write_token(draw, target: int = 0, act_id: int | None = None) -> PELocalWriteToken:
8989+ return PELocalWriteToken(
9090+ target=target,
9191+ act_id=draw(st.integers(min_value=0, max_value=7)) if act_id is None else act_id,
9292+ region=draw(st.integers(min_value=0, max_value=1)),
9393+ slot=draw(st.integers(min_value=0, max_value=63)),
9494+ data=draw(uint16),
9595+ is_dest=draw(st.booleans()),
9696+ )
9797+9898+9999+@st.composite
100100+def frame_control_token(draw, target: int = 0, act_id: int | None = None) -> FrameControlToken:
101101+ return FrameControlToken(
102102+ target=target,
103103+ act_id=draw(st.integers(min_value=0, max_value=7)) if act_id is None else act_id,
104104+ op=draw(st.sampled_from(list(FrameOp))),
105105+ payload=draw(uint16),
85106 )
8610787108