Personal fork of prime agent. Mirror of: https://mnemosyne.sigilzero.dev/sigilzero/prime-agent

make /reload re-read the session live inputs (#3393)

* make /reload re-read the session's live inputs instead of a wire-compat no-op The daemon 'reload' answered the TS success without re-reading anything, while its own doc promised settings, provider auth, and the MCP user-server config — after a /login from another process, the running session kept serving the stale credential from its build-time provider target. The handler now rebinds the request auth from the credential store (the same block a model switch runs) and re-reads the MCP manager's settings and auth store. Settings and the model catalog resolve per use and need nothing. * fix /reload: keep the live model, park the auth reload off the runtime The reload rebuilt the provider target from a fresh resolve_model call, which retargeted the live request path (a routed image episode lost its serving model; a changed settings default could switch an unflagged session) and called the MCP auth store blocking_lock on the async worker runtime (a panic). The reload now refreshes only the stored credentials and headers for the LIVE model, keeps its service tier, and parks the whole reload on a blocking thread like get_mcp_connections. The compile and fmt failures of the first push (private visibility, formatting) ride along; the test fixtures load a valid models.json (custom providers require apiKey) and build full wire models. * address review: commit the auth refresh on the live slot, refresh the armed route, guard the async path with a test The auth refresh now lands in place on the CURRENT slot contents under the write lock (a racing set_model, route swap, or retirement is never clobbered by an older snapshot; a changed model re-resolves for whatever is live now). A routed image episode re-applies its armed target on every model-turn attempt, so the reload refreshes that snapshot auth from the same store. A real-engine async dispatch test now guards the blocking thread posture: it fails with a runtime block panic if the reload moves back onto the executor. * address re-review: fence the route install against the reload, refresh the saved session target, clean up the test dir apply_armed_image_route now holds the route lock through the slot write (the clone-and-install is one critical section), and the reload refreshes the armed route BEFORE the live slot: an attempt either installs the refreshed route target or lands before the slot refresh, never over it. The armed route saved session-target fallback re-binds from the same store, so a failing route clear never restores the pre-login key. The real-engine async test builds its worker under a TempDir. * fence every armed-route install against the reload The failover switch and restore paths cloned the armed route under the route lock, released it, then wrote the clone into the provider slot: a /reload refreshing the route and the slot in between reinstalled the old credentials after the reload answered. Both installs now go through install_armed_route_target, which holds the route lock through the slot write (the same fence apply_armed_image_route uses). The restore path no longer holds the provider-slot lock across the route read either, so every path that touches both locks takes them route-then-slot. * re-resolve the failover primary request auth at restore The failover capture kept the pre-switch key and headers for the whole episode; a credential rotated (and reloaded) during the failover served the old auth again once the turn settled back on the primary. The restore now re-resolves the request auth from the store for the captured model and falls back to the capture only when nothing resolves — the same never-clear semantics the reload refresh uses. * retrigger ci: the heartbeat snapshot fallback flake (sigstop race; passes 3/3 locally on this head) * a resolved credential replaces the request headers, it never inherits the old ones A credential whose resolution succeeds rebinds the pair in place: the key AND the headers, so a rotated credential without a team header clears the stale X-Prime-Team-ID instead of serving it on the new key. The failover primary restore falls back to the captured pair only when the store resolves nothing - a resolved key must never be paired with the captured headers. * assert the reload auth with boolean guards: the resolved key never formats into a failure message CodeQL flags the resolved credential flowing into an assert_eq! failure format (cleartext logging of sensitive information): the new header tests compare without the formatting sink. * surface the auth store reload failure: the reload reports what the session applied reload_auth_storage returns the storage reload error (a malformed auth.json or an unacquirable lock leaves the MCP manager on its previous credentials), reload_live_inputs propagates it, and handle_reload answers the client with the failure instead of a success the session did not apply. The connections view keeps its best-effort posture. * gate the target rebinds on the auth store reload: a failed reload keeps the last-good request auth The auth store re-read now runs first in reload_live_inputs: a malformed document or an unacquirable lock fails the reload before any request-auth resolution, so the live target keeps its working stored credential instead of falling to the configured fallback key over the unreadable store. The settings re-read still applies on the failed path; the error still propagates to the client. * serialize /reload: the last reload to run leaves the target serving the newest store Overlapping reload_live_inputs calls each resolve the store at their own read times, so an earlier reload could install its already-resolved key/header pair over a later reload's fresher pair (the model-unchanged branch trusts its pre-read). A session-level reload lock serializes the whole refresh; the armed route and live slot keep their existing lock discipline. * retrigger ci: the concurrent harness-state refinement flake (passes 3/3 locally; main green at 49480faa0) * the failover restore keeps the captured pair when the auth store read fails An unreadable auth.json makes the fresh resolution fall to the configured fallback key, which is not a stored credential - restored_primary_target must not replace the captured last-good pair with it, exactly the overwrite /reload's gate avoids. The resolution now reports the store's read health; the capture backs the restore when nothing resolves or the store failed. The create-config key override stays store-independent. * retrigger ci: the supervisor tool-results stats flake (same test flaked on main's own PR f67c5cf9 and merged green after retrigger) * a failed store read keeps the override key with the captured headers, and the restore serializes with /reload The create-config key override stands in memory regardless of the store's health, but its resolved headers still ride the store read: on a failed read the restore keeps the captured headers (the last-good team context). The failover restore also takes the session reload lock across its resolve-and-install, so a /reload landing mid-restore is never clobbered by the restore's older pair - whichever runs last leaves the newest store standing. * the override test asserts without formatting the resolved pair: no sensitive data in failure messages CodeQL flags the restored pair (the registry-resolved auth) flowing into an assert message: the header comparison keeps its boolean guard without the formatting sink. * retrigger ci: the catalog-slices supervisor timing flake (same protocol_breadth_b10_b11 binary as the earlier flake; passes 3/3 locally) * document the restore's blocking-pool posture: the turn (this closure included) runs on spawn_blocking, never an async executor worker The serializer thread asked for the locked resolution-and-install to be offloaded to a blocking task: the whole turn already runs inside one (worker/turn.rs parks the turn on spawn_blocking), so the synchronous credential resolution, OAuth refresh included, never blocks an executor worker - only a concurrently reloading session waits on the lock span, which is the serialization the clobber-race finding required. * fence the route clear against /reload: the clear serializes its resolve-and-install like the failover restore clear_image_route resolved fresh auth and wrote the slot without the reload lock, so a /reload landing in that window was clobbered by the clear's older pair - a login that just reloaded never reached the next turn (the same race the failover restore's fence closed). The clear now takes the session reload lock across its resolve-and-install; it runs on the turn's blocking thread, so the synchronous resolution never touches an async executor worker. Also merges main 7f646a40f (tombstoned-worker retirement). * fence the failover switch against /reload: the last unfenced resolve-and-install in the turn path The failover switch resolved the backup model's key/headers and wrote the slot without the reload lock, so a /reload landing in that window was clobbered by the switch's older pair - the restore callback in the same function already takes the lock for exactly this. The switch now serializes its resolve-and-install the same way (blocking-pool turn, no executor contact). * retrigger ci: the owned-stop kill-timing flake (passes 3/3 locally on 784520b0f; main green at afe8d14c4) * retrigger ci: the mid-provider-wait abort flake (different timing test each run; green in 10+ prior CI runs of this code; 974/975 in the binary) * rebind only off live store auth: the models.json fallback key is never a stored credential The reload path and the failover restore treated any resolved key as store auth, so a healthy store that no longer holds the provider's credential (or a second read racing the gate's) resolved the configured models.json fallback and overwrote the target's last-good pair - violating the PR's own 'nothing stored never clears' contract. The resolution now reports whether the store actually holds a live credential for the provider on a healthy read (one read cycle, answering both questions); the fallback key never replaces a serving pair anywhere, and the override keeps the captured headers when the store cannot serve. * retrigger ci: the deny job hit Docker Hub's unauthenticated pull rate limit (429) — same infra failure main's own CI shows at 6808008ac and 763094e1b * fence the remaining slot writers against /reload and state the live-credential invariant switch_model, build_session, and arm_image_turn_route resolved credentials and wrote the slot (or the armed route) without the reload lock, so a concurrent /reload landing between their resolve and install was clobbered by the older pair - the same race the failover callbacks, route clear, and failover switch fences close. All three now serialize their resolve-and-install; every writer is verified non-reentrant with the lock (build_session's scoped guard never rides an await; the arm returns before the turn's fenced paths). The store-auth docs now state the true invariant: the rebind runs onto a live credential - stored, or the ambient environment/runtime override, like a fresh resolution after a logout - and never onto the models.json fallback placeholder. * retrigger ci: the post-switch TUI render flake (screen-content assertion; passes 3/3 locally on this exact head; no stall signature) * hold the reload fence through route publication, and from before the route take on clear The arm's lock was scoped inside the match arm and dropped before the armed route published, so a /reload in that gap refreshed credentials the arm then overwrote with its older pair; and clear took the route before locking, leaving a concurrent reload unable to refresh the detached pair while the clear's fallback restore could clobber the reload's slot refresh. The arm now holds the lock from before the resolution through the route install; the clear takes it before the route take. Lock order is reload -> route -> slot everywhere. * fence the apply's session-target capture: the last route writer joins the reload serialization apply_armed_image_route captured the session target and installed the route without the reload lock, so an apply landing in the reload's window between its route refresh and slot rebind captured the pre-reload pair - a later clear with a failed resolve would restore those stale credentials. The apply now holds the lock across capture and install; every provider-target and route writer now serializes under it. * retrigger ci: the loaded-runner perf flakes (drag_select frame cost + protocol linearity; both pass locally on this head; both green on the prior head's CI) * retrigger ci: the contended-runner drag_select perf flake, roll 5 (identical pa-tui source green on main and locally; the PR-runner pool is contended tonight) * retrigger ci: the contended-runner perf-band flakes, roll 6 (the failing timing tests are main's code, green on main and locally; PR pool still contended) * retrigger ci: the contended-runner perf-band flakes, roll 7 * merge-compat: the reload dispatch test builds the WorkerConfig the Decision API child flag added * the reload lock becomes a tokio mutex: the async build path parks instead of blocking the executor build_session took the std reload lock on an async path, so a concurrent /reload holding it across auth-file I/O and a possible OAuth refresh stalled the executor worker until the reload finished. The lock is now tokio::sync::Mutex: the async build locks by .await (the task parks, no worker stall), and every blocking-pool site (the reload's spawn_blocking, the turn machinery's fences, switch_model) uses blocking_lock - the same posture as the session and mcp locks. Comments tightened per review: the fence rationale stays, the step-order narration goes. * Revert "the reload lock becomes a tokio mutex: the async build path parks instead of blocking the executor" This reverts commit 81dc3619ef23fd2ac0d2caa60d7fa9cbcb9bc1b7. * the build's async path takes the reload lock by yielding, never by blocking the worker Reverting the tokio-mutex conversion: the turn machinery runs inside the engine's own runtime.block_on, so blocking_lock panicked in the failover path (provider_failover_e2e failed locally and on CI). The std lock returns everywhere; the build's async acquire becomes a try_lock-and-yield loop - a reload holding the lock across auth I/O parks the TASK, not the executor worker; a poisoned lock acquires instantly (the holder is gone). The comment tightening from the reverted commit is kept. * tighten the fence comments: the rationale stays, the step-order narration goes * retrigger ci: the refinement concurrency flake (passes 3/3 locally; main's code, untouched by this PR) * retrigger ci: the loaded-runner flake family (supervisor stats race + perf bands; all pass locally, all main's code) * retrigger ci: the tool_results stats count race (3-vs-4, the same flake main's own PR f67c5cf9 hit; passes 3/3 locally) * recover the poisoned lock's guard instead of deadlocking on it: the Poisoned error carries the acquired guard try_lock's Poisoned error already carries the acquired guard; calling .lock() inside the match arm waited on the guard the scrutinee still held - a self-deadlock after any reload_lock holder panicked. Recover the guard with poison.into_inner(). * retrigger ci: the contention-flake family, roll continues (the timing tests pass locally; pool still degraded) * the clear path restores the captured pair when the store holds no live credential clear_image_route installed its fresh resolution unconditionally, so an episode settling after a logout overwrote the serving pair with the models.json fallback key - the same install the reload's store-auth gate rejects. The clear now takes the resolution with its store-auth flag: a live credential installs the fresh pair, otherwise the captured session target restores (the resolve_model-Err arm's fallback). * the clear installs the fresh model with its chain-resolved auth; the capture only backs a failed rebuild The prior clear-path gating restored the captured session target whenever the store held no live credential - but the capture is from the episode's start model: a mid-episode switch was dropped, the create-config override was discarded, and a missing capture left the routed image target serving after the episode. The clear returns to the CURRENT session state instead: the freshly resolved model with its chain-resolved auth (override or configured fallback included, what a fresh session of that model serves); the captured target only backs a resolution that cannot rebuild the model. * park the build's lock retry, and read the store-auth flag after the resolution's own OAuth refresh The try_lock retry hot-spun with yield_now (the task stays immediately runnable while a reload holds the lock across auth I/O and a possible OAuth refresh) - it now sleeps 10ms per retry, parking the worker. The store-auth flag was read before the registry's own credential lookup, so a failed-then-refreshed OAuth token left the flag false and the callers discarded the fresh token; the check now runs after the resolution, on the same registry the refresh updated. * retrigger ci: two registered flake classes (refinement harness-state lock contention + wave_b10 heartbeat sigstop race; both previously flaked and cleared on retrigger, both pass 3/3 locally on this head) * merge-compat: drop the decision_child field the Decision API revert removed * retrigger ci: the owned-stop kill-timing flake (registered class, failed at 784520b0f and cleared; passes locally on this head) * retrigger ci: registration kill-timing family (4/4 local post-thaw) + the TS-side goal_backoff incomplete observation (receipt 11779755212) — both runner timing classes, branch code sound * retrigger ci: draft_disconnect ts-reference-side refusal timeout (pinned TS binary identical to main's green run; my Rust diff cannot touch the TS side) * retrigger ci: contributor-trust gate infra failure (setup-nu could not download a Nushell release; the audit then failed by design)


Author kt Committer GitHub Date Commit Parent b1d71636