# Harness laws: inference, local tiny models and reasoning depth

Shared rules (what a law secures, adversary, H1 to H6): [harness.md](harness.md).

### S6. The inference layer

Between harness and wire APIs; decides what a request may ask of a provider.

1. Quirks rulebook: family, model and host settings as data. Rule = layer, target id, directives with values. Compile rejects unknown directive, wrong value type, two rules of one layer+target giving one directive different values. Lookup: host > model > family. No rule → Unknown (no guess; wire API default applies). Explicit `model.compat` beats both. Rule order changes no answer.
2. One target per layer:
   - Family: the one family whose pattern matches the id (or id and name) and whose revision range, if any, holds. Revision (omp taxonomy): from first digit after the prefix in the bare name; a size (`8b`) is not one; none → 0. `qwen3.8-27b` → `qwen-3.8` (`>=3.8`); `qwen3-8b`, `qwq-32b` → `qwen` (`<3.8`).
   - Two families match → `QuirksError::FamilyOverlap`; none wins by id or order. Shipped-rulebook test rejects overlap on any compiled-catalog model. Out-of-catalog overlap → no family: family directives Unknown; error logged once per model id.
   - Model: model id.
   - Host: host naming the wire API (API one host speaks), else naming the provider id, else longest URL fragment in base URL (tie: smaller host id). One host; signals never combine: provider `openrouter` on a loopback base URL (local proxy) gets `openrouter`, not `loopback`. `openrouter` (provider match) adds Anthropic-format `cache_control` marks for `anthropic/` models; `openrouter-url` (custom provider, openrouter.ai base URL) gives other OpenRouter settings, no marks.
3. Forced tool call = tool + retry count. Each request ends with a soft prompt naming the tool. First request sets native forced choice only if supported and free (no prompt-cache invalidation); retries set it if supported. Support per request (`maps_forced_tool_choice`, wire API's predicate). Manual extended thinking: Anthropic and Claude on Bedrock reject forced choice → unsupported, no escalation to an unsent flag; a wire API given one there drops it and warns. No call → retry while retries remain, then `ForcedToolError::NotCalled` (names tool). Nothing forced → no change.
4. ≤ `strict_cap` strict schemas per request, highest `priority` first (default `0`; ties: earlier tool). Rest loose: client validates and repairs arguments. None dropped. Strict-required tool past cap → error; several → names the highest-priority one.
5. Repetition guard, text and thinking deltas of every stream:
   - Chunk = one complete non-blank output line; lines over 4096 bytes (`MAX_CHUNK_BYTES`) split there on a character boundary, so no-newline loops (`abab…`) are cut.
   - Window = last `k` chunks (`retry.provider.streamRepeatLimit`, default 64); periodic with period `p` if each chunk after the first `p` equals the one `p` earlier. Period 1 to `k / 4` (at least 1) → abort: cycle seen 4 times; period 1 = one chunk `k` times; default cuts ≤ 16-line blocks seen 4 times. Otherwise not cut.
   - State (compiled core, a stepper): the ids of the chunks read, newest first; per `p`, count of consecutive chunks up to the newest equal to the one `p` earlier; periodic at ≥ `k - p`. Rust holds one partial chunk per block and one id per distinct line of the attempt: memory grows with the attempt's output (one id per line in the core).
   - Cut → retryable `stream_repeating`; message gives period and last line; failure info records them (`RepeatedLines`, `repeated_lines`). Harness retries with a new request ending in an unlogged note (attempt, period, last line) (harness.md H4.7); a greedy model (`elpis-fast`, refuses temperature ≠ 0) loops on the same request.
   - Example: `qwen3.8-27b` thinking looped 1,359 lines on a 7-line cycle, no line twice in a row; a one-chunk-`k`-times check misses it; this guard cuts at line 77.
   - No text and no tool call = no answer: model error, retried with a note (harness.md H4.7). `length` stop, no answer → length note (harness.md H4.8). Noted retries (cut stream, no answer) spend a `MAX_NOTES` (8) note, not a retry, and go at once.
6. Compaction:
   - Threshold = limit (`contextWindow - reserveTokens`) − margin (`BackgroundMargin`: eukhe session 10% of window, `eukhe-core`; harness library default 32,768 tokens). Over threshold → background compaction; over limit → request blocks until done.
   - Ends mid-run → summary placed at next run boundary; wire `compaction_end` sent then with it, or as a skip if stale behind a later compaction. Queued summary survives run end (aborted run keeps queued writes). Summary write leaves queue unsettled, or a snapshot replaces events that could settle it → `compaction_end` unsuccessful. Summary queued (not placed, not stale) → no new background compaction.
   - Keeps newest `compaction.keepRecentTokens` (eukhe default 40,000; system entries count), ≤ half the threshold so history remains; small windows keep less.
   - Check uses a local input-token estimate. `compaction.providerTokenCount` (default off): ask provider `count_tokens` first (+1 request per generation); no counter or failure → warning, local estimate.
7. Provider operations beyond chat; eight capabilities: `count_tokens`, `list_models`, `embed`, `generate_image`, `tokenize`, `usage`, `web_search`, `compact`.
   - Directive named after the operation = endpoint dialect (`"list_models": "openai"`) or `false`; compile rejects a dialect lacking the operation.
   - Scope-`capabilities` hosts (`openai`, `google`, `llama-cpp`) set only capability directives and `prompt_progress`; wire lookup skips them (local `openai` provider keeps loopback settings).
   - `llama-cpp`: `"prompt_progress": true` → `openai-completions` sends `"return_progress": true`; stream emits `prompt_progress` (`total`, `cache`, `processed`, `time_ms`) per progress chunk before first token. `model.compat.supportsPromptProgress` wins.
   - Missing operation → typed `Unsupported { capability, provider }` before auth or request; never another provider or local estimate (callers request the estimate explicitly).
   - `eukhe provider caps [<provider>]`: support, no request. `eukhe provider models <provider>`: provider's model list.
8. Service tiers (`service_tiers`): entry = provider, wire API, tiers, optional family. Model honors `default` + tiers of entries matching provider, API, and family if named. Shipped:
   - `openrouter`/`openai-completions`: `flex`, `priority` (OpenRouter routes to tier endpoints, bills the serving tier).
   - `openai`/`openai-responses`, `openai-codex`/`openai-codex-responses`: `auto`, `scale`.
   - Families `gpt-priority`, `gpt-priority-explicit-cache`: + `flex`, `priority` on `openai`; + `priority` on `openai-codex` (ChatGPT backend: no flex).

   Readers (`quirks::service_tiers`): `/tier`, `/fast`, connection state (`availableServiceTiers`), image route, completions `service_tier`.
9. Thinking ladder (S10, ladder 0): `thinking_ladder` gives a family a `thinkingLevelMap`. Shipped:
   - `qwen-3.8`: `off` (wire `none`), `low`, `medium`, `xhigh`.
   - `qwen`: `off` (template toggle), `high` (omp's llama.cpp rules for Qwen templates).
   - `grok-4.3`: `off` as `none`; no `minimal`.
   - `grok-4.5`: no `off`, `minimal`.
   - `grok-4.6` (ids `grok-4.6`, `grok-4.7`): no `off`, `minimal`; + `xhigh`.

   Defaults (`thinking_defaults`) fill missing entries: `reasoning` = `off` to `high`; `plain` (no reasoning) = `off`. Stored xAI subscription moves xAI models to `openai-responses`; there, no `thinkingLevelMap` → model's rulebook ladder, else all levels null.
10. Tool calls read at stream end, any stop reason. Reply with tool calls runs them (vLLM, llama.cpp, Ollama stop with `stop` plus calls) (harness.md H4.1). `length` reply runs none (arguments may be cut); each gets `truncated`: repeat the call with complete arguments; write large content as a minimal version first, then extend it with edits (harness.md H4.1). Tool-call markup in text of a call-less reply (five families: Hermes and Qwen JSON, Qwen3-Coder XML, omp "qwen" form, Kimi section tokens, DeepSeek DSML; not in quoted code) → rebuilt if every leaked call parses and names an offered tool: markup leaves the text, stop reason becomes `toolUse`, calls run as tool calls; else retried with a leak note, ≤ `LEAK_NOTES` (2) in a row, then answers as text (harness.md H4.8.3; laws `run_calls_never_answer`, `rec_leak_rebuilt_is_calls`). Code: `eukhe-durable/src/harness/generation/leak.rs` `scan`, `generation.rs` `classify`, `generation/round.rs` `start_tool_round`.
11. Provider default model: `provider_defaults` (provider id → model id, `quirks::provider_default_model`). Read by startup model chain, fallback model template, onboarding after Prime Inference sign-in. Unlisted → no default.
12. Prices missing from catalog rows:
    - `service_tier_prices`: row = provider, wire API, tiers (response `service_tier`, else request's), multiplier, optional model id. Cost = base × multiplier of provider+API+id row, else provider+API row without id; none → base. Shipped: `openai` on `openai-responses`, `openai-completions`: `flex` 0.5; `priority`, `fast` (API's Fast-mode value) 2, `gpt-5.5` 2.5. `openai-codex`: `flex` 0.5; `priority` 2, `gpt-5.5` 2.5.
    - `cache_write_1h_prices`: row = regex on lowercased model id + input-rate multiplier; first match prices 1h cache writes; none → model's cache write rate. Shipped: `claude` 2 (Anthropic's price; Bedrock and gateways reselling Claude at Anthropic list rates charge it too).

    Read by `models::calculate_cost`, `models::apply_service_tier_pricing` (`quirks::cache_write_1h_multiplier`, `quirks::service_tier_multiplier`).
13. Every other model fact a wire API reads is rulebook data (S6.10); wire APIs never test the model id.
    - `adaptive_thinking` (`claude-adaptive`, `claude-adaptive-4-6`): `thinking.type: adaptive` + effort, Anthropic Messages and Bedrock (`models::adaptive_thinking`); explicit `compat.forceAdaptiveThinking` wins.
    - `thinking_block_binding` (`claude-adaptive`): Bedrock sends `thinking.block_binding` (not GovCloud).
    - `prompt_caching` (true: `claude-adaptive`, `claude-adaptive-4-6`, `claude-cached`; false: `claude-uncached`): Bedrock sends `cachePoint` blocks; unnamed models cache only with `AWS_BEDROCK_FORCE_CACHE=1`.
    - `explicit_cache_breakpoints` (`gpt-explicit-cache`, `gpt-priority-explicit-cache`): OpenAI Responses sends `prompt_cache_breakpoint`, `reasoning.context: "all_turns"`.
    - `google_thinking_level` (true: `gemini-3-plus`, `gemini-latest`, `gemma-4`; false: Gemini 2.5 families): Google sends `thinkingLevel`; unnamed → `thinkingBudget`.
    - `thinking_budgets` (`gemini-2.5-pro` 128, 2048, 8192, 32768; `gemini-2.5-flash` 128, 2048, 8192, 24576; `gemini-2.5-flash-lite` 512, 2048, 8192, 24576): Google per-level budget, Gemini API and Vertex. User `thinkingBudgets` wins, clamped to the rule's `minimal`..`high`. No rule → user budget, else `-1` (dynamic). Unordered rule budgets (not `minimal` ≤ `low` ≤ `medium` ≤ `high`) → error, never a clamp (rulebook compiler refuses).
    - `thinking_defaults.budgets` (1024, 2048, 8192, 16384): budget when neither user nor rule gives one, every token-budget wire API.

    Family `except` regex excludes matches (families stay disjoint). Bedrock effort = model's thinking-level map (native `xhigh`: `xhigh: "xhigh"`), else Bedrock default.

Model `bend/infer.bend`. Laws: `laws/infer.bend` (S6), `laws/infer.bend` and `laws/effort.bend` (S6.9). Proofs: `proof/infer/` and `proof/effort/`. Free inputs: rulebook, query, directive, model replies, provider support, tools, stream, context sizes.

| Law | Says |
| --- | --- |
| `quirk_unknown_rejected` | Unknown-directive rule → no compile (S6.1). |
| `quirk_clash_rejected` | Same layer/target/directive, different values → no compile (S6.1). |
| `quirk_unknown_when_none` | No rule applies → Unknown (S6.1). |
| `quirk_known_when_some` | Applying rule always answers (S6.1). |
| `quirk_highest_wins` | Compiled: highest applying layer answers (S6.1). |
| `quirk_order_free` | Compiled: swapping two rules changes no answer (S6.1). |
| `quirk_compile_order_free` | Swapping two rules keeps compile outcome (S6.1). |
| `quirk_ladder_explicit_first` | Explicit map beats rulebook ladder (S6.9). |
| `quirk_ladder_rule_second` | No explicit map → rulebook ladder (S6.9). |
| `quirk_ladder_default_last` | Neither → no entries; default ladder gives all (S6.9). |
| `quirk_ladder_offered_exact` | Reasoning model offers a level iff ladder gives a wire string, or no entry and default lists it (S6.9). |
| `quirk_ladder_plain_exact` | Non-reasoning offers exactly plain default (S6.9). |
| `quirk_ladder_shipped_is_s10` | Shipped defaults: offered = S10 ladder of chosen map; S10 laws hold (S6.9). |
| `force_bounded` | ≤ `1 + retries` attempts (S6.3). |
| `force_first_flag` | First attempt flags only if supported and free (S6.3). |
| `force_retry_flag` | Retries flag if supported (S6.3). |
| `force_reports` | Reports a call iff the model called the tool within its attempts (S6.3). |
| `strict_within_cap` | ≤ `cap` strict tools (S6.4). |
| `strict_keeps_every_tool` | Each tool strict or loose, multiplicity kept (S6.4). |
| `strict_by_priority` | Strict priorities ≥ loose ones (S6.4). |
| `guard_cuts_repeats` | One chunk `k` times in a row → cut (S6.5). |
| `guard_cuts_loops` | Periodic `k`-chunk window, period ≤ `k / 4` (at least 1) → cut (S6.5). |
| `guard_keeps_others` | No such window → not cut (S6.5). |
| `compact_starts_early` | Turn growth ≤ margin → compaction before any turn over limit (S6.6). |
| `infer_force_flag_exact`, `infer_force_after_exact` | The compiled forced call's steps: attempt 0 flags iff supported and free, a retry iff supported; a miss on attempt `i` retries iff `i + 1 ≤ retries`, else fails; a call ends it (S6.3). |
| `infer_force_steps` | Those steps driven over the replies = `force`: its laws hold for the harness (S6.3). |
| `infer_guard_start_exact`, `infer_guard_feed_exact` | The compiled guard's step: a chunk cuts, after one more chunk, with the shortest period `p ≤ max(1, k / 4)` whose newest `k - p` chunks each equal the chunk `p` before; else it is kept (S6.5). |
| `infer_guard_steps` | That step driven over a stream = `guard` (S6.5). |
| `infer_threshold_over_exact` | The compiled threshold: off or window 0 → none; `tokens + reserve > window` → blocking; else margin > 0 and `tokens + reserve + margin > window` → background (S6.6). |
| `infer_threshold_blocks_steps`, `infer_threshold_early_steps` | Over a run, the first request over each threshold = `first_at` at `limit + 1` and `limit − margin + 1`: `compact_starts_early` holds for the harness (S6.6). |
| `infer_force_ffi_*`, `infer_guard_ffi_*`, `infer_threshold_ffi_*` | The entries answer the model on every request whose numbers fit the wire; anything else gets the exact error form. |

Code:
- Rulebook: `eukhe-ai/src/quirks/mod.rs` (`compile`, `Rules::lookup`, `Rulebook::query`, `Rulebook::family`, `Rulebook::check_catalog`, `Quirks`, `ModelKey`, `ThinkingDefaults`), `quirks/revision.rs`, `quirks/tiers.rs` (`service_tiers`, `supports_service_tier`), `quirks/prices.rs` (`service_tier_multiplier`, `cache_write_1h_multiplier`), `quirks/rulebook.json` (shipped), `eukhe-ai/src/models/ladder.rs` (`thinking_level_map`, ladder functions).
- Resolvers (rulebook as detected input): `api/openai_completions/compat.rs` (`detect_compat`, `is_custom_qwen_template_model`), `api/anthropic_messages.rs` (`get_anthropic_compat`, `manual_thinking`), `api/openai_responses.rs` `get_compat`, `api/azure_openai_responses.rs`, `api/openai_codex_responses/request.rs`, `api/bedrock_converse_stream.rs` (strict default, `manual_thinking`), `api/google_shared/stream.rs` (validated function calling), `api/mistral_conversations/payload.rs`, `models/thinking.rs` `adaptive_thinking`.
- Strict, forced: `api/constrained_sampling.rs` (`strict_budget`, `StrictPlan`), `api/simple_options.rs` `maps_forced_tool_choice`; `eukhe-types/src/pi_ai/tool.rs` `Tool::priority`, `eukhe-types/src/pi_ai/settings.rs` `ToolChoice` (`Required`, `Tool(name)`; mapped in every wire API); `eukhe-durable/src/harness/generation/force.rs` (`ForceEscalation`, `Forcing`, `ForcedToolError`, `soft_prompt`), `eukhe-bend/src/rules/infer_force.rs` (`flag`, `after`, `Reply`, `Step`), `ConversationStreamOptions::force_tool`.
- Guard: `eukhe-ai/src/utils/stream_repeat.rs` (`RepetitionGuard`: deltas → chunks → ids, `MAX_CHUNK_BYTES`, `StreamRepeat`), `eukhe-bend/src/rules/infer_guard.rs` (`Guard`, a stepper), `api/lazy.rs` `watched_lazy_stream`.
- Compaction: `eukhe-durable/src/harness/generation/decide.rs` `threshold_for` (policy → whole tokens), `eukhe-bend/src/rules/infer_threshold.rs` (`Compaction::over`, `whole`), `types/settings.rs` (`BackgroundMargin`, `ThresholdTokens`), `eukhe-durable/src/harness/generation.rs` (`threshold_compaction`, `provider_tokens`: provider count, logged fallback); `eukhe-core/src/settings/cvar/table.rs` `COMPACTION_PROVIDER_TOKEN_COUNT`, `eukhe-core/src/durable/settings.rs` (`WindowPercent(10.0)`, repeat limit); `eukhe-daemon/src/worker/durable_host/translator.rs` (`summary_write`, `end_lost_placements`, held ends at a snapshot).
- Capabilities: `eukhe-ai/src/capabilities/` (`Capability`, `Dialect::endpoint`, `capabilities_of`, one module per dialect), `models/capabilities.rs` (`Models` operations), `Directive::Endpoint`, `eukhe-cli/src/provider_command.rs`.

Tie (bridge tests, transcripts byte for byte):
- `harness::generation::infer_bridge::the_rust_quirks_rulebook_equals_the_bend_model` vs `bridge/bridge_quirks.bend`: compile, lookup on generated rulebooks, each also with two rules swapped; directives = first of each value type in `Directive::ALL`; queries may name no family or host.
- Forced call, guard, compaction threshold run as their compiled models (`bend/ffi_infer_force.bend` 73, `bend/ffi_infer_guard.bend` 74, a stepper, `bend/ffi_infer_threshold.bend` 75; laws `infer_*_steps` tie the per-step calls to `force`, `guard`, `first_at`); no Rust copy, no bridge. Rust keeps byte work: counting requests; cutting deltas into chunks (lines, blank lines skipped, `MAX_CHUNK_BYTES`) and one id per distinct chunk; counting tokens and rounding reserve and margin (`BackgroundMargin::tokens`) up to whole tokens, exact for a whole reserve (a fractional reserve and margin together may start background compaction one token early; NaN reads 0; past 10^14 saturates).
- Strict plan and thinking ladder (section 6) run as their compiled models (`bend/ffi_infer.bend`, `bend/ffi_effort.bend`); no Rust copy.

Tests:
- `quirks::tests`: shipped rulebook compiles, no catalog overlap; `families_that_overlap_on_a_catalog_model_are_rejected`, `a_model_two_shipped_families_match_names_no_family`; errors, precedence, order, host/family queries, former hand-written checks; `the_shipped_qwen_families_split_at_the_revision_3_8`.
- `quirks::revision::tests`; `quirks::tiers::tests` (`the_table_keeps_every_answer_of_the_former_predicate`: every compiled-catalog model and tier vs the former hardcoded predicate, kept only in the test); `models::ladder::tests`.
- `openai_completions::tests::tool_choice` (`a_custom_qwen_3_8_model_sends_the_rulebook_ladder`), `openai_completions::tests::service_tier`, `openai_completions::tests::host_quirks` (`a_custom_provider_at_openrouter_ai_sends_no_cache_marks`, `the_openrouter_provider_at_a_loopback_url_keeps_the_openrouter_settings`).
- `constrained_sampling::tests` (`the_strict_budget_keeps_the_highest_priorities_and_the_earlier_of_equals`, `a_strict_plan_sends_tools_past_the_cap_loose_and_drops_none`), `generation::force::tests` (`manual_extended_thinking_has_no_native_forced_choice`), `anthropic_forced_tool_choice` `the_forced_choice_predicate_matches_the_wire`.
- `utils::stream_repeat::tests` (chunking and the stream): `a_line_is_a_chunk_only_once_its_newline_arrives`, `blank_and_whitespace_lines_are_not_chunks`, `interleaved_blocks_keep_their_own_partial_lines_and_share_one_run`, `a_new_attempt_restarts_the_guard`, `a_loop_with_no_newline_is_cut_in_bounded_chunks`, `a_long_line_is_chunked_on_character_boundaries`, `the_observed_thinking_loop_is_cut_at_its_77th_line`, `the_observed_thinking_loop_fails_as_repeating_by_default`; `eukhe-bend` `rules::infer_{force,guard,threshold}::tests` (the compiled core is reached).
- `generation::tests::threshold` (the policy in whole tokens, keep window); translator `a_summary_write_that_leaves_the_inbox_unsettled_ends_its_compaction`, `a_snapshot_ends_the_compactions_whose_end_it_may_have_replaced`; wire tests `cargo test -p eukhe-ai --test providers`.

Not modelled (tests tie them): model → targets map (S6.2); soft prompt, wire forms of forced choice; delta → chunk split.

#### S6 tool-call argument repair

Before validation (as omp), the client repairs arguments that are not strict JSON or spell a declared type differently.

1. Streaming preview closes incomplete text; final parse does not: `JSON.parse`; else escape raw control characters and invalid string escapes, retry; else relaxed dialect: single quotes, unquoted keys, comments, `True`/`False`/`None`, stray/trailing commas, `0x`/`0b` numbers, unquoted values (`src/main.rs`). Relaxed rejects truncation, trailing characters, non-finite numbers, unquoted values that can hide the next field.
2. Unparsable or non-object → `{ __parseError, __rawJson }` (error, first 512 characters); no text key survives, so half-formed calls never run. Result: `invalid_arguments` error with parse error and raw text.
3. JSON-string key (`"\"path\""`) → plain key, unless the plain key is present.
4. Alternate spellings → declared type: JSON string for object/array; single string for string array; number/boolean for string. String booleans/numbers → value, as before.
5. Each repair → `argument_repaired` warning on the result. Unknown key in a closed schema stays an error.
6. Unavailable tool → `tool_unavailable` naming ≤ 3 offered tools: small edit distance, nearest first; then names ending with the called name's last `_` or `__` segment.

Code: `eukhe-ai/src/utils/json_parse.rs` (`parse_json_with_repair`, `parse_tool_call_arguments`), `json_parse/relaxed.rs`, per-wire-API final parse (`anthropic_messages/events.rs`, `bedrock_converse_stream/events.rs`, `mistral_conversations/consume.rs`, `openai_completions/streaming.rs`, `openai_responses_shared/stream.rs`); `eukhe-durable/src/harness/tool/arguments.rs` (`repair_arguments`, `ArgumentError::NotJson`), `tool/result.rs` (`tool_unavailable_message`).
Tests: `utils::json_parse::tests`, `json_parse::relaxed::tests`, `tool::arguments::tests`, `tool::result::tests`.

#### S6 the built-in model catalog

Feeds `/model`, registry, daemon `get_model_catalog`; user `models.json` on top. Two local sources; no fetch:

1. Bundled `models.bundled.json` beside the executable; release builds it from `PrimeIntellect-ai/prime-agent-catalog` at the commit in `scripts/release/catalog-pin.json`, checking each sha256. Strict parse; entry transports must be compiled in (transport pinning).
2. Compiled: `eukhe-ai` pi-ai tables (`providers/data/*.json`) + Prime Inference entries.

Catalog = union; model = provider + id.

1. Every model of either source, exactly once; neither hides the other's.
2. In both: bundled gives api, `baseUrl`, data (`name`, `reasoning`, `input`, `cost`, `contextWindow`, `maxTokens`), its optional fields; compiled fills missing optional fields (`thinkingLevelMap`, `compat`, `featured`, `headers`) only on the same api.
3. In one: unchanged.
4. Order: bundled, then compiled-only in compiled order. Fields come only from each source's entry, so read order changes nothing.
5. Bundle missing or damaged → compiled catalog.

Example: Claude Haiku 5.5 (compiled, not pinned) is in `/model`. Move the pin to the catalog repo's newest commit:

```sh
make catalog-pin   # validates, records the commit and both sha256
git diff scripts/release/catalog-pin.json   # review, then commit
```

No code touches `~/.eukhe/models/provider-model-catalog.v1.json` (older builds' fetched catalog).

Laws: `laws/catalog_union.bend` (C1; model `bend/catalog_union.bend`; proofs `proof/catalog_union/`).
Code: `eukhe-models/src/effective.rs` (`effective_catalog`), `chain.rs` (`ModelCatalog::models`), `bundled.rs` (`load_bundled_models`), `pinning.rs`.
Union runs as the compiled model (`fast`, pinned to `union` by law `union_fast`; entry `bend/ffi_catalog_union.bend`, request laws `catalog_union_ffi_*`): `effective_catalog` projects both sources (key and api interned, each field = place of its source entry), calls it, assembles the models back.
Tests: `chain::tests::a_bundle_without_a_compiled_model_keeps_it`; `eukhe-daemon` `tests/model_catalog_e2e.rs` (`a_bundle_without_a_compiled_model_still_lists_it`: pinned snapshot as bundle, Anthropic login → Claude Haiku 5.5 listed).

#### S6 models.json and the saved default model

One `models.json` loader for every view: registry (`eukhe model list`, `/model`, daemon `get_model_catalog`, onboarding, auxiliary model, memory compactor, RLM, usage) and session model collection.

1. Each `providers.<id>` loads alone; bad entry (wrong type, missing `baseUrl`/`apiKey`, invalid model) drops only that provider, diagnostic names `providers.<id>`.
2. Unreadable/unparsable file → one diagnostic, no custom models.
3. Diagnostics: `eukhe model list` (stderr), `get_model_catalog` `modelsJsonErrors`, each session's settings-error row at open.

Unpinned sessions start on the saved default (`defaultProvider`/`defaultModel`), never silently replaced:

1. Not in catalog or outside a set `enabledModels` → next startup default; one display-only `eukhe.default_model` row names saved default, reason, replacement.
2. Provider lacks credentials → typed `DefaultUnusable` + login hint; no substitute provider; session opens modelless; row names saved default and reason.
3. No replacement → session opens modelless with the row.
4. Memory compactor (no `memory.model`) fails with saved default and reason.

Laws: `default_honoured_or_reported`, `default_quiet_when_honoured`, `default_quiet_when_unset`, `default_unkeyed_never_moves` (model `bend/default_model.bend`; proofs `proof/default_model/`).
Code: `eukhe-core/src/models/custom.rs` (`ModelsConfig::load`), `models/registry.rs` (`compose`), `durable/models/compose.rs`, `models/resolver.rs` (`saved_default_miss`, `resolve_default`), `durable/models.rs` (`resolve_session_model`, `resolve_open_model`). `resolve_default` runs `resolve` of `bend/default_model.bend` as the compiled Bend core (entry `bend/ffi_default_model.bend`).

### S9. Local tiny models

On-machine models per role (as omp): titles `tiny`, typed judgments `judge`, `translate`, text to speech `speech`, speech to text `dictation`.

Catalog `eukhe-tiny/src/catalog.rs` pins each file: Hugging Face repo, commit, size, sha256. Models: `lfm2.5-230m`, `lfm2.5-350m` (text, ONNX q4); `whisper-base`, `whisper-small` (speech to text); `kokoro` (Kokoro-82M, text to speech, misaki lexicon for phonemes).

Download: user only, `eukhe tiny-models download <id>` / `eukhe tiny-models download all`; `eukhe tiny-models list` shows installed, missing, incomplete. Nothing else downloads.

1. Files stage in `<agent dir>/models/tiny/.staging-<id>-<pid>`; sha256 computed while writing.
2. Wrong size or sha256 → stop, staging removed, earlier install untouched. Staging not removable → fails with its own error plus a warning naming the directory.
3. All correct → one rename to `<agent dir>/models/tiny/<id>`: complete or absent.

Worker: one process per model+backend, shared by all eukhe processes. Socket, bind lock, log in `<agent dir>/run/tiny/` (`<id>-onnx.sock`, `.bind.lock`, `.log`), made 0700 by worker and client even if present. Hidden `eukhe tiny-worker <id> --backend onnx --agent-dir <dir>`; agent dir from the starter's command line, never environment (daemon with `--agent-dir` and its worker agree).

1. Client connects, sends `ping`; worker answers launch tag `<build id>|onnx|<device>|<dtype>`.
2. Same tag → use it (joined).
3. Other tag → `shutdown`: worker removes socket, answers `bye`, refuses new connections/requests, finishes other processes' in-flight requests in full, exits; client starts a new one (replaced). New build or device/dtype change never hits a stale worker.
4. No live socket → client starts one (started), detached, own process group; it holds the bind lock only while checking and binding; a live worker answers → it exits, client uses the live one.
5. Exits after 15 idle minutes (`EUKHE_TINY_WORKER_IDLE_MS` for tests), socket removed first.
6. JSON-line requests: `load`; `chat` (messages, prefill, stop strings, max tokens); `transcribe` (audio, language); `speak` (text, voice, speed). One answer each, in order; client builds prompts, parses output. Worker dies mid-request → typed `Crashed` + log path. Lines ≤ 256 MiB each side: longer request → `bad_request`; longer answer → out of protocol; bytes discarded.

Engine `eukhe-tiny-onnx`: ONNX Runtime via `ort`, loaded at run time from `providers.tinyRuntimeLibrary`, else `ORT_DYLIB_PATH`, else `libonnxruntime.so` (nixpkgs `onnxruntime` ≥ 1.24). Nix wrapper and dev shell set `ORT_DYLIB_PATH` to nixpkgs (`--set-default`: user value wins). Build downloads nothing, links no runtime. CPU only. Own tokenizer (`tokenizer.rs`): byte-level BPE from `tokenizer.json` with added tokens, pre-tokenizer regex, ranked merges; other settings → load fails.

Roles (each names a chain):

1. Chain: `modelRoles.<role>` (if set), then `retry.fallbackChains.<role>`; explicit `[]` = primary only; unset → role's catalog models, then session provider model (text roles). `translate`: `lfm2.5-350m` before `lfm2.5-230m` (see Translation).
2. Available: local if installed; provider model if the session's usable model.
3. First available wins; none → `Unavailable`; never outside the chain.

Uses:
- Titles: first user message of a top-level session starts one in background (role `tiny`; `titles.generate: false` disables); turn never waits; failure → none; user names kept. `modelRoles.tiny` unset, no local model → one extra session-provider request. Scripted (faux-script test seam) sessions: no title.
- Judge: `local` provider exposes each text catalog model as classifier `local/<id>` on `classify`. Code-mode `models.classify()` without model (`null`/`undefined`) uses role `judge`: first chain model that is an available classifier (installed local, or provider classifier with configured auth); none → `Unavailable` naming role and chain.
- CLI: `eukhe translate <lang> <text>`, `eukhe say <text> [--out f.wav]`, `eukhe transcribe <file.wav>` → `translate`, `speech`, `dictation`.
- First paint never waits for a model load.

Translation: user turn `Translate from English to <Language>.\nEnglish: <text>`, assistant prefill `<Language>:`. One-line text stops at first line end. Empty reply, or same letters and digits as input (case-insensitive) → `TranslationError::Empty`/`TranslationError::Unchanged`, prints nothing (failing tiny models echo). Letterless input exempt.

Eval: 10 short English sentences (`hello`, `good morning, my friend`, ... `See you tomorrow.`) → German, French, Spanish, Portuguese; greedy; stop at first line end. Echoes = replies equal to input (of 40); chrF = mean (0 to 100) vs one reference:

| Variant | 230m echoes | 230m chrF | 350m echoes | 350m chrF |
| --- | --- | --- | --- | --- |
| system "translator" + user `Translate this text to <Language>:` (old) | 9 | 34 | 0 | 64 |
| instruction in the system prompt only | 24 | 12 | not run | not run |
| language code, not name | 27 | 13 | not run | not run |
| old + one example pair | 2 | 36 | 0 | 41 |
| old + prefill `<Language>: ` | 2 | 52 | 0 | 59 |
| old + one example pair + prefill `<Language>: ` | 1 | 20 | not run | not run |
| user `Translate this text to <Language>:` + prefill `<Language>:`, no system prompt | 16 | 26 | 2 | 53 |
| user `Translate from English to <Language>.\nEnglish: <text>` + prefill `<Language>:` (new) | 1 | 53 | 0 | 65 |
| new without "from English" (`Text:`) | 4 | 47 | 0 | 62 |
| new + the "translator" system prompt | 0 | 49 | 0 | 59 |
| new + one example pair | 0 | 59 | 0 | 59 |

- New recipe: best on `lfm2.5-350m`, near best on `lfm2.5-230m`.
- No example pair: needs one per target language.
- `lfm2.5-230m` is poor (`hello` → German `willkommen`), so built-in `translate` prefers installed `lfm2.5-350m`: `Hallo`, `Vielen Dank.`, `Où est la gare ?`, errors (`Gute Morgensinn, mein Freund`).
- Source English only; eval tested English only.

Dictation (into the TUI draft):

1. `stt.enabled` (default `false`) enables keys; read from the session at each take-starting press; `false` → key shows how to enable.
2. `app.stt.pushToTalk`, `app.stt.toggle`: no default key; bind in `binds.cfg` (S4.5), e.g. `bind f8 "action app.stt.toggle"`. Push-to-talk records while held (terminal resends on auto-repeat; gap > 300 ms, 800 ms after first press, = release). Toggle records until next press.
3. Recorder: first on `PATH` of `pw-record`, `parec`, `arecord` (macOS: `sox`, `ffmpeg`); raw 16 kHz mono 16-bit stdout → temp WAV. Failure → take fails with its stderr; no next program.
4. `dictation` role reads the file like `eukhe transcribe` (local only); text inserted at cursor; empty → "Dictation heard no speech".
5. Status `[REC] recording`, then `[STT] transcribing`. Esc mid-take stops recorder, removes file, shows "Dictation cancelled". Failure → error row.

No `cpal`: `alsa-sys` links `libasound` via `pkg-config` at build time; CI (`ubuntu-24.04`) lacks `libasound2-dev`; Nix would need `alsa-lib`, `pkg-config`. Recorder programs need nothing at build. Client input: no Bend model.
Code: `eukhe-tui/src/dictation.rs` (`Recorder`, `Transcriber`, `SubprocessRecorder`, `run_take`, `wav_bytes`), `session_ui/dictation.rs` (`handle_dictation_key`, `apply_dictation_update`), `status_line.rs`; `eukhe-cli/src/tiny_command.rs` (`RoleTranscriber`, `transcribe_file`).
Tests: `eukhe-tui` `dictation::tests` (fake recorder/transcriber: toggle, hold release, recorder failure, abort stops recorder and removes file; `/bin/sh` subprocess recorders: first found, stderr, no audio, no program), `keybindings::tests::dictation_actions_bind_only_through_binds_cfg`, `status_line::tests::the_dictation_phase_leads_the_line`, `tests/dictation_headless.rs` (TUI on mock daemon: toggle, push-to-talk, Esc, failure, `stt.enabled` off).

Model `bend/tiny.bend`. Laws: `laws/tiny.bend` (S9). Proofs: `proof/tiny/`. Free inputs: chain, available set, needs of any processes and tags, worker exits, any order.

| Law | Says |
| --- | --- |
| `tiny_pick_in_chain` | Only available chain models (roles 3). |
| `tiny_pick_first` | First available chain model (roles 3). |
| `tiny_pick_unavailable` | None available → Unavailable (roles 3). |
| `tiny_every_need_served` | Each need: exactly one of started, joined, replaced (worker 1-4). |
| `tiny_worker_current` | Worker has the tag of the last need since the last exit (worker 3). |
| `tiny_same_tag_joins` | Need with the running tag joins (worker 2). |

Code: `eukhe-tiny/src/roles.rs` (`chain`, `pick`, `Candidate`, `FallbackChain`), `election.rs` (`ensure`, `request`, `probe`, `Launcher`, `Got`), `worker.rs` (`launch_tag`, `run_worker`, `bind`, `serve`), `client.rs` (`TinyClient`, `TinyError`), `catalog.rs` (`CATALOG`, `installed`, `install_state`), `download.rs` `download`, `tasks.rs` (prompts, output parsing); `eukhe-tiny-onnx/src/text.rs`, `whisper.rs`, `kokoro.rs`, `runtime.rs`, `tokenizer.rs`; `eukhe-core/src/tiny/` (settings, `resolve`, `title.rs`, `judge.rs` `judge_classifier`), `eukhe-core/src/durable/codemode.rs`; `eukhe-durable/src/codemode/models.rs` `ClassifierDefault`; `eukhe-daemon/src/worker/title.rs`; `eukhe-cli/src/tiny_command.rs`, `tiny_models_command.rs`.

Tie: `pick` runs as the compiled model (entry `bend/ffi_tiny.bend`, request laws `tiny_ffi_*`; a candidate crosses as the index of its first equal one). Each choice of one process's election (join, replace or start from its ping; a shutdown's answer; a launch's outcome; the check after a start; a started worker's seat) runs as the compiled model too (entry `bend/ffi_tiny_elect.bend`, laws `tiny_*_spec`, `tiny_elect_*`); Rust connects, pings, shuts down, launches and reports. `eukhe-tiny` `bridge::the_rust_election_equals_the_bend_model` (real election: `ensure`, real `serve`, in process, fake engine, generated needs and exits) vs `bridge/bridge_tiny.bend` stays for the interleaving only: several processes and workers on one socket over time.

Tests:
- `client::tests`: real workers, temp run dir, started on the client's command line without agent dir variable, other `HOME`: started, joined, replaced, idle exit, crash, not installed, client's agent dir served.
- `worker::tests`: shutdown lets another client's in-flight request finish; existing run dir made 0700.
- `protocol::tests`: over-limit line skipped, next read.
- `download::tests` (mock HTTP): bad sha256, wrong size, broken transfer install nothing; partial install replaced only on success; unremovable staging keeps the download's own error.
- `eukhe-core` `tiny::tests` (role resolution; `translate` prefers `lfm2.5-350m`), `tiny::judge::tests` (`judge` model for model-less classify).
- `eukhe-tiny` `tasks::tests`: translate request; echo → `TranslationError::Unchanged`.
- `eukhe-durable` `codemode::tests::calls`: `models.classify(null, …)` asks the default classifier.
- `eukhe-daemon` `tests/session_title_e2e.rs`: supervisor on temp socket and agent dir, mock provider; first prompt titles; `titles.generate: false` titles nothing.
- `eukhe-tiny-onnx` `tokenizer::tests`: ids/text vs ids Hugging Face `tokenizers` once gave for the same files and texts; random text round-trips.
- Real-model (need models + runtime library, else print skip reason): `eukhe-tiny-onnx` `text::tests` (title from `lfm2.5-230m` and, if present, `lfm2.5-350m`, whose graph has a `num_logits_to_keep` input), `whisper::round_trip_tests` (Kokoro speaks, Whisper reads back), `tokenizer::tests` on model files; `eukhe-core` `tiny::smoke_tests` (title, `Models::classify` judgment on `local` via a real worker).

Not modelled: two processes starting a worker at once (bind lock orders; second uses the first's worker).

### S10. Reasoning depth

Thinking level = how much the model thinks before answering. Ladder order: `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`.

Ladder. All ladders and wire values are rulebook data (S6.9); code holds none. Offered levels keep ladder order:

0. Chosen ladder: model's explicit `thinkingLevelMap` (`models.json`, catalog); else, reasoning model of a custom provider: family's rulebook ladder; else none. Built-in providers keep catalog ladders. Custom `qwen3.8-27b`, no map → `off`, `low`, `medium`, `xhigh`.
1. No reasoning → plain default (shipped: `off` only); map ignored.
2. Reasoning, per entry: `null` removes; wire string keeps, sent as is; missing → default ladder (shipped: `off` to `high`, so `xhigh`/`max` need entries), sent in the wire API's own form.
3. So reasoning models may offer `off` only (rest `null`) or nothing (all `null`).
4. Daemon serves offered levels (`availableThinkingLevels`; each catalog model's levels to the picker); clients compute no ladder.

Clamp → offered level:

1. Offered → unchanged.
2. Else first offered above; none → first offered below.
3. None offered → `off`.

Cycle (`cycle_thinking_level`, forward/backward), one offered step:

1. < 2 offered (no reasoning, or `off` only) → no change; daemon answers `null`.
2. Else from clamp of current. Forward: next, last → first. Backward: previous, first → last.
3. Forward visits each offered level once per lap in ladder order; forward and backward cancel.

Session (main conversation stores its level; may be unset):

1. Open settles: stored, else default (`defaultThinkingLevel`, else `medium`); clamped if the session model is in the catalog, else kept.
2. Model switch clamps current level to the new model; set clamps the request (kept without catalog model); cycle sets the step's result.
3. Sent: stored level, else `off`; each provider clamps before sending.
4. Shown (daemon summary, connection state `thinkingLevel`): stored clamped to the model (unclamped without catalog model) = sent. Unset shows `default`, never `off`.

Model `bend/effort.bend`. Laws: `laws/effort.bend` (S10). Proofs: `proof/effort/`. Free inputs: model with or without reasoning, any map (entries missing, `null`, wire string); level; stored level and default; switches, sets, cycle steps in any order.

| Law | Says |
| --- | --- |
| `effort_offered_plain` | No reasoning → `off` only, any map (ladder 1). |
| `effort_offered_null` | `null` entry never offered (ladder 2). |
| `effort_offered_ext` | `xhigh`/`max` only with a wire entry (ladder 2). |
| `effort_offered_kept` | Reasoning model offers each kept level: wire entry, or missing below `xhigh` (ladder 2). |
| `effort_offered_asc` | Strictly ascending, no repeats (ladder). |
| `effort_clamp_offered` | Something offered → every clamp offered; switches land on one (clamp 2, session 2). |
| `effort_clamp_keeps` | Offered level unchanged (clamp 1). |
| `effort_clamp_idem` | Clamp idempotent, even with none offered (clamp 3). |
| `effort_cycle_still` | < 2 offered → no change (cycle 1). |
| `effort_cycle_next` | Forward → next offered (cycle 2). |
| `effort_cycle_wrap` | Forward from last → first (cycle 2). |
| `effort_cycle_prev` | Backward → previous offered (cycle 2). |
| `effort_cycle_wrap_back` | Backward from first → last (cycle 2). |
| `effort_cycle_inverse` | From offered, backward cancels forward (cycle 3). |
| `effort_cycle_inverse_back` | From offered, forward cancels backward (cycle 3). |
| `effort_unset_default` | Unset shows `default`, never `off` (session 4). |
| `effort_settle_offered` | Something offered → open settles on offered (session 1). |
| `effort_stored_is_sent` | After open and any actions, stored = sent (session 1-3). |
| `effort_shown_is_sent` | After open and any actions, shown = sent (session 4). |

Code: `eukhe-ai/src/models/ladder.rs` (`thinking_level_map`, `get_supported_thinking_levels`, `registry_thinking_levels`, `registry_clamp_thinking_level`, `clamp_thinking_level`, `cycle_thinking_level`), provider clamps in `eukhe-ai/src/api/`; `eukhe-core/src/durable/open.rs` `settle_main_agent`; `eukhe-daemon/src/model_switch.rs` (switch, set), `setting_switches.rs` `handle_cycle_thinking_level`, `worker/summary.rs` `thinking_level`, `rpc/model_commands.rs`.

Tie: offer, clamp and cycle run as the compiled model (`bend/ffi_effort.bend`, called from `eukhe-ai` `models/ladder.rs`); no Rust copy of the rule.
Tests: `eukhe-core` `durable::open::tests` (`open_settles_an_unset_thinking_level_on_the_clamped_default`); `eukhe-daemon` `setting_switches::tests` (`cycle_thinking_level` without reasoning → `null`; steps wrap both ways; unset shows `default`).
Not modelled: settings order for the default level; level → per-provider wire value.