# Harness laws: inference, local tiny models and reasoning depth The shared rules (what a law secures, the adversary, H1 to H6) are in [harness.md](harness.md). ### S6. The inference layer The inference layer is between the harness and the wire APIs. It decides what a request may ask of a provider. 1. The quirks rulebook gives the settings of a model family, a model, and a host as data. A rule has a layer, a target id of that layer, and directives with values. The compiler rejects an unknown directive, a value of the wrong type, and two rules of one layer and target that give one directive different values. A lookup takes the rule of the highest layer that applies: host over model over family. When no rule applies, the answer is Unknown. The rulebook does not guess. The wire API applies its own default, and the explicit `model.compat` wins over both. The order of the rules changes no answer. 2. A model names one target per layer. The family is the one family whose pattern matches the model id (or the id and name) and whose revision range, when it has one, holds for the revision that the id names (omp's taxonomy: the revision starts at the first digit after the prefix in the bare name; a size such as `8b` is not a revision; an id that names no revision has the revision 0). Thus `qwen3.8-27b` is in the family `qwen-3.8` (`>=3.8`) and `qwen3-8b`, `qwq-32b` are in `qwen` (`<3.8`). Two families that match one model are an error (`QuirksError::FamilyOverlap`). No family wins by its id or its order. The test of the shipped rulebook rejects two families that match one model of the compiled catalog. A model out of the catalog that two families match names no family: each family directive is then Unknown, and the log shows the error one time for each model id. The model is the model id. The host is the host that names the wire API (an API that only one host speaks), else the host that names the provider id, else the host with the longest URL fragment in the base URL (ties: the smaller host id). A model names one host only: the signals do not combine. For example, the provider `openrouter` with a loopback base URL (a local proxy) gets the settings of the `openrouter` host, not those of the `loopback` host. Before the rulebook, the code combined the provider and URL checks. The host `openrouter` (provider match) gives Anthropic-format `cache_control` marks to `anthropic/` models. The host `openrouter-url` (a custom provider with an openrouter.ai base URL) gives the other OpenRouter settings, but no marks. 3. A forced tool call names a tool and a count of retries. Every request of the call ends with a soft prompt that names the tool. The first request sets the native forced choice only when the provider supports it and it is free (it costs no prompt-cache invalidation). Each retry sets it when the provider supports it. The support is calculated for each request (`maps_forced_tool_choice`), with the predicate that the wire API uses. Under manual extended thinking, Anthropic and Claude on Bedrock reject a forced choice, so the request has no support: no attempt escalates to a flag that is not sent. A wire API that gets a forced choice under manual extended thinking removes it and logs a warning. A reply without the call starts a retry while retries remain. After the retries, the generation fails with `ForcedToolError::NotCalled`, which names the tool. When no tool is forced, nothing changes. 4. A provider takes at most `strict_cap` strict tool schemas per request. The tools of highest `priority` get them (default `0`: all equal; of equal tools, the earlier one first). The other tools are loose: the client validates and repairs their arguments. No tool is dropped. A tool that requires a strict schema and is past the cap is an error. 5. The repetition guard reads the text and thinking deltas of every stream. A chunk is one complete, non-blank output line. A line longer than 4096 bytes (`MAX_CHUNK_BYTES`) ends a chunk at that length, on a character boundary; the remaining text starts the next chunk. Thus a loop with no newline (`abab…`) is also cut. The guard keeps a window of the last `k` chunks (setting `retry.provider.streamRepeatLimit`, default 64). The window is periodic with period `p` when each chunk in it, after the first `p`, is equal to the chunk `p` before it. When the window is periodic with a period of 1 to `k / 4` chunks (at least 1), the guard aborts the request. Thus a cycle must come at least four times. Period 1 is one chunk `k` times in a row. With the default, a block of up to 16 lines that comes four times is cut. The stream then ends with the retryable `stream_repeating` failure. The failure message gives the period and the last line, and the failure info records them (`RepeatedLines`, read by `repeated_lines`). The harness retries the cut stream, but not with the same request: a model decodes the same request in the same way, and a model that decodes greedily (for example `elpis-fast`, which refuses a temperature other than 0) loops on it again. The retry's request ends with a harness note that gives the attempt, the period and the last line (docs/harness.md H4.7). The run does not log the note. A stream with no periodic window of `k` chunks is not cut. For each period `p`, the guard keeps a counter: the number of chunks in a row, up to the newest, that are equal to the chunk `p` before them. The window is periodic with period `p` when this counter is `k - p` or more. Thus the guard keeps at most `k / 4` chunks and `k / 4` counters, and one partial chunk for each block, for all lengths of the stream. Example: in a real session, `qwen3.8-27b` looped in its thinking for 1,359 lines on a cycle of 7 lines, and no line came two times in a row. The rule before this one (one chunk `k` times in a row) did not cut it. This guard cuts it at line 77. A reply that ends with no text and no tool call (for example only thinking, when the guard did not cut it) is no answer either: the harness fails it as a model error and retries it with a note that says that the reply gave no text (docs/harness.md H4.7). A `length` stop with no answer gets a length note (docs/harness.md H4.8). A noted retry (a cut stream, no answer) spends a note of the generation's own budget (`MAX_NOTES`, 8), not a retry of the retry policy, and goes out at once, with no backoff. 6. Compaction starts on the background path when the context is more than the threshold: the limit (`contextWindow - reserveTokens`) less a margin (`BackgroundMargin`). The eukhe session sets the margin to 10% of the context window (`eukhe-core`); the default of the harness library is 32,768 tokens. A context more than the limit blocks the request until compaction ends. A background compaction that ends during a run places its summary at the run's next boundary; its wire `compaction_end` goes out then, with that summary, or as a skip when the summary went stale behind a later compaction. The queued summary stays after a run end (an aborted run keeps its queued writes). When the summary write leaves the queue with no settlement, or a snapshot replaces the events that could settle it, the `compaction_end` goes out as unsuccessful. While a summary is in the queue (not placed, not stale), no background compaction starts. The compaction keeps the most recent `compaction.keepRecentTokens` of the context (eukhe default 40,000; system entries count). The kept part is at most half of the threshold, so a compaction always has history to summarize. With a small context window, the compaction thus keeps less than the setting. The threshold check uses a local estimate of the request's input tokens. When `compaction.providerTokenCount` is on (default off), the check asks the provider's token counter (`count_tokens`) first. This adds one request to the provider per generation. When the provider has no counter or the count fails, the check logs a warning and uses the local estimate. 7. A provider can have more operations than the chat stream. The capability surface names eight: `count_tokens`, `list_models`, `embed`, `generate_image`, `tokenize`, `usage`, `web_search` and `compact`. The rulebook gives the support as data: the directive with the name of the operation has the dialect of the endpoint (for example `"list_models": "openai"`), or `false`. The compiler rejects a dialect that has no implementation of that operation. A host of scope `capabilities` (`openai`, `google`, `llama-cpp`) serves only these lookups. It sets only capability directives and `prompt_progress`, and the wire lookup skips it, so it changes no other request setting: a local `openai` provider keeps the loopback settings. The `llama-cpp` host sets `"prompt_progress": true`: an `openai-completions` request then sends `"return_progress": true`, and the stream gives a `prompt_progress` event (`total`, `cache`, `processed`, `time_ms`) for each progress chunk, before the first token. `model.compat.supportsPromptProgress` wins over the rulebook. An operation that the provider does not have fails with the typed error `Unsupported { capability, provider }` before auth or a request. It never uses another provider or the local estimate. A caller that wants the estimate asks for it. `eukhe provider caps []` shows the support and sends no request. `eukhe provider models ` asks the provider for its model list. 8. The service tiers that a provider honors are rulebook data (`service_tiers`): each entry names a provider, a wire API, the tiers, and optionally a family. A model honors `default`, and each tier of each entry whose provider and API are the model's and whose family, when the entry names one, is the model's family. The shipped table: `openrouter` over `openai-completions` honors `flex` and `priority` (OpenRouter routes to tier endpoints and bills the tier that served the request); `openai` over `openai-responses` and `openai-codex` over `openai-codex-responses` honor `auto` and `scale`; the families `gpt-priority` and `gpt-priority-explicit-cache` add `flex` and `priority` on `openai` and `priority` on `openai-codex` (the ChatGPT backend has no flex tier). The `/tier` and `/fast` commands, the connection state (`availableServiceTiers`), the image route and the completions request (`service_tier`) all read this table (`quirks::service_tiers`). 9. The thinking ladder of a model is rulebook data (S10, the ladder 0). The directive `thinking_ladder` gives a family its ladder as a `thinkingLevelMap`. The shipped ladders: `qwen-3.8` offers `off` (wire `none`), `low`, `medium` and `xhigh`; `qwen` offers `off` (the template toggle) and `high` (omp's llama.cpp rules for the Qwen templates); `grok-4.3` sends `off` as `none` and removes `minimal`; `grok-4.5` removes `off` and `minimal`; `grok-4.6` (the ids `grok-4.6` and `grok-4.7`) removes `off` and `minimal` and adds `xhigh`. The default ladders (`thinking_defaults`) give the levels where a ladder has no entry: `reasoning` lists `off` to `high`, `plain` (a model without reasoning) lists `off`. A stored xAI subscription moves the xAI models to `openai-responses`; a model there without a `thinkingLevelMap` gets the rulebook's ladder of its model, else a map that sets every level to null. 10. The harness reads the tool calls at the end of a stream, whatever the stop reason. A reply with a tool call runs its calls: vLLM, llama.cpp and Ollama send the stop reason `stop` with tool calls (docs/harness.md H4.1). A reply that hit the output limit (stop reason `length`) runs none of its calls: their arguments can be cut. Each call gets a `truncated` result that tells the model to make the call again with complete arguments, and to write large content as a minimal version first, then extend it with edits. Some models write a tool call as markup in the text, not in the tool-call channel. In a reply with no tool call, the harness reads the markup of five families (Hermes and Qwen JSON, Qwen3-Coder XML, the omp "qwen" form, Kimi section tokens, DeepSeek DSML), but not in quoted code. When every leaked call parses and names a tool that the request offered, the harness rebuilds the calls: the markup leaves the text, the stop reason becomes `toolUse`, and the calls run as tool calls. Else the reply is retried with a leak note, at most `LEAK_NOTES` (2) in a row; then it answers as text (docs/harness.md H4.8, laws `run_calls_never_answer`, `rec_leak_rebuilt_is_calls`). Code: `eukhe-durable/src/harness/generation/leak.rs` (`scan`), `generation.rs` `classify`, `generation/round.rs` `start_tool_round`. 11. The default model of each provider is rulebook data (`provider_defaults`: provider id to model id, `quirks::provider_default_model`). The startup model chain, the fallback model template and the onboarding's model after the Prime Inference sign-in read it. A provider that the table does not name has no default. 12. The prices that a catalog row does not carry are rulebook data. `service_tier_prices`: each row names a provider, a wire API, the tiers it prices (the `service_tier` of the response, else of the request), a multiplier, and optionally a model id. A response at a tier costs the base price times the multiplier of the row of the model's provider, API and id, else of the row of its provider and API without a model id; with no row the cost stays the base price. The shipped table: `openai` over `openai-responses` and `openai-completions` prices `flex` at 0.5 and `priority` and `fast` (the value that the API reports for Fast mode) at 2, `gpt-5.5` at 2.5; `openai-codex` prices `flex` at 0.5 and `priority` at 2, `gpt-5.5` at 2.5. `cache_write_1h_prices`: each row names a regex on the lowercased model id and a multiplier of the input rate; the first row that matches prices the model's 1h cache writes, and a model with no row pays its own cache write rate for them. The shipped table: `claude` at 2 (Anthropic's price, which Bedrock and the gateways that resell Claude at Anthropic's list rates charge too). `models::calculate_cost` and `models::apply_service_tier_pricing` read these tables (`quirks::cache_write_1h_multiplier`, `quirks::service_tier_multiplier`). 13. Every other fact of a model that a wire API reads is rulebook data too (S6.10): a wire API asks the rulebook, never the model id. - `adaptive_thinking` (families `claude-adaptive`, `claude-adaptive-4-6`): the model takes `thinking.type: adaptive` with an effort, on Anthropic Messages and on Bedrock alike (`models::adaptive_thinking`). An explicit `compat.forceAdaptiveThinking` of the model wins over it. - `thinking_block_binding` (`claude-adaptive`): Bedrock sends `thinking.block_binding` (not on GovCloud). - `prompt_caching` (`claude-adaptive`, `claude-adaptive-4-6`, `claude-cached`: true; `claude-uncached`: false): Bedrock sends `cachePoint` blocks. A model that no rule names caches only with `AWS_BEDROCK_FORCE_CACHE=1`. - `explicit_cache_breakpoints` (`gpt-explicit-cache`, `gpt-priority-explicit-cache`): an OpenAI Responses request sends `prompt_cache_breakpoint` and `reasoning.context: "all_turns"`. - `google_thinking_level` (`gemini-3-plus`, `gemini-latest`, `gemma-4`: true; the Gemini 2.5 families: false): a Google request sends `thinkingLevel`; a model that no rule names sends `thinkingBudget`. - `thinking_budgets` (`gemini-2.5-pro` 128, 2048, 8192, 32768; `gemini-2.5-flash` 128, 2048, 8192, 24576; `gemini-2.5-flash-lite` 512, 2048, 8192, 24576): the Google budget of each level, on the Gemini API and on Vertex alike. The user's `thinkingBudgets` wins, held inside the rule's `minimal` to `high`. A model with no rule sends the user's budget, else `-1` (dynamic). - `thinking_defaults.budgets` (1024, 2048, 8192, 16384): the budget of a level where neither the user nor a rule gives one, for every wire API that sends a token budget. A family may give an `except` regex: a model that it matches is not in the family, so the families stay disjoint. The Bedrock effort is the model's thinking-level map (a native `xhigh` model says `xhigh: "xhigh"` there), else the Bedrock default effort. Model: `bend/infer.bend`. Laws: `LAWS.bend` section (S6) and `LAWS.bend` section `ladder` (S6.9). Proofs: `PROOF.bend` section `infer`, `PROOF.bend` section `quirks`. Every input is free: any rulebook, query and directive; any replies of the model; any provider support; any tools; any stream; any context sizes. | Law | Says | | --- | --- | | `quirk_unknown_rejected` | A rulebook with a rule of an unknown directive does not compile (S6.1). | | `quirk_clash_rejected` | Two rules of one layer, target and directive with different values do not compile (S6.1). | | `quirk_unknown_when_none` | With no rule that applies, the answer is Unknown (S6.1). | | `quirk_known_when_some` | A rule that applies always gives an answer (S6.1). | | `quirk_highest_wins` | In a rulebook that compiles, the rule of the highest layer that applies gives the answer (S6.1). | | `quirk_order_free` | In a rulebook that compiles, the order of two rules changes no answer (S6.1). | | `quirk_compile_order_free` | The order of two rules does not change whether the rulebook compiles (S6.1). | | `quirk_ladder_explicit_first` | The explicit thinking-level map is the chosen ladder, over the rulebook's ladder (S6.9). | | `quirk_ladder_rule_second` | With no explicit map, the rulebook's ladder is the chosen ladder (S6.9). | | `quirk_ladder_default_last` | With neither, the chosen ladder has no entry: the default ladder gives every level (S6.9). | | `quirk_ladder_offered_exact` | A reasoning model offers a level exactly when its chosen ladder gives it a wire string, or gives it no entry and the default ladder lists it (S6.9). | | `quirk_ladder_plain_exact` | A model without reasoning offers exactly the levels of the plain default ladder (S6.9). | | `quirk_ladder_shipped_is_s10` | With the shipped default ladders, the offered levels are the S10 ladder of the chosen map, so every S10 law holds for them (S6.9). | | `force_bounded` | A forced call makes at most `1 + retries` attempts (S6.3). | | `force_first_flag` | The first attempt sets the native flag only when it is supported and free (S6.3). | | `force_retry_flag` | Every retry sets the native flag when it is supported (S6.3). | | `force_reports` | A forced call reports a call exactly when the model called the tool within its attempts (S6.3). | | `strict_within_cap` | At most `cap` tools get strict schemas (S6.4). | | `strict_keeps_every_tool` | Every tool is strict or loose, as many times as it was given (S6.4). | | `strict_by_priority` | Every strict tool has a priority of at least every loose tool (S6.4). | | `guard_cuts_repeats` | A stream that repeats a chunk `k` times in a row is cut (S6.5). | | `guard_cuts_loops` | A stream with a window of `k` chunks that is periodic, with a period of at most `k / 4` (at least 1), is cut (S6.5). | | `guard_keeps_others` | A stream with no window of `k` chunks that is periodic, with a period of at most `k / 4` (at least 1), is not cut (S6.5). | | `compact_starts_early` | When each turn grows the context by at most the margin, compaction starts before any turn over the limit (S6.6). | Code: `eukhe-ai/src/quirks/mod.rs` (`compile`, `Rules::lookup`, `Rulebook::query`, `Rulebook::family`, `Rulebook::check_catalog`, `Quirks`, `ModelKey`, `ThinkingDefaults`), `quirks/revision.rs`, `quirks/tiers.rs` (`service_tiers`, `supports_service_tier`), `quirks/prices.rs` (`service_tier_multiplier`, `cache_write_1h_multiplier`), `eukhe-ai/src/models/ladder.rs` (`thinking_level_map`, the ladder functions) and `quirks/rulebook.json` (the shipped rules); the resolvers that take the rulebook as their detected input: `api/openai_completions/compat.rs` `detect_compat`, `is_custom_qwen_template_model`; `api/anthropic_messages.rs` `get_anthropic_compat`, `manual_thinking`; `api/openai_responses.rs` `get_compat`; `api/azure_openai_responses.rs`, `api/openai_codex_responses/request.rs`, `api/bedrock_converse_stream.rs` (strict default, `manual_thinking`), `models/thinking.rs` `adaptive_thinking`, `api/google_shared/stream.rs` (validated function calling), `api/mistral_conversations/payload.rs`; `api/constrained_sampling.rs` `strict_budget`, `StrictPlan`; `eukhe-types/src/pi_ai/tool.rs` `Tool::priority`; `eukhe-types/src/pi_ai/settings.rs` `ToolChoice` (`Required`, `Tool(name)`), mapped in every wire API; `api/simple_options.rs` `maps_forced_tool_choice`; `eukhe-durable/src/harness/generation/force.rs` (`ForceEscalation`, `ForceReply`, `Forcing`, `ForcedToolError`, `soft_prompt`) and the option `ConversationStreamOptions::force_tool`; `eukhe-ai/src/utils/stream_repeat.rs` (`RepetitionGuard`, `MAX_CHUNK_BYTES`, `StreamRepeat`), `api/lazy.rs` `watched_lazy_stream`; `eukhe-durable/src/harness/generation/decide.rs` `threshold_for`, `types/settings.rs` `BackgroundMargin`, `ThresholdTokens`; `eukhe-durable/src/harness/generation.rs` `threshold_compaction`, `provider_tokens` (the provider count and its logged fallback); `eukhe-core/src/settings/cvar/table.rs` `COMPACTION_PROVIDER_TOKEN_COUNT`; `eukhe-core/src/durable/settings.rs` (`WindowPercent(10.0)`, the repeat limit); `eukhe-daemon/src/worker/durable_host/translator.rs` (`summary_write`, `end_lost_placements`, the held ends at a snapshot); `eukhe-ai/src/capabilities/` (`Capability`, `Dialect::endpoint`, `capabilities_of`, one module per dialect), `models/capabilities.rs` (the `Models` operations), `Directive::Endpoint`; `eukhe-cli/src/provider_command.rs`. Tie: the bridge tests `harness::generation::infer_bridge::the_rust_quirks_rulebook_equals_the_bend_model` (compile and lookup on generated rulebooks, each also with two rules swapped, against `bend/bridge_quirks.bend`; the directives are the first of each value type in `Directive::ALL`, and a query can name no family or no host), `durable::effort_bridge::the_rust_reasoning_depth_equals_the_bend_model` (S10: the rulebook's default ladders and rule ladders against `bend/infer.bend` section 6) and `the_rust_inference_units_equal_the_bend_model` (the escalation, the strict budget, the guard on lines fed in random deltas: random chunks, cycles of 1 to 4 chunks, the same cycles with one chunk changed, and the observed loop of 7 lines against windows of 27, 28, 32 and 64 chunks, and the compaction threshold, against `bend/bridge_infer.bend`) compare the transcripts byte for byte. The code compares token counts strictly (`>`), so for whole token counts the model threshold is the limit less the margin, plus one. Regression tests: `quirks::tests` (the shipped rulebook compiles and no two families match one catalog model; `families_that_overlap_on_a_catalog_model_are_rejected`; `a_model_two_shipped_families_match_names_no_family`; errors; precedence; order; host and family queries; the former hand-written checks; `the_shipped_qwen_families_split_at_the_revision_3_8`), `quirks::revision::tests`, `quirks::tiers::tests` (`the_table_keeps_every_answer_of_the_former_predicate`: every model of the compiled catalog and every tier, against the former hardcoded predicate, kept only in the test), `models::ladder::tests`, `openai_completions::tests::tool_choice` (`a_custom_qwen_3_8_model_sends_the_rulebook_ladder`), `openai_completions::tests::service_tier`, `openai_completions::tests::host_quirks` (`a_custom_provider_at_openrouter_ai_sends_no_cache_marks`, `the_openrouter_provider_at_a_loopback_url_keeps_the_openrouter_settings`), `constrained_sampling::tests` (`the_strict_budget_keeps_the_highest_priorities_and_the_earlier_of_equals`, `a_strict_plan_sends_tools_past_the_cap_loose_and_drops_none`), `generation::force::tests` (`manual_extended_thinking_has_no_native_forced_choice`), the `anthropic_forced_tool_choice` test `the_forced_choice_predicate_matches_the_wire`, `utils::stream_repeat::tests` (`a_loop_with_no_newline_is_cut_in_bounded_chunks`, `a_long_line_is_chunked_on_character_boundaries`, `the_observed_thinking_loop_is_cut_at_its_77th_line`, `the_observed_thinking_loop_fails_as_repeating_by_default`, `a_cycle_longer_than_a_quarter_of_the_window_is_not_cut`, `a_single_differing_line_restarts_the_window`, `the_guard_keeps_a_quarter_of_the_window`, and `the_guard_cuts_where_the_model_does`: every stream of three chunk values up to 8 chunks, for every `k` up to 12), `generation::tests::threshold`, the translator tests `a_summary_write_that_leaves_the_inbox_unsettled_ends_its_compaction` and `a_snapshot_ends_the_compactions_whose_end_it_may_have_replaced`, and the provider wire tests (`cargo test -p eukhe-ai --test providers`). Not modelled: the query map from a model to its targets (S6.2), the soft prompt and wire forms of a forced choice, and the split of deltas into chunks. Their tests tie them. #### S6 tool-call argument repair A model can send tool-call arguments that are not strict JSON, or that use another spelling of a declared type. The client repairs these arguments before validation. It does as omp does. 1. The streaming preview of the arguments closes an incomplete text. The final parse does not. It reads the full text with `JSON.parse`. When that fails, it escapes raw control characters and invalid escapes in strings, and tries again. When that fails too, it reads a relaxed dialect: single quotes, unquoted keys, comments, `True`, `False` and `None`, stray and trailing commas, `0x` and `0b` numbers, and an unquoted value such as `src/main.rs`. The relaxed dialect rejects a truncated text, characters after the value, a number that is not finite, and an unquoted value that can hide the next field. 2. When no parse reads the text, or the value is not an object, the arguments become `{ __parseError, __rawJson }`. They keep the parse error and the first 512 characters of the text. No key of the text stays, so a half-formed call never runs. The tool result is an `invalid_arguments` error. It gives the parse error and the raw text to the model. 3. A key in a JSON string (`"\"path\""`) becomes the plain key. This does not occur when the plain key is also in the call. 4. A value in another spelling of its declared type becomes that type: a JSON string of a declared object or array, a single string for an array of strings, and a number or boolean for a string. The string spellings of a boolean or number become the value, as before. 5. Each repair is an `argument_repaired` warning on the tool result. An unknown key of a closed schema stays an error. 6. A call of a tool that is not available gets a `tool_unavailable` result. The message names at most three of the offered tools. First come the names at a small edit distance, nearest first. Then come the names that end with the last `_` or `__` segment of the called name. Code: `eukhe-ai/src/utils/json_parse.rs` (`parse_json_with_repair`, `parse_tool_call_arguments`), `json_parse/relaxed.rs`, the final parse of each wire API (`anthropic_messages/events.rs`, `bedrock_converse_stream/events.rs`, `mistral_conversations/consume.rs`, `openai_completions/streaming.rs`, `openai_responses_shared/stream.rs`); `eukhe-durable/src/harness/tool/arguments.rs` (`repair_arguments`, `ArgumentError::NotJson`); `tool/result.rs` (`tool_unavailable_message`). Regression tests: `utils::json_parse::tests`, `json_parse::relaxed::tests`, `tool::arguments::tests`, and `tool::result::tests`. #### S6 the built-in model catalog The built-in model catalog is the list of models that the `/model` menu, the registry and the daemon's `get_model_catalog` use. The user's `models.json` goes on top of it. It has two sources. Both are on this machine: eukhe does not fetch a catalog. 1. The bundled catalog, `models.bundled.json` beside the executable. A release makes it from the catalog repository (`PrimeIntellect-ai/prime-agent-catalog`) at the commit in `scripts/release/catalog-pin.json` and checks the sha256 of each file. The parse is strict, and each entry must use a transport that the client compiles in (transport pinning). 2. The compiled catalog, the pi-ai tables of `eukhe-ai` (`providers/data/*.json`) and the Prime Inference entries. The built-in catalog is the union of the two sources. A model is its provider and id. 1. Each model of either source is in the catalog. Neither source hides a model of the other. Each model is in the catalog one time. 2. When the two sources have a model, the bundled entry gives its api, its `baseUrl`, its data (`name`, `reasoning`, `input`, `cost`, `contextWindow`, `maxTokens`) and each optional field that it has. The compiled entry gives each optional field that the bundled entry does not have (`thinkingLevelMap`, `compat`, `featured`, `headers`), but only when the two entries use the same api. 3. A model that only one source has comes from that source with no change. 4. The order is the bundled entries in bundled order, then the compiled models that the bundle does not have, in compiled order. The fields of a model come only from its entry in each source, thus the order in which eukhe reads the sources changes no model. 5. Without a bundled catalog, or with a damaged one, the catalog is the compiled catalog. Thus a model that the compiled catalog has and the pinned catalog does not have (for example Claude Haiku 5.5) is in the `/model` menu. To move the pin to the newest commit of the catalog repository, run `make catalog-pin`. It runs the validation and records the commit and both sha256. Review the diff of `scripts/release/catalog-pin.json`. Then commit it. Older builds kept a fetched catalog in `~/.eukhe/models/provider-model-catalog.v1.json`. No current code reads or writes that file. Laws: `LAWS.bend` section `catalog_union` (C1, the model in `bend/catalog_union.bend`, the proofs in `PROOF.bend` section `catalog_union`). Code: `eukhe-models/src/effective.rs` (`effective_catalog`), `chain.rs` (`ModelCatalog::models`), `bundled.rs` (`load_bundled_models`), `pinning.rs`. Bridge: `effective::bridge::the_rust_union_equals_the_bend_model` runs the code against `bend/bridge_catalog_union.bend` on generated catalogs. Regression tests: `chain::tests::a_bundle_without_a_compiled_model_keeps_it`; `eukhe-daemon` `tests/model_catalog_e2e.rs` (`a_bundle_without_a_compiled_model_still_lists_it`: the pinned catalog snapshot as the bundle, an Anthropic login, and the daemon's catalog has Claude Haiku 5.5). #### S6 models.json and the saved default model One loader reads `models.json` for every view: the registry (`eukhe model list`, the `/model` menu, the daemon's `get_model_catalog`, onboarding, the auxiliary model, the memory compactor, RLM, usage) and the model collection of a session. 1. Each `providers.` entry loads alone. A bad entry (a wrong type, a missing `baseUrl` or `apiKey`, a model that is not valid) drops only that provider, and a diagnostic names `providers.`. The other providers stay. 2. A file that eukhe cannot read or parse is one diagnostic and gives no custom models. 3. The diagnostics show in `eukhe model list` (stderr), in the `modelsJsonErrors` list of the `get_model_catalog` reply, and in the settings-error row of each session at open. A session that no request pins starts on the saved default (`defaultProvider`/`defaultModel`). eukhe never moves past it in silence: 1. When the catalog does not have it, or a set `enabledModels` scope does not include it, the session runs on the next startup default. One `eukhe.default_model` row (display-only) names the saved default, the reason and the model in its place. 2. When its provider has no credentials, the resolution is the typed error `DefaultUnusable` with the login hint. No other provider takes its place: the session opens with no model and the row names the saved default and the reason. 3. When no other model can take its place, the session also opens with no model and the row. 4. The memory compactor (no `memory.model`) fails its call with the saved default and the reason. Laws: `default_honoured_or_reported`, `default_quiet_when_honoured`, `default_quiet_when_unset`, `default_unkeyed_never_moves` (the model in `bend/default_model.bend`, the proofs in `PROOF.bend` section `default_model`). Code: `eukhe-core/src/models/custom.rs` (`ModelsConfig::load`), `models/registry.rs` (`compose`), `durable/models/compose.rs`, `models/resolver.rs` (`saved_default_miss`), `durable/models.rs` (`resolve_session_model`, `resolve_open_model`). Bridge: `durable::models::default_bridge::the_rust_session_model_resolution_equals_the_bend_model` runs the resolution against `bend/bridge_default_model.bend`. ### S9. Local tiny models A local tiny model is a small model that runs on this machine. eukhe uses it for some workloads (roles): session titles (`tiny`), typed judgments (`judge`), translation (`translate`), text to speech (`speech`) and speech to text (`dictation`). It is done as omp does it. The catalog (`eukhe-tiny/src/catalog.rs`) pins each model. Each file names its Hugging Face repository, the commit, the size and the sha256. The models: `lfm2.5-230m` and `lfm2.5-350m` (text, ONNX q4), `whisper-base` and `whisper-small` (speech to text), `kokoro` (Kokoro-82M, text to speech, with the misaki lexicon for its phonemes). Download. Only the user starts a download: `eukhe tiny-models download ` or `eukhe tiny-models download all`. `eukhe tiny-models list` shows each model as installed, missing or incomplete. Nothing else downloads a model. 1. The download writes each file into a staging directory (`/models/tiny/.staging--`). It computes the sha256 while it writes. 2. A file of another size or another sha256 stops the download. The staging directory is removed. An earlier install does not change. If the staging directory cannot be removed, the download still fails with its own error, and a warning names the directory. 3. When all files are correct, one rename installs the directory as `/models/tiny/`. Thus a model is complete or absent. The worker. One worker process serves one model and backend, for every eukhe process. Its socket, bind lock and log are in `/run/tiny/` (`-onnx.sock`, `.bind.lock`, `.log`). The worker and the client make this directory 0700, also when it exists already. The worker is `eukhe tiny-worker --backend onnx --agent-dir `, a hidden command. It gets the agent dir of the process that starts it on its command line, never from its environment. Thus a daemon started with `--agent-dir` and a worker it starts use the same directory. 1. A process that needs a model connects to the socket and sends `ping`. The worker answers with its launch tag: `|onnx||`. 2. Same tag: the process uses the worker (joined). 3. Another tag: the process sends `shutdown`. The worker removes its socket and answers `bye`. It takes no more connections or requests. The requests of other processes that are in progress get their full answers; then the worker exits. The process starts a new worker (replaced). Thus a new build or a changed device or dtype setting never uses a stale worker. 4. No live socket: the process starts a worker (started). The worker starts detached, in its own process group. It takes the bind lock only while it checks the socket and binds it. If a live worker answers, the new worker exits and the process uses the live one. 5. The worker exits after 15 minutes without a request (`EUKHE_TINY_WORKER_IDLE_MS` changes the window for tests). It removes its socket first. 6. Requests are JSON lines: `load`, `chat` (messages, prefill, stop strings, max tokens), `transcribe` (audio, language), `speak` (text, voice, speed). Each request gets one answer, in order. The client makes the prompt and reads the output. If the worker stops during a request, the client gets a typed error (`Crashed`) with the log path. Each side reads a line of at most 256 MiB. The worker answers a longer request line with `bad_request`; the client treats a longer answer line as out of protocol. Neither side keeps the bytes of a longer line. The engine (`eukhe-tiny-onnx`) is ONNX Runtime through the `ort` crate. The worker loads the runtime library at run time: `providers.tinyRuntimeLibrary`, else `ORT_DYLIB_PATH`, else `libonnxruntime.so` (nixpkgs `onnxruntime`, 1.24 or later). The Nix package wrapper and the dev shell set `ORT_DYLIB_PATH` to the nixpkgs library (`--set-default`: a value the user sets wins). The build downloads nothing and links no runtime. Inference is on the CPU. The text tokenizer is eukhe's own (`tokenizer.rs`): the byte-level BPE of the model's `tokenizer.json`, with its added tokens, its pre-tokenizer regex and its ranked merges. A file with any other setting does not load. Roles. A role names a chain of models: 1. The chain is `modelRoles.` (when set), then `retry.fallbackChains.`. An explicit `[]` keeps the role on its primary model only. An unset chain is the built-in priority: the catalog models of the role, then the session's provider model (for a text role). For `translate`, `lfm2.5-350m` comes before `lfm2.5-230m` (see Translation). 2. A local model is available when it is installed. A provider model is available when it is the session's usable model. 3. A task uses the first available model of the chain. When no model of the chain is available, the task fails as `Unavailable`. It never uses a model outside the chain. Uses. The first user message of a top-level session starts a title in the background (role `tiny`; `titles.generate: false` turns it off). The turn does not wait for it. A failure leaves the session without a title. A user name is never replaced. With `modelRoles.tiny` unset and no local model installed, the title is one more request to the session's provider. A scripted session (the faux-script test seam) takes no title. The `local` provider serves each text catalog model as a classifier model (`local/`) on the `classify` path. A code-mode `models.classify()` call with no model (`null` or `undefined`) uses the `judge` role. Its model is the first model of the chain that is an available classifier model: an installed local model, or a provider classifier model with configured auth. When no model is available, the call fails with `Unavailable`, and the message names the role and its chain. `eukhe translate `, `eukhe say [--out f.wav]` and `eukhe transcribe ` use the roles `translate`, `speech` and `dictation`. First paint never waits for a model load. Translation. The request is one user turn, `Translate from English to .\nEnglish: `, and the assistant turn starts with `:`. A text of one line stops at the first line end. If the reply is empty, or has the same letters and digits as the input (case ignored), the translation fails with a typed error (`TranslationError::Empty`, `TranslationError::Unchanged`) and prints nothing. A tiny model that does not translate repeats its input, so such a reply is never the answer. An input with no letters is exempt. This recipe won a fixed eval: 10 short English sentences (`hello`, `good morning, my friend`, ... `See you tomorrow.`) into German, French, Spanish and Portuguese, greedy, each variant stopped at the first line end. The table gives the echoes (replies equal to the input) out of 40 and the mean chrF (0 to 100) against one reference translation: | Variant | 230m echoes | 230m chrF | 350m echoes | 350m chrF | | --- | --- | --- | --- | --- | | system "translator" + user `Translate this text to :` (old) | 9 | 34 | 0 | 64 | | instruction in the system prompt only | 24 | 12 | not run | not run | | language code, not name | 27 | 13 | not run | not run | | old + one example pair | 2 | 36 | 0 | 41 | | old + prefill `: ` | 2 | 52 | 0 | 59 | | old + one example pair + prefill `: ` | 1 | 20 | not run | not run | | user `Translate this text to :` + prefill `:`, no system prompt | 16 | 26 | 2 | 53 | | user `Translate from English to .\nEnglish: ` + prefill `:` (new) | 1 | 53 | 0 | 65 | | new without "from English" (`Text:`) | 4 | 47 | 0 | 62 | | new + the "translator" system prompt | 0 | 49 | 0 | 59 | | new + one example pair | 0 | 59 | 0 | 59 | The new recipe is the best on `lfm2.5-350m` and close to the best on `lfm2.5-230m`. The example pair needs one example per target language, so eukhe does not use it. `lfm2.5-230m` still translates badly even with the new recipe (`hello` into German gives `willkommen`). Thus the built-in `translate` chain takes `lfm2.5-350m` first when it is installed. `lfm2.5-350m` gives `Hallo`, `Vielen Dank.` and `Où est la gare ?`, but it also makes errors (`Gute Morgensinn, mein Freund`). The prompt names English as the source language. The eval tested English sources only. Dictation. The TUI types speech into the editor draft: 1. `stt.enabled` (default `false`) turns on the dictation keys. The TUI reads the value from the session at each key press that starts a take. When the value is `false`, the key shows how to turn it on. 2. The actions `app.stt.pushToTalk` and `app.stt.toggle` have no default key. The user binds them in `binds.cfg` (S4.5), for example `bind f8 "action app.stt.toggle"`. Push-to-talk records while the key is held: the terminal sends the key again at each auto-repeat, and a gap of more than 300 ms (800 ms after the first press) is the release. Toggle records until the next press of its key. 3. The recorder is the first program on `PATH` of `pw-record`, `parec` and `arecord` (on macOS: `sox`, then `ffmpeg`). It writes raw 16 kHz mono 16-bit samples to its stdout, and the TUI writes them to a temporary WAV file. When the program fails, the take fails with the stderr of the program. eukhe does not try the next program. 4. The `dictation` role reads the file as `eukhe transcribe` does (local models only). The text goes into the draft at the cursor. An empty text shows "Dictation heard no speech". 5. The status line shows `[REC] recording`, then `[STT] transcribing`. Esc during a take stops the recorder, removes the file and shows "Dictation cancelled". A failure shows as an error row. eukhe does not use the `cpal` crate. `cpal` needs `alsa-sys`, which links `libasound` through `pkg-config` at build time. The CI runners (`ubuntu-24.04`) do not install `libasound2-dev`, and the Nix build would need `alsa-lib` and `pkg-config`. A recorder program needs nothing at build time. Dictation is client input: it has no Bend model. Code: `eukhe-tui/src/dictation.rs` (`Recorder`, `Transcriber`, `SubprocessRecorder`, `run_take`, `wav_bytes`); `session_ui/dictation.rs` (`handle_dictation_key`, `apply_dictation_update`); `status_line.rs`; `eukhe-cli/src/tiny_command.rs` (`RoleTranscriber`, `transcribe_file`). Tests: `eukhe-tui` `dictation::tests` (a fake recorder and transcriber: toggle, the hold release, a recorder failure, an abort that stops the recorder and removes the file; the subprocess recorder with `/bin/sh` programs: the first program found, its stderr, no audio, no program), `keybindings::tests::dictation_actions_bind_only_through_binds_cfg`, `status_line::tests::the_dictation_phase_leads_the_line` and `tests/dictation_headless.rs` (the TUI on a mock daemon: toggle, push-to-talk, Esc, a failure, `stt.enabled` off). Model: `bend/tiny.bend`. Laws: `LAWS.bend` section (S9). Proofs: `PROOF.bend` section `tiny`. Every input is free: any chain, any set of available models, any needs of any processes with any tags, any worker exits, in any order. | Law | Says | | --- | --- | | `tiny_pick_in_chain` | A task uses only a model of its chain that is available (roles 3). | | `tiny_pick_first` | A task uses the first available model of its chain (roles 3). | | `tiny_pick_unavailable` | With no model of the chain available, the task fails as Unavailable (roles 3). | | `tiny_every_need_served` | Each need gets exactly one outcome: started, joined or replaced (worker 1-4). | | `tiny_worker_current` | After any events, the worker runs with the tag of the last need since the last exit (worker 3). | | `tiny_same_tag_joins` | A need with the tag of the running worker joins it (worker 2). | Code: `eukhe-tiny/src/roles.rs` `chain`, `pick`, `Candidate`, `FallbackChain`; `election.rs` `ensure`, `request`, `probe`, `Launcher`, `Got`; `worker.rs` `launch_tag`, `run_worker`, `bind`, `serve`; `client.rs` `TinyClient`, `TinyError`; `catalog.rs` `CATALOG`, `installed`, `install_state`; `download.rs` `download`; `tasks.rs` (prompts and output parsing); `eukhe-tiny-onnx/src/text.rs`, `whisper.rs`, `kokoro.rs`, `runtime.rs`, `tokenizer.rs`; `eukhe-core/src/tiny/` (settings, `resolve`, `title.rs`, `judge.rs` `judge_classifier`); `eukhe-core/src/durable/codemode.rs`; `eukhe-durable/src/codemode/models.rs` `ClassifierDefault`; `eukhe-daemon/src/worker/title.rs`; `eukhe-cli/src/tiny_command.rs`, `tiny_models_command.rs`. Tie: the bridge tests `bridge::the_rust_pick_equals_the_bend_model` and `bridge::the_rust_election_equals_the_bend_model` (`eukhe-tiny`) run `pick` on generated chains and available sets, and the real election (`ensure`, the real `serve`, in process, with a fake engine) on generated needs and exits, against `bend/bridge_tiny.bend`. Regression tests: `client::tests` (real worker processes in a temp run dir, started on the client's command line with no agent dir variable and another `HOME`: started, joined, replaced, idle exit, crash, not installed, the client's agent dir served); `worker::tests` (a shutdown lets another client's request in progress finish with its answer; an existing run dir is made 0700); `protocol::tests` (a line over the limit is skipped and the next line read); `download::tests` (a mock HTTP server: a bad sha256, a wrong size and a broken transfer install nothing; a partial install is replaced only on success; a staging dir that cannot be removed keeps the download's own error); `eukhe-core` `tiny::tests` (role resolution; `translate` takes `lfm2.5-350m` first) and `tiny::judge::tests` (the `judge` model of a classification with no model); `eukhe-tiny` `tasks::tests` (the translate request; a reply that repeats the input is `TranslationError::Unchanged`); `eukhe-durable` `codemode::tests::calls` (`models.classify(null, …)` asks the default classifier); `eukhe-daemon` `tests/session_title_e2e.rs` (a supervisor on a temp socket and agent dir, a mock provider: the first prompt titles the session, and `titles.generate: false` titles nothing); `eukhe-tiny-onnx` `tokenizer::tests` (ids and text against the ids that Hugging Face `tokenizers` gave once for the same files and texts, and random text that encodes and decodes back to itself). Real-model tests run only with the models and the runtime library given (else they print why they skip): `eukhe-tiny-onnx` `text::tests` (a title from `lfm2.5-230m` and, when present, from `lfm2.5-350m`, whose graph has a `num_logits_to_keep` input), `whisper::round_trip_tests` (Kokoro speaks a sentence, Whisper reads it back) and the `tokenizer::tests` on the model files; `eukhe-core` `tiny::smoke_tests` (a title and a `Models::classify` judgment on the `local` provider, through a real worker). Not modelled: two processes that start a worker at the same time (the bind lock orders them; the second one uses the first worker). ### S10. Reasoning depth The reasoning depth (thinking level) of a session sets how much the model thinks before it answers. The levels, in ladder order, are `off`, `minimal`, `low`, `medium`, `high`, `xhigh` and `max`. The ladder of a model. Every ladder and every wire value is rulebook data (S6.9); the code holds none. A model offers some levels of the ladder, in ladder order: 0. The chosen ladder is the explicit `thinkingLevelMap` of the model (`models.json`, the catalog); else, for a reasoning model of a custom provider, the rulebook's ladder for its family; else no ladder. Built-in providers keep the ladders of their catalog. Thus the custom model `qwen3.8-27b` with no map offers `off`, `low`, `medium` and `xhigh`. 1. A model without reasoning offers the levels of the plain default ladder (shipped: `off` only). Its map has no effect. 2. A reasoning model keeps each level by its entry in the chosen ladder. A `null` entry removes the level. A wire string keeps the level, and the request sends that string. A missing entry takes the default ladder (shipped: `off` to `high`, so `xhigh` and `max` need an explicit entry); such a level goes out in the wire API's own form. 3. Thus a reasoning model can offer `off` only (each other entry is `null`) or no level (each entry is `null`). 4. The daemon serves the offered levels (`availableThinkingLevels`, and each catalog model's levels to the model picker); clients compute no ladder. The clamp. The clamp moves a level to a level that the model offers: 1. An offered level does not change. 2. Another level goes to the first offered level up the ladder from it. When there is no such level, it goes to the first offered level down the ladder from it. 3. When the model offers no level, the clamp is `off`. The cycle. A cycle step (`cycle_thinking_level`, forward or backward) moves one level along the offered levels: 1. When the model offers fewer than two levels, the step changes nothing. This is true for a model without reasoning and for a model that offers `off` only. The daemon answers `null`. 2. Else the step starts at the clamp of the current level. Forward goes to the next offered level. From the last offered level it goes to the first. Backward goes to the offered level before. From the first offered level it goes to the last. 3. Thus forward visits each offered level one time in each lap, in ladder order. A backward step cancels a forward step, and a forward step cancels a backward step. The session. The main conversation stores its level. The level can be unset: 1. Open settles the stored level. It takes the stored level, else the default (`defaultThinkingLevel` in the settings, else `medium`). When the session model is in the catalog, open clamps this level to the model. Without a catalog model, open keeps the level. 2. A model switch clamps the current level to the new model. A set clamps the requested level to the model (without a catalog model it keeps the requested level). A cycle step sets the level that the step gives. 3. The sent level. The durable runtime sends the stored level, else `off`. Each provider clamps this level to the model before it sends the request. 4. The displayed level. The daemon summary and the connection state (`thinkingLevel`) show the stored level clamped to the model (the stored level without a catalog model). Thus they show the level that the requests send. An unset level shows as `default`, never as `off`. Model: `bend/effort.bend`. Laws: `LAWS.bend` section (S10). Proofs: `PROOF.bend` section `effort`. Every input is free: any model, with or without reasoning, with any map (each entry missing, `null` or a wire string); any level; any stored level and default; any model switches, sets and cycle steps, in any order. | Law | Says | | --- | --- | | `effort_offered_plain` | A model without reasoning offers `off` only, with any map (ladder 1). | | `effort_offered_null` | A reasoning model never offers a level with a `null` entry (ladder 2). | | `effort_offered_ext` | A model offers `xhigh` or `max` only with a wire entry for it (ladder 2). | | `effort_offered_kept` | A reasoning model offers each level that its map keeps: a wire entry, or a missing entry below `xhigh` (ladder 2). | | `effort_offered_asc` | The offered levels go up the ladder strictly: no level comes two times (ladder). | | `effort_clamp_offered` | When the model offers a level, the clamp of each level is an offered level. Thus a model switch lands on an offered level (clamp 2, session 2). | | `effort_clamp_keeps` | The clamp does not change an offered level (clamp 1). | | `effort_clamp_idem` | A second clamp changes nothing, also when the model offers no level (clamp 3). | | `effort_cycle_still` | With fewer than two offered levels, a cycle step changes nothing (cycle 1). | | `effort_cycle_next` | Forward from an offered level goes to the next offered level (cycle 2). | | `effort_cycle_wrap` | Forward from the last offered level goes to the first (cycle 2). | | `effort_cycle_prev` | Backward from an offered level goes to the offered level before it (cycle 2). | | `effort_cycle_wrap_back` | Backward from the first offered level goes to the last (cycle 2). | | `effort_cycle_inverse` | From an offered level, a backward step cancels a forward step (cycle 3). | | `effort_cycle_inverse_back` | From an offered level, a forward step cancels a backward step (cycle 3). | | `effort_unset_default` | An unset level shows as `default`, never as `off` (session 4). | | `effort_settle_offered` | When the model offers a level, open settles on an offered level (session 1). | | `effort_stored_is_sent` | After open and any actions, the session stores the level that it sends (session 1-3). | | `effort_shown_is_sent` | After open and any actions, the display shows the level that the session sends (session 4). | Code: `eukhe-ai/src/models/ladder.rs` `thinking_level_map`, `get_supported_thinking_levels`, `registry_thinking_levels`, `registry_clamp_thinking_level`, `clamp_thinking_level`, `cycle_thinking_level`; the provider clamps in `eukhe-ai/src/api/`; `eukhe-core/src/durable/open.rs` `settle_main_agent`; `eukhe-daemon/src/model_switch.rs` (model switch, set); `setting_switches.rs` `handle_cycle_thinking_level`; `worker/summary.rs` `thinking_level`; `rpc/model_commands.rs`. Tie: the bridge test `durable::effort_bridge::the_rust_reasoning_depth_equals_the_bend_model` (`eukhe-core`) runs `get_supported_thinking_levels`, `clamp_thinking_level` and `cycle_thinking_level` (forward and backward, from each ladder level) on catalog models against `bend/bridge_effort.bend`. The models are the edge models (no reasoning, no map, an `off`-only ladder, an empty ladder, one level, explicit `xhigh` and `max`) and random models (each entry missing, `null` or a wire string). The test requires that the models reach each rule. Regression tests: `eukhe-core` `durable::open::tests` (`open_settles_an_unset_thinking_level_on_the_clamped_default`: open settles an unset level on the clamped default); `eukhe-daemon` `setting_switches::tests` (`cycle_thinking_level` without reasoning answers `null`; forward and backward steps wrap; an unset level shows as `default`). Not modelled: the order of the settings that give the default level, and the mapping of a level to the wire value of each provider.