Open source agent hosting on the AT Protocol
languid docs design.md
23 kB · 168 lines
Markdown
at main

Design #

What languid is, the principles it follows, and how the pieces fit. This describes the design as it stands; where the code doesn't do something yet, the section says so. Why individual choices were made is in decisions.md, and what's left to build is in the roadmap.

What it is #

languid hosts agents on the AT Protocol. Each agent is its own atproto account, with its own DID and handle. Its state (blocks, conversations, tool calls and their results, configuration) lives as ai.languid.* records in a private Space that belongs to its owner. Tool calls run on the owner's own computers, through a small runner that dials out to the service. Inference goes to whatever model provider the service, or the owner, has keys for.

Because the state is records rather than rows in a private database, an agent isn't tied to the service that runs it. Another host, another harness, or the owner's own scripts can read the same records and carry the agent forward.

Principles #

  • Portability is the product. An agent's identity, blocks and history live in records that move with it. No host is load-bearing.
  • The agent is a first-class account. It authors its records under its own DID. The owner relationship is a link between two accounts, not a delegation.
  • Private by default. Blocks, messages and trajectories go in a space. Only what the agent chooses to publish goes in its public repo.
  • The harness is replaceable. Anything that can read the agent's space can run the agent.
  • Records are the source of truth; indexes are caches. Search indexes and pending-work queues are derived from records and can be rebuilt from them.
  • Access control is not confidentiality. The PDS host and every authorized reader see plaintext. Keys and secrets never go in records.

Accounts and identity #

  • Agents. Each agent is an account on the service's PDS, created by the service's manager through the PDS's invite codes. Its handle is <name>.<domain> (scout.languid.ai on the hosted service).
  • Owners. An owner is any atproto account on a PDS that runs Spaces. They sign in to the service with atproto OAuth. On the hosted service, invited people get an account on its PDS through a sign-up page.
  • The link. The agent's public repo holds ai.languid.agent.profile and ai.languid.agent.owner; the owner's repo can hold ai.languid.agent.link, so either side can prove the relationship. (A signed-in owner's repo doesn't get the link yet: the OAuth grant doesn't cover their public repo.)

Spaces #

Each agent has one space, of type ai.languid.space.agent, at at://<owner DID>/space/ai.languid.space.agent/<agent name>. It's a memberspace whose authority is the owner's own DID, with the owner and the agent as read/write members.

The authority is part of a space's identity, and whoever holds it controls who's in the space. Anchoring it on the owner's DID means the owner controls their agents' spaces outright, the space moves when the owner migrates their account, and no account exists only to hold authority. The service creates the space as the owner, through the OAuth grant, which is limited to creating and managing ai.languid.space.agent spaces.

The space type's lexicon must be published and resolvable, or authorization servers refuse the grant: languid admin publish-lexicon publishes it and prints the DNS record that points to it.

Planned: a shared space type, ai.languid.space.shared, for blocks several agents share.

Error reports #

Each agent also has a small space for its error reports, of type ai.languid.space.telemetry, at at://<agent DID>/space/ai.languid.space.telemetry/self (src/telemetry.ts). Its authority is the agent itself, since the service already holds the agent's login, so owners grant nothing new. The agent writes; the owner is a read-only member. It holds only error reports and the owner's sharing decision, so sharing it never shows anyone the agent's memory, messages or tool output.

Reports come from failed runs, model provider errors, failed wakeups, service errors in a call about the agent, and CLI crashes that look like bugs (the CLI sends them to the service, which checks every field). Before a report is written, anything that looks like a key, token, password or credentialed URL is replaced, home directories are shortened and every field is capped. Prompts, tool arguments and message text are never passed in. At most 60 reports an hour are kept per agent, and an identical one within a minute is dropped.

A run still going when the service stops (a deploy, a restart) is cut off, since runs live in memory; on SIGTERM or SIGINT the service writes a RunLost report for each one before exiting.

The service keeps its own log in an ops space, at://<ops DID>/space/ai.languid.space.ops/self (src/ops.ts). Its authority is a dedicated account, ops.<handle domain>, which the service makes on first use; the installation owner is a read-only member. It holds the sign-up funnel and ops events (boots, crashes, Languid's own bugs, purges), never agents' content, and /admin/logs shows them.

The maintainer can resolve or reopen shared feedback at /admin/feedback, with feedback_triage, or through ai.languid.telemetry.triageFeedback. The feedback inbox exposes the full record URI and CID and accepts a status filter (open, resolved, or all). Missing triage means open. Filtering happens before limiting, so older open reports remain findable. Each change writes an event in the private ops space, grouped by a hash of the full URI, not by a reporter-local record key. The latest event supplies status, note and fix link; prior events remain in the journal. The service verifies the original is still shared with a maintainer DID and the recipient still has read access before accepting a change. Notes never enter the reporter's space or the shared-report API. Machines do not get the triage mutation tool.

Triage is private; replies are what the agent hears. Resolving never sends a message, and a supplied reply is rejected. The installation owner lists shared feedback with languid feedback [--kind …] [--open] [--json] (--open: no reply yet) and answers one item with languid feedback reply <agent> <rkey> --status fixed|wontfix|needs-info|noted [--version vNNN] "<text>" (ai.languid.telemetry.replyFeedback). The service accepts a reply only from the installation owner and only while that agent's feedback is shared with a maintainer DID that still has read access, the same test as triage. It writes an ai.languid.telemetry.feedbackReply record into the agent's telemetry space with the agent's login, with its own delivery state (delivered). The agent isn't woken: its next turn anywhere (hosted, Discord, MCP, the pi chat, a local languid run through ai.languid.telemetry.takeReplies, a Machine) gets the reply once, in a <notices> section at the end of its system prompt, and the reply is marked delivered. The pi chat keeps the notes for every request of the turn that answers the thread's latest message. Sending the same reply twice writes two records, and the agent hears both. The feedback inbox, /admin/feedback (a Reply column and filter) and the agent's settings page show the latest reply under each item.

Agents can send feedback too: the feedback tool writes an ai.languid.telemetry.feedback record in the same space. If the space isn't shared, the tool's result tells the agent to ask its owner, and to call feedback_share only if they agree. feedback_share always asks the owner on an approval card; neither "allow all" nor a saved rule can approve it, and a run with no one to ask is refused. A local CLI run's feedback tools go through the service (ai.languid.telemetry.sendFeedback, setSharing, and feedbackInbox for the installation owner), and languid run asks about feedback_share at the terminal, never on --yes.

Sharing is opt-in. The CLI asks the first time a crash is recorded, and the settings page asks while the owner hasn't answered. Sharing adds the recipient as a read-only member; stopping removes them. The recipient is the ops account (ops.languid.ai, or LANGUID_TELEMETRY_RECIPIENT): Languid's one destination for feedback and error reports. On boot, spaces shared with the earlier recipient (cameron.languid.ai) move to it. The membership is the grant, and the sharing record remembers the answer so owners are asked once. The service lists only agents whose owner said yes and whose space has the recipient as a member: the installation owner reads feedback at /admin/feedback, error reports at /errors or with languid errors inbox, and the installation owner's own agents get a feedback_inbox tool that reads both.

Records #

All under ai.languid.*. Shapes are TypeScript in src/records.ts for now; published lexicons are on the roadmap.

NSID rkey lives in holds
ai.languid.agent.profile self agent's public repo name, description
ai.languid.agent.owner self agent's public repo owner DID, space
ai.languid.agent.link agent DID owner's public repo the agent, confirming the owner side
ai.languid.agent.config self space model, system prompt, compaction settings, script policy
ai.languid.memory.block label space label, description, value, limit, read-only, pinned, subject
ai.languid.conversation.thread tid space title; parent (starting thread, task call, wake) for a task's thread (agents.md)
ai.languid.conversation.message tid space thread, role, author DID, client, content
ai.languid.conversation.summary tid space thread, first message kept, summary
ai.languid.tool.definition tool name space schema, description
ai.languid.tool.call tid space tool, arguments, the message that issued it, placement, assigned computer
ai.languid.tool.result tid space call, computer, output or error, timing
ai.languid.tool.job tid space a background command's end: status, exit code, last output
ai.languid.computer.node computer name space a computer that runs the agent's tools: hostname, OS, capabilities (ai.languid.machine.node until Nov 15)
ai.languid.computer.permissions computer name space the owner's rules for that computer: tools always allowed (alwaysAllow) or never (deny), and bash commands that run without asking (allowCommands) or never (denyCommands); written only by the service, deleted with the computer
ai.languid.run.schedule tid space owner-authorized one-off or UTC fixed-interval review
ai.languid.run.wakeup occurrence hash space automated input and source revision; assistant response links back to complete work
ai.languid.telemetry.error tid telemetry space one error, scrubbed: source (cli, service, run, model, wakeup), class, message, stack frames, method, provider and model, languid commit, platform
ai.languid.telemetry.feedback tid telemetry space feedback the agent sent with its feedback tool: bug, idea or other, its own words (scrubbed), model, languid commit
ai.languid.telemetry.feedbackReply tid telemetry space the maintainer's answer to one feedback record (languid feedback reply): the feedback's rkey, status (fixed, wontfix, needs-info, noted), text (scrubbed), release, and whether the agent has been told
ai.languid.telemetry.sharing self telemetry space the owner's answer to sharing error reports: yes or no, with whom, when
ai.languid.ops.feedbackTriage full-URI SHA-256 + tid ops space private triage event: feedback URI/CID, open/resolved, note, HTTPS fix link, actor DID and time; original reports stay untouched
ai.languid.ops.event tid ops space one thing the service did or hit: level, kind (boot, crash, bug, pds, purge), scrubbed message and stack, agent or method, repeat count, languid commit; deleted after 90 days
ai.languid.ops.funnel tid ops space one invited owner reaching one sign-up step
ai.languid.ops.message tid ops space a message from Languid (title, body, optional to owner DID; none = every owner signed up by then), shown in Inboxes; written at /admin/messages
ai.languid.ops.messageRead owner DID ops space the messages one owner dismissed

One record outside ai.languid.*: an agent's static sites are place.wisp.fs records (rkey = site name) in its public repo, wisp.place's lexicon, so other wisp hosts can serve them. Each file is a gzipped blob. ai.languid.site.publish writes one with a single approval, and the service serves them on their own host (LANGUID_SITES_HOST), sandboxed and without cookies (src/sites.ts, Sites).

Planned: ai.languid.memory.passage (archival text), calendar/cron schedules and general model/tool work dispatch. Wakeup record lexicon drafts are in lexicons/; they are not published yet. So are drafts for evals (ai.languid.eval.*, see evals.md).

Model JSON (tool arguments, schemas) is stored as a JSON string, because the atproto data model has no floats. Each lexicon group (agent, memory, conversation, tool, computer, run, space) needs its own _lexicon.<group>.languid.ai TXT record once lexicons are published.

The service #

The languid service (languid serve) is the only reader and writer of agent records. Clients (pi, the CLI) call it over XRPC with a session token, and the web chat calls it with the browser's session cookie; none of them holds an agent's credentials. Every call is scoped to the caller's own agents.

One writer is a deliberate choice. The Spaces alpha has no compare-and-swap on writes, so two writers editing one block would silently overwrite each other. The service serializes edits to records an agent's threads share (blocks, config, permissions), never holding a lock for a whole turn, and conflict handling lives in its methods rather than in the storage layer.

Threads are the unit of concurrency. Each thread has one turn at a time, and an agent's threads run at the same time, over one shared store. A message sent into a busy thread joins the turn already going (see connectors.md).

The locks are in-process, which is right for one service instance. Running several needs a shared lock.

The web model picker supports search and starred favorites. Favorites are UI preferences stored in the service's state.json, keyed by the signed-in owner's DID, so they apply across that owner's agents and browsers. They don't change an agent's configured model. Unlike agent records, these preferences don't yet move between hosts.

Context #

Every model call's context is compiled from records, in a fixed order:

  1. the system prompt, from the config record;
  2. pinned blocks, in full;
  3. unpinned blocks, by label and description only, so the agent knows they exist and can read them (getRecord) when it needs one;
  4. the computers the agent can use, and which one it's on;
  5. the thread: its latest summary, if any, then every message after it.

Clients never assemble context themselves. pi fetches it from the service before every request, so pi's own session file is just a cache.

Blocks are the agent's editable memory: a persona, notes about its owner, a scratchpad. The agent edits them with tools (create, append, replace, rewrite), and each edit is a record update. Pinning (block_pin) is its own operation: it decides whether a block is shown every turn, the fixed part of the context (core memory, in Letta's terms), and changes nothing else. Reading a block is not pinning it. A block can name the account it's about (subject); the harness renders that account's current handle, so a block never holds a stale copy of one.

Messages are one record per turn. A thread groups them, so one agent can hold many conversations over the same blocks. Each user turn reaches the model in an envelope naming its sender, its relation to the agent and the client it came through. Messages go in rkey order, except that an assistant message goes right after the newest message its context had (contextEnd): a user message that arrived while it was being written comes after it, and the thread still owes it an answer.

Compaction never deletes messages. Near the context window, the service writes a summary record covering the older messages, and the compiled thread becomes that summary plus what follows. The full history stays in the space.

Lookups go through scripts. Rather than one tool per query, the agent writes a short JavaScript program against XRPC methods (com.atproto.repo.* and ai.languid.memory.*), run in a sandbox whose only capability is those calls, under a policy that decides by method, collection and repo. Scripts can also write records (createRecord, putRecord, deleteRecord), but only in the agent's own space or its own public repo, never in ai.languid.* collections, and only for collections the owner allows on the settings page ("Record writes"), with or without asking first. The policy lives in the agent's config record, which scripts can't write.

Tool calls and computers #

A tool call is a record. The service assigns each call to exactly one computer when it records the assistant message: the one the model or session names, otherwise an online computer. Runners never race for calls.

A runner is a small process on a computer the owner controls. It long-polls the service for its computer's calls, runs them with pi's tool implementations (read, bash, edit, write, grep, find, ls) and posts results back. It listens on nothing, so a computer needs no open port or public address. The owner approves bash, edit and write from the client; the others run without asking. The owner's rules for a computer are in its ai.languid.computer.permissions record, set on the agent's settings page or by "Always allow" on a question, and every client goes by them (src/permissions.ts): a call they allow needs no approval anywhere, one they deny is refused without asking (even under "Allow all"), and the rest are asked about.

Every tool call states its purpose in a required call_intention argument, listed first in each tool's schema, so a client can show what a call is for while it runs. The service checks it before a call is placed, asked about or run, and strips it from the arguments a tool receives. It is the model's own text: clients show it as a label, and approval questions show the command beside it. Tools are sent strictly where the provider can constrain the model to the schema (src/strict-tools.ts), so the argument can't be left out: OpenAI, Claude (as many tools as fit its strict limits, bash first), and Languid's own Inkling 2 Small and DeepSeek V4.1 Flash on Together or our GPUs (sent again without strict if the provider refuses); other Together and OpenRouter models get the plain schema, and the check is the backstop.

Tool calls run at most once: a call whose runner restarts mid-call fails rather than running again, since a shell command can't safely be replayed. A bash command still running after 10 seconds becomes a background job that outlives the call; its end is recorded as an ai.languid.tool.job record.

Planned:

  • Placement with preferences. A call's record already carries a mode (pinned, preferred or any) and computers; only pinning is enforced, and capability matching isn't done.
  • Tools defined by records, and MCP servers configured per runner: each computer declares its tools when it connects (connectors.md).

Inference #

Model access goes through pi's ModelRuntime, so every provider pi supports works, including subscription logins. The model is part of the agent's config record. Clients never call a provider themselves: they stream each model call through the service (ai.languid.model.stream), which uses its own credentials. That keeps keys in one place and lets the service meter use per owner.

Self-hosters bring their own keys. The hosted service can give an owner a budget in US dollars, metered from each call's cost; per-owner keys are on the roadmap.

Work items #

The first slice is implemented for record-backed wakeups: one-off/fixed-interval schedules and opt-in background-job reviews produce private event records. An event without a linked successful assistant response is pending work. A single dispatcher executes them. Machines build on it: owner-made workers (Memory keeper first) whose runs are wakeups with a short tool loop under narrowed permissions, parking on approvals; external claims are not implemented yet.

The broader design remains planned, and is the direction the runtime is heading. A work item is one piece of pending agent work: a tool call with no result, or a thread whose next step is a model call. Pending is derived from which records exist; nothing records a status, so a result and its "done" mark can never disagree.

Any worker that can read pending items and write results can move an agent forward: the service, the owner's own computer waking once a day, a provider's batch API, a pool of spare compute. The service arbitrates claims, so it remains the only writer. Each item can carry a latency class, which lets an owner trade speed for cost per item. Interactive chat is the same records, claimed immediately.

Today only tool calls can be pending. A call waiting for the owner's yes is a record pair, ai.languid.approval.request and the owner's ai.languid.tool.approval (connectors.md), so it outlives a restart; the rest of the queue is held in the service's memory. The roadmap has the steps. connectors.md proposes the first of them: one queue per thread derived from records, with the result of a tool call as its approval.

Moving an agent #

An agent moves to another host as an account migration: its public repo goes as a CAR file and its DID document points at the new PDS. Its space travels separately: the Spaces alpha exports a member's repo in a space (com.atproto.space.getRepo / importRepo) and the space's host-side state (exportSpaceMeta / importSpaceMeta). languid doesn't use these yet.

Out of scope for now #

  • End-to-end encryption of records.
  • Model training or fine-tuning.

Open questions #

  • Where the search index runs: in the service, on the PDS host, or on the owner's computer.
  • Access control beyond two members: when several agents and people share a space, whether it needs an arbiter layer of its own.
  • Spaces maturity: the alpha has no backups and may still change before its beta.