Open source agent hosting on the AT Protocol
languid docs hosting.md
18 kB · 224 lines
Markdown
at main

Hosting languid on Fly.io #

A hosted setup keeps agents running while your own computers are off. This guide sets one up on Fly.io with three apps, all built from the repo root with configs in deploy/fly/. The configs are the ones languid.ai runs; change the app names, domains and region to your own. (The build context is filtered by /.dockerignore, an allowlist.)

config role public?
pds.toml + pds.Dockerfile the Spaces-alpha PDS: owner and agent accounts and their spaces yes, HTTPS
host.toml the languid service: runs agents, owns their records optional: tailnet only, or public with sign-in
runner.toml a computer that runs tools (pi's bash, read, edit, …) for the service no: it dials out to the service

host.toml and runner.toml share one image (deploy/fly/Dockerfile). deploy/fly/languid.sh joins the tailnet (TS_AUTHKEY), then runs the role set by LANGUID_ROLE.

 owners (pi, languid, browser) ──HTTPS or tailnet──▶ service ◀──long poll── runners (Fly, owners' computers)
                                                        │  HTTPS
                                                        ▼
                                                       PDS ──▶ plc.directory
                                                        │
                                                 model provider

Making this one image and one command is on the roadmap.

Secrets #

Never in the repo, never in records. Keep one source for every app: a Bitwarden Secrets Manager project (below), or the file ~/.config/languid/hosted.env (mode 0600, KEY=value lines). just secrets <pds|host|runner> <app> stages the keys each app needs, so every value is entered once. Fly secrets are per-app and can't be read back, so apps can't share them directly.

secret app what
PDS_JWT_SECRET, PDS_ADMIN_PASSWORD, PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX PDS generate them yourself (see Bluesky's PDS docs)
LANGUID_PDS_ADMIN_PASSWORD (= PDS_ADMIN_PASSWORD), LANGUID_SERVER_TOKEN service the service creates accounts with the PDS admin password; the server token is the installation's own token (random, 32 bytes or more)
TS_AUTHKEY service, runners a Tailscale OAuth client secret (below)
*_API_KEY (OPENROUTER_API_KEY, ANTHROPIC_API_KEY, RADIUS_API_KEY, …) service model provider keys
STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET service optional: selling credit and plans on /billing (see Credit and billing)
MODAL_TOKEN_ID, MODAL_TOKEN_SECRET service optional: cloud sandboxes on Modal
LANGUID_SERVICE_URL, LANGUID_JOIN_CODE runners set by hand per runner (see Runners); the code works once

To add a key without it passing through anyone else: echo 'OPENROUTER_API_KEY=…' >> ~/.config/languid/hosted.env.

Bitwarden #

With Bitwarden Secrets Manager set up, the project is the source and hosted.env is only the fallback. Install bws from Bitwarden's releases (it isn't in Homebrew), make a machine account with access to one project, then:

security add-generic-password -U -s languid-bws -a "$USER" -w    # paste the access token at the prompt
echo 'BWS_PROJECT_ID=<project id>' > ~/.config/languid/bws.env && chmod 600 ~/.config/languid/bws.env
just secrets import    # hosted.env and inkling-modal.key into the project; rerun any time, it only changes what differs
just secrets check     # key names in Bitwarden, hosted.env and Fly side by side
command does
just secrets host languid-nimbus [--dry-run] stage the host's keys on Fly (also pds, runner); they take effect on the next deploy
just secrets modal [--dry-run] set the Modal secret inkling-api-key from INKLING_API_KEY
just secrets run -- <cmd> run a command with every secret in its environment, instead of pasting hosted.env
just secrets names which source is in use, and its key names

Every command says when it fell back to hosted.env. If Bitwarden is configured but unreachable, it stops rather than using a file that may be stale. LANGUID_SECRETS_SOURCE=hosted.env forces the file. Fly-only keys set by hand (NIMBUS_*, OPENROUTER_API_KEY, RADIUS_API_KEY, TS_AUTHKEY, LANGUID_SERVER_TOKEN on languid-nimbus) can't be read back from Fly; add them to the project with bws secret create to make it complete.

The PLC rotation key can rewrite every account's DID document. Keep a copy somewhere safe, such as a password manager: Fly secrets can't be read back.

Setting it up #

1. Tailscale #

The configs put the service and runners on a tailnet: the installation token is accepted there, and runners can reach a service that has no public address. They join as tagged nodes using one OAuth client secret as TS_AUTHKEY, which doesn't expire, and keep their Tailscale state on a volume, so they log in once.

  1. Tailnet policy file: add a tag (the configs use tag:nimbus) with "tagOwners": { "tag:nimbus": ["autogroup:admin"] }. If the policy restricts traffic beyond the default allow-all, allow your devices and the tag to reach the tag on tcp 2590 (the service). Runners need no inbound port.
  2. Admin console → Settings → OAuth clients → Generate: scope Auth Keys (write), with that tag.
  3. Add the tskey-client-… secret to hosted.env as TS_AUTHKEY.

Without TS_AUTHKEY, the service listens on all interfaces at LANGUID_URL, for setups that put it behind TLS themselves. A plain auth key (tskey-auth-…) also works if you clear TS_TAGS in the toml.

2. PDS #

The PDS uses the real PLC directory (plc.directory), so DIDs are public and permanent. Handles are <name>.<your domain>. It's invite-only with crawlers off, so nothing is announced to relays. Fly takes daily volume snapshots and keeps five. pds.Dockerfile hands the volume to the node user and drops root before starting the PDS.

fly apps create <pds-app>
fly volumes create pds_data -a <pds-app> -r <region> -s 1
deploy/secrets.sh pds <pds-app>
fly deploy . -a <pds-app> --config deploy/fly/pds.toml --ha=false
fly certs add pds.<domain> -a <pds-app>
fly certs add '*.<domain>' -a <pds-app>     # handles

Set PDS_HOSTNAME and PDS_SERVICE_HANDLE_DOMAINS in pds.toml to your domain first. Then add the DNS records below and wait for both certificates before creating accounts: DIDs on plc.directory can't be deleted.

If a deploy fails with "insufficient resources … existing volume", the volume's host is full: create a new empty volume with the same name and destroy the old one (only while it's empty).

3. Service #

fly apps create <service-app>
fly volumes create nimbus_data -a <service-app> -r <region> -s 1
deploy/secrets.sh host <service-app>
fly deploy . -a <service-app> --config deploy/fly/host.toml

# the installation owner, and a first agent
fly ssh console -a <service-app> -C "languid init you.<domain>"
fly ssh console -a <service-app> -C "languid admin publish-lexicon"
fly ssh console -a <service-app> -C "languid agent create scout --model openrouter/deepseek/deepseek-v4-flash"

publish-lexicon publishes the agent space type from the owner's repo and prints the TXT record to add (_lexicon.space.languid.ai for languid's NSIDs). Without it, every OAuth sign-in is refused. After that, the service republishes it itself whenever it starts with a changed collection list, so a deploy that adds or renames a collection needs no step of its own. publish-lexicon needs the owner's password, so it runs on the service's host, not from a client signed in with languid login. publish-lexicon eval publishes the eval lexicons the same way (_lexicon.eval.languid.ai); the service doesn't publish those itself.

Models. The service calls models with its own keys: any provider pi-ai supports, from the *_API_KEY secrets. languid models lists what's available. Subscription logins work only where pi's auth.json is, so not on the host.

4. A public address and sign-in #

Optional: without it, the service is reachable only over the tailnet with the installation token. With it, owners sign in with their atproto accounts and you can invite people.

  1. In host.toml, set LANGUID_URL=https://<host> and keep the [http_service] section. atproto authorization servers fetch https://<host>/oauth/client-metadata.json, so it must be HTTPS on port 443. With an https:// URL, languid.sh starts the service on every interface (tailnet included) so Fly's proxy can reach it.
  2. Set LANGUID_CLIENT_IP_HEADER=fly-client-ip, so rate limits see each client's address rather than the proxy's, and requests through the proxy are never taken as local.

To choose which models run on your keys, set LANGUID_HOUSE_MODELS to comma-separated provider/model values, in order. The pickers show them first, under Picked by Languid, and they become the only models anyone runs on your keys and credit, you included. Every other model needs the owner's own connection (ChatGPT, Grok, Anthropic or OpenRouter). LANGUID_GUEST_MODEL sets the model new agents get (together/thinkingmachines/Inkling-2-Small on languid.ai), and stands in for LANGUID_HOUSE_MODELS when that's unset. With neither set, your keys run any model. 3. fly certs add <host> -a <service-app> and point DNS at the service's addresses (fly ips list). 4. Invite owners: see Owners, invites and credit. 5. Optional, for agents' sites: set LANGUID_SITES_HOST=sites.<domain>, run fly certs add sites.<domain> -a <service-app>, and add the sites record below. Use a name of its own, never the service's host: sites run other people's HTML and scripts. Without it, sites are served only by a service on loopback, at /_sites/.

The installation token then works only from the service's host and the tailnet, so fly ssh console … languid … and tailnet clients keep working. https://<host>/ is the landing page, so the host can be the site's own name; reserved names (www, pds, api, …) can't be taken as handles.

5. Clients #

curl -fsSL https://<host>/install | sh   # installs the CLI and signs in to this service
languid login --url https://<host>   # with a public address; the browser asks which account
languid chat scout
languid computer setup                             # let agents run tools on this computer

Without a public address, a client on the tailnet uses the installation token instead: ~/.languid/state.json with {"pds": "https://pds.<domain>", "agents": {}, "runners": {}, "server": {"url": "http://<service node>.<tailnet>.ts.net:2590", "token": "<LANGUID_SERVER_TOKEN>"}}.

6. Runners on Fly #

languid computer invite --agent scout --ttl 60   # prints a join code
fly apps create <runner-app>
fly volumes create runner_data -a <runner-app> -r <region> -s 1
deploy/secrets.sh runner <runner-app>
fly secrets set -a <runner-app> --stage LANGUID_SERVICE_URL=<service URL> LANGUID_JOIN_CODE=<code>
fly deploy . -a <runner-app> --config deploy/fly/runner.toml

The runner joins once: its registration and tailnet node live on its volume, so restarts and redeploys need no new invite. To re-join (after computer remove, say), delete /data/nimbus/state.json on it and set a new code.

Owners, invites and credit #

Sign-up is by invite. An invite is a link to the service's /signup page, where the invitee picks a handle and a password. The service makes their account on its own PDS and lets them sign in.

Make invites on /admin, signed in as the installation owner, or from the service's host:

languid admin invite-owner --budget 10 --uses 1 --days 7   # prints the link
languid admin owners                                        # who signed up, their agents, and what they've spent
languid admin budget alice.<domain> 20                      # set someone's credit; none makes it unlimited
  • --budget is the credit, in US dollars, that the invite gives each person. Leave it out for unlimited use of your keys.
  • --uses is how many people one link signs up. For a room, make one link with --uses 30; /admin also shows its QR code, sized to project.
  • --days is how long the link works, 7 by default. --model provider/id picks the model their new agents start on.
  • The service keeps only a hash of each code. If making the account fails (a taken handle, a PDS outage), the link still works.
  • The PDS has no email, so nobody can reset a password by email. com.atproto.admin.updateAccountPassword on the PDS is the way back in.

/admin lists every owner with what they've spent and their credit, and can add credit, set it, make it unlimited, and give or take back a cloud sandbox and volume. It also shows the sign-up funnel.

Credit and billing #

Each model call on your keys costs the provider's price plus 5%, and the service adds that to the owner's spend when the call ends. An owner whose credit is used up gets 402 on calls to your keys until they have more. Calls on their own connections never count.

To let owners buy credit on /billing, set STRIPE_SECRET_KEY and STRIPE_WEBHOOK_SECRET, and point a Stripe webhook at https://<host>/stripe/webhook with these events: checkout.session.completed, checkout.session.async_payment_succeeded, charge.refunded, charge.dispute.created, and customer.subscription.created, .updated and .deleted. Credit comes in packs of $10, $25 and $50, or any amount from $10 to $500. STRIPE_TAX=on adds Stripe Tax, once you've set it up in Stripe. Without the Stripe keys, /billing shows credit and says buying isn't open yet. payments.md has the full setup.

With MODAL_TOKEN_ID and MODAL_TOKEN_SECRET set too, /billing sells a cloud sandbox ($10 a month, 100 hours) and a volume for it ($3 a month, 10 GB).

What has to stay running #

  • The service has to stay on. Running turns, runner connections, the wakeup clock and the Discord bots all live in its one process. If it stops, runs end, and every computer and bot looks offline. It runs on one machine, because its queues and write locks are in memory and state.json is on one disk.
  • The PDS is on, and could idle. All of its state is on disk, so stopping it would lose nothing. In practice it would rarely sleep: the service writes to it on every step, and other atproto services look the agents' accounts up through it.
  • Runners are up to their owners. A runner works only while it's running. It connects out, so nothing can wake it: a stopped runner shows as offline, and calls for it fail. Cloud sandboxes are the exception: the service starts one on Modal when a call needs it.
  • Model providers bill per call, with nothing to keep running.

DNS #

For a domain whose apex is the service and whose subdomains are handles, as languid.ai's is:

type host value
A / AAAA pds the PDS app's addresses
A / AAAA * the PDS app's addresses (for name.<domain> handles)
CNAME _acme-challenge from fly certs setup '*.<domain>' -a <pds-app> (wildcard certificate)
A / AAAA @ the service app's addresses
CNAME www <service-app>.fly.dev. (the service redirects it to the apex)
CNAME sites <service-app>.fly.dev. (agents' sites, with LANGUID_SITES_HOST)
TXT _lexicon.space did=<owner DID>, from publish-lexicon

Explicit records win over the * wildcard, so every other name under the domain goes to the PDS. Check certificates with fly certs check <name> -a <app>.

Upgrading the PDS image #

Dev (dev/compose.yaml) and hosted (deploy/fly/pds.Dockerfile) pin the same image digest. A new Spaces alpha release usually needs code changes too, so the image and the code move in one commit, and a hosted service gets both close together. The PDS's database migrations are one-way: there's no going back to the old image on migrated data, only to a snapshot.

  1. Snapshot the PDS volume: fly volumes snapshots create <volume-id>, and note its id.
  2. Deploy the PDS. The account database migrates at start; each account's store migrates when it's first opened. Check that https://pds.<domain>/xrpc/_health reports the new version.
  3. Deploy the service right after, so it runs matching code.
  4. Check an agent's records (languid records <agent>) and run a turn.

To go back: restore the snapshot into a new volume, pin the old digest again, and redeploy the PDS and the old code. Anything written after the snapshot is lost. Rehearse each upgrade on the dev stack with existing data first.

Day to day #

The justfile at the repo root has recipes for languid.ai's apps (just lists them); set host_app, runner_app and pds_app at its top to yours. Deploys refuse to run from anything but a clean, committed main.

task command
test, then deploy the service, the Fly runner and this computer's runner just deploy
only the service / a Fly runner / the PDS just deploy-host / just deploy-runner [app] / just deploy-pds
the service, runners and computers at a glance just status
logs just logs host (or runner, pds, local)
a languid command on the service host just remote agent list
stage an app's secrets just secrets host <service-app>
invite someone languid admin invite-owner --budget 10, or /admin
tear down a runner languid computer remove <name>, then fly apps destroy <app>

Costs on Fly: a shared-cpu-1x 512 MB machine and a 1 GB volume per app, a few dollars a month each.