radial
A collaborative coding-agent orchestrator for atproto radl.app
Star
Fork
Watch
radial docs operators.md
18 kB · 374 lines
Markdown
at tangled-round-integrity

Running a daemon alongside somebody else's #

Radial's claim is that agents from different operators can share a space without shared infrastructure — the atproto network is the whole bus. This is the document for the second operator: what changes when you are not the only daemon watching a space, what the knobs mean, and how to tell whether your view of the space and somebody else's agree.

Everything before this point works with one operator. docs/running-an-agent.md is the end-to-end walkthrough for that, and none of it changes here. docs/radial-json.md is the field-by-field configuration reference; this document is about the decisions behind the fields.

1. Joining somebody else's space #

A space is a trust boundary (design §3). Its creator is the root of trust, and membership is an append-only grant log in the admin's own repo — so joining is something the admin does, not something you do.

  1. You create the agent identities. They are ordinary atproto accounts on any PDS. radiald init never mints a DID; it registers an existing identity and publishes an agent record describing it.

    radiald config init your-agent.example      # writes ~/.config/radial/radial.json
printf '%s\n' "$APP_PASSWORD" | radiald init --password-stdin

  2. The space admin adds each agent DID as a member with kind: agent:

    radial member add --profile admin --space "$SPACE" --did did:plc:… --kind agent --role agent

    Until they do, every record your agents write is ignored by every materializer in the space, including your own. That is not a failure mode to debug: it is the trust rule working. radiald index <space> --records … shows them under ignored with the fold's own reason.

  3. You point your daemon at the space and run it:

    { "run": { "spaces": ["at://did:plc:theirs/com.disnetdev.radial.space/example"] } }

    radiald run

Your daemon now polls every active member's repo, including the other operator's. Nothing is shared but the records.

What you do NOT need #

No shared database, no shared queue, no coordinator, no agreement about intervals or images, no network path between the two daemons. Neither operator can see the other's ledgers, and neither needs to.

1a. The second daemon on your machine #

§1 is about the second operator. The same-machine case — two daemons, two agent identities, one host — is a different question, and the answer is not "run two daemons": several identities under one daemon already work, because each profile may carry its own identifier, and one process serves them all.

Run a second daemon when the two need to differ in something a profile cannot express: a separate lifecycle (restart one without the other), a different GitHub account, a different provider key, a different poll interval.

Then give each instance its own config naming its own data directory, and let --config select it:

radiald config init agent-b.example --config ~/radial/b.json --data-dir ~/radial/b-state
printf '%s\n' "$B_PASSWORD" | radiald init --config ~/radial/b.json --password-stdin
radiald run --config ~/radial/b.json

Under systemd, each unit carries its own environment — this is the part no config file can hold, because it is credentials:

[Service]
ExecStart=/usr/bin/radiald run --config /home/op/radial/b.json
Environment=GH_TOKEN=…
Environment=ANTHROPIC_API_KEY=…

gh auth token is per user, not per instance, so two daemons acting as two GitHub accounts each need GH_TOKEN in their own process — the daemon prefers it over gh when it is set. Profile names need not differ between instances; data directories must. radiald init refuses to re-register a profile under a different DID (naming both, and --replace-identity if you meant it), radiald run refuses to start when a configured DID does not match its stored session, and only one daemon may hold a run state directory at a time.

The field reference for all of this is radial-json.md, "More than one daemon on one machine". Note that two daemons polling one space duplicate the ingestion traffic — inherent to running two, not a cost of separating their state.

2. Routing: which of your agents answers what #

An agent only accepts requests for types it has an assignment for (design §11). That assignment is a profile in your radial.json, published as an agent record by radiald init:

{
  "identifier": "your-agent.example",
  "harness": "claude",
  "models": ["claude-opus-4-8=high"],
  "agents": {
    "planner":     { "artifactTypes": ["plan", "adr", "architecture"] },
    "implementer": { "artifactTypes": ["implementation"] },
    "reviewer":    { "harness": "pi", "models": ["openai/gpt-5=high"], "artifactTypes": ["review", "answer"] },
    "second-reviewer": { "harness": "codex", "models": ["gpt-5.6-sol=high"], "artifactTypes": ["review"] }
  }
}

Three things follow from that shape:

  • The agent RECORD is what a human's assignee menu offers, not this file. Editing artifactTypes and not republishing changes nothing anybody can see; radiald init <profile> --update is what publishes it. radiald run warns at startup when the two have drifted apart.
  • harness and models are per profile. That is the routing table: a pi or codex reviewer on one provider beside a claude implementer on another, under one identity. An unknown harness name is refused at startup, never silently downgraded to a different one.
  • A registry type nobody accepts sits open. A space admin can add a type by writing a record, and every client grows a button for it — but no agent grows. radiald run warns once per space about types no agent in the space accepts. Somebody else's daemon covering a type is a perfectly good answer; the warning is about types nobody covers.
  • review and answer are built-ins, and get no warning. Neither is registry data, so neither is counted by that coverage check — but a profile still has to list the name to be offered the work. Drop answer and the UI's "Ask an agent to reply" picker is simply empty in every space you serve.

3. Claims and leases #

A request with an assignee goes to that agent, and nothing in this section applies to it. A request with no assignee is open to whoever claims it, and claims are how two daemons avoid both answering it.

The protocol #

  1. Your daemon writes a claim record: the request, a lease expiry, the moment the lease was issued (renewedAt), and a createdAt.
  2. It waits until it has ingested its own claim and seen it win the tie-break — earliest createdAt read as an INSTANT (one instant has several legal RFC-3339 spellings), then lowest record URI by CODE POINT (not by locale collation). Every materializer computes that identically, which is what makes the answer the same on both operators' machines.
  3. Only then does it start a turn. It renews the lease while the turn runs.
  4. If somebody else's claim wins, it stops: the ledger row reads lost, and any container it had already started is killed.

Step 2 is why an open request takes one extra ingestion cycle to start. It is also what bounds wasted work: the most two daemons can duplicate is one cycle's worth.

The knobs #

Full table in docs/radial-json.md. What to actually change:

Symptom Change
Open requests take too long to start Lower --interval first (the confirm wait is measured in ingestion cycles). Do not shorten the lease to compensate.
Claims are being lost and reclaimed in a loop Your lease is too short for your poll interval. radiald run refuses leaseMs < 4 × interval outright; if you are near that line, raise the lease.
A daemon claims more than it can start Lower maxOutstanding, or raise concurrency. Claims are already capped by free turn capacity.
Claim visibility between operators is slow Raise confirmCycles, not the lease.

maxOutstanding and the confirm window are counted per space: a daemon polling two spaces runs the protocol independently in each, and its claim rows are reconciled only against the space they were made in. What bounds the daemon as a whole is concurrency — its free turn slots.

radiald claim reset <request-uri> clears where this daemon's claim rows for a request stand — the claim-side parallel of radiald turn reset. It forgets the protocol state, not the generations already spent (see "Claims and clocks" above): a request at the generation cap gets another sixteen, starting above the records it already wrote.

A daemon that restarts mid-lease #

The claim rkey is a pure function of the request and the claim's generation, and the claim ledger is on disk, so a restart re-derives the same record and renews it. It does not write a second claim, and it does not re-enter the race. A daemon that lost its ledger walks the generations from zero and lands on the same record, adopting the first one it could still use.

4. Jetstream #

Optional. Off by default.

{ "run": { "jetstream": { "endpoint": "wss://jetstream2.us-east.bsky.network/subscribe" } } }

Polling reads whole repos on an interval; Jetstream is the firehose filtered down to Radial's collections and the space's member DIDs, so another operator's record arrives in seconds instead of at your next poll. Three things are worth being explicit about:

  • It is a latency optimisation, never a correctness dependency. Polling stays the authority and the backfill. An endpoint that is down costs latency and nothing else — which is what makes it safe for it to be an instance somebody else runs.
  • Degraded mode is a first-class state. A socket that is open and silent for longer than two poll intervals is reported degraded, and polling returns to the short interval. It is not an error; a quiet stream and a dead stream look the same from outside, so the daemon assumes the worse one.
  • Deletes are counted and never applied. Radial has no record tombstones: the poller cannot express one either, and a delete that only stream-fed observers honoured would be a rule two materializers disagree about. It is a known gap, deliberately (design §12).

While the stream is healthy, polling drops to backfillIntervalMs (60 s by default). A new member is invisible to a live subscription until it reconnects with the new DID filter — the poller covers them in the meantime, which is one more reason polling never goes away.

5. Divergence: do our two views agree? #

Two independent folds of the same records must produce the same index. Nothing enforces that — there is no coordinator — so the way to check is for both sides to fingerprint what their fold decided and compare.

radiald index "$SPACE" --records ~/.local/share/radial/run/records.db --digest

{
  "space": "at://…",
  "overall": "9f2c…",
  "sections": {
    "members": "…", "openRequests": "…", "winningClaims": "…",
    "chains": "…", "records": "…"
  }
}

The web app shows the same digest in its Diagnostics disclosure, so an operator and somebody reading the space in a browser can compare by eye.

Equal digests mean both folds decided the same thing.

Different digests mean a sync gap, not a consensus failure (design §12). One side has ingested something the other has not. The section that differs says where to look:

Section What differs
members A membership grant or removal one side has not seen. Check this first — it changes what every other section counts.
openRequests A request, or a fulfilment/retraction of one.
winningClaims Who holds what. Usually transient — either a claim race, or one side having seen a lease arrive before the other, since a lease is honoured from first sighting (§3). Persistent disagreement is worth investigating.
chains Version chains — an artifact one side is missing.
records Everything else. A difference here with the others equal is an ordinary ingestion lag.

Two things the digest deliberately ignores, so that it does not flap: a live claim's CID, expiry and renewedAt (renewed every few minutes, so two observers legitimately hold different bytes at any instant — the tie-break's decision is what is fingerprinted instead), and the two observer-local stamps on every record — the repo revision it was observed at, and when this machine first saw it.

If the digests disagree and stay disagreeing, radiald index reset clears the local record store and re-ingests from scratch. It leaves the turn, check and claim ledgers alone. Stop radiald run first.

6. Tuning against real latency #

The defaults here were chosen against the failure drills, which run on a fake clock. Before trusting them on a live pair of PDSes, measure two numbers with run.claims left at its defaults:

  1. Claim visibility — from a human posting an open request to the other operator's daemon seeing your claim.
  2. Confirmation — from your claim being written to your own daemon marking it held.

Both are visible in radiald run's log (claimed …, claim confirmed …, and on the other side claim lost …). If confirmation takes more than about twice your poll interval, raise confirmCycles rather than shortening the lease: the lease is what protects work in flight, and the confirm window is what protects against duplicate work.

Status: this measurement has not been taken against two live PDSes yet. The defaults are the drill-derived ones (10-minute lease, renew every 200 s, one confirm cycle, one outstanding claim). Record the numbers here when you run it, and adjust the shipped defaults before tagging a release that claims otherwise.

7. The walkthrough #

The Phase 7 exit criterion, as a thing to actually do:

  1. Two operators, two machines, two model providers — one claude profile, one pi profile. One space; neither operator has any infrastructure in common with the other.
  2. A human posts an open (unassigned) review request on a landed artifact.
  3. Both daemons see it. Both may claim it. Exactly one claim wins the fold on both operators' indexes.
  4. The loser's ledger reads lost within one ingestion cycle, and it never launches a container — or kills one within a cycle if it had already started.
  5. The winner produces the verdict, and it appears in the other operator's UI with correct attribution.
  6. radiald index --digest matches on both sides once ingestion settles.
  7. Repeat with the roles reversed — stagger which daemon starts first — to confirm neither side is privileged.

Steps 3–6 are covered offline by the failure drills in packages/daemon/test/multi-operator.test.mjs: crash mid-turn, competing claims, non-member injection, removed-member-mid-claim, and digest divergence. They run on two independently-configured daemon stacks over one fake network, with no Docker and no wall-clock waits.

The same file's F6–F8 give each operator its own CLOCK as well, which is the only way to ask the questions §3's "Claims and clocks" answers: a reader ten minutes fast, a writer that jumps a week forward mid-lease (and the poisoned record it leaves behind), and the tie-break a slow clock wins every time.

8. Known gaps #

  • No record deletes anywhere. The poller never removes a record and stream deletes are ignored, so a deleted record lives forever in every index. Deterministic claim rkeys bound claim accumulation to one record per (agent, request); tombstones are a follow-up needing changes to the record store's collapse rule and a delete on the repo writer.
  • A lease cannot be shortened while the fold still honours it. See §3. Release is passive; the only sanctioned shortening is a repair of a version that is already out of contract.
  • The claim tie-break is self-reported and uncorroborated. Earliest createdAt, then lowest record URI — and the URI half is weaker than it reads: the rkey is claim-<hash of the request>, identical for every operator, so it reduces to lowest DID. A daemon whose clock runs behind its peers therefore wins every race, forever, and a fixed DID ordering breaks the remaining ties the same way every time. The damage is starvation and unfairness, not divergence — membership gating still holds and the loser abandons cleanly — which is why it ranks below the lease bounds that shipped in this release. Closing it means reintroducing a corroborating time source (rev is observer-local by construction; the rkey carries no time), which is its own decision with its own wire-contract change. radiald run warns about the skew it can measure; the drill that asserts the current behaviour is F8 in packages/daemon/test/multi-operator.test.mjs.
  • A claim ingested long after it was written gets a full lease from first sight. Bounded by one lease, and strictly better than honouring the writer's clock — but radiald index reset re-ingests from scratch, so it restarts every live claim's lease on this machine.
  • Jetstream's DID filter is fixed per connection. A new member is covered by polling until the subscription reconnects.
  • Container behaviour is not exercised by this repo's test gate. No harness's argv is validated against a real binary in CI; see the header of docker/Dockerfile.