code-browser docs integrations

2 files

at main

mads.havmand.eu · An attached destination's threads in the code and under Review (cb-dv5.7) · 1mo ago

d301bb40

github.md

GitHub fetch: the pull request's description and its head's checks (cb-dv5.3)

1mo ago

README.md

An attached destination's threads in the code and under Review (cb-dv5.7)

1mo ago

README.md

Integrations #

Status: draft, part of code-browser/review@2.

This document specifies how reviews are exported to other systems and how the app fetches what happens there. The review model and the store are in REVIEW-FORMAT.md ("format §N" below). Each adapter has its own document: github.md. It does not specify a user interface.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be read as in RFC 2119.

1. Scope #

This release does three things:

Export. The user sends one submitted review, with its threads, to one destination, once (§6).
Attach. The user attaches a destination to the change they are reviewing, without exporting anything (§3.1). Opening a pull request's URL attaches it.
Fetch. For a destination a review was exported to or that is attached, the app reads the destination itself (its description and checks) and everything in it (reviews, threads, replies, resolutions, edits and deletions, by anyone), and shows it beside the local review (§7).

Fetching is read-only: it never writes to the destination. The app sends nothing after an export. Replies, resolutions, edits and retractions made in the app are local entries (format §10) and stay local; to answer at the destination, the user goes there. Sending them is later work (§9).

2. Terminology #

Term	Meaning
adapter	Code that maps reviews onto one external system, named by a lower-case `system` such as `github`.
destination	One place at a system that reviews are exported to or that is attached, such as one pull request (§3).
export	Sending one review, with its threads, to one destination (§6).
attachment	The record that a change (a repository and a change key, format §11.2) is shown with a destination, whether or not anything was exported there (§3.1, §5.5).
description	The destination's own text, author and state: for GitHub, the pull request's body (§5.1).
check	One CI result the destination reports for a commit, such as a GitHub check run or a GitLab pipeline job (§5.6).
link	The record that a local review or thread was sent to a destination, with its external id (§5.2).
remote entry	An item fetched from a destination: a review, thread or comment there (§5.3).
viewer	The destination account the app acts as.
marker	A hidden line in a sent body that names the local item (§6.3).

3. Adapters and destinations #

An adapter maps reviews onto one external system: a forge's pull request reviews, tracker issues, SARIF, or (later) git refs. The core never requires one.

A destination is one place at a system, named by the adapter with a canonical locator (for GitHub, <owner>/<name>/pull/<number>) on a host. Each adapter declares the syntax of its locators and external ids. A host is lower-case ASCII (IDN names in punycode), with no port and no user info, and is always reached over https.

Only submitted reviews are exported. A review is exported to at most one destination, once. Several reviews MAY share one destination (a review and its re-review, on one pull request).
Exporting is always an explicit action by the user that names the destination. The system, host, locator and account come only from the user's choice or the app's own settings. A value derived from the clone (its remotes, files or config) MAY be offered as a suggestion, shown in full, and is never used without the user confirming it. Repository settings never set any of them (SETTINGS.md).
A review is linked to a destination when it has a link there or an export there that is not cancelled.
A destination is in use when a review is linked to it or it has an attachment (§3.1). Only destinations in use are fetched, apart from planning (§6.1).
Each adapter declares what it can do: export, fetch, describe (read the destination's description, §5.1) and checks (§5.6). An adapter that cannot export can still be attached and fetched.

An adapter that can be opened by URL also declares:

Declaration	GitHub	GitLab (for example)
its URL syntax: which URLs name a destination, and the `host` and `locator` each names	`https://<host>/<owner>/<name>/pull/<n>`	`https://<host>/<group>/<name>/-/merge_requests/<n>`
the destination's canonical URL, from `host` and `locator`	the same, without a suffix	the same
the refs that hold its head and base, fetched from the repository's remote when it is opened, and the remote-tracking ref the head goes to	`refs/pull/<n>/head` → `refs/remotes/<remote>/pull/<n>`	`refs/merge-requests/<n>/head` → `refs/remotes/<remote>/merge-requests/<n>`
its names: the system's display name, what a destination is called, the short kind for a tab, and the short form of a locator	GitHub · pull request · `pr` · `#1287`	GitLab · merge request · `mr` · `!342`

The app tries each adapter's URL syntax on what is typed in the omnibar; opening one is the same flow for every system: find or clone the repository, fetch the declared refs, open the subject, attach (§3.1) and show the destination's tab. The core never names a system, a kind of destination or a ref layout of its own; it shows the adapter's names.

3.1 Attachments #

An attachment says: the change with this repository and change key is shown with this destination. It is how the user sees a pull request's description, checks and discussion while reviewing it, before, or without, exporting there.

The change key is the subject's, as format §11.2 defines it: for a subject opened from a destination's URL, that destination's canonical URL (§3), lower-cased; for a change review of a branch, the branch. A subject without a change key, and an audit (base = head), cannot be attached.
A change has at most one attachment. A destination MAY be attached to several changes of its repository (a branch, and the same pull request opened by URL).
Attaching is an explicit action, and the rules of §3 apply: the system, host and locator come from the user's choice. Two actions attach:
- Opening a destination's URL (any adapter's URL syntax, §3, typed or pasted in the omnibar). The URL is the user's choice; it is attached with no further question, and confirmed is set.
- Attach on a change review. The app MAY suggest destinations from the clone's remotes, the one whose head is the subject's head first; each is shown with its system, host, locator and account, and nothing is attached until the user picks one. The pick sets confirmed.
Detaching deletes the attachment. The destination's row and its remote entries and checks are then deleted if nothing else uses it (§3): no link, no export, no other attachment. Detaching never touches reviews.
An attachment changes nothing about reviews: no review or draft is created by attaching, and a draft's change key is not changed by it. Browsing a pull request by its URL therefore leaves an attachment and the fetched copy in the store, and nothing else.
Exporting a review of an attached change offers the attached destination first (§6.1). The user still confirms the plan.
A restore clears confirmed on every destination (§5.1), attached ones included.

4. Authority #

Data	Authoritative	Rule
reviews, threads, evidence, coverage, entries	the store	never changed by a fetch
what exists at a destination: its description, checks, reviews, threads, comments, their text and state	the destination	the fetched copy is overwritten by each fetch
which destination a change is shown with	the store (attachments)	changed only by the user attaching or detaching (§3.1)
a thread's status in the store	local `resolve` entries	a remote resolution does not create or change one
a linked thread's status at the destination	the destination	shown beside the local status
a linked review's standing and verdict	the store, for the local standing; the destination, for the verdict it shows	shown side by side

A linked item therefore has two states: its local state, and its state at its destination. Readers show both where they differ (for example "resolved here, open on GitHub", "edited on GitHub", "deleted on GitHub", or "superseded here, still requesting changes on GitHub"). Neither is folded into the other.

5. Tables #

These tables are part of the store and follow format §4: its conventions (-- time, -- oid, -- bool, -- json stand for the checks of format §4.3), migrations, transactions and backups. Migration 1 creates §5.1 to §5.4 without the description columns of §5.1; migration 2 adds those columns, and §5.5 and §5.6.

5.1 Destinations #

CREATE TABLE destinations (
  id           INTEGER PRIMARY KEY,
  repo         TEXT NOT NULL REFERENCES repos(id),
  system       TEXT NOT NULL CHECK (system NOT GLOB '*[^a-z0-9-]*' AND system <> ''),
  host         TEXT NOT NULL CHECK (host NOT GLOB '*[^a-z0-9.-]*' AND host <> ''),  -- §3
  locator      TEXT NOT NULL,            -- adapter-defined, canonical (§3)
  url          TEXT,                     -- §5.3 URL rule
  title        TEXT,
  head         TEXT,                     -- oid: the destination's head at the last fetch
  viewer       TEXT,                     -- the account's public name, e.g. its login
  viewer_id    TEXT,                     -- the account's stable id at the system
  confirmed    INTEGER NOT NULL DEFAULT 0,  -- bool: the user confirmed this destination
  generation   INTEGER NOT NULL DEFAULT 0,  -- §7
  fetched      TEXT,                     -- time of the last complete fetch
  fetch_error  TEXT,
  paused_until TEXT,                     -- time (§6.6)
  data         TEXT,                     -- json: adapter-defined; never an endpoint (§8)
  -- the description (§7 step 1), for adapters that declare `describe`:
  body         TEXT,                     -- markdown as the destination returns it; NULL when it has none
  truncated    INTEGER NOT NULL DEFAULT 0,  -- bool: body was cut to fit
  author       TEXT,                     -- display name, as for remote entries (§5.3)
  author_id    TEXT,
  state        TEXT CHECK (state IN ('open', 'draft', 'merged', 'closed')),  -- as the adapter maps it
  remote_state TEXT,                     -- the destination's own state word, verbatim
  base_ref     TEXT,                     -- the branch it merges into, as the destination names it
  head_ref     TEXT,                     -- the branch it merges from
  created      TEXT,                     -- time, as the destination claims
  updated      TEXT,                     -- time, as the destination claims
  checks_omitted INTEGER NOT NULL DEFAULT 0,  -- checks of the head not stored (§5.6)
  UNIQUE (system, host, locator)
) STRICT;

The description columns follow the bounds and checks of §5.3: body is cut at 1 MiB, the others at 1 KiB, and all of them are untrusted text, displayed as format §5.3 requires. Each complete fetch overwrites them; the store keeps no history of them.

A destination belongs to one repository. A review is exported only to a destination of its own repo.
confirmed is set by confirming an export plan (§6.1) or by re-confirming the destination: an action that shows its system, host, locator and account and sets confirmed = 1. A restore clears it (format §4.6): a restored store may come from anywhere, so no destination in it is fetched until the user confirms it again.

5.2 Links #

CREATE TABLE links (
  id          INTEGER PRIMARY KEY,
  destination INTEGER NOT NULL REFERENCES destinations(id),
  review_id   TEXT NOT NULL REFERENCES reviews(id),
  thread_id   TEXT,
  external_id TEXT NOT NULL,             -- the item at the destination
  url         TEXT,
  sent        TEXT NOT NULL,             -- time
  sent_digest TEXT NOT NULL,             -- 'sha256:' + hex of the body sent, marker removed (§6.3)
  FOREIGN KEY (review_id, thread_id) REFERENCES threads(review_id, id),
  UNIQUE (destination, external_id)
) STRICT;

CREATE UNIQUE INDEX links_local ON links(destination, review_id, ifnull(thread_id, ''));
CREATE INDEX links_by_review ON links(review_id);

CREATE TRIGGER links_same_repo BEFORE INSERT ON links
WHEN (SELECT repo FROM reviews WHERE id = NEW.review_id)
     IS NOT (SELECT repo FROM destinations WHERE id = NEW.destination)
BEGIN SELECT RAISE(ABORT, 'a review is linked only to a destination of its repository'); END;

CREATE TRIGGER links_frozen_update BEFORE UPDATE ON links
BEGIN SELECT RAISE(ABORT, 'a link cannot change'); END;
-- and the same BEFORE DELETE ON links.

A link says: this review, or this thread of it, was sent to this destination and is external_id there.
A local item has at most one link per destination (links_local), and an external item belongs to at most one local item.
Links are written once, when an export is sent (§6.4), and never change or get deleted. A link whose external item was deleted stays; the fetched copy says it is gone (§5.3).

5.3 Remote entries #

CREATE TABLE remote_entries (
  id                 INTEGER PRIMARY KEY,
  destination        INTEGER NOT NULL REFERENCES destinations(id) ON DELETE CASCADE,
  external_id        TEXT NOT NULL,       -- adapter syntax (§3)
  kind               TEXT NOT NULL CHECK (kind IN ('review', 'thread', 'comment')),
  parent_external_id TEXT,                -- a comment's thread, if any; a thread's review, if any
  author             TEXT NOT NULL,       -- display name, as the destination gives it
  author_id          TEXT,                -- the author's stable account id
  edited_by          TEXT,                -- account id of the last editor, when the destination says
  body               TEXT,                -- NULL for a thread with no text of its own, and once deleted
  truncated          INTEGER NOT NULL DEFAULT 0,  -- bool: body was cut to fit
  verdict            TEXT CHECK (verdict IN ('approve', 'request-changes', 'comment')),  -- reviews, as mapped
  remote_state       TEXT,                -- the destination's own state word, verbatim
  resolved           INTEGER,             -- bool: threads only
  resolved_by        TEXT,
  anchor_path        TEXT,                -- as the destination anchors it, checked (below)
  anchor_side        TEXT CHECK (anchor_side IN ('head', 'base')),
  anchor_commit      TEXT,                -- oid, when the destination gives one
  anchor_start       INTEGER,
  anchor_end         INTEGER,
  outdated           INTEGER,             -- bool: the destination says so
  url                TEXT,
  created            TEXT,                -- time, as the destination claims
  updated            TEXT,                -- time, as the destination claims
  first_fetched      TEXT NOT NULL,       -- time
  last_fetched       TEXT NOT NULL,       -- time
  seen_generation    INTEGER NOT NULL,
  missed             INTEGER NOT NULL DEFAULT 0,  -- complete fetches in a row that did not see it
  deleted            TEXT,                -- time the store learnt of the deletion
  data               TEXT,                -- json: adapter-defined, at most 64 KiB
  UNIQUE (destination, external_id),
  CHECK (kind = 'thread' OR resolved IS NULL),
  CHECK (kind = 'review' OR verdict IS NULL),
  CHECK (anchor_start IS NULL OR (1 <= anchor_start AND anchor_start <= anchor_end))
  -- plus a byte limit on every text column: 1 MiB for body, 64 KiB for data, 1 KiB for the rest
) STRICT;

CREATE INDEX remote_by_parent ON remote_entries(destination, parent_external_id);

A remote entry is what the destination holds, by anyone, the viewer included. The fetched copy of an item the app sent is the remote entry with the link's external_id. "By the viewer" means author_id equals the destination's viewer_id; it is compared when needed, never stored.
Bounds. Adapters fit fetched values to the column limits before writing: a body over 1 MiB is cut at a UTF-8 boundary and truncated set; other text is cut likewise. A fetch never fails because an item is too large.
Checks. An anchor path that does not meet format §5.2, or lines that are not a valid range, are stored as NULL, with the raw value in data. A remote anchor is displayed only when its lines are inside the blob. While a thread is not outdated, its lines are those of the destination's current head: adapters map it so (GitHub's line is), and the app places it in the code only when that head is the one shown. A URL column holds only an https URL on the destination's host (or a host the adapter declares); anything else is stored as NULL. The app checks a URL the same way again before opening it.
Remote entries are ordered by created, then external_id.
parent_external_id is text, not a foreign key: a reply may arrive before its thread. Readers resolve it within the destination.
A remote verdict is shown as that account's verdict at the destination. It never counts as the user's verdict (format §11.1).
Edits at the destination overwrite the fetched columns. The store keeps no history of remote text. For an item the app sent, readers compare the link's sent_digest with the digest of the fetched body (§6.3) and show "edited at the destination" when they differ.
Deletions (§7) set deleted and clear body to NULL, keeping the row (its author, times and position) as a tombstone. An item seen again by a later fetch is back: its upsert clears deleted and missed and restores body.
Everything here is untrusted and displayed as format §5.3 requires.

Readers show a remote entry that is a link's fetched copy as the destination state of that local item, not as a separate item. Other remote entries of a destination (other people's reviews, threads and replies) are shown with the reviews linked to it.

A view that needs one order of a thread's comments (such as whether a thread waits on the user) takes the remote entries in their order, then the posted local entries about that thread in pos order. A comment is the user's when it is local or by the viewer.

5.4 Exports #

CREATE TABLE exports (
  id          INTEGER PRIMARY KEY,
  destination INTEGER NOT NULL REFERENCES destinations(id),
  review_id   TEXT NOT NULL REFERENCES reviews(id),
  plan        TEXT NOT NULL,              -- json: the adapter's placement, as the user saw it
  progress    TEXT,                       -- json: steps done, with the external ids they created
  state       TEXT NOT NULL CHECK (state IN ('sending', 'sent', 'failed', 'cancelled')),
  maybe_sent  INTEGER NOT NULL DEFAULT 0, -- bool: an attempt may have created something (§6.5)
  retryable   INTEGER NOT NULL DEFAULT 0, -- bool: the last failure may succeed on retry (§6.6)
  account     TEXT,                       -- account id of the last attempt
  attempts    INTEGER NOT NULL DEFAULT 0,
  last_error  TEXT,                       -- no credentials (§8)
  created     TEXT NOT NULL,              -- time
  finished    TEXT                        -- time
) STRICT;

CREATE UNIQUE INDEX exports_once ON exports(review_id) WHERE state <> 'cancelled';
CREATE INDEX exports_by_destination ON exports(destination);

-- links_same_repo's check, BEFORE INSERT ON exports, and:
CREATE TRIGGER exports_submitted BEFORE INSERT ON exports
WHEN (SELECT state FROM reviews WHERE id = NEW.review_id) <> 'submitted'
BEGIN SELECT RAISE(ABORT, 'only submitted reviews are exported'); END;

exports_once keeps a review to one export. A sent export is the record of what went where, and is kept.

From	To	When
(new)	`sending`	the user confirms the plan (§6.1)
`sending`	`sent`	the adapter confirms every step (§6.4)
`sending`	`failed`	any failure (§6.6), or the app finds it `sending` at start
`failed`	`sending`	the user retries it
`failed`	`cancelled`	the user cancels it, after the adapter has undone what it left unpublished (§6.4)

An export found sending when the app starts was interrupted: it becomes failed, with maybe_sent = 1 and retryable = 1.

5.5 Attachments #

CREATE TABLE attachments (
  repo        TEXT NOT NULL REFERENCES repos(id),
  change_key  TEXT NOT NULL CHECK (change_key <> ''),  -- format §11.2
  destination INTEGER NOT NULL REFERENCES destinations(id),
  attached    TEXT NOT NULL,             -- time
  via         TEXT NOT NULL CHECK (via IN ('url', 'pick')),  -- §3.1: opened by URL, or picked
  PRIMARY KEY (repo, change_key)
) STRICT;

CREATE INDEX attachments_by_destination ON attachments(destination);

CREATE TRIGGER attachments_same_repo BEFORE INSERT ON attachments
WHEN NEW.repo IS NOT (SELECT repo FROM destinations WHERE id = NEW.destination)
BEGIN SELECT RAISE(ABORT, 'a change is attached only to a destination of its repository'); END;

Attaching inserts the repos row for the repository when there is none (format §6.1 inserts it with the first review otherwise).
Attaching a change that already has an attachment replaces it, in one transaction that also applies the detach rule of §3.1 to the destination it had.

5.6 Checks #

CREATE TABLE remote_checks (
  id          INTEGER PRIMARY KEY,
  destination INTEGER NOT NULL REFERENCES destinations(id) ON DELETE CASCADE,
  commit_oid  TEXT NOT NULL,             -- oid the checks are for: the destination's head at the fetch
  name        TEXT NOT NULL,             -- as the destination names it, e.g. "ci / test (node 22)"
  state       TEXT NOT NULL CHECK (state IN ('pending', 'running', 'passed', 'failed', 'skipped', 'cancelled', 'neutral')),
  remote_state TEXT,                     -- the destination's own words, verbatim
  url         TEXT,                      -- §5.3 URL rule
  started     TEXT,                      -- time, as the destination claims
  finished    TEXT,
  pos         INTEGER NOT NULL,          -- the destination's order
  data        TEXT,                      -- json: adapter-defined, at most 64 KiB
  UNIQUE (destination, pos)
  -- plus the byte limits of §5.3
) STRICT;

Checks describe one commit: the destination's head. Each complete fetch of an adapter that declares checks replaces all of the destination's rows in the transaction of §7 step 3. Checks have no tombstones and no history: a check that is gone is simply not there.
Readers show checks with their commit_oid, and say so when it is not the subject's head ("checks are for 9a0b7c2, a newer head").
Only status, name, times and a link are stored. Logs are never fetched: they are the destination's, opened there through url.
A destination holds at most 500 checks: the first 500 in its order. The number of the head's checks not stored (past the cap, or of a kind the adapter does not map) is the destination's checks_omitted, replaced with the checks, so that readers can say "and 12 more".

6. Exporting #

6.1 Planning #

Planning creates the destination row, or reuses the one with the same (system, host, locator), and runs a complete fetch (§7): the user naming the destination confirms it for that fetch. The adapter then computes a plan from the review, its threads and the fetched state. When the review's change is attached (§3.1), its destination is the one offered first.

A review cannot be planned when it is not submitted, already has an export that is not cancelled, or has a repo other than the destination's. The app says why.

The plan has:

The review: what it becomes there, with its verdict (§6.2).
Threads. Each thread that is not private, with its placement: its own item there (on lines or on a file), part of the review's body, or dropped.
The code. "The diff" is the destination's diff at planning time. A thread whose anchor_commit is not in the destination's history is placed as a file or body thread, with a shape row. The plan records the destination's head.
Evidence is not quoted unless the user opts in for this export. When they do, the plan shows the exact text that will be sent for each quoted log.
Supersedes. When the review supersedes one linked to the same destination, the plan says which remote review it replaces, and whether the destination still shows that one's verdict (§4). Only an approval or a request for changes is a verdict there; an earlier comment (or a review whose verdict is gone) stays there as a comment.
Entries. Posted entries about the review are not sent. The plan says how many, as an info row.
A loss report: rows of kind drop (not sent), shape (sent in another form) or info (sent as expected, worth knowing).

The app MUST show the loss report before sending, and MUST NOT send what the user has not seen planned. When the user confirms, in one transaction, the destination's confirmed is set and the export is inserted as sending, and sending starts (§6.4). When the user cancels, the destination's row and its remote entries are deleted if nothing uses it (§3); otherwise the row is kept as it was.

6.2 Mapping declaration #

Each adapter MUST declare, in its code and its document, how it maps:

Local	The adapter declares
change review and verdict	what it becomes, and what happens to each verdict
audit	what it becomes, or that audits are refused (most forges have no equivalent)
thread, by anchor	where it goes: on lines, on a file, in the review's body, or dropped
thread kind, severity, suggestion, symbol	how they are rendered
evidence	how it is quoted when the user opts in, and how much of the log
`generated_by`	how it is shown
coverage, entries, private threads	never sent
each step	which remote state counts as done (§6.5)

and how it fetches:

Remote	The adapter declares
each kind of item it reads	which `kind` of remote entry it becomes, and its `parent_external_id`
review states	the `verdict` each maps to, if any
thread state, anchors, outdated	how they fill the columns of §5.3

Everything a mapping drops or reshapes MUST appear in the loss report.

6.3 Markers and quoted text #

Every body an adapter sends SHOULD carry a marker naming the local target (format §3.4), in a form the destination keeps but does not show, when the destination has one. Markers let the adapter find what it sent when a confirmation was lost (§6.5).

A body carries exactly one marker, as its final line. Only the final line of a fetched body is read as a marker.
Text the user did not write (evidence logs and commands, excerpts, paths, symbol names) MUST be quoted so that it cannot end its quote. Text that may span lines (logs, commands, excerpts) goes in a fence longer than any run of backticks inside it. Paths and symbol names go in a code span likewise, after every control character in them (LF, CR, TAB and the rest) is replaced by a visible escape such as \n, and | by \| inside a table. It then cannot add a marker, a mention, a link, an image or markup.
Digest. The digest of a body, sent or fetched, is sha256: and the hex SHA-256 of its UTF-8 after converting it to LF line endings (format §5.1) and, if its final line is a marker, removing that line and the LF before it. Nothing else is changed. A sent body is digested as sent, with the adapter's prefix lines and after any cap. Adapters SHOULD test the digest on a real round trip to their destination.

A marker proves nothing: anyone can write one. Markers are read only by the check of §6.5. A marker anywhere else is ignored, and a fetch never creates links.

6.4 Sending #

In one transaction: set state = 'sending', account to the viewer's id, and increment attempts.
If maybe_sent is set, run the check of §6.5 first, as the account of the attempt that may have sent.
Before the first step, the adapter reads the destination's head. If it differs from the plan's, the export fails with last_error "head moved" and retryable = 0: the anchors were planned against another diff. The way on is to cancel it and plan again.
The adapter renders each request from the stored rows and plan. Each finished step is recorded in progress, with the external ids it created, in a transaction of its own. An attempt resumes after the last finished step.
On confirmation, in one transaction: set state = 'sent' and finished; insert the links (the review, and each thread placed as its own item, with every thread's placement in progress); and upsert the returned items as remote entries, with seen_generation set to the destination's generation. Then fetch (§7).
On failure, as §6.6 says.

Sending never retries by itself. A failed export is retried or cancelled by the user.

Before an export with progress is cancelled, the adapter MUST undo the remote state its steps left unpublished (for GitHub, delete the pending review). Until that succeeds, the export stays failed, with the reason in last_error. Cancelling an export with maybe_sent set leaves at the destination whatever it may have created; a fetch shows it as a remote entry by the viewer.

6.5 Uncertain sends #

Before a step with maybe_sent set is sent again, the adapter runs a complete fetch (§7) and looks for its marker.

If the fetch does not complete, the export fails again with retryable = 1. It is never sent without a completed check.
A match is a remote entry of that destination whose marker names the step's local item, whose author_id equals the export's account, and which, where the destination tells, no one else has edited (edited_by). With several matches, the one with the earliest created, then the lowest external_id, is used, and the others are reported to the user.
If the match is in the adapter's done state for that step (§6.2), the step is finished with it. If not (for GitHub, a pending review), it is recorded in progress, and sending resumes at the next unfinished step.
With no match, maybe_sent is cleared and the step is sent.
If the destination cannot hold markers, the app asks the user whether to send again, saying it may duplicate what was sent.

6.6 Failures and rate limits #

An adapter reports each failure as one of:

Kind	Examples	Result
not sent	DNS failure, connection refused, TLS failure, rate limit, a refusal before any write	`failed`, `retryable = 1`
maybe sent	timeout or reset after the request was written, a 5xx	`failed`, `retryable = 1`; on a creating step also `maybe_sent = 1`
permanent	validation, permission, gone	`failed`, `retryable = 0`, with the reason in `last_error`

The app offers retry only when retryable is set, and cancel always.

A rate limit pauses the host: every destination of that system and host gets paused_until (the retry-after time when given, else 60 s), and neither fetches nor sends until then.

7. Fetching #

A fetch reads one destination and upserts what it finds. It runs when a linked review is opened, when a subject with an attachment is opened, when the user asks, when attaching (§3.1), when planning (§6.1), after an export is sent (§6.4), and MAY run periodically (no more than once a minute per destination). Periodic fetches of a destination that is in use only through attachments run only while a window shows a subject attached to it. A fetch never runs for a destination that is paused or not confirmed, except when planning.

A fetch never writes to the destination, and MUST NOT change reviews, threads, evidence, coverage, entries, links, exports or attachments.

Every fetch is complete: it reads the whole destination.

The adapter reads the destination itself (for GitHub, the pull request) and records url, title, head, viewer and viewer_id, and, when it declares describe, the description columns (§5.1). If that fails with "not found" or "forbidden", the adapter cannot tell a deletion from lost access (an expired token, another account). The fetch stops with fetch_error, and nothing is marked deleted.
It increments generation, then reads every item, page by page. Each page is written in one transaction: upsert every item by (destination, external_id), setting last_fetched and seen_generation to the current generation and missed to 0. An item whose id does not match the adapter's syntax (§3) stops the fetch with fetch_error.
When every page has been read without error, and the checks for the head of step 1 (adapters that declare checks), in one transaction that also sets fetched, clears fetch_error and replaces the checks (§5.6): every row that is not deleted and has seen_generation below generation gets missed incremented, and those that reach 2 are marked deleted. Lists can skip an item (a write between pages, a lagging index), so one miss is not a deletion. An adapter MAY mark an item deleted at once when a direct read of it says "not found" while the destination itself is readable. A fetch that did not complete MUST NOT change missed or mark anything deleted; the next fetch starts again from step 1.
A destination holds at most 20000 remote entries. A fetch that would go past that stops with fetch_error, marks nothing deleted, and stops periodic fetches of it until the user asks.

Upserting by (destination, external_id) makes a fetch idempotent: fetching the same state twice changes nothing but last_fetched and seen_generation. Incremental fetching is not specified; an adapter that needs it comes with a migration.

After a fetch that marks deletions, the app SHOULD checkpoint the WAL, so that deleted text leaves the live files (backups keep it, format §4.6).

8. Credentials #

Credentials are never in the store. Tokens stay where they are kept today (the github.auth setting): the OS keychain, or gh auth token, read by the main process when needed and never sent to the renderer.

A token is looked up by the destination's exact host (for example gh auth token --hostname <host>, or the keychain entry for that host). A host with no token of its own gets none.
Adapters MUST build every request from host, locator and external ids that match the adapter's syntax. They MUST NOT send credentials to a URL taken from fetched data or from data, plan or progress, and MUST NOT follow a redirect to another host with them.
Credential lookup MUST NOT use git credential helpers or any repository-local config. Tools such as gh are found on the user's PATH (never in the current directory) and run with a fixed working directory outside every repository.
The store, its backups, the app's logs, error reports and anything sent to the renderer MUST NOT contain tokens, authorization headers or cookies. Adapters MUST scrub these from last_error, fetch_error, data and progress. destinations.viewer and viewer_id hold only public account names and ids.
Fetched bodies and URLs are stored as the destination returns them: they are what anyone with access there sees. Adapters SHOULD remove query parameters they know to grant access (for GitHub, jwt on private attachment links) from fetched URL columns.

An adapter's plan, progress and data JSON are private to it. It MUST read what its own earlier versions wrote, or fail the export with retryable = 0 for the user to cancel.

9. Later #

Not in this release, each coming with a migration:

Sending entries. Replies, resolutions, edits and retractions sent to a destination, and replies to fetched items. This needs an outbox of operations per destination, with the uncertain-send check of §6.5 on every creating operation, detection of edits made at the destination before an edit is sent, and rules for targets that were never sent, were placed in a review body, or were deleted there.
Exporting again. Sending threads added to, or changed in, a review after it was exported. Reviews are immutable, so today this is a re-review.
Drafts as pending reviews. Sending a draft to a destination that has pending reviews (GitHub's are visible only to their author), and replacing it as the draft changes.
Check logs. Fetching a failed check's log to show beside the code. It is more untrusted text, and larger, than anything fetched today.
More of the destination. Labels, assignees, requested reviewers, milestones, linked issues.
Reactions, and a local "concur", mapped to each other.
Several accounts per host. One viewer per destination is assumed.
Remote images and attachments in fetched bodies: they are not loaded (format §5.3), and not stored.
Retention of sent exports and tombstones.