at-sdp
A CLI tool for Statically Defined PDSes — ship an atproto account & your lexicons with your wisp.place static site.
Star 8
Fork 0
Watch

Support wisp-based Websocket firehose! #8

Merged byjp.me
targeting main from native-firehose
Merged opened by and assigned to byjp.me
targeting main from native-firehose

Relays can now subscribe to an at-SDP repo when hosting with wisp.place as a wisp supporter.

build writes the repo's event log into the site (under xrpc/_sdp/events/) and adds wisp.place's opt-in line to the _redirects block; wisp.place then serves com.atproto.sync.subscribeRepos from that log, for sites on a supporter's custom domain.

What changes for people using it:

  • at-sdp build always writes the event log (--events auto|sync|commit, --event-retention and --event-bytes tune it), and warns when the domain is <something>.wisp.place, where the firehose won't be served (only custom domains are supported)
  • a #commit over wisp's 256 KiB frame cap goes out as a #sync instead
  • a build without the private key carries the live log forward, but fails rather than quietly dropping a log it can't read
  • at-sdp check --firehose dials the site and expects exactly the frames the build made, retrying until --timeout, as wisp can take up to a minute to pick up a new opt-in
  • at-sdp notify asks relays (bsky.network & atproto.africa by default; --relay to change) to crawl the site, but only once its own dial has upgraded — bsky.network only takes 50 new hosts a day across everyone, and 16 failed dials mark a host offline, so a site that isn't enabled shouldn't cost a relay anything
Commits 38
feat: notify asks Blacksky's relay as well as bsky.network
docs: say how to recover from a lost private key
feat: check says what it verified
chore: drop the exit-2 guard, which no CI depends on
docs: write at-SDP in the firehose docs
feat: wisptest sends the checkpoint after wisp's grace at an unservable frame
docs: wisp waits at a missing or oversized frame, as wisptest does
fix: only an HTML head.json means the site has no log
docs: name the bridge spec's superseded requirements
docs: relays dial subscribeRepos with a cursor but no cache-buster
refactor: unexport repogen's #commit limits
fix: check gives up on a request that never answers
fix: build doesn't retry a live file over wisp's limit
fix: build resets a live log holding a frame check would reject
fix: a keyless build fails rather than drop a live log it can't read
fix: a site's fallback page for head.json means no log, not an unusable one
docs: bring over the firehose spec and scrap SDP-7e3j
docs: describe wisp's native firehose
feat: hold every stored frame to wisp's 256 KiB cap
fix: check --firehose and notify dial past a cached 404
test: notify's usage errors never exit 2
fix: only wisp's 404 fall-through gets the supporter hint
fix: check says a build without an event log needs this at-sdp
test: check --firehose waits on an open socket for wisp's push
fix: check --firehose reports why its last full attempt failed
feat: add notify to request a crawl once the firehose answers
feat: check --firehose dials the site's firehose
feat: check requires wisp's firehose opt-in in the built _redirects
feat: check verifies the event log
test: wisptest's firehose tests don't race their read deadline
fix: a keyless build gives one warning, true whether or not a log is live
refactor: define the firehose rule in pkg/events
fix: MaxIndexBytes is 1,000,000 bytes
fix: build doesn't retry a live event log that hangs
test: wisptest serves subscribeRepos as wisp does
feat: build writes the firehose event log and wisp's opt-in rule
feat: describe commits as firehose events
feat: add the event log's shared types