scatter#

A service that broadcasts the requestCrawl HTTP request out to multiple known relays.

Overview#

In the #relay-operators channel in the ATProto Touchers Discord there have been discussions around building some tooling to help with doing admin tasks for indie relays.

Some of the issues we face today are:

• bootstrapping and continuously discovering the pds host list
• getting it to reconnect to pds hosts
• detecting and adjusting account limits per pds host (h/t fig for the above list)

The solution or part of the solution could be any one or all of these:

so the first form on my mind is a small server-side service that talks to the relay over the admin api. it can be on the same host as the relay, but doesn’t have to be.

it could be just a terminal ui, or a web app (even atproto oauth if you really wanted)

• fig

maybe even a common account that posts on bluesky whenever a new pds is about to hit 100 users (the default limit). operators could reply in the thread and decide if it’s spam or real.

• sri.xyz

yeah this is about what I was thinking, just a little service that deals with the scheduled items, and, personally, i think it'd be useful to have some form of better access control than a shared secret (admittedly this is of limited use when most relays atm are just operated by one person but could be useful in the future), then a web UI for controlling all this. to expand on the limit problem, would a 'trusted' PDS list that auto bumps be useful - could be good for cases like blacksky or northsky?

• Mia

For the scope of this experiment I'll just be playing around with the ideas fig presented in a thread within the channel, which are:

requestcrawl can be called for any host without any authentication, there could be a community service instance that handles periodically calling requestcrawl on your relay if you sign up for it

on the flip side, this service could also accept and broadcast requestCrawls on behalf of pds hosts to any registered relays

I think both ideas are achievable in one service so I'll try both and hopefully relay (heh) my findings back in the channel for feedback and iterating on this idea.

Shoutout to Mia, Fig and Sri !

Features#

• Broadcast to multiple relays - Forward requests to all healthy relays in parallel
• Automatic relay discovery - Pull relay list from a git repository
• Health checking - Periodic health checks mark unhealthy relays
• Retry logic - Automatic retries with configurable backoff
• Structured logging - JSON logs for production observability
• Graceful shutdown - Proper cleanup on SIGINT/SIGTERM

Usage#

Command Line Flags#

Environment variables are also supported (e.g., ADDR, GIT_REPO_PATH, etc.).

Running#

./scatter --git-repo-path=/path/to/relay-repo

Or with environment variables:

export RELAY_URLS_LIST=/url/to/list-of-relays
./scatter

API Endpoints#

GET /health#

Liveness probe. Returns 200 OK with body OK.

GET /relays#

List all registered relays with health status.

curl http://localhost:8080/relays

Response:

[
  {"url":"https://relay1.example.com","last_check":"2024-01-10T12:00:00Z","is_healthy":true},
  {"url":"https://relay2.example.com","last_check":"2024-01-10T12:00:00Z","is_healthy":false}
]

POST /xrpc/com.atproto.sync.requestCrawl#

Forward a request to all healthy relays.

curl -X POST http://localhost:8080/xrpc/com.atproto.sync.requestCrawl \
  -H "Content-Type: application/json" \
  -d '{"hostname":"example.com"}'

Response:

[
  {"url":"https://relay1.example.com","status":200,"attempts":1},
  {"url":"https://relay2.example.com","error":"connection timeout","attempts":2}
]

Architecture#

                    ┌─────────────┐
                    │   Client    │
                    └──────┬──────┘
                           │
                           ▼
                    ┌─────────────┐
                    │   Scatter   │
           /xrpc/com.atproto.sync.requestCrawl
                    └──────┬──────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
        ┌─────────┐  ┌─────────┐  ┌─────────┐
        │ Relay 1 │  │ Relay 2 │  │ Relay 3 │
        └─────────┘  └─────────┘  └─────────┘

Development#

# Build
go build ./...

# Test
go test ./...

# Run
go run .

TODO#

• [] pull and do crawl on service startup if no relay list present
• [] fallback to backup list???
• [] add some more logging
• [] more tests possibly